there are a few ways: using .pivot: >>> origin.pivot(index=’label’, columns=”type”)[‘value’] type a b c label x 1 2 3 y 4 5 6 z 7 8 9 [3 rows x 3 columns] using pivot_table: >>> origin.pivot_table(values=”value”, index=’label’, columns=”type”) value type a b c label x 1 2 3 y 4 5 6 z 7 8 … Read more
From data.table 1.14.1 (in development; installation), the new function measure makes it much easier to melt data with concatenated variable names to a desired format (see ?measure. The separator argument is used to create different groups of measure.vars. In the … argument, we further specify the fate of the values corresponding to the groups generated … Read more
You can melt your dataframe: >>> keys = [c for c in df if c.startswith(‘key.’)] >>> pd.melt(df, id_vars=”topic”, value_vars=keys, value_name=”key”) topic variable key 0 8 key.0 abc 1 9 key.0 xab 2 8 key.1 def 3 9 key.1 xcd 4 8 key.2 ghi 5 9 key.2 xef It also gives you the source of the … Read more
reshape does this with the appropriate arguments. varying lists the columns which exist in the wide format, but are split into multiple rows in the long format. v.names is the long format equivalents. Between the two, a mapping is created. From ?reshape: Also, guessing is not attempted if v.names is given explicitly. Notice that the … Read more
There is no built-in function (if you work with SQL and Hive support enabled you can use stack function, but it is not exposed in Spark and has no native implementation) but it is trivial to roll your own. Required imports: from pyspark.sql.functions import array, col, explode, lit, struct from pyspark.sql import DataFrame from typing … Read more
Note for users with pandas version under < 0.20.0, I will be using df.melt(…) for my examples, but your version would be too low for df.melt, you would need to use pd.melt(df, …) instead. Documentation references: Most of the solutions here would be used with melt, so to know the method melt, see the documentaion … Read more