Reshape Data in Polars: Pivot, Melt & Transpose

Data rarely comes in the shape you need. You often have to restructure it. Polars offers three powerful tools for this: pivot, melt, and transpose. Each solves a different problem. This guide explains them with clear examples.

Understanding reshaping is key to clean data analysis. It helps you prepare data for visualizations, machine learning, or reports. Let's dive into each method.

What is Reshaping?

Reshaping changes the structure of a DataFrame. It does not change the data itself. You move values between rows and columns. This makes data easier to work with.

Polars has two main categories: wide to long and long to wide. melt goes from wide to long. pivot goes from long to wide. transpose swaps rows and columns completely.

1. Pivot: Long to Wide

pivot turns unique values from one column into multiple columns. It aggregates data along the way. This is useful for creating summary tables.

Imagine you have sales data. Each row is a product and a quarter. You want one column per quarter. pivot does that.

Example: Pivot Sales Data

import polars as pl

# Sample long-format data
data = pl.DataFrame({
    "product": ["A", "A", "B", "B", "C", "C"],
    "quarter": ["Q1", "Q2", "Q1", "Q2", "Q1", "Q2"],
    "sales": [100, 150, 200, 250, 300, 350]
})

print("Original Data:")
print(data)

# Pivot: quarters become columns
pivoted = data.pivot(
    index="product",
    columns="quarter",
    values="sales",
    aggregate_function="sum"
)

print("\nPivoted Data (Wide Format):")
print(pivoted)

Original Data:
shape: (6, 3)
┌─────────┬─────────┬───────┐
│ product ┆ quarter ┆ sales │
│ ---     ┆ ---     ┆ ---   │
│ str     ┆ str     ┆ i64   │
╞═════════╪═════════╪═══════╡
│ A       ┆ Q1      ┆ 100   │
│ A       ┆ Q2      ┆ 150   │
│ B       ┆ Q1      ┆ 200   │
│ B       ┆ Q2      ┆ 250   │
│ C       ┆ Q1      ┆ 300   │
│ C       ┆ Q2      ┆ 350   │
└─────────┴─────────┴───────┘

Pivoted Data (Wide Format):
shape: (3, 3)
┌─────────┬──────┬──────┐
│ product ┆ Q1   ┆ Q2   │
│ ---     ┆ ---  ┆ ---  │
│ str     ┆ i64  ┆ i64  │
╞═════════╪══════╪══════╡
│ A       ┆ 100  ┆ 150  │
│ B       ┆ 200  ┆ 250  │
│ C       ┆ 300  ┆ 350  │
└─────────┴──────┴──────┘

The index parameter keeps rows unique. columns creates new column headers. values fills the cells. aggregate_function handles duplicates. Use "sum", "mean", "first", or "last".

Note:pivot is great for reporting. But it can create many columns if your column has many unique values. Use it wisely.

2. Melt: Wide to Long

melt is the opposite of pivot. It collapses multiple columns into key-value pairs. This is ideal for plotting or modeling.

You specify id_vars (columns to keep) and value_vars (columns to melt). The result has two new columns: "variable" and "value".

Example: Melt Wide Data

# Wide data from previous example
wide_data = pl.DataFrame({
    "product": ["A", "B", "C"],
    "Q1": [100, 200, 300],
    "Q2": [150, 250, 350]
})

print("Wide Data:")
print(wide_data)

# Melt back to long format
melted = wide_data.melt(
    id_vars="product",
    value_vars=["Q1", "Q2"],
    variable_name="quarter",
    value_name="sales"
)

print("\nMelted Data (Long Format):")
print(melted)

Wide Data:
shape: (3, 3)
┌─────────┬──────┬──────┐
│ product ┆ Q1   ┆ Q2   │
│ ---     ┆ ---  ┆ ---  │
│ str     ┆ i64  ┆ i64  │
╞═════════╪══════╪══════╡
│ A       ┆ 100  ┆ 150  │
│ B       ┆ 200  ┆ 250  │
│ C       ┆ 300  ┆ 350  │
└─────────┴──────┴──────┘

Melted Data (Long Format):
shape: (6, 3)
┌─────────┬─────────┬───────┐
│ product ┆ quarter ┆ sales │
│ ---     ┆ ---     ┆ ---   │
│ str     ┆ str     ┆ i64   │
╞═════════╪═════════╪═══════╡
│ A       ┆ Q1      ┆ 100   │
│ A       ┆ Q2      ┆ 150   │
│ B       ┆ Q1      ┆ 200   │
│ B       ┆ Q2      ┆ 250   │
│ C       ┆ Q1      ┆ 300   │
│ C       ┆ Q2      ┆ 350   │
└─────────┴─────────┴───────┘

If you omit value_vars, Polars melts all non-id columns. Use variable_name and value_name to rename the new columns. This is cleaner than the defaults.

Tip:melt is often used before GroupBy & Aggregations in Polars. Long format is standard for grouping operations.

3. Transpose: Swap Rows and Columns

transpose flips the DataFrame. Rows become columns and columns become rows. It is a full rotation of the data.

Use this when you need to switch the orientation. For example, when a dataset has observations as columns.

Example: Transpose a Small DataFrame

# Sample data
data = pl.DataFrame({
    "name": ["Alice", "Bob"],
    "age": [25, 30],
    "city": ["NYC", "LA"]
})

print("Original DataFrame:")
print(data)

# Transpose
transposed = data.transpose(
    include_header=True,
    header_name="attribute",
    column_names=["person1", "person2"]
)

print("\nTransposed DataFrame:")
print(transposed)

Original DataFrame:
shape: (2, 3)
┌───────┬─────┬──────┐
│ name  ┆ age ┆ city │
│ ---   ┆ --- ┆ ---  │
│ str   ┆ i64 ┆ str  │
╞═══════╪═════╪══════╡
│ Alice ┆ 25  ┆ NYC  │
│ Bob   ┆ 30  ┆ LA   │
└───────┴─────┴──────┘

Transposed DataFrame:
shape: (3, 3)
┌───────────┬─────────┬─────────┐
│ attribute ┆ person1 ┆ person2 │
│ ---       ┆ ---     ┆ ---     │
│ str       ┆ str     ┆ str     │
╞═══════════╪═════════╪═════════╡
│ name      ┆ Alice   ┆ Bob     │
│ age       ┆ 25      ┆ 30      │
│ city      ┆ NYC     ┆ LA      │
└───────────┴─────────┴─────────┘

include_header=True keeps the original column names as a new column. header_name renames that column. column_names sets new column headers for the transposed data.

Caution:transpose changes data types. All values become strings if columns have mixed types. Check your schema after transposing. Use Polars Data Type Casting & Schema Management to fix types.

When to Use Each Method

Choose the right tool for your task:

• Pivot: Create summary tables. Turn categories into columns.
• Melt: Prepare data for plotting or modeling. Stack multiple columns into one.
• Transpose: Flip the entire dataset. Use when rows and columns are swapped.

These methods are often combined. You might melt data, then pivot it differently. Polars makes this chain easy.

Performance Tips

Polars is fast, but reshaping can be heavy. Keep these in mind:

• Use pivot with aggregate_function to avoid errors from duplicates.
• For large data, melt is very efficient. It is a streaming operation.
• transpose loads the entire DataFrame into memory. Only use it on small data.

Combine reshaping with other operations. For example, filter rows first with Select Columns & Filter Rows in Polars before pivoting. This reduces data size.

Conclusion

Reshaping data is a core skill in Polars. pivot, melt, and transpose cover almost every need. Practice each method with your own data. Start with small examples. Then apply them to real datasets.

Remember: wide format is for humans, long format is for machines. Use pivot for reports. Use melt for analysis. Use transpose for quick orientation changes.

Polars makes reshaping simple and fast. Master these tools to handle any data shape. Your analysis will become cleaner and more powerful.