Reshaping Data

Long vs Wide Format

Data comes in two shapes:

# LONG format (each observation is a row)
  name     subject  score
  Alice    Math     90
  Alice    Science  85
  Bob      Math     78
  Bob      Science  92

# WIDE format (each subject is a column)
  name     Math  Science
  Alice    90    85
  Bob      78    92

Long format is better for analysis (filtering, grouping). Wide format is better for reports and display. You’ll constantly convert between them.

Creating Pivot Tables

pd.pivot_table(
    orders,
    index="cust_id",
    values="total_order_cost",
    aggfunc=["sum", "count"]
)

Melting Wide to Long

pd.melt() is the reverse of pivoting. It takes columns and unpivots them into rows:

# Wide format: one column per metric
# employee has salary, bonus as separate columns
pd.melt(
    employee,
    id_vars=["first_name", "department"],
    value_vars=["salary", "bonus"],
    var_name="metric",
    value_name="amount"
)

The key parameters:

• id_vars — columns to keep as-is (identifier columns)
• value_vars — columns to unpivot into rows
• var_name — name for the new column holding the old column names
• value_name — name for the new column holding the values

When to Pivot vs Melt

• Data has one row per observation, you want a summary table → pivot_table()
• Data has metrics spread across columns, you want one row per metric → melt()
• Need to aggregate while reshaping → pivot_table() (has aggfunc)
• Just rearranging, no aggregation → melt() or pivot() (without _table)

Titanic Survivors and Non-Survivors

Inspections by Risk Category

An inspection can have several violations in different risk categories. First collapse the data to one row per inspection, keeping only its highest-severity category — rank the categories by severity, sort, and drop duplicates per inspection. Then build the pivot with pd.pivot_table(), add a total column, and sort by it descending.

Key Takeaways

• pd.pivot_table() converts long → wide with aggregation.
• pd.melt() converts wide → long (unpivoting).
• pd.crosstab() is a shortcut for frequency pivot tables.
• Long format for analysis; wide format for reports.
• index, columns, values, aggfunc are the four pivot_table parameters to know.

What’s Next

You’ve completed Module 4. You can now break complex analyses into clean steps, apply custom logic, and reshape data for any format. Module 5 covers dates, strings, and conditional logic — the functions that handle messy real-world data.