Ciaren

Window function

Window function — windowFunction

Compute a window/analytics value into a new column, optionally scoped to a partition and ordered within it. Row order is preserved; the result is added as a new column.

Use cases

  • Rank rows within each group (top product per region).
  • Running totals, cumulative max/min over an ordered key.
  • Compare a row to the previous/next one with lag/lead.

What it does

A window function adds a new column computed from a window of rows — scoped to a partition (group) and ordered within it. The original row order is preserved; the calculation happens internally and the result is added alongside the existing columns.

Below: cumsum partitioned by region and ordered by date adds a per-region running total without collapsing rows.

Before
regiondateamount
North2024-01-01100
North2024-01-02150
South2024-01-0180
South2024-01-02200
4 rows · 3 cols
Window: cumsum (partition_by=region, order_by=date, target=amount) → running_total
After
regiondateamountrunning_totalnew
North2024-01-01100100
North2024-01-02150250
South2024-01-018080
South2024-01-02200280
4 rows · 4 cols

Configuration

Config keyTypeRequiredDescription
functionstringYesrow_number, rank, dense_rank, cumcount, cumsum, cummax, cummin, lag, lead
new_columnstringYesName of the column to add
partition_bystring[]NoRestart the window within each group (empty = whole table)
order_bystring[]ConditionalRow order within the window; required for rank/dense_rank
targetstringConditionalValue column; required for cumsum/cummax/cummin/lag/lead
offsetintNoShift distance for lag/lead (default 1)
descendingboolNoOrder descending (default false)

Generated Python code

# function: cumsum, partition_by: ['region'], order_by: ['date'], target: 'amount'
df_2 = df_1.assign(running_total=lambda _d: _d.sort_values('date', kind='stable').groupby('region', sort=False)['amount'].cumsum())

Tips & common mistakes

  • Each function needs its own inputs: ranking needs order_by; value functions (cumsum, cummax, cummin, lag, lead) need a target.
  • rank/dense_rank rank by the first order_by column.
  • For lag/lead, rows at the window edge with no neighbor are null.
  • Use partition_by to restart the calculation per group; leave it empty to run across the whole table.

Row order in the output is preserved — the window sorts internally and restores the original order, so this node is safe to place anywhere.

See also