Ciaren

Engineering notes and announcements

Ciaren blog

Why the project exists, how it's built, and where it's going, written by the people building it.

Follow along

RSS feed

Copy this URL into your feed reader to get every new post.

https://ciaren.com/blog/rss.xml

All posts

2 min readexcelparquet

How to Check an Excel to Parquet Conversion

Check that an Excel to Parquet conversion preserves leading zeros, keeps the right rows, and writes the types your downstream joins and reports expect.

Read post
3 min readdeduplicationdata-quality

Deduplicate Revisions with a Clear Winner

Keep one revision per supplier and SKU with a multi-column sort, an explicit tie-breaker, and checks that catch conflicting revisions before deduplication.

Read post
2 min readdata-cleaningnull-values

When Blank Values Should Trigger a Fallback

Choose a fallback column using a ticket-routing example. See how nulls, empty strings, and whitespace affect which source value wins in each row.

Read post
3 min readjoinsdata-quality

Why a Left Join Inflates Your Order Totals

Trace inflated order totals to repeated customer keys, fix the lookup before the left join, and check that each order still contributes its amount once.

Read post
3 min readpivot tablesaggregation

How to Pivot Monthly Return Rates

Pivot monthly return rates in Ciaren using units sold and returned. See why two stores with rates of 50% and 10% can have a combined return rate of 14%.

Read post
5 min readannouncementopen-source

Visual ETL That Exports Python: Why I Built Ciaren

Visual data tools are quick to start and hard to leave. Why I built Ciaren, an open-source visual ETL tool that exports plain pandas or Polars scripts.

Read post
5 min readlocal-firstprivacy

Local-First Data Tools: What It Means in Practice

What local-first means for a data tool, checked setting by setting: where it listens, where state and secrets live, what works offline, and its limits.

Read post
5 min readcode-exportpython

Visual ETL to Python: What a Good Code Export Looks Like

How to judge a visual ETL tool's Python export: no hidden runtime, readable names, secrets kept out, and tests that it matches the engine. Real examples.

Read post
6 min readarchitectureengineering

pandas vs Polars Code Generation: Ciaren's Architecture

How Ciaren runs one visual flow on pandas or Polars and exports matching code for both: the engine protocol, per-node codegen, run records, and plugins.

Read post