Engineering notes and announcements
Ciaren blog
Why the project exists, how it's built, and where it's going, written by the people building it.
Follow along
RSS feed
Copy this URL into your feed reader to get every new post.
https://ciaren.com/blog/rss.xmlAll posts
How to Check an Excel to Parquet Conversion
Check that an Excel to Parquet conversion preserves leading zeros, keeps the right rows, and writes the types your downstream joins and reports expect.
Read postDeduplicate Revisions with a Clear Winner
Keep one revision per supplier and SKU with a multi-column sort, an explicit tie-breaker, and checks that catch conflicting revisions before deduplication.
Read postWhen Blank Values Should Trigger a Fallback
Choose a fallback column using a ticket-routing example. See how nulls, empty strings, and whitespace affect which source value wins in each row.
Read postWhy a Left Join Inflates Your Order Totals
Trace inflated order totals to repeated customer keys, fix the lookup before the left join, and check that each order still contributes its amount once.
Read postHow to Pivot Monthly Return Rates
Pivot monthly return rates in Ciaren using units sold and returned. See why two stores with rates of 50% and 10% can have a combined return rate of 14%.
Read postVisual ETL That Exports Python: Why I Built Ciaren
Visual data tools are quick to start and hard to leave. Why I built Ciaren, an open-source visual ETL tool that exports plain pandas or Polars scripts.
Read postLocal-First Data Tools: What It Means in Practice
What local-first means for a data tool, checked setting by setting: where it listens, where state and secrets live, what works offline, and its limits.
Read postVisual ETL to Python: What a Good Code Export Looks Like
How to judge a visual ETL tool's Python export: no hidden runtime, readable names, secrets kept out, and tests that it matches the engine. Real examples.
Read postpandas vs Polars Code Generation: Ciaren's Architecture
How Ciaren runs one visual flow on pandas or Polars and exports matching code for both: the engine protocol, per-node codegen, run records, and plugins.
Read post