ArchitectureApril 202611 min read
Lakehouse migrations: what breaks in month four
Cost, not correctness, is what usually ends these programmes. Here is where the spend hides.
Lakehouse migrations rarely fail on correctness. The tables land, the queries return the right answers, and the first two months look like a success story. Month four is when the finance business partner asks why the cloud bill has tripled.
The spend usually hides in three places. Small-file writes from streaming ingestion, which inflate storage operations far beyond what the data volume suggests. Unpartitioned or badly partitioned tables that force full scans for queries that should touch a fraction of the data. And compute left running because nobody owns the cluster policy.
All three are fixable, and all three are much cheaper to fix before the migration than after. Compaction and partition strategy should be decided alongside the ingestion design, not retrofitted once the tables are large enough to be painful to rewrite.
The organisational fix matters as much as the technical one. Put cost on the same dashboard as freshness and quality, owned by the same team. A platform where cost is somebody else's report will drift, every time.
If this describes where you are, the first conversation is a diagnosis rather than a pitch.
Book a consultation