Archive - DataEngineer.io Newsletter

The 2025 AI-enabled Data Engineering roadmap

AI is making manually writing complex data pipelines a thing of the past!

Apr 25 •

March 2025

The Battle of Data Lakes: Iceberg vs Delta vs Hudi

We’ve seen the rise and fall of Apache Hive as the default choice when it comes to data lakes.

Mar 20 •

How to pass data engineering SQL interviews in big tech

SQL and data engineering are like peanut butter and jelly.

Mar 5 •

How to design resilient and large scale data systems

Whenever you’re building a data system, there’s a million things to consider.

Mar 1 •

February 2025

January 2025

How to build scalable data lakes with Apache Iceberg

Tabular.io was acquired by Databricks back in June 2023 for two billion dollars.

Jan 12 •

December 2024

Year in Review: 2024 at DataExpert.io

2024 for a year of intense growth for me and DataEngineer.io!

Dec 27, 2024 •

November 2024

October 2024

September 2024

August 2024

When to pick SparkSQL vs DataFrame vs Dataset

Spark offers so many different APIs and languages that it can be overwhelming which way is “best.”

Aug 23, 2024 •

Engineering cultures: Meta vs Netflix vs Airbnb

I spent seven years working at the behemoths Facebook, Netflix, and Airbnb and I learned a ton about culture over that period of time.

Aug 14, 2024 •

Data products = the future of data engineering

Data products are the systems and machinery that print big tech billions of dollars every year!

Aug 3, 2024 •

July 2024

How to pass data engineer interviews in 2024

Data engineering interviews can feel unpredictable and you don’t quite know what to expect!

Jul 1, 2024 •

June 2024

May 2024

April 2024

The 2024 breaking into data engineering roadmap

Getting a data engineering job is complicated.

Apr 11, 2024 •

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts