Archive - DataExpert.io Newsletter

How I got a 12x speed up in a 50 TB pipeline at Meta

Data Orchestration Challenges I Faced at Airbnb, Netflix & Facebook – Part III

Aug 4 •

Error

Error

July 2025

Scaling Netflix's threat detection pipelines without streaming

Data orchestration challenges I faced at Netflix, Airbnb, & Facebook (Part II)

Jul 25 •

Error

Error

How I cut Airbnb's Pricing pipeline backfill time 95%

Data Orchestration challenges I faced in my years at Airbnb, Netflix & Facebook (Part I)

Jul 18 •

Error

Error

June 2025

How to join the Free 6-week Data Engineer Boot Camp!

I’m releasing a free six week program for data engineers to level up and get better!

Jun 18 •

Error

Error

April 2025

The 2025 AI-enabled Data Engineering roadmap

AI is making manually writing complex data pipelines a thing of the past!

Apr 25 •

Error

Error

March 2025

The Battle of Data Lakes: Iceberg vs Delta vs Hudi

We’ve seen the rise and fall of Apache Hive as the default choice when it comes to data lakes.

Mar 20 •

Error

Error

How to pass data engineering SQL interviews in big tech

SQL and data engineering are like peanut butter and jelly.

Mar 5 •

Error

Error

How to design resilient and large scale data systems

Whenever you’re building a data system, there’s a million things to consider.

Mar 1 •

Error

Error

February 2025

Unstructured Data is the Future of Data Engineering!

Data engineering is changing very quickly in 2025!

Feb 22 •

Error

Error

Change Data Capture (CDC) is taking over the world!

If you go back ten years, you’ll see that daily dimensional snapshotting ruled (and continues to rule) the world.

Feb 2 •

Error

Error

January 2025

How to build scalable data lakes with Apache Iceberg

Tabular.io was acquired by Databricks back in June 2023 for two billion dollars.

Jan 12 •

Error

Error

December 2024

Year in Review: 2024 at DataExpert.io

2024 for a year of intense growth for me and DataEngineer.io!

Dec 27, 2024 •

Error

Error

November 2024

How I prepared for a staff data engineer interview at Airbnb

Two weeks before the pandemic started in 2020, I decided to quit my job at Netflix to travel the world.

Nov 27, 2024 •

How to enroll in my Free Data Engineer Boot Camp

On November 15th, I launched a free 6 week data engineering boot camp! It’s self-paced and you can enroll here.

Nov 17, 2024 •

October 2024

September 2024

My learnings from 7 failed tech interviews

Job interviews are stressful.

Sep 19, 2024 •

How to avoid pipeline backfill nightmares

Tired of the backfill process? Checkout these best practices to help you ease the pain of backfilling the data.

Sep 16, 2024 •

and

August 2024

When to pick SparkSQL vs DataFrame vs Dataset

Spark offers so many different APIs and languages that it can be overwhelming which way is “best.”

Aug 23, 2024 •

Engineering cultures: Meta vs Netflix vs Airbnb

I spent seven years working at the behemoths Facebook, Netflix, and Airbnb and I learned a ton about culture over that period of time.

Aug 14, 2024 •

Data products = the future of data engineering

Data products are the systems and machinery that print big tech billions of dollars every year!

Aug 3, 2024 •

July 2024

How to pass data engineer interviews in 2024

Data engineering interviews can feel unpredictable and you don’t quite know what to expect!

Jul 1, 2024 •

June 2024

How I built a huge graph database of Netflix's cloud infrastructure

Microservices are amazing and painful at the same time!

Jun 20, 2024 •

ErrorError

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts