DataEngineer.io Newsletter

Home
Data Engineering Academy
Videos
Archive
Leaderboard
About
Top 3 LLM frameworks that you should know
There are a lot of libraries and frameworks coming out in the LLM space.
 • 
Zach Wilson
 and 
Filip Makraduli
How GenAI will impact data engineering
GenAI is catching fire over the last two years.
 • 
Zach Wilson
6
The Hard Truth About Bad Data Models
And how to fix them!
 • 
Zach Wilson

September 2024

My learnings from 7 failed tech interviews
Job interviews are stressful.
 • 
Zach Wilson
8
How to avoid pipeline backfill nightmares
Tired of the backfill process? Checkout these best practices to help you ease the pain of backfilling the data.
 • 
Zach Wilson
 and 
Junaid Effendi
6

August 2024

When to pick SparkSQL vs DataFrame vs Dataset
Spark offers so many different APIs and languages that it can be overwhelming which way is “best.”
 • 
Zach Wilson
1
Engineering cultures: Meta vs Netflix vs Airbnb
I spent seven years working at the behemoths Facebook, Netflix, and Airbnb and I learned a ton about culture over that period of time.
 • 
Zach Wilson
6
Data products = the future of data engineering
Data products are the systems and machinery that print big tech billions of dollars every year!
 • 
Zach Wilson
9

July 2024

How to pass data engineer interviews in 2024
Data engineering interviews can feel unpredictable and you don’t quite know what to expect!
 • 
Zach Wilson
3

June 2024

How I built a huge graph database of Netflix's cloud infrastructure
Microservices are amazing and painful at the same time!
 • 
Zach Wilson
3
How to choose between batch, micro-batch, and streaming when building a data pipeline
Speed is all the rage in data engineering nowadays.
 • 
Zach Wilson
4

May 2024

Why it's hard for data engineers to get promoted after senior engineer
And things you can do to overcome it!
 • 
Zach Wilson
22
© 2024 Zach Wilson
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great culture