Archive - DataEngineer.io Newsletter

How to pass data engineer interviews in 2024

Data engineering interviews can feel unpredictable and you don’t quite know what to expect!

Jul 1 •

June 2024

How I built a huge graph database of Netflix's cloud infrastructure

Microservices are amazing and painful at the same time! On the amazing side, they allow teams to operate more independently. On the painful side, they…

Jun 20 •

How to choose between batch, micro-batch, and streaming when building a data pipeline

Speed is all the rage in data engineering nowadays.

Jun 4 •

May 2024

Why it's hard for data engineers to get promoted after senior engineer

And things you can do to overcome it!

May 21 •

How to learn data engineering more effectively

When building a career in data engineering, you’ll find yourself in a sea of technological choices and soft skills.

May 2 •

April 2024

The 2024 breaking into data engineering roadmap

Getting a data engineering job is complicated.

Apr 11 •

March 2024

How to convert your yearly review to a promotion in big tech

The dreaded review cycle usually comes once or twice a year at companies.

Mar 24 •

How to save millions by optimizing data pipeline shuffling

Shuffling isn’t just a fancy dance move! Shuffling is caused when you try to aggregate or join datasets in distributed environments like Spark or…

Mar 11 •

February 2024

How to go from senior to staff data engineer in big tech

In big tech, less than 3% of engineers make it to the staff level! If you want to beat the odds and actually become an L6+ engineer, this newsletter is…

Feb 25 •

How I scaled my $1m/year revenue startup's data model

Startups move quickly and we cut corners to get things out in front of people as quickly as we can to get feedback and iterate!

Feb 10 •

January 2024

How to design resilient and large scale data systems

Whenever you’re building a data system, there’s a million things to consider. Gone are the days of just shoving everything into MongoDB and calling it…

Jan 29 •

How to craft the perfect data engineer resume and LinkedIn profile in 2024

Layoffs are picking up this January. I’m sure y’all saw the layoffs announced by Twitch and Discord. There will be many more to come over the next few…

Jan 15 •

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts