In big tech, less than 3% of engineers make it to the staff level! If you want to beat the odds and actually become an L6+ engineer, this newsletter is for you. In my time in big tech, I went from junior to staff data engineer in four years. I’ll unveil the learnings I have from that journey.
In this article we’ll talk about:
How staff data engineer is different from senior
What behaviors you should change to get to the next level
What strategies you can employ to get promoted faster
What things should staff+ data engineers know
How is staff engineer different from senior
Once you get to a certain point in your engineering career, you start to realize that you’re limited by how much code you can output in a given time frame. This realization is why big tech companies have “terminal levels.” At most companies, the terminal level is senior engineer.
What terminal level means is you can be a senior engineer for life without pressure to continue to climb up the ladder. And this is a great position to be in. Senior engineers in big tech make $300k-400k/year!
Because there isn’t pressure on engineers to get promoted beyond senior, it becomes exponentially more difficult to get the higher-on-the-ladder roles!
Senior engineers on the road to staff recognize that their code alone won’t get them there. They need to find LEVERAGED opportunities for impact.
For senior engineers, they are trusted to execute and deliver on a large-scale project without much help.
For staff engineers, they are trusted to IDENTIFY the opportunities for their team to deliver value.
What behaviors you should change to get to the next level
The change from execution-focus to impact-identifying-focus is a brand new skill set that many engineers do not care to develop.
After you’ve identified large opportunities for your team, you should DELEGATE the pieces that can be delivered by other engineers.
For example, at Airbnb, I worked on refactoring the core pricing and availability pipelines. While I delegated the pricing settings and availability settings pipelines to other engineers on my team.
Consistently finding new leveraged opportunities is THE key skill for being successful as a staff engineer.
These leverage opportunities can be both horizontal and vertical!
Horizontal
Horizontal impact is all about addressing the HOW work gets done. Maybe you employ a new process for your team that minimizes data quality errors. Getting your team to ADOPT these processes is how you leverage your impact and scale across many engineers.
Horizontal impact also looks like interviewing and setting the standard for who gets hired at the company. Preventing bad hires and finding the rockstars is sometimes a thankless but extremely important task that will demonstrate next level impact!
Vertical
Vertical impact is all about addressing business problems in a scalable way. For example, at Airbnb, I found that the availability definition we used was incomplete both in terms of legacy and new capabilities coming online. Building out a new pipeline that managed the new capabilities allowed our smart pricing machine learning to scale to new customers like large hotels! Unlocking new business opportunities through your code is how you build leveraged vertical impact!
What strategies you can employ to get promoted faster
The very first thing you need to realize about getting promoted is that your manage doesn’t really care about that. You need to be your biggest advocate! Talking about promotion timelines with your manager on a monthly basis is so important so “nobody is surprised.”
I made this mistake at Facebook in 2018. I thought I was 2-3 months away from senior engineer and my manager told me it was more like a year away and this caused me to get really angry and leave. I ended up getting a nice senior role at Netflix from it so this strategy ended up working for me!
The first things you need to ask yourself are:
Can I get realistically get promoted internally with my current manager’s support?
If the answer to this is no, you should find a new team (either internally or externally) as quickly as possible. But also remember this promotion takes on average three years to get!
If the answer is year, then you should build a promo plan with them and be grateful that you have a solid manager!
How can I build my brand to have leadership trust me with more important decisions?
As much as engineers hate it, politics is part of staff+ engineering. You should be brand building right now. Both internally by helping others in your company go further. And externally by giving talks at conferences and/or making content!
If you’re interviewing outside, do you have a compelling narrative for the interviewers?
If you’re trying to get a staff+ position, you better have an impact narrative from your previous companies that will help you get there. This is my number one rule for job hopping!
What things should staff+ data engineers know
Staff engineers don’t generally focus on a single pipeline. Or if they do, it’s a business critical one!
Things you’ll want to know to make it this far:
You’ve mastered distributed compute. You know how to optimize all types of pipelines of arbitrary scales
You know how to insert data quality into every type of pipeline whether that be streaming or batch.
You are both a software engineer AND a data engineer. You know the fundamentals around continuous integration and unit testing. You catch data quality errors BEFORE they enter production
You know data modeling. You know the tradeoffs between dimensional data modeling and one big table data modeling.
You know how to pick the right technology for the job. Should you ACTUALLY use streaming for that use case or is that bringing only marginal value?
You know about data architecture
Can you sanely talk about lambda vs kappa architecture?
What about the ins and outs of CAP theorem?
You know how to sell the impact of things and build a compelling narrative
If you’re good at identifying leveraged impact but not at selling your team on executing on it, you won’t be successful as a staff data engineer!
What other things do you think are important to get to that staff engineer level? If you liked this content, please share it with your friends!
I do weekly free lives on Thursday from 5:00 to 6:30 PM Pacific about SQL, data modeling, data lakes, etc. To get the most out of these lives, having a free DataExpert.io account is important.
Thank you so much for your support. I love building this data engineering community!
This is an insanely useful post. I had one question though - What does internal brand building look like?
I read the whole thing and still says 14% read lol.
Anyways, do you think its possible to become Staff without making org wide impact?