Year in Review: 2024 at DataExpert.io

2024 for a year of intense growth for me and DataEngineer.io! In this article, we’ll be going over each quarter of the business and how we pivoted and executed for another successful year!

By the numbers:

• Made 1,994 commits on Github

• Made 594 LinkedIn posts

• Wrote 23 newsletter articles

  • Top Articles

    • The 2024 Breaking into Data Engineering roadmap

    • My learnings from 7 failed tech interviews

    • Engineering cultures Meta vs Airbnb vs Netflix

    • How to design large scale and resilient data systems

• Delivered 4 keynotes in Australia and New Zealand

• Uploaded 50+ YouTube videos

• Gained 25,000 stars on Github:

  • The Data Engineer Handbook (+23,000 stars)

  • LLM Driven Data Engineering (+900 stars)

  • Cumulative Table Design (+600 stars)

  • Little Book of Pipelines (+200 stars)

• Taught 1500+ students in paid boot camps

• Taught 30,000+ students in free boot camps

• Gained 350,000 followers across platforms

• Hired 3 people:

  • Aida Bozulan to do front end work for DataExpert.io

  • Wan Woo Chang to work on data platform stuff for DataExpert.io

  • Bruno Souza de Lima to be the Snowflake and dbt instructor for DataExpert.io

First quarter of 2024

In January 2024, this is what our architecture looked like!

Our priorities in Q1 were:

• Start building a broader online presence on YouTube

  • We did many free live streams covering topics:

    • Self-joins in Trino

    • Advanced Window functions in Trino

    • Data Lake Fundamentals with Iceberg

    • Dimensional Data Modeling with Parquet and Iceberg

• Refining the TechCreator platform

  • We greatly improved the Query editor experience

  • We AB tested logged out vs logged in purchases (logged out purchases are better)

  • We migrated off of jwt authentication onto Clerk.com authentication after a script kiddie abused our sign up form!

• Make May boot camp to be 100% in the cloud

  • We set up partnerships with Astronomer for managed Airflow and Starburst for managed Trino.

  • We moved off of Heroku Kafka onto Confluent Kafka and breathed a sigh of relief!

Second quarter of 2024

By the end of Q2 2025, this is what our architecture looked like!

Our priorities in Q2 were:

• Delivered a solid May + June boot camp

  • We didn’t do as great of a job as I would have liked. Reviews dropped a bit here from 4.8 stars to 4.5 stars.

    • The main reason for this is because:

      • The boot camp was the biggest we’ve ever done, over 400 students

      • We shifted too much of the curricula at once

• Launched LinkedInExpert.io with Stephanie Nuesi and Carly Taylor

  • Our students gained over 3000 followers on average over the 6 week boot camp!

• Built AI-based autograding into the boot camp for a better student experience

  • We built a Github based example here

• Get subscriptions working for more consistent revenue

• Moved from teaching Flink to teaching Spark Streaming

  • This was a last minute decision that definitely hurt boot camp reviews. Although getting the boot camp 100% off docker was a great feeling!

• Fired half my team to focus more on profitability vs revenue

  • I was very stress at the end of June and decided to downsize my company quite a bit to refocus on the parts we’re doing right!

Third quarter of 2024

Our Q3 priorities were:

• Launched FullStackExpert.io with Dan Bowden

  • This was a big success with a cohort of over 50 students!

• Do creator houses to spark creative juices:

  • First one in Salt Lake City, Utah with Fritz Johnson, Carly Taylor, Arin Goldsmith, and Annie Nelson

  • Second one in Murray, Utah with Fritz Johnson and Madeline Zhang

  • Third one in Pacifica, California with Delia Lazarescu, Li Yin, Clarissa Eyu, and Madeline Zhang

• Solidify subscription revenue and slow down (Q3 was the slowest quarter by far)

• Launched an Analytics Engineering boot camp in September

  • The Capstone projects were insane!

Fourth quarter of 2024

Our Q4 priorities were:

• Launched the free data engineering boot camp and 12 week data engineering challenge

• Scaled up the TechCreator.io platform to handle 30,000 free students using it!

• Grew DataExpert.io daily active users from 300 to ~3000!

  • 

    You can see exactly the date when I launched the free boot camp!

• Launched the January 6th New Year Boot camp

  • Gathered amazing guest speakers

    • Joe Reis - author of best selling book Fundamentals of Data Engineering

    • Sundas Khalid - principal analytics lead at Google, 300k+ on IG, 250k+ on YouTube

    • Jason Reid - cofounder of Tabular, the Apache Iceberg company

    • Shachar Meir - ex Data Engineering Director at Meta

    • Yuzheng Sun - data scientist at Statsig

  • Decided to commit to using both Snowflake and Databricks for V5

• Adding Snowflake capabilities to the Query Editor page (launching soon)

Looking forward

I’m excited for 2025 to scale up and have more cohorts and more boot camp!

Some things on the horizon:

• I bought AIExpert.io as a domain for a more LLM and AI-focused boot camp!

• I bought MLOpExpert.io and SQLExpert.io for more focused boot camps!

• If you’re an educator interested in learning how to use TechCreator.io for your cohorts, please reach out and talk with me!

• If you’re still interested in joining the January 6th boot camp, you can use code LAUNCH20 for 20% off here