Phase 1 – Python & SQL Foundations (Days 1–20)

Python Core (Days 1–10)

Note

• Use Google Colab so you can avoid local installation and IDE setup.

• Emphasis is on data cleaning, validation, and transformation, not building full applications.

Day 1

• Variables and assignment

• Core data types: str, int, float, bool

• Using type() to inspect objects

Day 2

• Common string operations

• Basic numeric operations

• Type conversions between str, int, and float

Day 3

• Boolean logic and truth values

• if / elif / else control flow

• Writing clear conditional checks

Day 4

• for loops

• while loops

• Loop control with break and continue

Day 5

• Lists and when to use them

• Tuples and immutability

• Indexing and slicing sequences

• Intro to list comprehensions

Day 6

• Dictionaries (key–value mappings)

• Sets and set operations

• Working with nested data structures (lists of dicts, dicts of lists, etc.)

Day 7

• Defining functions with def

• Function parameters and arguments

• Returning values and understanding function scope

Day 8

• Basic exception handling with try / except

• Reading files from disk

• Writing data back to files

Day 9

• Loading and working with JSON data

• Reading and writing CSV files

• Introduction to Python’s logging module

Day 10

• Build a small end-to-end Python script

• Add robust error handling

• Integrate logging

• Walk through and explain your code in detail

SQL for Data Engineers (Days 11–20)

Day 11

• SELECT, WHERE, ORDER BY

• Thinking through filtering logic

Day 12

• GROUP BY and aggregate functions

• Using HAVING to filter aggregated results

Day 13

• Different types of joins (INNER, LEFT, RIGHT, FULL)

• Practicing how to interpret join outputs

Day 14

• Basic subqueries

• Correlated subqueries and when they’re useful

Day 15

• Common Table Expressions (CTEs) with WITH

• Comparing CTEs vs subqueries in terms of readability and reuse

Day 16

• Window functions such as ROW_NUMBER and RANK

• PARTITION BY fundamentals

Day 17

• Windowed SUM() OVER() calculations

• Implementing running totals and similar patterns

Day 18

• Views vs materialized views

• Typical use cases and performance implications

Day 19

• Timed SQL practice similar to interviews

• Aim for 5–8 questions under time pressure

Day 20

• Consolidated SQL revision

• Explaining a single, reasonably complex SQL query end to end

Phase 2 – Pandas & Core Data Engineering Concepts (Days 21–30)

Pandas Data Manipulation (Days 21–26)

Day 21

• Difference between Series and DataFrame

• Reading data from CSV and JSON into Pandas

Day 22

• Selecting rows and columns

• Filtering data with conditions

• Column-level operations and derived columns

Day 23

• Handling missing data (drop vs fill strategies)

• Working with datetime columns

• Useful string operations in Pandas

Day 24

• merge, join, and concat for combining datasets

• groupby with aggregations for summaries

Day 25

• Writing data out as CSV and Parquet

• Reading large files in chunks

Day 26

• Mini Pandas project:

  • Clean raw data

  • Apply transformations

  • Write the final dataset to disk

Core Data Engineering Concepts (Days 27–30)

Day 27

• ETL vs ELT: what they mean and when to use each

• OLTP vs OLAP workloads and characteristics

Day 28

• Basics of dimensional modeling

• Difference between fact and dimension tables

• Slowly Changing Dimensions (SCD Type 1 & Type 2)

Day 29

• Introduction to orchestration tools (focus on Airflow concepts)

• Reading and understanding an Airflow DAG and its tasks

Day 30

• Comparing Spark and Pandas (when Spark is the better choice)

• PySpark basics: read → transform with groupBy → write

Phase 3 – Cloud & Modern Data Stack (Days 31–40)

Day 31

• Docker fundamentals

• Containerizing a simple Python ETL script

Day 32

• Core ideas of cloud computing (IaaS, PaaS, SaaS)

• High-level AWS overview

Day 33

• Concepts of AWS S3 and IAM

• Comparing RDS with Redshift and when to use each

Day 34

• Snowflake basics

• Snowflake architecture and common use cases

Day 35

• Databricks and the Lakehouse paradigm

• Explanation of Delta Lake and why it matters

Day 36

• Infrastructure as Code (Terraform basics)

• Small example: provisioning S3 and IAM with Terraform

Day 37

• CI/CD concepts for data pipelines (e.g., GitHub Actions)

• Running and testing pipelines on each commit

Day 38

• Designing config-driven pipelines

• Managing secrets and environment variables safely

Day 39

• Monitoring and observability for data systems

• Core metrics: duration, failures, retries, and alerting

Day 40

• Cloud and architecture revision

• Explaining an end-to-end data pipeline spanning ingestion to consumption

Phase 4 – Project & Interview Readiness (Days 41–60)

Project Build (Days 41–50)

Day 41

• Define project scope, data sources, and overall design

Day 42

• Implement data ingestion logic (batch or streaming, as appropriate)

Day 43

• Build transformation logic in Pandas

• Add tests around transformations

Day 44

• Design incremental load logic

• Ensure the pipeline can be restarted safely

Day 45

• Write output data as Parquet

• Apply sensible partitioning strategies

Day 46

• Learn streaming and Change Data Capture (CDC) concepts

• Kafka fundamentals: producers, consumers, and topics

Day 47

• Load data into a warehouse or database (e.g., Postgres or Snowflake)

Day 48

• Strengthen logging and error handling in the project

• Introduce basic data governance concepts (PII handling, masking)

Day 49

• Write a clear README and project documentation

• Optionally explore dbt models and dbt docs

Day 50

• Finalize the project

• Perform a structured review and note improvements

Interview Preparation (Days 51–60)

Day 51

• Python interview-style questions (syntax, logic, data structures)

Day 52

• Pandas-focused interview problems and small exercises

Day 53

• Timed SQL practice similar to real interviews

Day 54

• Spark / PySpark conceptual and practical questions

Day 55

• Cloud-related scenario questions (AWS, data platforms, architecture)

Day 56

• ETL/ELT design questions and trade-off discussions

Day 57

• Mock interview (self-recording or with a peer)

Day 58

• Identify weak areas based on mock interviews

• Do targeted revision on those topics

Day 59

• Refine and polish your resume

• Write strong project bullet points that highlight impact

Day 60

• Final confidence check and quick revision

• Start applying to data engineering roles

Key Skills You Should Be Ready to Explain by Day 60

By the end of this plan, you should be able to confidently discuss:

• Python fundamentals and file handling patterns

• SQL, including window functions, CTEs, and views

• How to choose between Pandas and Spark for different workloads

• ETL/ELT pipeline design, from ingestion to serving

• Orchestration concepts (e.g., Airflow DAGs and tasks)

• Cloud data platforms such as Snowflake, Databricks, Redshift, and S3

• A mini data pipeline using Docker, CI/CD, and structured logging

• Core monitoring and observability ideas for data pipelines

• Streaming basics, including Kafka and CDC-style architectures