All Data Engineering Courses - Spark, Kafka, Airflow, dbt, Snowflake &

1. Data Engineering Basics

Introduction to Data Engineering

Introduction to Data Engineering with over 1 hour of videos including my journey here.

Andreas Kretz

Computer Science Fundamentals

A complete guide of topics and resources you should know as a Data Engineer.

Andreas Kretz

Introduction to Python

Learn all the fundamentals of Python to start coding quick

Amit Jain

Amit Jain

Python for Data Engineers

With this comprehensive course, you will expand your Python skill set to tackle complex Data Engineering projects with ease.

Andreas Kretz

Data Preparation & Cleaning for Machine Learning

All you need for preparing data to enable Machine Learning

Andreas Kretz

Docker Fundamentals

Learn all the fundamental Docker concepts with hands-on examples

Andreas Kretz

SQL for Data Engineers

Learn fundamentals and advanced techniques for real-world data challenges

Andreas Kretz

Successful Job Application

Everything you need to successfully apply for Data Engineering jobs

Andreas Kretz

Becoming a Better Data Engineer

Master the core principles of Data Engineering and build smarter pipelines

Andreas Kretz

The Hidden Foundation of GenAI

Learn Embeddings - The Core of Search, RAG, and LLMs

Andreas Kretz

2. Platform & Pipeline Design Fundamentals

Data Platform And Pipeline Design

Learn how to build data pipelines with templates and examples for Azure, GCP and Hadoop.

Andreas Kretz

Platform & Pipelines Security

Learn the important security fundamentals for Data Engineering

Andreas Kretz

Choosing Data Stores

Learn the different types of data stores and when to use which.

Andreas Kretz

Data Modeling 1

Learn to create the data models for SQL, NoSQL databases and Data Warehouses

Andreas Kretz

Data Modeling 2 - Relational Data Modeling

Learn how to model your data for relational databases

Andreas Kretz

Data Modeling 3 - Dimensional Data Modeling

Learn how to model your data for analytical data stores with dimensional modeling

Andreas Kretz

3. Fundamental Tools

Building APIs with FastAPI

Learn the fundamentals of designing, creating and deploying APIs with FastAPI and Docker

Andreas Kretz

Apache Kafka Fundamentals

Learn the fundamentals of Apache Kafka

Andreas Kretz

Apache Spark Fundamentals

Apache Spark quick start course in Python with Jupyter notebooks, DataFrames, SparkSQL and RDDs.

Andreas Kretz

Data Engineering on Databricks

Everything you need to get started with Databricks. From setup to building ETL pipelines & warehousing.

Andreas Kretz

MongoDB Fundamentals

Learn how to use MongoDB

Andreas Kretz

Log Analysis with Elasticsearch

Learn how to use Elasticsearch to monitor and debug your pipelines through log indexing

Andreas Kretz

Airflow Workflow Orchestration

Learn how to orchestrate your data pipelines with Apache Airflow

Andreas Kretz

Snowflake for Data Engineers

Everything you need to get started with Snowflake

Andreas Kretz

dbt for Data Engineers

Everything you need to transform data with dbt and Snowflake

Andreas Kretz

Apache Iceberg Fundamentals

Get hands-on with Apache Iceberg and build a complete local Lakehouse setup using Spark, Docker, and MinIO.

Andreas Kretz

DuckDB for Data Engineers: From Local to Cloud with MotherDuck

Learn how to build data workflows that run on your laptop, in the cloud, or across both using DuckDB and MotherDuck.

Andreas Kretz

Spark Declarative Pipelines & Lakeflow Designer on Databricks

Learn how to move from notebook-based workflows to structured, production-ready pipelines in Databricks using a modern declarative approach.

Andreas Kretz

High-Performance Data Analysis with Exasol

Exasol is a massively parallel analytical database built for speed. This course gives you a practical, hands-on introduction to it. You'll deploy Exasol Personal on AWS, run complex analytical queries on realistic datasets, and write Python UDFs that execute custom logic directly inside the database. You'll also explore in-database ML and LLM inference, so you can apply models to your data without exporting it. By the end, you'll know where Exasol fits in a modern data architecture and how to use its core features in real analytics workflows.

Andreas Kretz

FREE

4. Full Hands-On Example Projects

Data Engineering on AWS

Full 5 hours course with complete example project. Building stream and batch processing pipelines on AWS.

Andreas Kretz

Data Engineering on Azure

Ingest, Store, Process, Serve and Visualize Streams of Data by Building Streaming Data Pipelines in Azure.

Kristijan Bakarić

Kristijan Bakarić

Data Engineering on GCP

Build a full end-to-end project will all important GCP tools.

Andreas Kretz

Building a Lakehouse on AWS and GCP

How to integrate a Data Lake with a Data Warehouse and query data directly from files

Andreas Kretz

Machine Learning & Containerization On AWS

Build a app that analyzes the sentiment of tweets and visualizing them on a user interface hosted as container

Andreas Kretz

Contact Tracing with Elasticsearch

Track 100,000 users in San Francisco using Elasticsearch and an interactive Streamlit user interface

Andreas Kretz

Streaming with Kafka & Spark

Streaming of JSON documents with FastAPI, Kafka, Spark Streaming, MongoDB and Streamlit

Andreas Kretz

Storing & Visualizing Time Series Data with InfluxDB and Grafana

Learn how to use InfluxDB to store time series data and visualize interactive dashboards with Grafana

Andreas Kretz

Data Engineering with Hadoop

Hadoop Project with HDFS, YARN, MapReduce, Hive and Sqoop!

Suyog Nagaokar

Dockerized ETL With AWS, TDengine & Grafana

Everything you need to find your dream job

Andreas Kretz

Azure Data Pipelines with Terraform and Databricks

Build an ETL pipeline with Terraform, Data Factory and Synapse

Andreas Kretz

Semantic Log Indexing & Search

Build a Dockerized, log search app powered by embeddings and Qdrant, with a FastAPI backend and Streamlit UI for private, natural-language queries.

Andreas Kretz

GenAI RAG with LlamaIndex, Ollama and Elasticsearch

Learn how to build a Generative AI Platform with local Retrieval Augmented Generation (RAG) with Elasticsearch, LlamaIndex, Ollama, and Mistral.

Andreas Kretz

Building Advanced Pipelines with Kestra on GCP

Workflow Orchestration Without Limits. Learn how to orchestrate scalable, production-ready workflows with Kestra on GCP, including retries, triggers, and monitoring.

Andreas Kretz

5. Roadmaps

Roadmap Data Analyst

Our 10 Week Data Engineering Roadmap for Data Analysts

Andreas Kretz

Roadmap Data Scientist

Our 14 Week Data Engineering Roadmap for Data Scientists

Andreas Kretz

Roadmap Software Developer

Our 12 Week Data Engineering Roadmap for Software Developers

Andreas Kretz

Roadmap Beginners

Our 11 Week Data Engineering Roadmap for Beginners and Graduates

Andreas Kretz

Apache Spark & Kafka Bootcamp

Learn how to build an end-to-end streaming pipeline with Apache Spark, Kafka and more

Andreas Kretz

AI Engineering Track

Everything you need to become an AI Engineer

Andreas Kretz

6. Certifications

Associate Data Engineer

Certify that you are job ready with the Associate Data Engineer exam

Andreas Kretz