Course level
Beginner Friendly
Single Courses To Complete Project Examples
Duration
On-Demand Access To Content
Updates
New Content Every Week
Included In The Academy
- Huge Step by step Data Engineering Course
- Access to our Data Engineering Club Zoom calls every other week
- Unlimited access for 12 month (incl. future courses)
- Access to all courses and capstone projects in the Academy
- Data Engineering on AWS E-Commerce capstone project
- Microsoft Azure capstone project
- Document Streaming capstone project with Docker, FastAPI, Apache Kafka, Apache Spark, MongoDB and Streamlit
- Lifetime access to the private Slack Workspace
- Course certificates
- Money back guarantee if you don't like it
From Basics To Full Capstone Projects
Our Data Engineering Academy teaches you all aspects of Data Engineering.
We start with the basics you need to learn Data Engineering. From what is Data Engineering, the computer science fundamentals, to how document your journey.
The second part teaches you the fundamentals of platform and pipeline design, how to incorporate security, the different types of data stores and schema design.
Module three contains fundamental tools who you are going to see all the time. From data ingestion, message queues, processing frameworks to visualization.
To apply what you have learned you can do a variety of capstone projects on AWS, Azure and a Document Streaming (NEW)
Apply What You Have Learned
AWS Capstone Project
AWS is the most used cloud platform out there. In this course, with 4 hours of video material, Andreas shows you how to do Data Engineering on AWS.
From the basics of setting up your account to building streaming and batch pipelines. You will learn working with the major tools like API Gateway, Kinesis, Lambda, AWS Glue, S3, DynamoDB, Redshift as well as connecting Power BI
The dataset we use is an e-commerce dataset
Azure Capstone Project
In this capstone project Kristijan Bakaric (Follow on LinkedIn) shows you how to start working with Microsoft Azure and how to build a complete end to end streaming pipeline.
The project uses major Azure components like API Management, Event Hub, Azure Functions, CosmosDB as well as PowerBI
This project uses Twitter data in combination with images of hurricane damages
NEW!!!
Document Streaming Capstone Project
If you are on the cloud or on premise. These awesome Open Source tools like FastAPI, Apache Kafka, Apache Spark streaming, MongoDB and Streamlit will help you get into Data Engineering without a cloud platform subscription
In this capstone project we build an end to end streaming pipeline. All in a dockerized environment so you can learn locally, but also deploy this on the cloud.
What our students say:
Stephen V.
I'm a full-time Data Engineer and I've found Andreas's videos very helpful. If I'm about to embark on a new project I will always check out one of Andreas's videos to get a good overview.
Seemanthini N.
I was always in the search for a good resource for Data Engineering hands on/knowledge and found Learn Data Engineering. This is the perfect course for someone who has coding experience and is looking to transition to a date engineering role. Making my way through the course and hopefully land into my dream role soon.
Ritesh C.
Data Engineering is a vast field with new advancements being made frequently. I appreciate the way Andreas has been able to identify and bring forward real concepts that help understand the overall subject better. He covers a lot of contemporary tools and tech in small capsule courses that are easy to understand and build upon. Glad to have opted in.
Group Discounts
If you're interested in signing up with your team - ask about our group discounts here: Group Discount Application
Abiodun A.
No regrets subscribing for the Data Engineering Academy, great contents Andreas put together. He is also very approachable. Good job!
Arockia N.
Teaching methods of Andreas are great. Very clear and concise. Personal coaching is tailored to the aspirations of the students. I would highly recommend him to anyone who wants to get to know data engineering better!
Yashovardhan M.
First of all, I would like to thank Andreas for compiling all the critical components required to be a great data engineer. There are only few courses on Data Engineering but this is an excellent course. The course concentrates on the concepts rather than sticking to a particular cloud. Andreas covers the basic details in his course material that every trainee would easily understand. I would truly recommend this course for not only the beginners but experienced people as well
Full Curriculum of the Learn Data Engineering Course
- Introduction
- My Journey as a Data Engineer
- Data Science Jobs
- Full Stack Data Scientists?
- Science and Engineering
- Who are Data Engineers
- Engineering Skills
- Data Platform & Tools
- Skills Engineers, Analysts, Scientists
- Engineering Tools in the Blueprint
- Data Engineers and Machine Learning
- ML Only a Small Part of Data Science
- Phases of Data Science Projects
- Jobs Within DS Project Phases
- Process to Learn Data Engineering
- How We Can Help You
- Conclusion
- Coding
- UML Diagrams
- Git
- Agile Roles
- Agile Process & Tools
- Azure DevOps
- DevOps
- Software Development Links
- OLTP vs OLAP
- Why Relational DBs and SQL is so important
- Data Modeling
- ER Model
- Primary & Foreign Keys
- Building your DB with dbdiagram.io
- Important SQL Queries
- Computer Networking Links
- Linux Links
- REST API Links
- Introduction & Contents
- The Platform Blueprint
- End to End Pipeline Example
- Push Ingestion Pipelines
- Pull Ingestion Pipelines
- Batch Pipelines
- Streaming Pipelines
- Stream Analytics
- Lambda Architecture
- Visualization Pipelines
- Visualization with Hive & Spark on Hadoop
- Visualization Data via Spark Thrift Server
- AWS Example
- Azure Example
- GCP Example
- Hadoop Example
- What are APIs and what are they used for
- Hosting vs using APIs
- Methods and Media Types
- HTTP response code
- Setup environment with WSL2, VS Code & FastAPI
- Testing FastAPI
- The dataset we use
- API Design
- Schema implementation preview
- OpenAPI & Swagger
- POST Customer API
- Get Customer API
- POST Create Customer Invoice API
- GET Invoice API
- GET All Invoices for Customer API
- Setup Docker and Deploy on WSL2
- Testing the APIs with Postman
- Security
- Conclusion
- Introduction & Contents
- Why Spark - Vertical vs Horizontal Scaling
- What Spark Is Good For
- Spark Driver, Context & Executors
- Cluster Types
- Client vs Cluster Deployment
- Where to Run Spark
- Tools in the Spark Course
- Docker Setup
- Jupyter Notebook Setup & Run
- RDDs
- DataFrames
- Transformations & Actions Overview
- Transformations
- Actions
- Link to GitHub Repository
- Download Datasets
- Notebook 1: JSON Transformations
- Notebook 2: Working with Schemas
- Notebook 3: Working With DataFrames
- Notebook 4: SparkSQL
- Notebook 5: Working with RDDs
- New Lecture
- Introduction
- What is Kafka
- Basic Kafka Parts
- Message Queue Basics
- Message Types
- Topics Partitions & Brokers
- Brokers & Zookeeper
- Development Environment
- Bitnami Docker Setup
- Basic Topic Commands
- Kafka Producer
- Kafka Consumer
- Testing Producer & Consumer
- Working with Consumer Offsets
- Examples How Kafka Fits in Data Platforms
- Conclusion
- Introduction
- Relational Schemas
- MongoDB Documents Explained
- Setup of Development Environment
- Introduction to Mongo-Express
- The Dataset
- MongoDB Schema Design
- Basic CLI Commands
- Installing pymongo
- Write Documents
- Read Documents
- Update Documents
- Delete Documents
- Working With Sub-Documents
- Working with Arrays of Sub-Documents
- Operators & Transactions
- How MongoDB fits into Data Science Platforms
- Summary & Outlook
Frequently Asked Questions (FAQs)
Is Learn Data Engineering right for me?
Our academy is right for you if you are interested in the plumbing of data science. We have helped many members and coaching students who work as Data Scientist, Data Analyst, Database Administrator, Software Developer as well as graduates who are searching for Data Engineering jobs or want to apply the skills in the future.
How long do I have to complete the course?
The courses in our academy are all completely on-demand, which means you can advance the courses and their modules at your own pace. You can start whenever you want, and there are no deadlines to finish. If you have questions either use our Slack group or the weekly live stream to get help.
Why join Learn Data Engineering
Unlike typical online course, the content on Learn Data Engineering grows every week. Andreas and other trainers are constantly adding new topics, videos, worksheets as well as a private weekly Q&A video.
Is there a monthly payment option?
Not right now. The academy is a one time payment with unlimited access.
How does the money back guarantee work?
We are absolutely confident that you like what you get with the academy. However, if you don't like what you get and you contact Andreas within the 30 days after the purchase to get a full refund.
Can I upgrade to the Coaching as an Academy member?
Yes, this is no problem!
Just send an email to [email protected] Then you will receive a coupon from us, which reduces the price of the coaching by the price of the annual access to the Academy, which you have already paid.
Is there a free trial available?
No, sorry there is no trial. We have a huge collection of over 200 videos you can watch for free on YouTube.
Here's the link: http://bit.ly/YoutubeAndreaskayy
Or read the data engineering cookbook. It's going to help you a lot learning more about Data Engineering.
Download now.
Where can I get even more information?
If you have specific questions regarding Learn Data Engineering, the academy or the coaching, please feel free to contact Andreas directly. Just click here