Course level
Beginner Friendly
Single Courses To Complete Project Examples
Duration
On-Demand Access To Content
Updates
New Content Every Week
Main Academy Features
The step by step course to learn Data Engineering with Capstone projects and sub courses on specific topics.
Plus our Data Engineering Club where we meet once a week on Zoom.
New content every week!
From Basics To Full Capstone Projects
Our step by step course starts with teaching you the basics you need to learn Data Engineering from what is Data Engineering, the fundamentals, to how document your journey.
The second part teaches you the fundamentals of platform and pipeline design, how to incorporate security and the different types of data stores.
Module three contains fundamental tools who you are going to see all the time. From data ingestion, message queues, processing frameworks to visualization.
To apply what you have learned you can do a variety of capstone projects on AWS, Azure and with Open Source Tools (July 2021)
Apply What You Have Learned
Spotlight: AWS Capstone Project
AWS is the most used cloud platform out there. In this course, with 4 hours of video material, Andreas shows you how to do Data Engineering on AWS.
From the basics of setting up your account to building streaming and batch pipelines. You will learn working with the major tools like API Gateway, Kinesis, Lambda, AWS Glue, S3, DynamoDB, Redshift as well as connecting Power BI
The dataset we use is an e-commerce dataset
Spotlight: Azure Capstone Project
In this capstone project Kristijan Bakaric (Follow on LinkedIn) shows you how to start working with Microsoft Azure and how to build a complete end to end streaming pipeline.
The project uses major Azure components like API Management, Event Hub, Azure Functions, CosmosDB as well as PowerBI
This project uses Twitter data in combination with images of hurricane damages
Spotlight: Document Processing Capstone Project (July 2021)
Andreas is currently working on this capstone project. With it he shows you how to use open source tools to build a data science platform.
If you are on the cloud or on premise. These awesome Open Source tools like FastAPI, Apache Kafka, Apache Spark streaming, MongoDB and Streamlit will help you get into Data Engineering without a cloud platform subscription
Current Progress:
Publish Building APIs with FastAPI and Docker course (DONE)
Publish Apache Kafka course (DONE)
Publish MongoDB course (DONE)
Development of Capstone Project (In Progress)
Included In The Academy
- Huge Step by step Data Engineering Course
- Access to our weekly Data Engineering Club Zoom call
- Unlimited access for 12 month (incl. future courses)
- Access to all sub-courses and capstone projects in the Academy
- Data Engineering on AWS E-Commerce capstone project
- Lifetime access to the private Slack Workspace
- Money back guarantee
Group Discounts
If you're interested in signing up with your team - ask about our group discounts here: Group Discount Application
Full Curriculum of the Learn Data Engineering Course
- Introduction
- My Journey as a Data Engineer
- Data Science Jobs
- Full Stack Data Scientists?
- Science and Engineering
- Who are Data Engineers
- Engineering Skills
- Data Platform & Tools
- Skills Engineers, Analysts, Scientists
- Engineering Tools in the Blueprint
- Data Engineers and Machine Learning
- ML Only a Small Part of Data Science
- Phases of Data Science Projects
- Jobs Within DS Project Phases
- Process to Learn Data Engineering
- How We Can Help You
- Conclusion
- Coding
- UML Diagrams
- Git
- Agile Roles
- Agile Process & Tools
- Azure DevOps
- DevOps
- Software Development Links
- OLTP vs OLAP
- Why Relational DBs and SQL is so important
- Data Modeling
- ER Model
- Primary & Foreign Keys
- Building your DB with dbdiagram.io
- Important SQL Queries
- Computer Networking Links
- Linux Links
- REST API Links
- Introduction & Contents
- The Platform Blueprint
- End to End Pipeline Example
- Push Ingestion Pipelines
- Pull Ingestion Pipelines
- Batch Pipelines
- Streaming Pipelines
- Stream Analytics
- Lambda Architecture
- Visualization Pipelines
- Visualization with Hive & Spark on Hadoop
- Visualization Data via Spark Thrift Server
- AWS Example
- Azure Example
- GCP Example
- Hadoop Example
- What are APIs and what are they used for
- Hosting vs using APIs
- Methods and Media Types
- HTTP response code
- Setup environment with WSL2, VS Code & FastAPI
- Testing FastAPI
- The dataset we use
- API Design
- Schema implementation preview
- OpenAPI & Swagger
- POST Customer API
- Get Customer API
- POST Create Customer Invoice API
- GET Invoice API
- GET All Invoices for Customer API
- Setup Docker and Deploy on WSL2
- Testing the APIs with Postman
- Security
- Conclusion
- Introduction & Contents
- Why Spark - Vertical vs Horizontal Scaling
- What Spark Is Good For
- Spark Driver, Context & Executors
- Cluster Types
- Client vs Cluster Deployment
- Where to Run Spark
- Tools in the Spark Course
- Docker Setup
- Jupyter Notebook Setup & Run
- RDDs
- DataFrames
- Transformations & Actions Overview
- Transformations
- Actions
- Link to GitHub Repository
- Download Datasets
- Notebook 1: JSON Transformations
- Notebook 2: Working with Schemas
- Notebook 3: Working With DataFrames
- Notebook 4: SparkSQL
- Notebook 5: Working with RDDs
- New Lecture
- Introduction
- Basic Kafka Parts
- What is Kafka
- Message Queue Basics
- Message Types
- Topics Partitions & Brokers
- Brokers & Zookeeper
- Development Environment
- Bitnami Docker Setup
- Basic Topic Commands
- Kafka Producer
- Kafka Consumer
- Testing Producer & Consumer
- Working with Consumer Offsets
- Examples How Kafka Fits in Data Platforms
- Conclusion
- Introduction
- Relational Schemas
- MongoDB Documents Explained
- Setup of Development Environment
- Introduction to Mongo-Express
- The Dataset
- MongoDB Schema Design
- Basic CLI Commands
- Installing pymongo
- Write Documents
- Read Documents
- Update Documents
- Delete Documents
- Working With Sub-Documents
- Working with Arrays of Sub-Documents
- Operators & Transactions
- How MongoDB fits into Data Science Platforms
- Summary & Outlook
Frequently Asked Questions (FAQs)
Is Learn Data Engineering right for me?
Our academy is right for you if you are interested in the plumbing of data science. We have helped many members and coaching students who work as Data Scientist, Data Analyst, Database Administrator, Software Developer as well as graduates who are searching for Data Engineering jobs or want to apply the skills in the future.
How long do I have to complete the course?
The courses in our academy are all completely on-demand, which means you can advance the courses and their modules at your own pace. You can start whenever you want, and there are no deadlines to finish. If you have questions either use our Slack group or the weekly live stream to get help.
Why join Learn Data Engineering
Unlike typical online course, the content on Learn Data Engineering grows every week. Andreas and other trainers are constantly adding new topics, videos, worksheets as well as a private weekly Q&A video.
Is there a monthly payment option?
Not right now. The academy is a one time payment with unlimited access.
How does the money back guarantee work?
We are absolutely confident that you like what you get with the academy. However, if you don't like what you get and you contact Andreas within the first five days after the purchase you get a full refund.
Can I upgrade to the Coaching as an Academy member?
Yes, this is no problem!
Just send an email to [email protected] Then you will receive a coupon from us, which reduces the price of the coaching by the price of the annual access to the Academy, which you have already paid.
Is there a free trial available?
No, sorry there is no trial. We have a huge collection of over 200 videos you can watch for free on YouTube.
Here's the link: http://bit.ly/YoutubeAndreaskayy
Or read the data engineering cookbook. It's going to help you a lot learning more about Data Engineering.
Download now.
Where can I get even more information?
If you have specific questions regarding Learn Data Engineering, the academy or the coaching, please feel free to contact Andreas directly. Just click here