Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Data Engineering on AWS
01 Introduction
Important: Before you start! (0:30)
Introduction (2:21)
Data Engineering (4:15)
Data Science Platform (5:20)
02 The Dataset
Data Types You Encounter (3:03)
What Is A Good Dataset (2:54)
The Dataset We Use (3:16)
Defining The Purpose (6:27)
Relational Storage Possibilities (3:46)
NoSQL Storage Possibilities (6:28)
03 Platform Design
Selecting The Tools (3:49)
Client (3:05)
Connect (1:18)
Buffer (1:28)
Process (2:42)
Store (3:41)
Visualize (3:00)
04 Data Pipelines
Data Ingestion Pipeline (3:00)
Stream To Raw Storage Pipeline (2:19)
Stream To DynamoDB Pipeline (3:09)
Visualization API Pipeline (2:56)
Visualization Redshift Data Warehouse Pipeline (5:29)
Batch Processing Pipeline (3:19)
05 AWS Basics
Create An AWS Account (1:58)
Things To Keep In Mind (2:45)
IAM Identity & Access Management (4:06)
Logging (2:22)
AWS Python API Boto3 (2:57)
06 Data Ingestion Pipeline
Development Environment (4:02)
Create Lambda for API (2:33)
Create API Gateway (8:30)
Setup Kinesis (1:38)
Setup IAM for API (5:00)
Create Ingestion Pipeline (Code) (6:09)
Create Script to Send Data (5:46)
Test The Pipeline (4:53)
07 Stream To Raw S3 Storage Pipeline
Setup S3 Bucket (3:42)
Configure IAM For S3 (3:21)
Create Lambda For S3 Insert (7:16)
Test The Pipeline (4:01)
08 Stream To DynamoDB Pipeline
Setup DynamoDB (9:00)
Setup IAM For DynamoDB Stream (3:36)
Create DynamoDB Lambda (9:20)
09 Visualization API
Create API & Lambda For Access (6:10)
Test The API (4:47)
10 Visualization Pipeline Redshift Data Warehouse
Setup Redshift Data Warehouse (8:08)
Security Group For Firehose (3:12)
Create Redshift Tables (5:51)
S3 Bucket & jsonpaths.json (3:02)
Configure Firehose (7:58)
Debug Redshift Streaming (7:43)
Bug-fixing (5:58)
Power Bi (12:16)
Batch Processing Pipeline
AWS Glue Basics (5:14)
Glue Crawlers (13:09)
Glue Jobs (13:43)
Redshift Insert & Debugging (7:16)
12 Conclusion
What We Achieved & Improvements (10:40)
Data Ingestion Pipeline
Complete and Continue
Discussion
0
comments
Load more
0 comments