Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Data Engineering on GCP
Introduction
Introduction (1:13)
GitHub & the team (1:30)
Data & Goals
Architecture of this project (3:19)
Introduction Weather API (2:18)
Setup Google Cloud Account (2:12)
Project Setup
Creating the project (2:35)
Enabling the required APIs (1:34)
Configure scheduling (2:20)
Pipeline Creation - Extract from API
Setup VM for database interaction (2:53)
Setup mysql database (2:16)
Setup vm client and create database (2:46)
Creating pub/sub message queue (1:41)
Create cloud function to pull data form API (4:17)
Explanation code pull from API (4:20)
Pipeline Creation - Write to db
Create function to write to db (7:47)
Explanation code write data to db (5:56)
Testing the function (5:51)
Create function write data to db - pull (3:53)
Explanation code write data to db - pull (4:33)
Visualization
Setup Looker Studio and create bubble chart (2:20)
Setup Looker Studio and create time series chart (1:57)
Pipeline Monitoring (6:20)
Conclusion & Outlook
Conclusion & Challenges (3:19)
Explanation code write data to db - pull
Lesson content locked
If you're already enrolled,
you'll need to login
.
Enroll in Course to Unlock