Course level

beginner-intermediate

Duration

85 Minutes of Videos


Last update

June 29, 2021


About the Author

Kristijan Bakarić


Kristijan is a hybrid of a scientist and an engineer working for an international energy company since 2013.

He worked in the business of oil and gas exploration as an explorationist for half of his career and the second half with a variety of subsurface data (analytics, science and engineering) projects, and most recently in digitization and digitalization of operations.

He holds a wide, generalist knowledge in a variety of domains, tools and technologies with a current focus in Data Engineering and Azure.

He is passionate about data, technology and learning, and especially sharing his knowledge and experience with others.

Connect with Kristijan on LinkedIn or Twitter

Description

In this course, you will build a Streaming Data Pipeline in Azure by working on a hands-on project in Azure. Project introduces a set of Azure services to ingest (APIM), store (Blob Storage), process (Azure Functions), serve (Cosmos DB) and visualize (Power BI) flows of twitter data coming in as JSON messages. 

After project introduction and a high level overview of the solution architecture, we will build its components and connect them into a data pipeline. Language of choice for local data preprocessing and developing Azure Functions is Python.


What's included?
  • Video Lessons
  • Source Codes
  • GitHub Documentation


Requirements
  • Azure Account
  • Software Development Basics
  • Basic Python Skills
  • Fundamentals of Data Stores (See 2.3 Choosing Data Sources Course in Academy)
  • Fundamentals of APIs (See 3.1 API Design & Development with FastAPI)
  • Fundamentals of Message Queues (See 3.3 Apache Kafka Course in Academy)


Course Curriculum


  01 Project Introduction
Available in days
days after you enroll
  02 Datasets and Local Preprocessing
Available in days
days after you enroll
  03 Azure Functions and Blob Storage
Available in days
days after you enroll
  04 Add Azure Function to Azure API management APIM
Available in days
days after you enroll
  05 Create and Combine Event Hubs, Azure Function and Cosmos DB
Available in days
days after you enroll
  06 Write Tweets to Cosmos DB (Core SQL) from Event Hub
Available in days
days after you enroll
  07 Connect Power BI Desktop to your Cosmos DB
Available in days
days after you enroll