Microsoft Azure is a cloud platform in which you can store and manage data and deploy virtual machines. It consists of more than 200 individual products and cloud services and enables companies to develop, run and manage applications in the cloud. Azure also provides many frameworks and tools to run applications in multiple clouds, locally or at the edge.
What you will learn
In this course, Kristijan Bakarić teaches you how to build a Streaming Data Pipeline in Azure by working on a hands-on project. This project introduces a set of Azure services to ingest (APIM), store (Blob Storage), process (Azure Functions), serve (Cosmos DB) and visualize (Power BI) flows of twitter data coming in as JSON messages.
After the project introduction and a high level overview of the solution architecture, you will build its components and connect them into a data pipeline. Language of choice for local data preprocessing and developing Azure functions is Python.
During this course, you create a JSON file that contains messages and write a Python program that will send objects in JSON as messages via HTTP requests to Azure API Management. You also learn how to develop and deploy Azure functions by using Python and Visual Studio Code and create an Azure function project with one default dummy function.
Furthermore, you are going to create and combine Event Hubs, Azure Functions and Cosmos DB. In this context, you will also write tweets to Cosmos DB from Event Hub and connect a Power BI Desktop to your Cosmos DB.
- Azure Account
- Software Development Basics
- Basic Python Skills
- Fundamentals of Data Stores (See 2.3 Choosing Data Sources Course in Academy)
- Fundamentals of APIs (See 3.1 API Design & Development with FastAPI)
- Fundamentals of Message Queues (See 3.3 Apache Kafka Course in Academy)