Cloud-Based ETL With Azure Data Factory And Azure Functions
Today, many ETL workloads and Data Warehouses move to the cloud. But also the data sources and requirements have changed. How can you process streaming data for a cloud-bourne DWH? How can you pull data from REST APIs in batches? What tools do you have for processing data in the cloud? In my session I will discuss scenarios that can be solved using the Azure Data Factory, scenarios that are beyond ADF capabilities and how Azure Functiosn can be employed in both scenarios. I will show real-life examples utilizing Storage queues for asynchronous processing, durable entities for streaming data and orchestrator functions.
#ETL, #Azure, #ADF, #AzureFunctions, #ModernDWH, #DWH, #Cloud