Azure Data Solutions
Azure Data Solutions
Azure Data Solutions
Introduction to Azure
Introduction to Storage
Azure Storage
o Azure Blob
o Table
o Message
o Queue
Azure Data Lake Store Gen I & Gen II
o What is Data Lake
o Data Lake vs. Hadoop
o Blob Storage vs. Data Lake
o Hierarchical Namespace
o Ingestion through different tools i.e.; Azure Data Explorer, AzCopy, Azure CLI,
Powershell
Introduction
What is Azure Synapse Analytics
How Azure Synapse Analytics works
When to use Azure Synapse Analytics
Create Azure Synapse Analytics workspace
Exercise - Create and manage Azure Synapse Analytics workspace
Describe Azure Synapse Analytics SQL
Explain Apache Spark in Azure Synapse Analytics
Exercise - Create pools in Azure Synapse Analytics
Orchestrate data integration with Azure Synapse pipelines
Exercise-Identifying Azure Synapse pipeline components
Visualize your analytics with Power BI
Understand hybrid transactional analytical processing with Azure Synapse Link
Use Azure Synapse Studio
Understand the Azure Synapse Analytical processes
Explore the Data hub, Develop hub, Integrate hub
Explore the Monitor hub, Manage hub
Describe a modern data warehouse
Define a modern data warehouse architecture
Exercise - Identify modern data warehouse architecture components
Design ingestion patterns for a modern data warehouse
Understand data storage for a modern data warehouse
Understand file formats and structure for a modern data warehouse
Prepare and transform data with Azure Synapse Analytics
Serve data for analysis with Azure Synapse Analytics
Introduction
Why Warehouse in cloud
Traditional vs. Modern Warehouse architecture
What is Synapse Analytics Service
Create Dedicated SQL Pool and Spark Pool
Create Azure Synapse Analytics Studio Workspace
Analyze Data using Dedicated SQL Pool and Spark Pool
Analyze Data using Apache Spark Notebook
Analyze Data using Serverless SQL Pool
Azure Synapse Benefits
Azure Databricks
Spark Basics
Why Spark is difficult? Why Databricks Evolved?
Why Databricks in Cloud? Introduction to Azure Databricks
Demo
Provision Databricks, Clusters and workbook
Mount Data Lake to Databricks DBFS
Explore, Analyze, Clean, Transform and Load Data in Databricks
Azure Databricks Clusters
Azure Databricks other Important Components
Databricks - Monitoring
How to create Cluster
How to work with Databricks File System
How to create notebooks and Integrate with ADF
How to import and export the Notebooks
How to connect to blob, SQL DB from Databricks
How to read data files from Azure Blob and Azure Data Lake Store
Using Scala, R, Python, Spark SQL Language
Creating Data Frames
Converting Data Frames into Temporary Table or Temporary View
Incremental and Full Load with Azure SQL Data Warehouse
Understand the architecture of Azure Databricks spark cluster
Understand the architecture of spark job
Read data in CSV format
Read data in JSON format
Read data in Parquet format
Read data stored in tables and views
Write data
Describe a DataFrame
Use common DataFrame methods
Use the display function
Exercise: Distinct articles
Describe the difference between eager and lazy execution
Describe the fundamentals of how the Catalyst Optimizer works
Define and identify actions and transformations
Describe the column class
Work with column expressions
Perform date and time manipulation
Use aggregate functions
Exercise: Deduplication of data
Describe the Azure Databricks platform architecture
Perform data protection
Describe Azure key vault and Databricks security scopes
Secure access with Azure IAM and authentication
Describe security
Exercise: Access Azure Storage with key vault-backed secrets
Describe the open source Delta Lake
Exercise: Work with basic Delta Lake functionality
Describe how Azure Databricks manages Delta Lake
Exercise: Use the Delta Lake Time Machine and perform optimization
Describe Azure Databricks structured streaming
Perform stream processing using structured streaming
Work with Time Windows
Process data from Event Hubs with structured streaming
Describe bronze, silver, and gold architecture
Perform batch and stream processing
Schedule Databricks jobs in a data factory pipeline
Pass parameters into and out of Databricks jobs in data factory
Integrate with Azure Synapse Analytics
Understand workspace administration best practices
List security best practices
Describe tools and integration best practices
Explain Databricks runtime best practices
Understand cluster best practices
Azure Cosmos DB
Introduction to NoSQL DB
Introduction to NoSQL
SQL vs. NoSQL
Types of NoSQL
NoSQL Offerings by Microsoft
Introduction to Cosmos DB
Cosmos DB Features
Cosmos DB - Multi Model 5 APIs
Table Storage vs. Cosmos DB
Provision Cosmos DB Account