Azure Synapse Analytics Course

Azure Synapse Analytics Training Content

Introduction & Overview

  • Azure Synapse Analytics Overview
  • Anzure Synapse Analytics Architecture
  • Create Azure Free Account for Synapse

Overview of pools in Synapse Analytics

  • Dedicated SQL pools
  • Serverless SQL pool
  • Apache Spark pools
  • Data Explorer pools

Using Azure Synapse Analytics to Query Data Lake

  • Creating Azure Synapse Analytics Workspace
  • Uploading Sample Data into Data Lake Storage
  • Exploring Azure Synapse Workspace and Studio
  • Querying a Data Lake Store using serverless SQL pools in Azure Synapse Analytics
  • Creating a View for CSV Data with a Serverless SQL Pool

Azure Storage Account Integration with Azure Synapse

  • Copy multiple files from blob to blob using wildcard file options
  • Copy multuple folders from blob to blob using dataset parameters
  • Get File Names from Folder Dynamically and copy latest file from folder

Azure Synapse Triggers

  • Schedule Trigger in Azure Synapse
  • Event Based Trigger in Azure Synapse
Azure SQL Database integration with Azure Synapse
  • Azure SQL Databases – Introduction __ Relational databases in Azure
  • Copy data from SQL Database to ADLS Gen2 using table, query and stored procedure
  • Overwrite and Append Modes in Copy Activity in Azure Synapse
  • Use Foreach loop activity to copy multiple Tables- Step by Step Explanation
Incremental Load to Azure Synapse in Azure Synapse
  • Incremental Load or Delta load from SQL to blob Storage in Azure Synapse
  • Multi Table Incremental Load or Delta load from SQL to to Azure Synapse
  • Incrementally copy new and changed files based on Last Modified Date
Logging and Notification and keyvault integration __ Azure Logic Apps
  • Log Pipeline Executions to SQL Table using Azure Synapse
  • Custom Email Notifications and keyvault integration with Linked Service
  • Send Error notification with logic app
  • Use Foreach loop activity to copy multiple Tables with pipeline logs logic and notifications
Deep dive into Copy Activity in Azure Synapse
  • Load data from on premise sql server to Azure Synapse
  • Copy Data from sql server to to Azure Synapse with polybase & Bulk Insert
  • Copy Data from on-premise File System to Azure Synapse
  • Loop through REST API copy data 2 ADLS Gen2 Linked Service Parameters
Data Flows Introduction
  • Azure Data Flows Introduction
  • Setup Integration Runtime for Data Flows
  • Basics of SQL Joins for Azure Data FlowsServerless SQL Pool Demo
  • Joins in Azure DataFlowsDedicated SQL Pool Demo
  • Difference Between Join vs.Lookup Transformation& Merge Functionality Spark Pool Demo
  • Dataflows – select the rows
  • Dataflows – Filter the rows
  • Dataflows – join Transformations
  • Dataflows – union Transformations
  • Dataflows – look up Transformations
  • Dataflows – window functions transformations
  • Dataflows – pivot, unpivot transformations
  • Dataflows – Alter rows transformations
  • Dataflows – Removing Duplicates transformations
Spark Pool Introduction in Azure Synapse
  • Spark Introduction and components
  • Spark Architecture
  • Create notbook and notebook option and create notebook in different langauges
  • MSSparkUtils for file system
  • MSSparkUtils for for creating notebook parameters
  • Magic commands and calling one synapse notebook from another and returning output of synapse notebook
  • Configure keyvault in azure synapse notebook
  • Different ways to connect to ADLSGen2 from synapse notebook
  • Different ways to connect to Blob from synapse notebook
  • Different ways to connect to Azure SQL Database from synapse notebook
  • Different ways to connect to on premise SQL Server from synapse notebook
  • Optimization while Reading and writing CSV files from Azure Synapse
  • Reading and writing parquet files from Azure Synapse
  • Reading and writing JSON files from Azure Synapse
  • Reading and writing avro and orc files from Azure Synapse
  • Reading and writing EXCEL files from Azure Synapse
  • Different ways to create RDD in synapse notebook
  • Different ways to create dataframes in synapse notebook
  • When to use repartition and coalesce
  • Joins in Synapse Notebook
  • Broadcast Joins in Synapse Notebook and configuration of spark for optimization
  • what is catalyst optimiser and skewness issue in spark
  • Optimization techniques in pyspark
  • Implementing SCD1 in Synapse Notebook
  • Implementing SCD2 in Synapse Notebook
  • Executing synapse notebooks from synapse pipleines with input and output parameters

Project: End to End DataMigration using Synapse Analytics