This is an Intro to Synapse session I hosted for the PowerBI Turkey Meetup group.
Data Lake Storage
Common Data Model
Optimized for Analytics
Python .NET Java Scala R
Experience Azure Synapse Studio
Artificial Intelligence / Machine Learning / Internet of Things
Intelligent Apps / Business Intelligence
• Linked services define the connection
information needed to connect to
• Easy cross platform data migration
• Represents data store or compute
Once a dataset is defined, it can be used in pipelines and sources
of data or as sinks of data.
• 90+ Connectors
• Various activities
• Supports common loading patterns.
• Fully parallel loading into data lake or SQL tables.
• Graphical development experience.
• Handle upserts, updates, deletes on sql sinks
• Commonly used ETL patterns(Sequence generator/Lookup
• Add file handling (move
files after read, write files
to file names described
in rows etc)
Triggers represent a unit of processing
that determines when a pipeline
execution needs to be kicked off.
• Event based
• Tumbling window
Modern Data Warehouse
INGEST PREPARE TRANSFORM &
AZURE SYNAPSE ANALYTICS
Exploratory Data Analysis
Preparing To Transform
Code based transformations - Spark
Starting from a table, auto-
generate a single line of
PySpark code that makes it
easy to load a SQL table into a
Code based transformations - SQL
Transform with Notebooks
• Allows to write multiple languages in one notebook
• Offers use of temporary tables across languages
• Language support for Syntax highlight, syntax error, syntax
code completion, smart indent, code folding
• Export results
Transform with Pipelines and Data
Transform with Serverless
• An interactive query service that provides T-SQL queries over
high scale data in Azure Storage.
• No infrastructure
• Pay only for query execution
• T-SQL syntax to query data
• Supports data in various formats (Parquet, CSV, JSON)
• Support for BI ecosystem
Machine Learning in Azure Synapse Analytics
Making Predictions with T-SQL
Azure Machine Learning or
Azure Synapse Spark
Convert to ONNX
Export to Storage
Azure Synapse SQL
Load into Table
Azure Synapse SQL
Load from Table
Create the model Register the model Use the model
From ingestion to visualization.
Databricks vs Synapse
• If you are primarily looking for a Data Warehousing solution, go
with Azure Synapse Analytics.
• If looking for a Spark solution and don’t have data warehousing
needs, go with Azure Databricks. In case of Spark based ML
scenarios, include Azure Machine Learning from within Azure
Databricks for experiment tracking, automated machine learning
• If heavily invested in Spark and have data warehousing needs, go
with both Azure Databricks and Azure Synapse.
Links worth sharing
Microsoft Cloud Workshop for Azure Synapse Analytics and AI
Data and AI Engagement Accelerators for PoC
http://daron.me | @daronyondem
Download slides here;