SQL Server Ground To Cloud
SQL Server Ground To Cloud
ms/sqlworkshops
Module 01
Module 02
Module 03
Module 04
Module 05
Module 06
Module 01
1. Discovery
2. Envisioning
3. Architecture Design Session (ADS)
4. Proof-Of-Concept (POC)
5. Implementation
6. Handoff
Module 02
Gain Performance with no
• Intelligent Query Processing
application changes
https://aka.ms/bobwardms
• https://github.com/microsoft/sqlworkshops/tree/master/SQLGroundToCloud/slides
SQL
SQL
Performance Security Availability Developer Modern
• Query Store • Always • Clusterless • JSON Platform
• Adaptive Query Encrypted Availability • Temporal Tables • Linux and
Processing • Row Level Groups • Graph Database Containers
• Automatic Security • Distributed • Machine
Tuning • Dynamic Data Transactions for Learning Services
• Columnstore and Masking Availability with R and
In-Memory OLTP Groups Python
• “It Just Runs • Resumable Index
Faster” Maintenance
SQL Server 2019
Solving Modern Data Challenges
ciphertext
SQL
1010
0101
0110
plaintext
Enclave
R
The Intelligent Query Processing feature family
The intelligent database
Intelligent QP
Gain performance without
changing the application Table Variable Batch Mode for Scalar UDF
Adaptive QP Approximate QP
Deferred Compilation Row Store inlining
The Solution(s)
140
• Build intelligent, adaptable operators
• Modify query plans in cache based 140 150
on previous execution
• Expand batch mode 150
Confidential computing
plaintext ciphertext
Enhanced
client driver
plaintext
Enclave
ADD SENSITIVITY CLASSIFICATION TO
dbo.sales.price, dbo.sales.discount
WITH ( LABEL='Highly Confidential',
INFORMATION_TYPE='Financial' )
data_sensitivity_information
Undo dependent
on oldest active
transaction
“Long” recovery
usually due to
long undo
Persisted
Version Store
Logical
Revert
sLog
Cleaner
“It’s all about
Data Virtualization”
Distributed compute engine integrated with SQL Server
Combine data from many sources without Store high volume data in a data lake and access Easily feed integrated data from many sources to
moving or replicating it it easily using either SQL or Spark your model training
Scale out compute and caching to boost Management services, admin portal, and Ingest and prep data and then train, store, and
performance integrated security make it all easy to manage operationalize your models all in one system
System resource &
All other systems
latency sensitive code paths
SQL OS v2
Windows Host Ext. Linux Host Extension
Windows Linux
Host extension mapping to OS system calls
(IO, Memory, CPU scheduling)
Efficient
Faster deployment, reduced patching, and
less downtime Container Container Container
SQL Server Containers
Built-in HADR orchestration with no clustering required
Pod Pod
User
Kubernetes
Pod
SQL Server
SELECT page_info.*
FROM sys.dm_exec_requests AS d
CROSS APPLY
sys.fn_PageResCracker(d.page_resource) AS r
CROSS APPLY sys.dm_db_page_info(r.db_id,
r.file_id, r.page_id,'DETAILED')
AS page_info;
SQL SQL
SQL Server
Microsoft Access
Certify once, run on-premises and in the cloud with Compatibility
Certification
Upgrade & modernize your SQL Server database on-premises, in the cloud and on the edge with Compatibility
Certification that eliminates risks of application compatibility
They also allow longer historical data storage, and as time goes on become
part of that storage lineage
Spark
Hadoop
Virtualization
Hardware Abstraction Physical Computer
Hypervisor
This allows for scale by ring-
fencing OS-level dependencies Operating System
Containers
Abstracting the OS, Allowing complete portability Physical Computer
Operating System
Scale is achieved through
multiple containers
Node
• Container Cluster Orchestration
Master
Node
• Node
Pod
• Cluster
Pod Pod
Node Node
Volume
Claim Node
Cluster Orchestration
Master
Web Tier
Control BDC
SQL Data Pool
Cluster
Orchestration SQL Server
Master
App Pool
SQL Server SQL Server
Master ML Server
Compute Pool
(Web Apps)
HDFS
Tools and Process Flow
Cluster Orchestration
Management Proxy
Master
Controller
Kibana Grafana
Operator InfluxDB
Elastic Search
Telegraf
CollectD
Fluentbit
Controller
Listener
Operator (Readable Secondary)
Cluster Orchestration
SQL Server SQL Server
Master SQL Server
AG
Secondary Secondary
Primary
(Becomes Primary)
Replication Controller
Listener
(Primary)
Knox
Kibana Grafana
Livy
ACLS
YARN Controller
HDFS
Spark
Takeaways
• SQL Server 2019 Big Data cluster includes SQL Server together with the HDFS and Spark Compute
engine as one package for big data processing, Machine Learning and AI
• Spark is a distributed compute engine that provides a unified framework for E2E big data processing
pipeline including Machine learning and AI
• You can use SQL Server 2019 to create a secure, hybrid, machine learning architecture starting with
data preparation, training a machine learning model, operationalizing your Model and using it for
scoring
• Go Do > Practice with installing, configuring, and operating SQL Server 2019
• Go Do > Download this deck and practice a demo on Big Data Clusters on SQL Server
• Go Do > Follow a complete workshop
Resources
Breakthrough
Seamless and Industry-leading Built-in
Competitive TCO productivity &
compatible security intelligence
performance
SQL virtual machines Managed instances Databases
• SQL Server and OS • SQL Server surface area • Hyperscale storage (up • Resource sharing
server access (vast majority) to 100TB) between multiple
• Expansive SQL And OS • Native virtual network • Serverless compute databases to price
version support support optimize
• Fully managed service
• Automated • Fully managed service • Simplified performance
manageability features management for
for SQL Server multiple databases
• Fully managed service
Provisioned General Purpose Hyperscale Business Critical
VM NA NA NA
Managed Instance
Single Database
Elastic Pool
In-Memory Database
Data Classification
UTF-8
Extensibility Framework
Self-installed VMs registered Azure VMs are now discoverable Self-installed VMs with RP Self-installed VMs with RP
with RP now can access on the new Azure SQL blade can be easily converted automatically indicate usage
automation features in in Azure Marketplace to PAYG images of Azure Hybrid Benefit
Azure Marketplace images
2. Once subscription registration is complete, register your VM with the SQL VM Resource Provider:
# Register your existing SQL Server VM with the new resource provider
az sql vm create -n <VMName> -g <ResourceGroupName> -l <VMLocation>
storm with Azure weather information, and Azure provides a more scalable, cost-effective way to manage this service.
Next the company moved its big data storage and processing, using services such as Azure Blob
storage, Azure Data Factory, and Azure SQL Database.
When AccuWeather set out to build an automated, scalable weather prediction service for business
customers, it turned to Azure as well. To create highly accurate forecasts, Radich’s team needed
sophisticated machine learning tools that it could customize using R and Python code.
“With Azure, we have access to sophisticated data science services in a real-time, on-demand basis, which helps us innovate faster.”
- Chris Patti: Chief Technology Officer, AccuWeather
Adapts compute resources to the Pay only for compute resources you Fully-managed and intelligent
workload without sacrificing performance consume, on a per-second basis database service
Automatically pauses and resumes Further optimize costs with configurable Built-in 99.99% availability
compute thresholds
Page Servers
<0.5ms for all data access
Full coverage
RBPEX
Azure Storage
Azure Storage
~2ms for all data access Attached SSD
HA 1 replica, no read-scale
I/O TBD
Gen4: 1 to 24 vCore Gen4: 4 to 24 vCore Gen4: 1 to 24 vCore Gen4: 4 to 24 vCore Gen4: 1 to 24 vCore
Compute tiers Gen5: 2 to 80 vCore Gen5: 4 to 80 vCore Gen5: 2 to 80 vCore Gen5: 4 to 80 vCore Gen5: 2 to 80 vCore
https://azure.microsoft.com/en-us/pricing/details/sql-database/
Instance
As a result a number of features are available in MI that are not in DB – for example:
Because you control the instance, a number of features are not in MI that are in DB - for example:
Only a detailed analysis of requirements can tell you which is the best fit:
𝑆𝑄𝐿𝐷𝐵𝑀𝐼 ≠ 𝑆𝑄𝐿𝐷𝐵 + +
SQL
What tools can help a evaluate databases against target platforms and
identify potential issues?
Can we have network isolation and secure channel access from on-
premises?
Service tier General purpose Business critical Hyperscale
Gen4: 1 to 24 vCore Gen4: 4 to 24 vCore Gen4: 1 to 24 vCore Gen4: 4 to 24 vCore Gen4: 1 to 24 vCore
Compute tiers Gen5: 2 to 80 vCore Gen5: 4 to 80 vCore Gen5: 2 to 80 vCore Gen5: 4 to 80 vCore Gen5: 2 to 80 vCore
https://azure.microsoft.com/en-us/pricing/details/sql-database/
License trade-in values
Take an inventory of on-premises licenses
SQL Server cores vCores on Azure
to determine potential for conversion with SA license SQL Database
Business critical
vCore
SQL Server
Enterprise Edition
4x General purpose or
Hyperscale vCores
Module 05
Migrating to Azure SQL
Demo: Database discovery and assessment
Demo: Migrating to Azure SQL
Demo Advanced Data Security
Discover and assess • Azure Migrate
what and where to
migrate to Azure • Data Migration Assistant
https://aka.ms/azuresqlslides
Data migration Process
NEW
Azure Migrate + DMS + DMA
Azure Migration Program (AMP)
Available to all Azure customers, scaled through specialized migration partners
Reduce cost and risk | Accelerate migration velocity | Deliver customer success
GW
mysqldbsrv.database.windows.net `
westus1-a.control.database.windows.net
104.44.238.205
GW
Node Node
1433 TR 3
proxy GW
CR1
SQL VNet
G
W mymi.<clusterid>.database
primary Node
.windows.net
node TLS
G ILB
Windows Firewall W TDS endpoint
Node (Private IP)
SQL
TLS SQL
Manageme
Engine
nt
TLS Node Agent
LB Node
G
SQL Management W
(Public IP)
Virtual cluster
VNet
GW
primary node Node mi_name.<dns_zone>.database.windows.net
TLS
ILB
GW
Windows Firewall TDS endpoint (private IP)
Node
SQL
TLS
SQL Engine
Management
Node Agent
TLS
LB Node
GW
SQL Management (public IP)
Virtual cluster
Legend:
Data
VNet
Azure
VNet
Azure
VNet
Certificate
Revocation
List
Service dependencies
Azure
Customer applications & tools
Maintenance
On-Premises + Azure VNet
Corpnet
Legend:
Data
Management
Certificate
Azure
VNet
TLS
TLS Certificate
Revocation
List
TLS
VIP
56.226.155.47
Public IP
3342 52.246.255.77 Allow 52.246.255.77 3342
Out
Application VNet
App Service
6
Environment
Managed Instance #1 Managed Instance #2
7 On-premises app
Express Route / VPN Gateway
Use our free training at https://aka.ms/sqlworkshops
https://azure.com/migration/
Module 06
Project Process
1. Discovery: The original statement of the problem from the customer
2. Envisioning: A "blue-sky" description of what success in the project would
look like. Often phrased as "I can..." statements
3. Architecture Design Session: An initial layout of the technology options and
choices for a preliminary solution
4. Proof-Of-Concept (POC): After the optimal solution technologies and
processes are selected, a POC is set up with a small representative example of
what a solution might look like, as much as possible. If available, a currently-
running solution in a parallel example can be used
5. Implementation: Implementing a phased-in rollout of the completed
solution based on findings from the previous phases
6. Handoff: A post-mortem on the project with a discussion of future
enhancements
https://microsoft.sharepoint.com/teams/BrandCentral/
www.microsoft.com/en-us/news
mediaacq
http://news.xbox.com/media/ www.superstock.com/
www.gettyimages.com/creativeimages/royaltyfree
Avoid intellectual property theft: Do not use photos
https://www.featureddevices.com/ or graphics copied from the web in presentations.
When in doubt, consult with CELA.
https://microsoft.sharepoint.com/teams/BrandCentral/P
ages/Presentations.aspx
https://microsoft.sharepoint.com/teams/BrandCentral/Search
Pages/BCGeneralResults.aspx?k=Device%20images
https://www.featureddevices.com/
here
here
7
5 Series4
Series3
4
Series2
3 Series1
0
1 2 3 4