Module 8:

CI/CD with Azure DevOps

Source Control vs Developer UI
ARM Templates Deployments
PowerShell Deployments
Source Control vs
Developer UI
Data Factory Continuous Integration

Debug
branch Service
Developer
Git save
merge

Deployed
Components
master publish
Dev Prod
Developer

Developer
PaulsFunFactor
y

Documentation Author & Monitor

When connected to View JSON for
source control current component.
(Git/GitHub) switch
between Repository code
ZIP extract/import of
development and branches. Simple mandatory complete data factory
deployed instances. key value pair with parameters file.
Save, creates a commit to the checking of JSON

connected repository. components.

Discard dirty panels
(unsaved changes) .
Deploy ARM template from Source code ‘Undo’.
development environment
to deployed service.
 Debug the Data Flow.

Debug the Control Flow. Get a cluster ready.

Run the pipeline.

Debug the Control Flow.
Run the pipeline.
ARM
Template
Deployments
Data Factory Continuous Delivery

Debug
branch Service
Git save
merge

Deployed
Components
master publish
Dev Prod
Developer
Data Factory Continuous Delivery
Option 1 – Single Data Factory Service
Debug
branch Service
Git save
merge

Deployed
Components
master publish
Dev Prod
Developer

Option 2 – ARM Templates for Multiple Data Factory Services

Debug
branch Service
Git save
merge
ARM Params ARM Params

master ARM
Template
Developer Dev
Test Prod
 Debug
branch Service
Git save
merge

Deployed
Components
master publish
Dev Prod
Developer
Getting Our ADF Source Code
Template Parameters
Debug
branch Service ARM Template Export
Git save
merge

Deployed
Components
master publish
Debug Prod
Dev
Developer

adf_publish

ARMTemplateForFactory.json
Template Parameters
Data Factory Continuous Delivery
Azure DevOps

Build Artifacts

Releases Scoped Variables

Test

Azure Resource Group Deployment ARMTemplateForFactory.json

 Override Template Parameters
Release -factoryName "$(DataFactory.Name)"
Approver
Production

Azure Resource Group Deployment

Data Factory Continuous Delivery 1 Linked Services
Azure DevOps
2 Datasets
Build Artifacts
3 Activities

Releases Scoped Variables 4 Pipelines

Test 5 Triggers

Azure Resource Group Deployment ARMTemplateForFactory.json

 Override Template Parameters
Release -factoryName "$(DataFactory.Name)"
Approver
Production

Azure Resource Group Deployment

Data Factory Continuous Delivery 1 Linked Services
Azure DevOps

Build Artifacts

Releases Scoped Variables

Test

Azure Resource Group Deployment ARMTemplateForFactory.json

 Override Template Parameters
Release -factoryName "$(DataFactory.Name)"
Approver
Production

Azure Resource Group Deployment

PowerShell Deployments
Data Factory Continuous Delivery - Option 3
1 Linked Services
Azure DevOps
2 Datasets
Build Artifacts
3 Activities

Releases Scoped Variables 4 Pipelines

Test 5 Triggers
Run PowerShell params.json linkedservices.json
ASezt- A pi pe lin e s & a c t iv it e
R M T em p l at e F o
datasets.json
uSet-
AzrDeatRaFeacstooryuV2rPcipeeliGneroup
triggers.json
s.
rjsFoanctory.json madafs_tpe
Release
ODeployment
AzvDeatrarFiadcteoryTVe2mLinpkeldaSteervPicearameters urblish
Set-AzDataFactoryV2Dataset
Approver -factoryName "$(DataFactory.Name)"

Production
Run PowerShell params.json
Se t-
ASet-AzDataFactoryV2Trigger
z
Data Factory Continuous Delivery - Option 3
1 Linked Services
Azure DevOps
2 Datasets
Build Artifacts
3 Activities

Releases Scoped Variables 4 Pipelines

Test 5 Triggers
Run PowerShell params.json linkedservices.json
Set-AzDataFactoryV2Pipeline pipelines & activites.json
Set-AzDataFactoryV2Dataset datasets.json
Set- triggers.json
AzDataFactoryV2LinkedServic
Release e
Approver
Production
Run PowerShell params.json
Set-AzDataFactoryV2IntegrationRuntime
Set-AzDataFactoryV2Trigger
 Data Factory Continuous Delivery - Option 4
1 Linked Services
Azure DevOps
2 Datasets
Build Artifacts
3 Activities

Releases Scoped Variables 4 Pipelines

Test 5 Triggers
Run PowerShell linkedservices.json
Import-Module -Name "azure.datafactory.tools" pipelines & activites.json
Publish-AdfV2FromJson -RootFolder $AdfPath ` datasets.json
-ResourceGroupName $resourceGroupName `
triggers.json
-DataFactoryName $dataFactoryName `
-Location $region `
Release
Approver -Stage $configFilePath

Production

Publish Azure Data Factory

https://www.powershellgallery.com/packages/azure.datafactory.tools
Deployment Options Summary
Option 1 – Use a single Data Factory service.

Option 2 – ARM Templates for multiple Data Factory services (environments).

Option 3 – Use PowerShell cmdlets for each ADF artifact.

Option 4 – Use a PowerShell module or custom Azure DevOps task.

 Data Factory DevOps Story Summary
What is your Which source How many What deployment What artifacts are we
code branching control tool to environments method do we going to use?...
strategy? use? do we want? want to use? OR
How much control do
Feature you want?

ARMTemplate
ForFactory.json
Release

linkedservices.json
pipelines &
activites.json
Feature
datasets.json
triggers.json

Master
Module 8:
CI/CD with Azure DevOps
Source Control vs Developer UI
ARM Templates Deployments
PowerShell Deployments