SQL and System Center 2012 R2
SQL and System Center 2012 R2
Deploy SQL 2012 in various different configurations to support System Center 2012 R2
running on Server 2012 R2
CENTER CENTER
Learn how to deploy
This guide is going to be a group effort, I wrote the last guide by myself, but felt it was better to include
some of the seasoned expert MVP's out there to help with this guide as there is a lot to take in. The idea
behind this guide was to discuss any time SQL comes in contact with System Center, of course the
design and setup are the start to it, but then we want to look at how System Center interacts with SQL
and give examples of it. So how do we build a SQL template in VMM, how do we monitor it in SCOM,
how can we automate things in SQL with SCO, how should we be backing up SQL with DPM. This guide
is going to go through a lot more walk thoughts on these and discuss different options. The people
involved with this guide were Pete Zerger did Azure, Robert Hedblom did DPM, Matthew Long did the
SQL MP chapter, Craig Taylor did the SQL VMM template section, Richard Green and Craig worked with
me on the general editing and work on the cluster builds etc.
Why are these community guide becoming more popular? As we all know things are changing really fast
in IT at the moment, long release cycles are just not cutting it when it comes to keeping up to day with
industry trends, a simple example of this was where the VMM team would release with a support for
versions of VMware that were already two versions old, and so Microsoft is releasing software faster and
faster. As Microsoft continues down the path of "rapid release" whereby R2 will follow SP1 with a year or
so it's going to be harder and harder to write conventional print books. They require too many edits and
can't keep up with the pace of production.
Lastly this guide is a community effort and so it may not be perfect and is not an official Microsoft guide.
It is written by people who work with System Center and SQL every day, and it is our way to share with
you and give a little back for all the hard work others give to us. If you find any errors, have thoughts of
your own or questions please forward them to paul.keely@infrontconsulting.com.
Updates to this guide should be fairly frequent and we will tweet this in the usual groups.
Can I install this on Server 2008 with SQL 2008? The answer to this questions is more should you rather
than can you. Deploying SC on a VM on day 1 on older OS’s and SQL versions is certainty feasible. A
better discussion is would it not be better to deploy your new SC apps in such a way so that you can
leave it in place for 3-5 years from now. If you choose to deploy on Server 2008 you are deploying on
really old code. If you decide to use Windows Clustering in 2008 you are just starting off on a poor footing
compared to Server and SQL 2012. If you are running on an older version of VMware that does not
support deploying Server 2012 then you may be in a bit of a bind. It is a best practice to install your
systems and then just leave them alone, performing either OS upgrades on the Database servers, SQL
upgrades or DB moves in years to come is not the best approach for an enterprise management tool. We
are just going to cover a short overview of the SQL requirements for each product in the suite. There is
also a short commentary on each DB here.
9
1.6 SQL Server on Windows Server 2012
Server 2012 has been a real game changer in many ways. The areas that will spring to mind are:
Better availability
As Server 2012 can use a whole lot more CPU and RAM as a VM it’s possible to have very
large and powerful SQL servers. Server 2012 now supports 640 processors over 64 sockets
with up to 4TB of RAM.
Storage Spaces often now referred to as “Spaces” is a new paradigm in storage and has a massive
impact on how we can build SQL. When we look at the SQL cluster build below we are going to be
using a Scale Out File Server (SOFS) Cluster. This cluster uses Storage Spaces as a storage target.
This is the very latest way to use the Server 2012 storage features to build a cluster and present that to
an application cluster as shared storage.
*some points of note are CSV’s and Resilient File System are not supported with SQL 2012*
10
2.0 BUILDING A SQL CLUSTER IF YOUR
The preferred design for any SQL infrastructure that is important is to
cluster the SQL servers. This helps to secure against the failure of a DB’S ARE
SQL compute node, it does not however help with a failure with the
backend storage. In the configuration I am using here we are using a IMPOR-
Scale Out File Server Cluster using storage paces as the backend
infrastructure. What this means is that I have a two node cluster TANT
presenting storage, and a two node cluster presenting the SQL
application layer. With this configuration I can lose one node THEN YOU
presenting storage and one node presenting SQL. The last thing I am
then going to do is add a last layer of HA to the mix by using SQL NEED TO
AlwaysOn (AO) You can just use a SQL cluster and not use AO, the
cost of AO is not for the fait hearted as you will need at least 3 copies CLUSTER
of SQL Enterprise .
YOUR SQL
Let’s first start off with a high level overview of what we are hoping to AlwaysOn.
achieve. We are going to use a 2 node SOFS cluster presenting
storage to a 2 node Hyper-V cluster. On the Hyper-V cluster we have a
VM on each node, SQL1 and SQL2.
So the first thing I do is create a load of VHDX's on my storage and present that storage to the SQL
nodes, as you can see I have 31 drives and assigning a drive letter to each volume isn't gonna fly.
12
What I am going to do is create a top level volume, assign it a drive letter and then mount the 4 SQL
drives for each instance inside that drive letter. This is all easier to do from a script but I am going to
show you a run through from a GUI, so you get to understand the process.
Work out how many instances you want and then go and create 5 VHDX's for each instance with the
relative size you want.
I ran this script to create the following drives for a SCOM instance, it’s pretty simple and from it you can
see the disk sizes we are going to use. I ran a script to create 29 drives, I am separating for example
the SCOM DB and the DW, and I am going to do the same for SCSM.
13
And in server manager I send up with this
14
Just a quick note, you need a volume for the OS and if you want to separate
the SQL bits from the OS (and you really should) then you need to add a disk
to each node and ideally make it the same drive letter to host your SQL
instance bits
Disk 2-5 are what I need to use for SCOM. Online the 4 disks (better to just do the 4 you want right
now because if you bring in 30 it’s difficult to manage that in the GUI.
When they come online you need to format them and give them some form of workable name, I’m not
going to go through all the screen’s but as an example I’ll take the first drive (it’s going to be my empty
root) format it as a 64K drive and name it SCOM_ROOT
15
Why are you making an empty root top level folder that is going to hold the
SQL disks 25GB, surely we only need a 1 GB drive? Well this is a good
point however if you take the SCOM installer it looks at the size of this top
level folder and if its 1 GB then it won’t let you install the SCOM DB’s
Then go to the file explorer and create a top level folder called SCOM_ROOT and in it 4 folders to hold
the drives you are about to mount. (This is of course just a personal preference, you can mount the
drives anywhere you like, just be aware of the complexity of VHDX sprawl)
16
Then go and mount the drives into each folder
17
And all the drives are now sitting on the empty root drive on the SQL node 18
2.4 Create a Windows Cluster
As was mentioned earlier, this guide is not going to go into all the details and
nuances of building a Windows cluster as there are so many great references
for that elsewhere
Install the Failover Clustering Feature on each of the nodes.
Then once the feature has been added, launch the Failover Cluster Manager.
19
Strictly speaking you don’t need to validate a cluster but you would be mad not to. Clustering in 2012
has become such a joy to work with and that is especially so when your clusters pass their validation.
20
Add each of the nodes in the cluster (I am going to add another node SQL3 to act as a target for the
AlwaysOn node in the next chapter)
22
I give the cluster a name and IP address and these are the Windows cluster details and are separate
from the SQL cluster details.
23
The cluster has been deployed and you should be able to ping the cluster name and IP address.
(This is not the place to go into complex cluster setup troubleshooting, but the main reason for me that
cluster wont deploy in production is that the cluster instance needs to create a computer object in AD,
you need to either have the rights to do that or have it done in advance for you. Get this sorted, provide
shared storage of some description, you can now ping that cluster name (SQLSC in my case) and it
should be ready to go for a SQL deploy.
24
The last step in this process is the add the storage you created in the first step
Now as you can see the cluster brings in the disks but doesn’t name them the way I did in server
manager so I need to sort that out, I start off with finding the root
25
Gen2 VM’s and disk importing. During the build of this guide I created a disk
for the SQL bits and assigned it the D drive, as its hanging off the SCSI
controller. However as it’s a SCSI drive Failover Cluster Manager will allow
you to import this disk into a cluster as it sees this as shared. If you do this
by mistake (and I did) each time you import new disks into the system the
cluster will take ownership of this disk, or at least try to, and you will lose it as
a shared drive and as it’s got the SQL bits your SQL cluster will fail too . So
top tip is only import the number of drives you need.
26
Rename the drive to SCOM_ROOT and do the same with all the other drives
As all of these disks are mounted in the root of SCOM_DATA I need to make them all dependent on
SCOM_DATA so that the cluster knows to bring up the parent drive first, otherwise in a failover the
drives would fail.
27
2.5 SQL Cluster Installation
Now I am ready to go an install SQL on node 1 in the cluster, I am going to name it SCOM and place
the SQL files in the correct drives that we have already prepared.
To begin with I am going to create an empty role on the cluster and assign the disks to that role.
From the SQL media select Installation and then New SQL Server failover cluster installation (I always
run this with admin rights)
29
30
31
32
33
34
35
Now when I go to SQL management studio I open on SCOM
36
The secondary node install is a few clicks and go, as you are just joining an existing cluster there is
almost no configuration, from the SQL media choose “Add node to a SQL failover cluster”
37
38
39
3.0 SQL ALWAYSON
SQL Server AlwaysOn is a new feature of SQL Server 2012. It allows us to replicate a database or a
group of databases to another server. The additional server could be located in a remote location from
the primary as there is no need for shared storage. AlwaysOn is only available with SQL Enterprise so
it’s not cheap. In our example above we have built out a multi instance SQL cluster that we are going
to install the full suite of SC applications. As we have built out the cluster using a clustered storage
configuration (SOFS) we have removed a single point of failure in the storage and SQL host level, what
we are now going to do is install SQL on a 3rd server (SQL3) and we are going to enable and configure
AlwaysOn. Ideally you would configure this in advance of installing the SC applications as you are
effectively generating a new instance, name and port for the applications to connect to.
It’s a full install, however I am only going to cover the bare number of screens below as you have seen
them in the previous chapters. The important thing to remember for the install is that the product key
you use to install SQL3 must be Enterprise Edition to access the AlwaysOn feature. 40
The whole point of AlwaysOn is that it abstracts the back-end and so you do not need use the same
instance name. Obviously it will make it easier for the purposes of administration and troubleshooting if
you can use the same instance names on both the primary and secondary copies in the AlwaysOn
Availability Group but it’s not a hard and fast requirement.
41
Now for the really important part. All of the Data Directories listed here are using
the exact same drive letter and path as were configured previously on the SQL
cluster with the shared storage. AlwaysOn assumes that we have the same drive
mappings and folder naming in place on all servers in the solution.
42
3.2 Enabling AlwaysOn Availability Groups on the Cluster
Once the installation is complete, head over to the cluster nodes previously deployed, SQL1 and SQL2.
On each of the servers, open SQL Server Configuration Manager to enable AlwaysOn for each of
the SQL Server Database Engine instances which has been installed. In this case, this applies to
SCOM and the SCOM Data Warehouse SQL Server instances.
As you can see from the screens above and below, the Windows Failover Cluster name for the cluster
which this node is a member is populated for us. Tick the Enable AlwaysOn Availability Groups
check box to enable the feature and then hit the OK button.
Rinse and repeat the above for all of the instances you will be enabling AlwaysOn for. The next step is
43
to remove the dynamic port mode and list a static port instead. This is done in the SQL Server
Network Configuration portion of SQL Server Configuration Management on the TCP/IP Protocol
Name for each instance.
Scroll all the way to the end of the IP Addresses tab to reach the IPAll section and clear any value
from the TCP Dynamic Ports field and add a port number value to the TCP Port field. In this example,
I’m using 15100 and 15101 for SCOM and SCOMDWR respectively.
Once again, rinse and repeat these steps for the remaining instances and remember that these steps
need to be completed for all of the cluster node members (SQL1 and SQL2 in this example).
44
Now that AlwaysOn Availability Groups has been enabled you will need to restart the SQL Server
Database Engine service for each of the instances. You will only need to do this on the node which
currently owns the SQL Server database instance as the other node will have its service already in a
stopped state.
45
3.3 Adding the Replica to the Failover Cluster
We now need to add the Failover Clustering feature to SQL3 and add it to our cluster.
To do this, head back over to SQL3 and either from the Server Manager GUI as shown below or using
PowerShell add the Failover Clustering feature. For PowerShell, use the Cmdlet Import-Module
ServerManager | Install-WindowsFeature –Name Failover-Clustering –IncludeManagementTools.
46
Once the feature is enabled, the easiest way to add SQL3 to the cluster is to connect to an existing
node in the cluster, SQL1 in my case. Once connected, use the Add Node option from the GUI to add
SQL3 to the Windows Failover Cluster.
47
On the Select Servers pane of the wizard, enter SQL3 as the server name and then click Add.
On the Confirmation pane of the wizard, be sure to un-check the Add all eligible storage to the
cluster option. Leaving this ticked will attempt to add the local disks from SQL3 to the cluster which
wouldn’t work too well.
48
Once the wizard is completed, you should see SQL3 as a third node in the cluster from the Node view
in the Failover Cluster Manager.
Once you have verified the above, make the changes to the SQL Server services that we previously
made to the clustered services to enable the AlwaysOn High Availability feature on SQL3.
Open SQL Server Configuration Management and use the SQL Server Services view and select
Enable AlwaysOn Availability Groups on the SQL Server Database Engine services for each
instance.
49
With AlwaysOn now enabled for the instances, use the SQL Server Network Configuration view to
set the port for each of the instances to a static port for the TCP/IP Protocol.
Use the same port number for each instance as you used on the clustered instances and be sure to
clear any values from the TCP Dynamic Ports field so that Dynamic Port is disabled.
50
51
Once you’ve finished enabling the feature and changing the port number to the new static port for the
instances, restart the SQL Server Database Engine service for each of the instances for the changes to
take effect.
With the above now done, we need to setup the SQL endpoints on SQL3 which will pair with those
created previously on the clustered SQL instances.
Once the database is created, we must take an initial Full backup of it. This is required because the
database is set to Full Recovery Model (required for AlwaysOn protection).
53
Make a full backup of this DB
54
As the database is the blank sql_guide database we created earlier, this completes in a second or two.
Once you have enabled the share and set the share permissions, the same permissions need to be
applied to the NTFS folder. Use the Security tab on the folder to add the NTFS ACEs for the same
service accounts, the SQL Server Database Engine and the SQL Server Agent service accounts.
56
3.7 Creating an AlwaysOn Availability Group
Once all of the above is done, we’re ready to start the AlwaysOn Availability Group wizard. From the
server which you want to be used as the initial primary replica (probably the clustered instance), login to
SQL Server Management Studio.
Right-click on the AlwaysOn High Availability folder and select the New Availability Group Wizard
option.
57
After the introductory step, we are asked what to use as the Availability Group Name. This name will be
added to the Windows Failover Cluster as a new Resource Group with a Virtual IP Address and a
Network Name which will be used going forwards as the network name for our client and application
connections to the database instance.
58
Next, select the databases to protect with AlwaysOn. As this is a new instance with only our sql_guide
database, select this. If you omitted the steps earlier to take a backup of the database, you will not be
able to proceed past this point until it has been done.
With our database now selected, we need to get the clustered SQL Server instance and the standalone
instance talking. Click the Add Replica button and you will be presented with a SQL Server login
dialog. Enter the server and instance name (SQL3\SCOM in this example) and login with Windows
Authentication using your current credentials.
59
On the Endpoints tab, you should see the name Hadr_endpoint and the Port Number 5022 shown. If
these look familiar, good. These are the ports and names we specified earlier using the PowerShell
Cmdlets.
On the Backup Preferences tab you can configure how SQL Server databases on this instance will be
backed up. The default option is Prefer Secondary which is ideal as it allows our primary to continue
servicing client and applications requests for database records while a backup runs from our replica on
the secondary.
60
On the Listener tab we configure the DNS Name (Network Name) which will be configured and the IP
Address and TCP Port. In this example, SCOMAO is the network name, 15100 is the TCP Port and a
Static IP Address of 10.1.10.211 has been defined.
61
Once you have moved on to the Data Synchronization panel, ensure that the Full option is selected to
get things going immediately. Here we specify the path to the file share created earlier through which
the initial synchronization is performed. 62
63
After the summary, SQL Server will setup the Availability Group and make the sql_guide database
highly available for the SCOM instance.
Repeat the steps given in the Creating a Blank Database to Protect with
AlwaysOn and the Creating an AlwaysOn Availability Group sections for your
SCOM Data Warehouse instance.
64
3.8 Deploying SCOM into the AlwaysOn Availability Group
Now when it comes to installing a System Center product database into the AlwaysOn Availability
Group, we need to point the setup to the AlwaysOn listener (SCOMAO in my case) and not the
clustered SQL Server instance name. The same process applies to all SC product installations.
65
Assuming your SCOM installation completes without a hiccup or an issue, you’ve now got SCOM
installed and configured to use the AlwaysOn Listener name which means it’s setup to use AlwaysOn
however SQL itself isn’t quite ready so we need to dive back into SQL to make the SCOM and SCOM
Data Warehouse databases highly available.
Connect to your SQL Server which is currently the Primary for the Availability Group. You can verify
the primary or secondary status from the AlwaysOn High Availability folder in the SQL Server
Management Studio.
66
Once connected and verified that you are connected to the Primary, we need to make changes to the
SCOM database.
Expand the Databases folder and access the Properties for the OperationsManager database.
From the Recovery Model drop-down menu, change the option from Simple to Full as required by
AlwaysOn.
With the Recovery Model now set, we need to take a full backup of the database before processing
with the AlwaysOn wizards.
67
With the database now ready, return to the AlwaysOn High Availability folder and right-click the
SCOMAO Availability Group name and select the Add Database option from the menu.
68
This will open the Add Database to Availability Group wizard.
On the Select Databases pane, you will see the sql_guide databases already a part of the availability
group and the OperationsManager database should report the status of Meets prerequisites.
Select the check box for the OperationsManager database and hit Next.
On the Select Data Synchronization pane, leave the default option of Full as is and use the same
shared file share which we created earlier in the guide for the initial data transfer.
69
On the Connect to Replicas pane, you don’t need to specify any replicas this time as the connections
are already configured in the Availability Group but you do need to connect to the instance.
Click the Connect button and use Windows Authentication to connect to the remote SQL Server
instance.
On the Validation pane, you should hopefully get green for all of the pre-flight checks.
On the final pane before synchronization begins, the Summary will confirm all of your selections from
the wizard. Select Finish and let the replication begin.
70
All going well, you should get a confirmation that the wizard completed successfully.
Viewing the list of Availability Databases back in the SQL Server Management Studio folder view
should now show the OperationsManager database as per the screen below.
71
Repeat the steps in this section for your SCOM Data Warehouse instance.
Once you have completed this section and repeated the steps against your SCOM Data Warehouse
instance, you’re all set. You’re now using SCOM 2012 R2 with SQL Server 2012 AlwaysOn technology,
protecting your databases across a multi-node, multi-instance SQL cluster and a standalone SQL
server.
72
3.10 Deploying Additional System Center Products
When it comes to deploying the rest of the suite you can present your AlwaysOn instance and
sometimes the port is requested and sometimes it doesn’t. Before we go any further please
understand that SCCM 2012 does not support AlwaysOn, no question. For some time the reason this
was not supported was just that it hadn’t been tested, however as far as I know it has been tested (or
considered) and it has been ruled out by the product group.
And SCSM does not ask for the port it queries the AlwaysOn listener.
73
74
4.0 HYPER-V REPLICA
Hyper-V Replica (HVR) is a host based Hypervisor centric replication process, it is an in-box feature
from Microsoft and is a fairly simple replication setup. Unfortunately HVR is not supported with other
types of SQL replication. There is no native automated failover like there is with a SQL clustering. So
why might I use HVR and not SQL clustering…
Its host based so the fabric team can setup and configure it
Let's start with HV1 that is going to be the replica target for the VM's, I have added a 500Gb drive
called HVR and I am going to use this as my target location.
75
76
77
Now we go to HV4 and select the SQL server, as you can see from the screen below the SQL server is hosting a
ton of System Center DB's, it’s not clustered and we want to have some warmish copy of the data
78
79
80
81
82
83
84
85
4.1 Testing HVR
When you want to test the replication an easy way is to place a file on the desktop and check it go
migrated
86
I am also going to create a new DB called SQL guide
87
Now back on SQL 1 (the replica host I should be able to test the replication, to do that I simply go to re
plication and select "test failover". This will create a new virtual machine with a snapshot of the VHD's.
88
5.0 DEPLOYING SCOM WITH DIFFERENT HA OPTIONS
So we have explored a number of configurations possible when it comes to how you can configure SQL
in terms of HA for the SQL server and for the site where your database or databases might run. So in
this section we are going to walk through the different options for installing SCOM in a multi-site
environment and what considerations you may need to take into account in designing and
implementing your SQL server or servers. SCOM is a great product in the System Center suite to
discuss because a lot of companies require multi-site monitoring and want to have the ability to have a
HA SCOM whereby if they lose a primary date center they want monitoring to failover to a secondary
datacenter or DR site.
We are going to start off with the most basic SCOM deployment and work our way up from there. If you
are a seasoned SCOM pro please excuse the basic nature of
this, but it will be helpful for others to understand where it’s all
building on.
The other scenarios covered in this chapter are quite advanced and many of them are only possible with
either Server 2012 and or SQL 2012. What is important to keep in mind is that any solution that involves
HA and no single point of failure comes at a price, and usually a high one.
89
5.1 Basic SCOM Design
SCOM is installed on a
An agent is installed on single VM with a SQL
a server. instance on a separate
VM.
91
Important points of note here are:
We are using a SCOM MS and SQL server in the main EU data center
In the small satellite office where they are only a small number of servers they can connect directly
across the WAN to report to a MS in the main data center
In one of the additional EU data center where we placed an additional MS we consider this a
poor design as the network latency across the WAN is greater than 20ms
In the US data center a GW server is deployed. As the minimum latency between the US and EU
is about 60ms the GW server will collect agent data in the US, compress the data and then forward
it to a MS in the EU
There is no HA or DR in this configuration
GW servers bring a layer of complexity and add an additional server so in small satellite offices a
calculation needs to be made to decide at what point you place a GW server versus just allowing
agents to report across the WAN. Where a small office has a domain controller and a multi role
file and print server adding a GW server may add little to the configuration.
This design is full of single points of failure
The GW server cannot write direct to the SQL DB, and has to proxy to a MS who in turn writes to
the SQL DB.
There is no licensing cost for SQL if we are using SQL standard and it’s just for System Center
92
5.3 Local Site HA
In this configuration we are going to overview basic HA with SCOM in a single site. The reason we are
using a single site is to get the basic tenets of design across, to facilitate a discussion on HA, before we
get into more complex multi-site options.
Using two MS’s in a pool allows agents to fail over from one MS to another
Using a SQL cluster the failure of a single SQL node means the MS can fail over to another node
in the cluster
This configuration would be considered a basic HA option as we are still using a single copy of
93
We are using a SQL cluster in the main data center, as we are using AlwaysOn in the remote site
we need then need to use SQL Enterprise for all nodes.
94
The 3rd SQL node in the remote site can be used for actions like backup etc.
This is an automated fully supported configuration
We are using double MS’s and GW servers in each site
In this environment we can still monitor under the following situations
o Loss of the production cluster in the primary site
o Loss of both MS’s in the production site
o In the event of the complete loss of the primary site we can still monitor the secondary
data center
This is a reasonably complex configuration with an amount of planning and design necessary
This is a reasonably expensive configuration due to the SQL Enterprise licensing
It would be possible to collocate other databases on the cluster especially other System Center
products and use the cluster as a management cluster
If you locate other products on the 2 node cluster in the primary data center the database does
not necessarily need to take part in the AlwaysOn replication node.
This type of configuration would typically be deployed in a large multi-site environment
This configuration does not require storage replication
You can add more than one location for the backup SQL node
95
5.5 SQL Stretched Cluster
In this example we are using a 2 node cluster using a replicated SAN
This design is based on using storage replication from your storage vendor
The SQL cluster is based on 2 nodes one in each location
The SQL cluster needs only SQL Standard
The cluster requires a stretched subnet spanning the two sites
In the event of a failover in terms of the SQL environment the MS in the primary data center needs
to be powered off to force the agents in the primary data center to now a GW server in that data
96
center
It is essential to perform component failure analysis, to understand what happens in the event of
failure of individual infrastructure components. This ensures the supporting team knows what is
required to bring the solution back to operation in all failure scenarios
This is not an easy setup
While there is a reduction in cost for the SQL in terms of not needing/using SQL enterprise, there
is the cost of storage mirroring.
97
Important points of note here:
98
5.7 Using Hyper-V Replica
Deploying the SQL servers as VM’s
100
6.0 SQL IN AZURE TO SUPPORT SYSTEM CENTER
This section is intended to discuss how to implement your System Center SQL Server instances in
Windows Azure virtual machines (VMs). While many of the guidelines for configuring Microsoft SQL
Server are the same, the architecture of VMs in Windows Azure is quite different. Rather than
duplicating information found elsewhere in this guide, this chapter will serve primarily to clarify those
differences and how to optimize performance for your SQL Server instances hosting databases in
Windows Azure.
With this in mind, this chapter does not aim to teach you Windows Azure Virtual Machines (Windows
Azure IaaS) from the ground up. For training on Windows Azure IaaS, see the “Windows Azure for IT
Pros Jump Start” course on the Microsoft Virtual Academy at
http://www.microsoftvirtualacademy.com/training-courses/windows-azure-for-it-pros-jump-
start#?fbid=8YnntofK5KO
Network Connectivity
System Center Support in Windows Azure
VM Configuration
Optimizing SQL Configuration in Windows Azure VMs
High Availability and Disaster Recovery
A full discussion of Windows Azure virtual networking and site-to-site VPN is beyond the scope of this
section, but you can learn more with these resources:
Guidelines for Deploying Windows Server Active Directory on Windows Azure Virtual Machines
Install a Replica Active Directory Domain Controller in Windows Azure Virtual Networks
Windows Azure VMs are dynamically addressed and you cannot override this by setting a static address.
Any attempt to do so will result in loss of connectivity to the VM.
101
Virtual machines within a VNet will have a stable private IP address. Azure assigns an IP address from
the address range you specify and offer an infinite DHCP lease on it. So the IP address will stay with
the virtual machine for its lifetime. The exception to this is when a virtual machine is in a Stopped
(Deallocated) state, in which case the IP address is returned to the pool allocated to the virtual network
you defined. You should avoid placing production VMs in this state to prevent an unwanted IP address
change.
App Controller
Operations Manager
Orchestrator
Server Application Virtualization
Service Manager
You will notice that Virtual Machine Manager is not mentioned is not listed, which is not surprising,
given VMM manages the hypervisor….a layer not under our control in Windows Azure.
For a current list of System Center and Windows component support in Windows Azure, see
http://support.microsoft.com/kb/2721672
6.3 VM Configuration
The considerations for SQL Server running in Windows Azure VMs are largely due to differences in
Windows Azure VM and platform architecture versus that of Hyper-V and vSphere. Considerations
include:
VM Sizing
Network Throughput
Disk Subsystem
Database File Placement
However, some aspects of VM architecture in Azure differ from the Hyper-V or VMware equivalents and
therefore will be addressed in this chapter.
102
6.4 VM Sizing
The first rule of virtualizing SQL Server is to ensure you have enough compute, storage and compute
resources to match what you used in the physical server. If you review Microsoft documentation around
Windows Azure VMs, the documentation says “Review options and choose a size that matches the
workload”. In Windows Azure, you cannot simply set the CPU cores and memory to exactly the values
you wish. Instead, you must choose one of the available VM template sizes. Even if you upload one of
your own Hyper-V VM templates to Azure, you will have to choose one of these template sizes. There
are currently eight (8) VM templates in Windows Azure, which are shown here.
Based on your desired VM size, you should pick the VM that most closely matches that of your sizing.
The right template may have a slightly more or slightly less resources (in terms of CPU and memory)
than what you would have chosen in an on-premise situation. Since Windows Azure costs are
calculated on a pay-for-what-you-use basis, be sure not to choose a larger VM than necessary!
The bottom line is the CPU and memory resources available in VM templates will almost certainly be
the driver of your decision, and the larger VM sizes provide greater bandwidth.
103
Important: Network communications generated by accessing data disks and the operating system disk
attached to the virtual machine are not considered part of the bandwidth limits.
Operating system disk (persistent): Every virtual machine has one attached operating system
disk (C: drive) that has a limit of 127 GB. You can upload a virtual hard disk (VHD) that can be used
as an operating system disk, or you can create a virtual machine from a platform image, in which
case an operating system disk is created for you. An operating system disk contains a bootable
operating system. It is mostly dedicated to operating system usage and its performance is optimized
for operating system access patterns such as boot up.
Data disk (persistent): A data disk is a VHD that you can attach to a virtual machine to store
application data. Currently, the largest single data disk is 1 terabyte (TB) in size. You can specify
the disk size and add more data disks (up to 16, depending upon virtual machine size). This is
useful when the expected database size exceeds the 1 TB limit or the throughput is higher than
what one data disk can provide. Data disks are where your SQL database, log files and SQL
application files will reside.
Temporary local disk (non-persistent): Each virtual machine that you create has a temporary
local disk, the D: drive and which is labeled as TEMPORARY STORAGE. This disk is used by
applications and processes that run in the virtual machine for transient storage of data. It is also
used to store page files for the operating system. Temporary disk volumes are hosted in the local
disks on the physical machine that runs your virtual machine (VM). Temporary disk volumes are not
persistent. In other words, any data on them may be lost if your virtual machine is restarted.
Temporary disk volumes are shared across all other VMs running on the same host.
Each data disk can be a maximum of 1 TB in size. Depending upon your virtual machine size, you can
attach up to 16 data disks to your virtual machine. I/Os per second (IOPS) and bandwidth are the most
important factors to consider when determining how many data disks you need. Storage capacity
planning is similar to the traditional ‘spindle’ count in an on-premises storage design. A single Azure
data disk delivers approximately 500 IOPS, and you can have up to 16 data disks in a single Azure
storage account. Disk configuration and sizing for I/O is covered in "Optimizing SQL Configuration in
Windows Azure VMs" later in this chapter.
of temporary unavailability. For example, if you want to leverage geo-replication at the Storage Account
level to automatically replicate your data disk to a secondary region, using a single data disk can be
helpful.
IMPORTANT: You cannot use geo-replication with multiple data disks configurations, because consistent
write order across multiple disks is not guaranteed.
Frequently accessed data is stored in the memory (RAM) of the host machine.
Less recently accessed data is stored on the local hard disks of the host machine. There
is cache space reserved for each virtual machine operating system and data disks based
on the virtual machine size.
Cache settings help reduce the number of transactions against Windows Azure Storage and can
reduce disk I/O latency in some scenarios. Windows Azure disks support three different cache settings:
Read Only, Read Write, and None (disabled). There is no configuration available to control the size of
the cache.
Read Only: Reads and writes are cached for future reads. All writes are persisted
directly to Windows Azure Storage to prevent data loss or data corruption while still
enabling read cache.
Read Write: Reads and writes are cached for future reads. Non-write-through writes are
persisted to the local cache first, then lazily flushed to the Windows Azure Blob service.
For SQL Server, writes are always persisted to Windows Azure Storage because it uses
write-through. This cache setting offers the lowest disk latency for light workloads. It is
recommended for the operating system and data disks with sporadic disk access. Note
that total workload should not frequently exceed the performance of the physical host
local disks when this cache setting is used.
None (disabled): Requests bypass the cache completely. All disk transfers are
completed against Windows Azure Storage. This cache setting prevents the physical
host local disks from becoming a bottleneck. It offers the highest I/O rate for I/O intensive
workloads. Note that I/O operations to Windows Azure Storage do incur transaction
costs but I/O operations to the local cache do not.
Important notes:
105
Cache can be enabled for up to 4 data disks per virtual machine (VM).
Changes to caching require a VM restart to take effect.
Since “Read Write” cache is enabled by default, no action is typically needed here. Just don’t disable this
setting without good reason!
If the workload demands a high rate of random I/Os (such as a SQL Server OLTP workload) and
throughput is important to you, the general guideline is to keep the cache set to the default value of
“None” (disabled). Because Windows Azure storage is capable of more IOPS than a direct attached
storage disk, this setting causes the physical host local disks to be bypassed, therefore providing the
highest I/O rate. This is the setting that will apply to your larger System Center databases, including;
For smaller databases with typically lower IO, including Orchestrator and App Controller, you may
consider placing them on an existing SQL instance, but still placing the database files on a data disk is
recommended.
While the VMM database typically falls into the “small with low I/O category, VMM is not supported in
Windows Azure VMs and therefore is not applicable to this discussion.
The same rules apply for TempDB when running SQL in Azure VMs. TempDB should be placed on the
operating system disk or a separate data disk (preferably a data disk if possible). Do not be tempted to
place this database on a temporary disk!
IMPORTANT: Using files and filegroups across multiple data disks does not help scaling transaction log
throughput and bandwidth (sequential writes) in the same way it does for random IO activity. If you need
106
to scale your transaction log performance, MS recommends use a striped volume instead, such as
Storage Spaces in Windows Server 2012 R2. For step-by-step guidance in using this technology in
Windows Azure, see “Use Windows Azure to learn Windows Server 2012 Storage Spaces”.
NOTE: With regards to Windows Azure disk performance, you may read that Microsoft have observed
a “warm-up effect” that can result in a reduced rate of throughput and bandwidth for a short period of
time. In situations where a data disk is not accessed for a period of time (approximately 20 minutes),
adaptive partitioning and load balancing mechanisms kick in. This warm-up effect is unlikely to be
noticed on systems with workloads in continuous use, such as System Center SQL instances.
For write intensive workloads (which is descriptive of the System Center SQL instances), adding more
data disks increases performance in a nearly linear fashion according to Microsoft testing.
For a sample T-SQL script for splitting (striping) a SQL database into multiple files on multiple drives, see
“Create Database (Transact SQL)” at http://msdn.microsoft.com/en-US/library/ms176061.aspx.
To update an existing database to stripe the database across multiple files, you will need to use the
ALTER DATABASE statement with the FILE option. T-SQL samples are available at
http://msdn.microsoft.com/en-us/library/bb522469.aspx.
IMPORTANT: While you could potentially scale beyond the 4,000 IOPS limit of disks in a single storage
account by attaching disks from multiple storage accounts, Microsoft recommends avoiding this if
possible. Attaching data disk from multiple Azure storage accounts increases complexity and the
likelihood of data consistency issues in some failure scenarios.
Log Shipping
Backup and Restore with Windows Azure Blob Storage Service
It is possible to not only to use these technologies in Windows Azure, but to combine them to
implement a SQL Server solution that has both high availability and disaster recovery capabilities,
whether in a private cloud, public cloud or hybrid cloud environment. Depending on the technology you
use, a hybrid cloud deployment option may require a site-to-site VPN tunnel with the Windows Azure
virtual network.
For step-by-step guidance in implementing these SQL Server HADR technologies, see the following
tutorials:
AlwaysOn Availability Groups - see Tutorial: AlwaysOn Availability Groups in Windows Azure
(GUI)
Database Mirroring - see Tutorial: Database Mirroring for High Availability in Windows Azure
Log Shipping - see Tutorial: Log Shipping for Disaster Recovery in Hybrid IT
Backup and Restore with Windows Azure Blob Storage Service - see Backup and Restore
for SQL Server in Windows Azure Virtual Machines
For additional reading on the currently available options and example HADR architectures for SQL
Server 2012 in Windows Azure virtual machines, see
High Availability and Disaster Recovery for SQL Server in Windows Azure Virtual Machines
108
7.0 BACKING UP SQL WITH DPM
1. File Services
2. Hyper-V
3. Exchange Server
4. SQL Server
5. SharePoint Server
6. Windows Servers in trusted domains, untrusted domains & Microsoft workgroups
7. Windows Clients
8. System State
System Center Data Protection Manager 2012 R2 will also protect any Windows Failover Server
Cluster based solution, being a cluster-aware backup product.
For a full installation guide and further useful information regarding DPM please visit the blog “Robert
and DPM” at http://robertanddpm.blogspot.com/.
1. SAN
2. NAS
3. DAS
4. VHDX
For further information and guidelines regarding how to design and deploy DPM please visit ‘Robert
and DPM’ at http://robertanddpm.blogspot.com/ or read the ‘DPM 2012 R2 Cookbook’ which will be
released during 2014 by Packt Publishing.
This can be done from the DPM console but also other System Center components such as System
Center Configuration Manager or System Center Orchestrator. You can also use simple batch files and
GPOs within Active Directory to deploy the agent.
7.7 DPMDB
The most critical part of the SCDPM server platform is the local database, called DPMDB. All
configurations made for Protection Groups, SCDPM agent deployment & backup schedules are stored
within the DPMD. In a high-end implementation of SCDPM it is strongly recommended to place the
DPMDB on a dedicated volume that has the correct sector sizes to ensure high performance and
scalability. The recommended NTFS unit allocation size is 64 Kb since this is matches SQL write
patterns.
110
The location of the DPMDB database on disk can be specified two ways:
1. As a configuration option during the installation of the DPM software
2. As a re-configuration, performed at any later time, using SQL Management Studio
The recommended option is to place the DPMDB on the correct volume during the installation of the
software, to minimize rework.
Establish a two-way transitive trust between the domains: The author would not casually
recommend this approach due to security considerations.
Using Workgroup protection: This will not protect any clustered or mirrored databases
Using Certificate Based Authentication (CBA): This will also protect clustered SQL databases
As a DPM administrator you can’t be selective with this behavior if you would like to exclude some SQL
databases from the auto-protection. If you enable this feature, System Center Data Protection Manager
2012 R2 dos it for all databases within the SQL Server instance.
113
To enable the Auto-Protection for a SQL Server instance you check the checkbox next to the SQL
Server instance name. A text (Auto) will appear in front of the SQL Server instance name and then you
know that the Auto-Protection feature is enabled.
By collocating the replica volumes in the DPM disk pool the number of possible protectable SQL Server
databases will increase from approximately 1,000 to 2,000 SQL Server databases. Now keep in mind
that this is a theoretical number meaning that the size of the databases will also play an important role.
The best way to find out how many databases a DPM server can protect is to begin to calculate the
sizes of the databases. More information regarding this can be found on the blog
http://robertanddpm.blogspot.se
114
7.9 Windows Azure
System Center Data Protection Manager 2012 R2 can connect to the Windows Azure Backup Service
to provide off-site archiving for up to 120 days. The supported workloads to be archived into Azure are:
7.10 Pre-Configuration
In this part of the chapter you will get both a walkthrough of how to setup a protection group and also
special pre-configuration tasks that may need to be performed dependent on your version of SQL
Server.
SIMPLE
FULL
BULK-LOGGED
The major difference between the SQL database recovery models are that FULL and BULK-LOGGED
recovery models works with transaction logs whereas a database configured with SIMPLE recovery
mode does not use transaction logs. Since System Center Data Protection Manager can interrogate
SQL Server and see what recovery model a database is configured to use, System Center Data
Protection Manager 2012 R2 will provide the appropriate restore options permissible for the given
recovery model.
In the scenario where a SQL Server failover cluster is performing an unplanned failover in response to
a failure, the DPM server will generate an alert to indicate that a consistency check needs to be
performed. This results in an inconsistent replica. Replica inconsistency can remember, be easily
resolved using the built-in auto-heal function. If you don’t want to wait three hours for the auto-heal
function to trigger you can alternatively create a Runbook in System Center Orchestrator 2012 R2 and
clear any replica of a data source that is inconsistent.
For System Center Data Protection Manager 2012 R2 to be able to protect a SQL Server failover
cluster built on the supported protectable versions of SQL Server a DPM agent must be deployed to all
servers participating in the failover cluster.
The deployment of DPM agents can be automated for a modern dynamic datacenter by combining the
power of System Center Data Protection Manger 2012 R2, System Center Operations Manager 2012
R2 and System Center Orchestrator 2012 R2.
As you also can see System Center Data Protection Manager 2012 R2 will also provide you with the
information of what you should do to get this up and running. You must provide the DPMRA service
sysadmin rights on the SQL Server instance that the database resides in.
Open the SQL Server 2012 Management Studio and log on to the instance that contains the databases
that you would like to protect using System Center Data Protection Manager 2012 R2.
Create a New Login… by right click the Logins folder.
Add the NT SERVICE\DPMRA account under the Login name. You can also search for it by clicking
the Search… button on the right side of the Login name field.
118
Click on Server Roles page on the left side of the window and add the sysadmin rights for DPMRA.
119
When you now execute the Configure New Protection Group job again in System Center Data
Protection Manager 2012 R2 you will now be able to protect the SQL Server 2012 databases.
120
7.17 Configuring Protection Groups for SQL Server
For System Center Data Protection Manager 2012 R2 to be able to start protect a SQL Server, you
need to create a Protection Group. A Protection Group is a set of instructions that will provide the DPM
agent with the backup schedule and workload information for a successful backup.
To configure a Protection Group you click on the New button in the top left part of the DPM console.
You now need to choose if you want to protect Servers or Clients. Select Servers and click Next >.
121
Now it’s time to choose Group Members of the Protection Group.
122
In this example I will expand the server SCSQL and all the present workloads will be presented.
123
Expand All SQL Servers and all SQL Server Instances will be displayed.
124
Now you have two choices, you can:
Shot-term protection using disk, it’s not a recommendation to use tape though it’s a possible
approach.
Online protection using Azure
Long-term protection using tape
In this example we will mark Short-term protection for on-premise restore operations and Azure for long
term protection (archiving). Click Next > to continue.
126
Now you must choose your Short-Term recovery goals. Initially, you should consider for how many
days you wish to store the protected data source recovery points in your DPM disk pool. This is called
Retention Range, the default value is five (5) days.
127
Next you must consider the Synchronization frequency by choosing a synchronization interval in the
dropdown list. The last part is to select when Recovery points should be created. By default recovery
points are created at 8 PM every day, to change this click the Modify button.
128
Now you can choose the recovery point times that meet your organizations recovery point objectives
(RPOs). Mark the times and click on the Add > button, also choose the appropriate weekdays for
creating recovery points. When you are done you click on OK.
129
Now you will get an updated summary for the sort-term recovery goals, verify it and click Next > to
continue.
130
On the Review Disk Allocation step, DPM will present a summary of the actual disk allocation that
need to be made for the workload to be protected according to your recovery point schedule. Please
note the checkbox that says “Automatically grow the volumes”. This is one of System Center Data
Protection Manager 2012 R2 auto-heal functions.
You can alter the disk allocation configuration by clicking the Modify button.
131
When you click the Modify button, you will be presented of the actual allocation for the collocated
volumes. You can alter the value that is proposed however if DPM detects that this will not result in a
successful modification you will be prompted with a warning.
132
When you are done altering the disk allocation click the OK button and you will return to the Review
Disk Allocation step. Click Next > to continue to the Choose Replica Creation Method step.
133
In the Choose Replica Creation Method step you have three different options that will provide you
with suggestions how to create your replica, these are:
Now, replica will be created during the creation of the Protection Group
Later, replica will be scheduled for creation
Manually, replica will be created manually by the DPM administrator
In this guide we will create the replica Now. Click the Next > button to continue.
134
The next step is the Consistency check option where you specify another of DPM’s auto-heal
functions called “Run a consistency check if a replica becomes inconsistent”. You can also choose if
the Protection Group you are designing will schedule a daily Consistency Check. This means that
DPM will synchronize any changes from the production workload to the DPM server and making the
replica consistent once again.
When you are finished you click Next > to get to the next step of the configuration.
135
In the next step called Specify online protection data you choose which workloads should be
archived to Windows Azure Backup. The supported workloads are:
SQL
Hyper-V
File Services
Check the checkbox for the workload you would like to put in Azure and click on Next >.
136
In the Specify Online Protection Goals window, you have two options. You can archive recovery
points every day or on a weekly basis. For the weekly archiving basis see the next screenshot.
137
Specify when the DPM server should synchronize the recovery point and click on Next > to continue.
138
Verify the configuration in the Summary step and click on the Create Group button.
139
In the Status step you will see when the configuration is successfully finished and the Close button will
be clickable. Click Close to end the setup wizard for the Protection Group.
140
7.18 Restoring SQL Server Data from DPM
There are three media types that DPM can restore SQL data to:
To restore data from System Center Data Protection Manager 2012 R2 you must click the Restore in
the navigation bar (down left in the console).
With transactional log systems like SQL (recovery model FULL or BULKED-LOGGED only) and
Exchange server you have the possibility to restore the SQL database or Exchange mailbox database
to its last good transaction using the Latest recovery time option. You will only be able to restore the
data to its original location using the Latest restore recovery option.
141
If you would like to restore from a valid recovery point click start by choosing the correct date all dates
that contains valid recovery points have bold numbers. Now you must choose a valid Recovery time in
the dropdown list, choose the time that you would like to perform a restore from.
After you have specified the date and time you right-click the database that you would like to restore
and choose Recover.
If you would like to see all of the available recovery points for restore for the data source choose Show
142
143
7.19.1 RECOVERY TO THE ORIGINAL INSTANCE OF SQL SERVER
This recovery type will restore the SQL database to its original location and overwrite the database
files. This option is useful in the case of for example corruption or accidental deletion.
In this recovery type you have two different database states that you can choose:
144
The second database state mentioned (Leave database non-operational but able to restore additional
transactional logs) will make it possible to restore additional log file data. These additional transaction
log must be in a folder on the original SQL Server.
If you have chosen to restore a Synchronization, DPM will prompt you with the following screen:
146
Now DPM presents the valid recovery points that you can choose to restore to a network folder. When
you have a valid recovery point you will be prompted to select where to restore the database. You can
restore the files to any server that DPM protects and choose a specific folder for the restore.
147
You need to specify the primary library and a copy library if you have one configured. You can choose
to compress or encrypt the data written to the tape. One important thing to keep in mind is if you
choose to encrypt the data written to the tape you will always need that specific certificate that
encrypted the data. If the certificate is lost or expires, you will be unable to restore this data from the
tape.
Right click the database and choose Recover and the Recovery Wizard starts. Keep in mind that only
Express-Full recovery points will be written to tape.
You have the same options as you had when you restored from the short-term recovery goals. There is
one thing which differentiates the two options and that is the Specify database state option. If you
choose to restore the database and set the “Leave database non-operational but able to restore
additional transaction logs” option you won’t be able to restore additional transactional log files, just the
database.
148
7.21 Cloud Recovery Restore
When you choose to restore databases from Azure you need to start by pointing out a recovery point
that was stored in Azure by choosing the right date and time in the Recovery workspace.
Right-click the database and choose Recover and the Recovery Wizard starts. You will have three
options:
Also available are System Center Data Protection Manager blogs. The ideal starting place is the official
product group blog from Microsoft at http://blogs.technet.com/b/dpm.
There are also several MVP blogs dedicated to DPM such as http://robertanddpm.blogspot.se.
150
8.0 MONITORING SQL WITH SCOM - MATTHEW LONG
System Center Operations Manager is Microsoft’s enterprise monitoring tool, and through its
Management pack system has a deep level of knowledge and monitoring insight on the health of a wide
variety of applications (both Microsoft and otherwise).
One such freely available management pack is the System Center Management Pack for SQL Server.
This management pack provides SCOM with the capabilities to automatically find SQL Servers in your
environment, proactively monitor SQL components for health state changes, and provide dashboards,
architecture diagrams and common task shortcuts for SQL administrators and operations staff.
As of the current release (6.4.1.0) the management pack provides support for:
As such we recommend you download the SQL MP from Microsoft download if you are using SCOM
2012 and wish to get the full experience. In addition the SQL mp guide is extremely details and covers
the prerequisites and monitoring capabilities in full. This guide is only available from Microsoft
download and must be read and understood by your SCOM administrators prior to importing the SQL
management pack.
You can download the current SQL management pack release from http://www.microsoft.com/en-
gb/download/details.aspx?id=10631.
This in turn releases resources that the SQL administration team may otherwise have to spend
investigating issues or fielding enquiries into SQL health and defending the platform when issues arise
for which the root cause is not apparent.
The SQL management pack also collects a huge amount of performance information, which is then
stored and aggregated in the SCOM Data Warehouse for long term reporting. Whilst the performance
counter collection interval may be too low to troubleshoot certain issues (with most performance
counters sampled on a 5 or 15 minute interval) this is more than acceptable for trending and reporting
across the last year of operation.
If your organization restores these rights to SQL roles as part of their build process then you will not
need to provision SCOM Run as Accounts. However as this is generally not considered a secure
configuration it is more likely that you will need to provide SCOM with an account that has access to
SQL resources in order to correctly discover and monitor SQL components.
The Low-privilege Environments section details what the necessary accounts and rights should be.
Note that by default Local System (the default agent action account) will likely already have access to
WMI, performance counters and the Windows registry. If you do not use one global account for
monitoring all SQL servers, you may have to provision and configure multiple accounts within
Operations Manager. Take note of the Run As Account distribution model you are using for these
accounts; if you are using the “Most Secure” method you will need to configure each agent monitoring
SQL components to be able to make use of that account. This may impact your system onboarding
process into operations manager.
If you are making use of SQL Clusters or AlwaysOn capabilities, you must ensure that you enable the
Agent Proxy capability on all SCOM agents monitoring those SQL components. Failure to do so will
152
prevent correct discovery of Clustered or AlwaysOn components. Whilst SCOM will raise alerts for
agents missing Run As Accounts or Agent proxy settings, the alerts will not appear in SQL specific
views and can only be rectified by SCOM administrators.
However, there are many situational or scenario specific monitors and rules that are disabled out of the
box as they are only of use in certain configurations, or require thresholds specified by your SQL
administrators before they can provide accurate heath calculations.
The following table lists all content in the SQL 2012 mp (as of 6.4.1.0) that is disabled by default.
Consider enabling these elements in your environment if they would prove useful (these can be
enabled either environment wide or only for specific SQL instances via SCOM Overrides).
SQL Server 2012 Discovery This discovers replication SQL Server 2012 DB
Replication publications and subscriptions for Engine
Publications and each SQL Server 2012 database.
Subscriptions
Discovery
Provider
SQL Server Full Monitor This monitor checks the status of SQL Server 2012 DB
Text Search the SQL Full Text Search service. Engine
Service
SQL User Monitor This monitor analysis the user SQL Server 2012 DB
Connections connections to the SQL database Engine
Performance engine over time and calculates a
baseline over the initial learning
153
period.
Blocking Monitor Monitors blocked sessions for a SQL Server 2012 DB
Sessions SQL instance. Engine
Database Backup Monitor This monitor checks the status of SQL Server 2012 DB
Status the database backup as reported by
Microsoft® SQL Server™.
Auto Close Monitor Monitors the auto close setting for SQL Server 2012 DB
Configuration the database
Auto Create Monitor Monitors the auto create statistic SQL Server 2012 DB
Statistics setting for the database
Configuration
Auto Shrink Monitor Monitors the auto shrink setting for SQL Server 2012 DB
Configuration the database
Auto Update Monitor Monitors the auto update statistics SQL Server 2012 DB
Statistics setting for the database
Configuration
Auto Update Monitor Monitors the auto update statistics SQL Server 2012 DB
Statistics Async async setting for the database
Configuration
DB Chaining Monitor Monitors the DB chaining setting for SQL Server 2012 DB
Configuration the database
Recovery Model Monitor Monitors the recovery model setting SQL Server 2012 DB
Configuration for the database.
Page Verify Monitor Monitors the page verify setting for SQL Server 2012 DB
Configuration the database.
Trustworthy Monitor Monitors the trustworthy setting for SQL Server 2012 DB
Configuration the database
DB Total Space Monitor Monitors the space available in the SQL Server 2012 DB
database and on the media hosting
the database in percentage terms.
DB Space Monitor Monitors for a large change in value SQL Server 2012 DB
Percentage of database available space over a
Change set number of sample periods.
Long Running Monitor This monitor checks for long running SQL Server 2012 Agent
Jobs SQL Agent jobs.
Disk Read Monitor Disk Read Latency monitor for 2012 SQL Server 2012 DB
Latency DBs
Disk Write Monitor Disk Write Latency monitor for 2012 SQL Server 2012 DB
Latency DBs
SQL Re- Monitor SQL Re-Compilation for 2012 DB SQL Server 2012 DB
Compilation Engine Engine
Transaction Log Monitor Transaction Log Free Space (%) SQL Server 2012 DB
Free Space (%) monitor for 2012 DBs
154
SQL Server 2012 Rule Detects SQL Server 2012 DB SQL Server 2012 DB
DB Engine is Engine restart. Engine
restarted
SQL Server Rule Detects that at least one endpoint in SQL Server 2012 DB
Service Broker or a SQL Server broker conversation Engine
Database has stopped listening for
Mirroring connections.
Transport
stopped
SQL Server Rule SQL Server Service Broker SQL Server 2012 DB
Service Broker transmitter stopped due to an error Engine
transmitter shut or a lack of memory
down due to an
exception or a
lack of memory
SQL Server Rule SQL Server Service Broker or SQL Server 2012 DB
Service Broker or Database Mirroring is running in Engine
Database Federal Information Processing
Mirroring is Standard (FIPS) compliance mode
running in FIPS
compliance
mode
The SQL Server Rule The SQL Server Service Broker or SQL Server 2012 DB
Service Broker or Database Mirroring endpoint is Engine
Database disabled or not configured.
Mirroring
transport is
disabled or not
configured
SQL Server Rule SQL Server Service Broker SQL Server 2012 DB
Service Broker Manager has shut down Engine
Manager has
shutdown
An SNI call failed Rule SQL Server Service Broker or SQL Server 2012 DB
during a Service Database Mirroring attempted to Engine
Broker/Database access the transport layer through
Mirroring the SQL Network Interface (SNI).
transport SNI returned an error. Transport
operation cannot continue.
An SQL Server Rule A procedure that SQL Server SQL Server 2012 DB
Service Broker Service Broker internally activated Engine
procedure output output results. Internal procedures
results should not output results.
155
The Service Rule SQL Server Service Broker or SQL Server 2012 DB
Broker or Database Mirroring transport has Engine
Database started
Mirroring
Transport has
started
Process Worker Rule This error indicates that there is a SQL Server 2012 DB
appears to be possible problem with a thread not Engine
non-yielding on yielding on a scheduler.
Scheduler
IO Completion Rule I/O completion ports are the SQL Server 2012 DB
Listener Worker mechanism by which SQL Server Engine
appears to be uses a pool of threads that was
non-yielding on created when the service was
Node started to process asynchronous I/O
requests.
An SQL job failed Rule A SQL Server Agent Job failed. SQL Server 2012 Agent
to complete
successfully
IS Service has Rule The Integration Services service SQL Server 2012
attempted to was used to send a request to the Integration Services
stop a running Integration Services runtime to stop
package a running package.
SQL Server Rule SQL Server is stopping because the SQL Server 2012 DB
terminating server is shutting down. Engine
because of
system
shutdown
Table: Creating Rule sp_createstats has generated SQL Server 2012 DB
statistics for the statistics for each eligible column in Engine
following the current database
columns
In addition to the disabled features above, it may be necessary to turn off functionality which is enabled
out of the box because it is noisy, not applicable, duplicating an existing alert, or not violating a best
practice of your organization. The SQL management packs provide a series of version specific and
generic groups that you can use for this purpose.
In addition, it may be worthwhile creating a series of custom dynamic groups that automatically
populate with SQL components matching certain criteria. These groups can then be used as the target
for monitoring overrides, in order to reduce SCOM administrative overhead and shorten the alert tuning
process. Common groups may include:
156
SQL Express/Developer edition Instances and databases (products may install SQL Express
DBs and self-manage these systems, so SCOM alerting is not required).
SQL DR systems (these systems are often inactive and will generate false alerts until they are
enabled for production use).
Clustered SQL Servers (monitoring configuration may differ significantly for a cluster. For
example, the cluster will manage which SQL services should be started at any given time).
The following table illustrates what classes exist in the SQL management packs, which may prove
useful when deciding how to target your custom monitors and rules.
Services
SQL Agent SQL DB Engine The SQL Agent is discovered once
per instance of a DB Engine.
SQL Agent Job SQL Agent Discovery is disabled by default.
One instance will be discovered per
job once enabled.
SQL Database SQL DB Engine One instance will exist for each
database. Note that system
databases will also be discovered as
this class. If monitors/rules are
specific for a certain database, leave
them disabled by default and enable
via overrides.
SQL Server DB File SQL Server DB File One instance discovered for each
Group DB file in the file group.
SQL Server DB Log SQL Database One instance discovered per
File database.
158
8.3.2 USEFUL MODULES WHEN AUTHORING SQL MONITORING
Once you have identified the appropriate class that you wish to target with your custom SQL
monitor/rule, you will need to build a module in your custom management pack that can be used to
query SQL and process whatever health check you wish to perform.
There are three main ways that you can easily query SQL using the built in SCOM modules:
Each of these modules has its own set of set of knowledge prerequisites (VBScript skills, Powershell
script skills, SCOM Composite module creation) so choosing the appropriate method will largely be pre-
deteremined by your available skillset. If you are comfortable with many of these methods, it should be
noted that whilst the OleDbProbe has the lighest performance impact on the SCOM agent, it will require
knowledge of other SCOM modules in order to process the returned data and generate a
health/performance state.
All three modules will allow you to specify a SQL query that will be executed against the targeted
system. If you are creating a Powershell or VBScript, you will need to ensure that you can build a
suitable connection string to connect to the SQL DB in question.
The database name can be retrieved using the name parameter on the SQL Database class.
The SQL Server/instance can be retrieved using the “Connection String” property of the SQL
DB Engine hosting the SQL database.
The connection string should specify one of:
o Integrated security (to run as the user account the script is executing under). You should
specify that your script module needs to use the SQL Server Monitoring Run as profile
(Microsoft.SQLServer.SQLProbeAccount) if the default SCOM agent account does
not have access to query SQL.
o If you are using SQL authentication you will need to create a Run As account and Profile
and use the SecureInput optional parameter of the Script module.
159
8.4 Sample VBScript
The following sample comes from the SQL Server 2012 MP. This script provides the state of each
Database which can then be filtered upon in a monitor to provide health state information.
Option Explicit
SetLocale("en-us")
Function Quit()
WScript.Quit()
End Function
If IsObject(oObject) Then
If Not oObject Is Nothing Then
IsValidObject = True
End If
End If
End Function
Class Error
Private m_lNumber
Private m_sSource
Private m_sDescription
Private m_sHelpContext
Private m_sHelpFile
m_sSource = ""
m_sDescription = ""
m_sHelpContext = ""
m_sHelpFile = ""
End Sub
Sub HandleError(customMessage)
Dim localLogger
If Not (Err.number = 0) Then
Set localLogger = new ScriptLogger
localLogger.LogFormattedError(customMessage)
Wscript.Quit 0
End If
End Sub
Function HandleErrorContinue(customMessage)
Dim localLogger
HandleErrorContinue = False
If Not (Err.number = 0) Then
Set localLogger = new ScriptLogger
localLogger.LogFormattedError(customMessage)
Err.Clear
HandleErrorContinue = True
End If
End Function
'#Include File:ConnectionString.vbs
Const SQL_DISCOVERY_CONNECT_FAILURE = -1
Const SQL_DISCOVERY_QUERY_FAILURE = -2
Const DB_PROPERTY = "Status"
Call GetDBHealth()
Sub GetDBHealth()
If WScript.Arguments.Count = 1 Then
Dim oAPI, oBag
Set oAPI = MOMCreateObject("MOM.ScriptAPI")
Set oBag= oAPI.CreatePropertyBag()
Dim nResult
nResult = DBHealth(WScript.Arguments(0), oBag)
Call oAPI.Return(oBag)
Else
Wscript.quit()
End If
End Sub
Dim e
Set e = New Error
Dim cnADOConnection
Set cnADOConnection = MomCreateObject("ADODB.Connection")
cnADOConnection.Provider = "sqloledb"
cnADOConnection.ConnectionTimeout = 30
Dim sProv
sProv = BuildConnectionString(sSQLConnectionString, "master")
162
e.Clear
On Error Resume Next
cnADOConnection.Open sProv
e.Save
On Error Goto 0
If 0 <> Err.number then
DBHealth = SQL_DISCOVERY_CONNECT_FAILURE
ThrowScriptErrorNoAbort "Query execution failed for '" & sSQLConnectionString &
"' SQL Server", e
Exit Function
End If
Dim oResults
e.Clear
On Error Resume Next
' query for the list of databases which are not database snapshots
Set oResults = cnADOConnection.Execute("SELECT name , ISNULL
(DATABASEPROPERTYEX(name," & "'" & DB_PROPERTY & "'),'N/A') from sys.databases")
e.Save
On Error Goto 0
oResults.MoveNext
Loop
cnADOConnection.Close
End Function
# ActiveConnectionsDataSource.ps1
# Active Connections Property Bag Script
#
param($computerName, $sqlInstanceName)
$res = "";
if ($InstanceName -eq "MSSQLSERVER")
{
$InstanceName = "DEFAULT"
}
$principalName = $ComputerName
if ($InstanceName -ne "DEFAULT") {
$principalName = "{0}\{1}" -f $principalName, $InstanceName
}
$UserPolicies = @{}
$msg = "User Policies: {0}" -f [Environment]::NewLine
$SqlCmd.CommandText = $query
$SqlCmd.Connection = $SqlConnection
$SqlAdapter.SelectCommand = $SqlCmd
$SqlAdapter.Fill($DataSet)|out-null
$res = $DataSet.Tables
}
catch
{
$SqlConnection.Close()
throw $_
}
$SqlConnection.Close()
return $res
}
function Main {
param(
$computerName,
$sqlInstanceName
)
#
# Prepare MOM API and property bag object
#
$api = New-Object -comObject "MOM.ScriptAPI"
try
{
$msg = [string]::Empty
GROUP BY dbid;"
$res = SqlQueryTables $computerName $sqlInstanceName $query
$dbCount = 0;
$res[1] | foreach {$dbConections[$_.DBName] = [double]$_.NConnections }
$dbConections.GetEnumerator() | foreach {
$bag = $api.CreatePropertyBag()
$bag.AddValue("Name", $_.Key)
$bag.AddValue("Value", $_.Value)
$dbCount++;
$bag
}
$msg = "Active connections were got for {0} DBs on {1}" -f $dbCount, $InstancePath
}
catch
{
$header = "Managegement Group: $Target/ManagementGroup/Name$. Script: {0}" -f
($MyInvocation.MyCommand).Name.ToString()
$msg += "Error occured during Active connection data source
executing.{0}Computer:{1} {0}Reason: {2}" -f [Environment]::NewLine, $env:COMPUTERNAME,
$_.Exception.Message
$api.LogScriptEvent($header, $SCRIPT_EVENT_ID, $ERROR_EVENT_TYPE, $msg)
}
}
It also has the nifty ability to retrieve the database settings from specified registry keys, which can avoid
the need to go out and discover those pieces of information. This makes it quite suitable for attaching
onto existing classes from other management packs.
<RowLength></RowLength>
<Columns>
<!-- Data for first row returned -->
<Column>Data in first column</Column>
<Column>Data in Second column. </Column>
</Columns>
<Columns>
<!-- Data for Second row returned -->
<Column>Data in first column</Column>
<Column>Data in Second column. </Column>
</Columns>
This can make processing the results in further modules a pain, since your XPath Query is going to
have to specify which row and column specifically you want to access. If you instead set
OneRowPerItem to true then you’ll get multiple return items and can filter them using an Expression
filter with a simple syntax such as $Data/Columns/Column[1]$ You may also wish to filter on the
RowLength property to establish if any rows were returned. Remember that the module will return a
data item if it succeeded to connect but doesn’t have rights to query, so check that data was returned
before you try to do something with it!
166
167
8.6 Example Configurations
Normally if I’m going to use an OleDBProbe to access a database repeatedly I’ll create my own probe
module that sets up the settings I’m going to need and is already set to use my MP’s run as profile for
DB access. That way I don’t have to keep specifying it over and over again. Below is a sample where
I’ve done this, and configured everything other than my query to pass in for a SQL database
probe. Now all my monitors and rules that make use of this know where to locate the DB and what
other query options to use automatically (along with credentials).
<DatabaseServerNameRegLocation>SOFTWARE\MyRegKey\Database\DatabaseServerName</DatabaseServerName
RegLocation>
</ProbeAction>
</MemberModules>
<Composition>
<Node ID="OledbProbe">
<Node ID="PassThru" />
</Node>
</Composition>
</Composite>
</ModuleImplementation>
<OutputType>System!System.OleDbData</OutputType>
<TriggerOnly>true</TriggerOnly>
</ProbeActionModuleType>
Here I’ve done the same thing, only without using registry keys to specify the location of my
DB. Normally I’d pass the DB details from my targeted class as I’ll have some property that has been
discovered defining where the database is. Also note that the module has been configured to use the
SQL Monitoring account Run as Profile automatically.
<ModuleImplementation Isolation="Any">
<Composite>
<MemberModules>
<ProbeAction ID="PassThru" TypeID="System!System.PassThroughProbe" />
<ProbeAction ID="OledbProbe" TypeID="System!System.OleDbProbe">
<ConnectionString>Provider=SQLOLEDB;Server=$Config/DatabaseServer$;Database=$Config/DatabaseName
$;Integrated Security=SSPI</ConnectionString>
<Query>$Config/Query$</Query>
<GetValue>true</GetValue>
<IncludeOriginalItem>false</IncludeOriginalItem>
<OneRowPerItem>$Config/OneRowPerItem$</OneRowPerItem>
</ProbeAction>
</MemberModules>
<Composition>
<Node ID="OledbProbe">
<Node ID="PassThru" />
</Node>
</Composition>
</Composite>
</ModuleImplementation>
<OutputType>System!System.OleDbData</OutputType>
<TriggerOnly>true</TriggerOnly>
</ProbeActionModuleType>
The syntax for this is shown below. Obviously replace the text in italics with your values.
170
9.0 BUILDING OUT A SQL SERVER TEMPLATE WITH VMM –
CRAIG TAYLOR
9.2 Introduction
In System Center Virtual Machine Manager (SCVMM or VMM), Microsoft have provided the capability
to deploy SQL Server as part of a Service Template, allowing for the more rapid deployment of multi-
tier applications dependent on the SQL Server database engine.
SQL Server 2008 R2 and later provide a Sysprep functionality as an advanced installation option. This
two-step installation process is similar to the Windows Sysprep tool in use. The two steps are:
Through embedding a Sysprep installation into an OS image, it is possible for an IT practice to both
accelerate and standardise their SQL Server deployments, whilst retaining the flexibility of
customisation to allow the deployed SQL Server instance to support any given applications specific
database requirements, (e.g. collation setting or FILESTREAM).
171
9.3 Preparing VMM
Preparing Virtual Machine Manager to make use of SQL Server Sysprep is also a two-stage process:
This chapter will outline the steps required to make use of SQL Server Profiles in Virtual Machine
Manager.
172
9.4 Creation of a Sysprep SQL Server Virtual Machine Template
Deploy a virtual machine from the base template you wish to fork. In this instance we are deploying our
standard Windows Server 2012 R2 template to the 'Production' VMM cloud
173
So that the VHDX files are meaningfully labelled, once captured to template during the VMM template
generalisation phase, we provide a Virtual Machine name that mentions SQL - in our instance -
WS2012R2_DC_SQL
174
175
It is important to ensure that the virtual machine is deployed as a workgroup server. This will prevent
configuration changes being applied to the Windows OS (say, via Group Policy) upon which we will
base our template disk
Once the virtual machine we plan to base our new template on has been deployed, we may run the
required SQL Server Sysprep image preparation.
NB: It is the author's recommendation that installation of the SQL Server pre-requisite, .NET 3.5, be
performed in advance of running the SQL installer, due to the unreliability inherent to this feature's
activation.
Following the activation of .NET 3.5, open the ‘Advanced’ section of the SQL Server Installation Center
and then click 'Image Preparation of a stand-alone instance of SQL Server'
176
It is possible to perform Sysprep-ed installs of Server Database Engine Services and SQL Server
Reporting Services. In this guide we cover preparation of the DB engine.
Select the features required for our template and then configure the installation directories
177
Provide the SQL installation with an Instance ID and Instance root directory, where the instances
system databases and logs will be deployed.
Please note that this Instance ID is immutable and cannot be altered during the Image Completion
process undertaken via the application of the VMM SQL Server profile.
178
179
Once the preparation has succeeded, copy the contents of the SQL media to a drive on the virtual
machine. In our example we copy the binaries to a subfolder of D:\Sources. Then using the VMM
administrators console, create a generalised VM Template based on the virtual machine
180
181
9.5 Creation of the SQL Server Profile
SQL Server Profiles are created in the Library section of the VMM administration console. Much like OS
& HW profiles, they are ultimately stored in the VMM database. No files on the VMM library server are
created as a consequence of their commission.
To create a SQL Server Profile, click the 'Create' drop down button on the Library section of the VMM
console. Then click 'SQL Server Profile'.
182
Under the SQL Server Configuration parent node, we must provide the profile with:
A name
An Instance name - This does not have to match the instance ID specified during the SQL Server
sysprep image preparation
An Instance ID - Note that this instance ID MUST match the one specified during the Sysprep-ed SQL
Server image preparation
A product key - the Image Completion process will fail without this being entered. It is normally found in
the DefaultSetup.ini file found on the installation media under the subfolder 'x64'
A RunAs account with administrative rights on our VM guest. In this instance, we have specified an
account that will be seeded as a local administrator on our server when it is joined to the domain. This
was achieved via the application of Group Policy
183
Under the SQL Server configuration ‘Configuration’ node, we must:
Provide a path for the SQL Server installation media accessible from the deployed virtual machine.
Note we have configured this to be a local path to minimize external dependencies during Service
Template deployment. Specifically, we have configured the path to which we copied the SQL Server
binaries during the template preparation.
Provide the users and groups you wish to provision SA rights on the SQL Server instance.
Specify the security mode you wish the SQL Server instance to operate under, either Windows
Authentication or Mixed Authentication
Define if TCP/IP & named pipes are going to be allowed for remote connections
184
Finally, configure an SA password by selecting a VMM RunAs profile containing the required credential
185
Under the SQL Server configuration ‘Service Accounts’ node, specify the Service Accounts under
which the SQL Server service & SQL Server Agent service will operate under, again using VMM RunAs
accounts:
If a SQL Server installation INI file is used as part of a profile, the administrator will be asked if they
wish to populate the SQL Server deployment profile with the settings contained in the INI file.
Should the administrator accept this warning, it is recommended to revisit all sections of the profile
wizard to ensure they have retained the expected settings:
188
9.7 Deployment of the Service Template
SQL Server Profiles can only be used as part of a Service Template. In this tutorial, we configure a
single tier service template, containing our SQL Server VM Template.
To apply a our created SQL Server profile to the VM tier right click on the tier and then select
'properties'. Following this, select the ‘SQL Server Configuration’ section of the properties window and
then select the profile we have configured from the SQL Server profile dropdown menu
189
Deploying this service will create a single virtual machine with SQL Server installed to our specification.
190
191
Upon deployment, one should attempt to connect to the deployed SQL instance using SQL
Management Studio, to confirm it meets the defined functional specification.
9.8 Conclusion
Virtual Machine Manager SQL Server profiles provide an easily managed means of rapidly provisioning
192
consistently configured SQL Server Virtual Machines as part of multi-tier Service Templates. The ability
to customise the functional specification, using an administration GUI for common settings and script
files for advanced settings, provide the cloud administrator with a flexible, low-overhead means of
providing standardised SQL infrastructure to their customers, be it for test, development or production
purposes. As many administrators can attest, SQL Server installations are typically a reasonably time-
consuming endeavour. As such, it is the author’s observation that time invested in educating oneself in
automating the process is typically pays dividends in a reasonably short period.
193
10.0 SPECIAL THANKS
This guide is nearly 130 pages of extra content over the last version, this would not have been possible
without Pete’s Azure experience when it comes to System Center in the cloud. Robert showed us why
he still loves DPM. Matthew who is a great MP dev gave a solid explanation of the SQL MP . Craig and
Richard gave a ton of help with the clustering, AlwaysOn and full guide edits.
All of the contributors hope to do a V2 of this guide over the next few months so check back soon, we
will announce the next edition on Twitter @paul_keely
194