Configure Data Center Process Automation Using System Center 2012 R2 and Windows Server 2012 R2

There is a joke that I heard at the Microsoft Management Summit a few years back on the subject of datacenter automation. When asked how many people would work at a new datacenter, the designer replied, “Only two, a security guard and his dog. And the job of the dog is to bite the security guard if he tries to touch anything.” The point that the presenter was trying to make is that the modern datacenter is so highly automated that it requires few actual physical staff to keep things running. Another benefit of automation is that complex repetitive tasks are handled by pre-configured workflows. Automating a complex process provides you with repeatable results. When you perform complex processes manually, there is always the chance that things will go off the rails should you get distracted. In this chapter you’ll learn about data center process automation using System Center 2012 R2 and Windows Server 2012 R2.

Objective 1.1: Implement workflows

Part of an effective private cloud deployment means automating any task that is repeatable using the tools at your disposal. In terms of the 70-246 exam, this means using products in the System Center 2012 R2 suite. In this section, you’ll learn how you can leverage the System Center suite to create complex automation for your organization’s private cloud.

Implementing runbook automation

With runbook automation you can automate complicated workflows. Runbooks represent a set of procedures that a server administrator performs on a regular basis. Originally, runbooks were actual physical books. These books contained documentation that described to the server administrator how to perform specific procedures. Today runbooks are software parts that, when triggered, actually perform the procedures with little or minimal direct input from the server administrator. Runbook automation is important in Microsoft private cloud environments because it allows you to automate complex tasks. The System Center product that you use to create runbook automation is System Center 2012 R2 Orchestrator.

Orchestrator

Unlike Windows PowerShell, which requires you to write scripts using an editor like Windows PowerShell ISE, Orchestrator allows you to build automation using a drag and drop interface called the Runbook Designer. Orchestrator can still call Windows PowerShell scripts, but it also integrates with many other products, including products within the System Center suite through integration packs. An integration pack is a collection of product-specific tasks that you can trigger through Orchestrator. You can download integration packs from the Internet, import them using the System Center 2012 R2 Orchestrator Deployment Manager as shown in Figure 1-1, and then deploy them to your runbook servers.

An Orchestrator deployment consists of the following parts:

• Management server This server manages the runbook servers. You use the management server to distribute integration packs to runbook servers and runbook designers. The management server also manages communication between the runbook designers, runbook servers, and the orchestration database. There is only one management server in an Orchestrator deployment
• Runbook server This server runs Orchestrator runbooks. Each runbook server can run up to 50 runbooks concurrently. You can alter this number using the Runbook Server Runbook Throttling tools, but should monitor the runbook server’s resource requirements. You can have multiple runbook servers in an Orchestrator deployment, with no maximum limit to the number of runbook servers specified in the Orchestrator documentation.

• Runbook Designer This designer allows you to build and test runbooks. The interface allows you to build runbooks by dragging and connecting activities that are available in integration packs. The Runbook Designer is shown in Figure 1-2.

  

  FIGURE 1-2 Runbook Designer

• Orchestration database Hosted on a Microsoft SQL Server instance, the orchestration database stores configuration data, policies, and log information.
• Orchestration console A web interface that users can use to list, control, and view runbooks.
• Orchestrator Web Service This web service allows custom applications, third-party tools, and other System Center items such as Service Manager, to connect to Orchestrator and to interact with runbooks.
• Deployment Manager The Deployment Manager allows you to deploy integration packs, Runbook Designers, and runbook servers. You use the Deployment Manager to import and deploy integration packs that you’ve downloaded from the Internet.

Runbooks

Runbooks are collections of linked activities that perform a procedure. You build runbooks in Orchestrator by dragging activities from integration packs to the designer workspace. For example, the runbook shown in Figure 1-3 uses two activities. The first activity, named Monitor Service, checks the state of a specific service on a specific computer and triggers if the service is in a specific state (started, stopped, or paused). The second activity, named Start/Stop Service, allows you to start, stop, pause, or restart a service. When the runbook is deployed, it will be triggered when the monitored service is in the state specified in the Monitor Service activity. After being triggered, the runbook will perform the task defined in the Start/Stop Service activity.

This example is very basic. When creating Orchestrator runbooks to perform sophisticated automation tasks, you are likely to use multiple activities and include conditional branches, loops, and error handling tasks. Each integration pack that you import into Orchestrator increases the number of activities that you can include in your runbooks.

Keep the following in mind when creating Orchestrator runbooks:

• Provide meaningful names for activities. You can rename activities after you drag them to the designer workspace. By renaming activities with descriptive names, then you can quickly understand what tasks a runbook is designed to accomplish. For example, with the runbook in the example above, you might rename the Monitor Service activity “Is the VMM Service Stopped” and the Start/Stop Service activity “Start the VMM Service.”
• Minimize the number of activities that are performed in a runbook. You can call runbooks from within runbooks. This modular approach to creating runbooks will simplify the process of troubleshooting them.
• Configure runbooks to write logs to external files rather than to the orchestration database.

Orchestrator runbooks run according to configured schedules. You create each run separately, and then assign the schedule to the runbook. You create runbook schedules in the Schedules node, under Global Settings, in the Runbook Designer as shown in Figure 1-4. Creating a runbook schedule involves assigning a name to the schedule, specifying what days of the week or days of the month the schedule applies to, and specifying which hours the schedule applies to.

Once you’ve created the schedule, you can apply it to a runbook. You do this by selecting the schedule on the General tab of the runbook’s properties, as shown in Figure 1-5.

You check out a runbook to make changes to the runbook. When you check in a runbook, the runbook will be deployed to runbook servers. Checked-in runbooks will also synchronize to Service Manager if you have configured a connector between Service Manager and Orchestrator.

Automating remediation of incidents

As anyone who has worked on a service desk can tell you, there are certain types of problems that users report to the service desk, or which occur in the infrastructure which are easily remediated by performing a specific set of actions. For example, a service might fail, just needing a manual restart. Using the capabilities of the System Center suite, it’s possible to detect these commonly occurring problems and automatically perform the steps required to remediate them without requiring direct manual intervention by members of the IT team.

Incidents

Service Manager incidents, which you might call trouble tickets or service desk jobs in non-Service Manager environments, describe an issue with some aspect of the server, client, network, or software infrastructure that requires resolution. In the context of the 70-246 exam, a Service Manager incident would describe an issue with some aspect of the private cloud deployment that requires resolution by the IT team.

You can create an incident manually using the Service Manager console by performing the following steps:

1. In the Configuration Items workspace of the Service Manager console, select the Computer or User for which you want to manually create the incident.
2. In the Tasks pane, click Create Related Incident.

3. In the Tasks pane of the Incident, click Apply Template. Depending on the issue, you can select one of the default templates shown in Figure 1-6. The default templates are as follows:

   • Default Incident Template
   • Generic Incident Request
   • Hardware Issue Incident Template
   • High Priority Incident Template
   • Networking Issue Incident Template
   • Printing Issue Incident Template
   • Software Issue Incident Template
   

   FIGURE 1-6 Incident templates

4. Click OK and the New Incident dialog box opens. The selection of the template causes certain fields of the incident to be automatically populated. For example, choosing the Networking Issue Incident Template causes the Classification category of the incident to be set to Networking Problems as shown in Figure 1-7.

   

   FIGURE 1-7 Networking incident

5. After selecting an incident template, you should provide the following additional information and then click OK:

   • Affected User This is the user who reported the incident.
   • Title Allows you to provide a name for the incident.
   • Description A description of the incident.
   • Other information as necessary based on the incident itself. Some information will automatically be included with the template.
6. On the Activities tab of the New Incident dialog box, you can add activities such as Manual Activities or Runbook Automation Activities that are related to the incident.
7. On the Related Items tab, you can add Work Items, Configuration Items, Knowledge Articles, and Attached Files.
8. On the Resolution tab, you provide information about how the incident was resolved, how much time it took, and specify a resolution category.
9. The Service Level tab allows you to view service level information.
10. The History tab allows you to view the history of the incident.

You can also automate the Service Manager email messages sent by users indirectly by having the users submit a form through the Service Manager Self-Service Portal, or by configuring the Operations Manager Alert connector to automatically generate incidents based on Operations Manager alerts.

Automatic incident creation

The Operations Manager alert connector for Service Manager allows you to automatically create Service Manager incidents based on Operations Manager alerts. An Operations Manager alert is created in Operations Manager when an object that Operations Manager monitors experiences a change that is deemed worthy of attention, such as a hardware or software failure occurring on a monitored server. There are two types of Operations Manager connectors for Service Manager: the alert connector, and the configuration item (CI) connector. The CI connector imports objects that Operations Manager has discovered into the Service Manager database. Alert connectors bring alert information into Service Manager.

To create the alert connector, perform the following steps:

1. In the Administration workspace of the Server Manager console, click Connectors.
2. On the Tasks pane, click Create Connector, and then click Operations Manager Alert Connector.
3. On the General page of the Operations Manager Alert Connector Wizard, provide a name for the alert connector.

4. On the Server Details page, shown in Figure 1-8, specify the name of the Operations Manager server and a Run As account that has permission to connect to Operations Manager. Ensure that you use the Test Connection button to verify that the account works and has appropriate permissions.

   

   FIGURE 1-8 Alert connector configuration

5. On the Alert Routing Rules page, click Add to add an alert routing rule. An alert routing rule allows you to specify which Service Manager incident template will be used to create an incident based on an Operations Manager alert.

6. In the Add Alert Routing Rule dialog box, shown in Figure 1-9, provide the following information:

   • Rule Name The name of the alert routing rule.
   • Template The Service Manager incident template that will be used when creating the Service Manager incident.
   • Criteria Type Here you can select the conditions that trigger the alert routing rule. You can choose between the alert being generated by a specific Operations Manager management pack, being generated by a specific computer or security group, a custom field, or an Operations Manager monitoring class.
   • Select Alert Severity And Priority Allows you to specify the alert priorities and severities that will trigger the alert routing rule.
   

   FIGURE 1-9 Alert routing rule

7. As Figure 1-10 shows, alerts that don’t match any of your configured rules will automatically be created as incidents using the Operations Manager Incident Template.

   

   FIGURE 1-10 Routing rules

8. On the Schedule page, select the frequency at which Service Manager will query the Operations Manager server for alerts. You can also configure the connector so that alerts within Operations Manager will be closed when the incident that relates to the alert is resolved or closed in Service Manager. You can also configure Service Manager to automatically mark incidents as Resolved if the incident that triggered the alert in Operations Manager is closed. Figure 1-11 shows these settings.

   

   FIGURE 1-11 Schedule settings

9. On the Summary page, review the connector setup, and then create the connector.

Once the connector is created, you can modify the alert routing rules by editing the properties of the connector as shown in Figure 1-12.

Integrating Orchestrator with Operations Manager and Service Manager

You can configure Orchestrator to integrate with Operations Manager by configuring a connection to the Operations Manager server from the Orchestrator Management server. When you do this, you can monitor and collect information from Operations Manager alerts, which you can use when building Orchestrator runbooks. To integrate Orchestrator with Operations Manager, first install the Operations Manager integration pack. You can download this integration pack from Microsoft’s website. You’ll also need to install the Operations Manager console on the server that hosts the Runbook Designer and verify that you can use it to make a connection to the Operations Manager server.

Once you’ve performed that step, you configure a connection from the Orchestrator Management server to the Operations Manager Management Group by performing the following steps:

1. In the Runbook Designer’s Options menu, click SC 2012 Operations Manager.
2. On the Connections tab of the SC 2012 Operations Manager dialog box, click Add.

3. In the Connection dialog box, shown in Figure 1-13, type the name of the connection, the IP address or FQDN of the Operations Manager server, and then provide the credentials of an account that has access to the Operations Manager server.

   

   FIGURE 1-13 Connection configuration

4. On the SC 2012 Operations Manager dialog box, shown in Figure 1-14, click Finish.

   

   FIGURE 1-14 Operations Manager connections

Once you have configured the connection, you’ll be able to use the activities that are included in the Operations Manager integration pack when building Orchestrator runbooks. These activities are shown in Figure 1-15, and have the following functionality:

• Create Alert This activity allows you to create an alert in Operations Manager.
• Get Alert This activity allows you to extract data from an Operations Manager alert. Use this activity as the basis of creating runbooks that create incidents in Service Manager by extracting relevant information from alerts and using that information when creating incidents.
• Get Monitor Use this activity to collect monitoring data. You can take the data extracted from this activity and use it to populate incidents in Service Manager.
• Monitor Alert Use this activity to watch for specific new or updated Operations Manager alerts. You might use this when configuring a runbook to have additional steps taken when specific alerts are raised in Operations Manager during runbook intiation.
• Monitor State Use this activity to monitor and run when an object managed by Operations Manager has its state changed to Warning or Critical. You might use this when configuring a runbook to have additional steps taken when the state of specific Operations Manager monitored objects changes during runbook initiation.
• Start Maintenance Mode This activity allows you to put an Operations Manager managed object into maintenance mode. Maintenance mode is a special state that suppresses alerting. For example, you would put a server into maintenance mode when applying software updates so that Operations Manager alerts aren’t generated by the software update process.
• Stop Maintenance Mode This activity allows you to take an Operations Manager managed object out of maintenance mode, so that Operations Manager alerts are no longer suppressed.

• Update Alert Use this activity to update an Operations Manager alert with data. For example, you could update an Operations Manager alert with information provided in a Service Manager incident.

  

  FIGURE 1-15 Operations Manager activities

You configure integration between Orchestrator and Service Manager by performing the following steps:

1. Ensure that the Service Manager integration pack is installed on the management server.
2. Click SC 2012 Service Manager in the Options menu of the Orchestrator Runbook Designer console.
3. On the Connections tab of the SC 2012 Service Manager dialog box, click Add.

4. In the Connection dialog box, shown in Figure 1-16, provide the following information. Ensure that you click Test Connection to verify that the connection to the Service Manager server functions correctly.

   • Name Name of the connection to the Service Manager server
   • Server FQDN of the Service Manager server
   • Credentials Credentials of an account that has permission to access the Service Manager server
   

   FIGURE 1-16 Connection properties

5. On the SC 2012 Service Manager dialog box, shown in Figure 1-17, click Finish.

   

   FIGURE 1-17 Service Manager connection

Once the connection between the Orchestrator and Service Manager server is established, you can use the integration pack activities, shown in Figure 1-18, to build workflows.

These activities allow you to do the following:

• Create Change With Template Use this activity to create a change record using an existing change template. When you use this activity, mandatory fields in the service manager change record need to be configured using Orchestrator when you use this activity.
• Create Object Use this activity to create a Service Manager object based on a defined class. For example, you could use this activity to create a Service Manager incident, change, or problem record.
• Create Incident With Template Use this activity to create a Service Manager incident based on an existing template. When you use this activity, mandatory fields in the Service Manager incident record need to be configured using Orchestrator.
• Create Related Object Use this activity to create new Service Manager objects that have relationships to existing Service Manager objects.
• Create Relationship Use this activity to create relationships between Service Manager elements. For example, you could use it to create a relationship between an incident and a computer or user. You can also use it to relate multiple incidents with a Service Manager problem record.
• Delete Relationship Use this activity to remove a relationship between Service Manager elements.
• Get Activity Use this activity to instruct Orchestrator runbook to collect activity records based on specific criteria.
• Get Object Use this activity to search for a Service Manager activity, incident, or change records based on specific criteria.
• Get Relationship Use this activity to have Orchestrator generate a list of objects from separate classes that are related by specific criteria.
• Monitor Object User this activity to configure Orchestrator to find new and updated records based on specific criteria.
• Update Activity Use this activity to update Service Manager activity records.
• Upload Attachment Use this activity to upload a file to an existing Service Manager object. For example, you might use this activity to upload a log file so that it can be stored with the incident generated automatically by an Operations Manager alert.
• Update Object Use this activity to modify the values of a Service Manager object’s properties.

Automatic incident remediation

Automatic incident remediation involves applying a specific solution to a known problem. You can configure Orchestrator runbooks triggered by specific Operations Manager alerts. Using some of the Orchestrator activities detailed earlier in this chapter, you can take the data contained in the alert and use it to populate a new Service Manager incident. The Orchestrator runbook can then perform the tasks necessary to automatically remediate the incident. For example, the Orchestrator runbook could run an activity that restarts the service that caused the original Operations Manager alert. Once the Operations Manager alert has been dealt with, the Orchestrator runbook could then update the Service Manager incident, closing both the incident and the Operations Manager alert once the issue that caused the alert has been resolved.

Change and activity management workflows

Workflows allow you to automate processes within Service Manager, making interactions with Service Manager more efficient. For example, you can configure workflows that will automatically close completed change requests, or configure workflows that will automatically notify Service Manager users when approvals are required. Using the Server Manager console, you can configure change management workflows that configure change request conditions and apply change request templates. You can also configure activity management workflows to configure activity management conditions and apply activity templates.

Change request templates

Change request templates store a common set of settings, applying these settings to new change requests. For example, you can create a change request template related to adding a new database to a SQL Server instance that includes commonly used properties, minimizing the amount of information that a user is required to enter when requesting such a change.

To create a change request template, perform the following steps:

1. In the Library workspace of the Server Manager console, click Templates, and then in the Tasks pane, click Create Template.

2. On the Create Template dialog box, specify a name for the template. Select the Change Request Class as shown in Figure 1-19, and select a Management Pack in which to store the new template.

   

   FIGURE 1-19 Create change request template

3. When you click OK, the Change Request Template form will be displayed. In this form, provide information that will be pre-populated on a change request template. As shown in Figure 1-20, this can include the area of the organization that the template applies to, the priority the change request should be assigned by default, as well as default impact and risk values.

   

   FIGURE 1-20 Configure change request template

4. On the Activities tab, you can add activities to the template. These additions can include any configured activity including runbook automation activities. Usually with Change Requests, you’d add a Default Review Activity as shown in Figure 1-21, which would allow another user to review and authorize the change request.

   

   FIGURE 1-21 Change request template activities

Change management workflows

You can use change management workflows to automate the process of dealing with change management requests. To create a change management workflow, perform the following steps:

1. In the Administration workspace or the Service Manager console, expand the Workflows node, and click Configuration.
2. In the Configuration pane, click Change Request Event Workflow Configuration, and in the Tasks pane, click Configure Workflow Rules.
3. In the Configure Workflows dialog box, click Add.

4. On the Workflow Information page of the Configure Workflows For Objects Of Class Change Request dialog box, shown in Figure 1-22, specify a name, whether the event that triggers the workflow is when an object is created, or updated, and a management pack in which to store the workflow.

   

   FIGURE 1-22 Workflow information

5. On the Specify Criteria page, ensure that Change Request is selected. In the list of available properties, select a criteria that will determine whether the change management workflow is applied. For example, in Figure 1-23, the change management workflow will be applied if the change request area is set to Security.

   

   FIGURE 1-23 Workflow criteria

6. On the Apply Template page, click Apply The Selected Template. You can then choose one of the existing change management templates to apply. Figure 1-24 shows the Security Release Change Request template selected.

   

   FIGURE 1-24 Apply template

7. On the Select People To Notify page, specify whether users should be notified when this change management workflow is triggered.
8. On the Summary page, review the settings, and click Create to create the change management workflow.

Activity management workflows

Activity management workflows allow you to automate the management of activities based on the properties of the activity. For example, you might create a workflow to assign all unassigned manual activities to a particular member of the IT staff. To create an activity management workflow, perform the following steps:

1. In the Administration workspace of the Server Manager console, click Configuration under the Workflows node.
2. In the Configuration pane, select the Activity Event Workflow node, and then click Configure Workflow Rules in the tasks pane.

3. On the Select A Class dialog box, shown in Figure 1-25, click the activity class to which you want the workflow to apply.

   

   FIGURE 1-25 Activity class

4. On the Configure Workflows dialog box, click Add.
5. On the Workflow Information page of the Configure Workflows For Objects Of Class, specify a name for the activity management workflow, a management pack to store the workflow, and whether the workflow will be triggered upon object creation or object modification.

6. On the Specify Criteria page, select a property and criteria that will trigger the workflow. For example, in Figure 1-26, the criteria is that the Activity Status equals Failed.

   

   FIGURE 1-26 Activity criteria

7. On the Apply Template page, you can choose to apply a template.
8. On the Select People To Notify, you can choose to notify specific people. When you choose to notify a person, you select who is to be notified and the message template.
9. On the Summary page, click Create.

Objective summary

• Orchestrator allows you to create runbook automation. You do this by linking activities from integration packs.
• You can configure Operations Manager to automatically create Service Manager incidents from alerts generated in Operations Manager.
• You can configure an Orchestrator runbook to create Service Manager incidents using the Service Manager integration pack.
• You can configure a Service Manager incident to trigger an Orchestrator runbook, which you can use to automatically resolve some types of issues.
• Change request templates store a common set of settings, applying these settings to new change requests.
• You can use change management workflows to automate the process of dealing with change management requests.

Objective review

Answer the following questions to test your knowledge of the information in this objective. You can find the answers to these questions and explanations of why each answer choice is correct or incorrect in the “Answers” section at the end of this chapter.

1. You want to create a runbook in System Center 2012 R2 Orchestrator that creates Service Manager incidents in response to Operations Manager alerts. Your organization has one Operations Manager server, one Orchestrator server, and one Service Manager server. Which of the following steps should you take?

   1. Configure a connection from the Operations Manager server to the Orchestrator server. Install the Orchestrator management pack on the Operations Manager server.
   2. Configure a connection from the Orchestrator server to the Operations Manager server. Install the Operations Manager integration pack on to the Orchestrator server.
   3. Configure a connection from the Orchestrator server to the Service Manager server. Install the Service Manager integration pack on to the Orchestrator server.
   4. Configure the Operations Manager connector on the Service Manager server. Configure alert routing rules for the connector on the Service Manager server.

2. You want to have alerts from any of the SQL Server 2012 instances monitored by your organization’s Operations Manager deployment automatically assigned as Service Manager incidents to Barry the SQL Server administrator. All SQL Server alerts on the Operations Manager server are triggered by rules stored within a SQL Server 2012 management pack. Your organization has one Operations Manager server and one Service Manager server. You have not deployed any other System Center products. Which of the following steps would you take to accomplish this goal?

   1. Configure the Operations Manager connector on the Service Manager server.
   2. Deploy the Operations Manager agent on the Service Manager server.
   3. Create an incident template for SQL Server events that assigns the incident to Barry. Create an Alert Routing rule for alerts generated by the SQL Server 2012 Management pack that applies this incident template.
   4. Create an Orchestrator runbook that creates an incident on the Service Manager server when an alert is raised on the Operations Manager server related to the SQL Server 2012 management pack.

3. You want to configure Service Manager so that Barry the SQL Server Administrator is notified when a SQL Server related change request is entered into the Service Manager database. Which of the following would you configure in Service Manager to accomplish this goal?

   1. Configure a change request workflow.
   2. Configure an incident event workflow.
   3. Configure an activity event workflow.
   4. Configure a desired configuration management event workflow.

4. You are creating a new change request template in Service Manager. Which class should you select when creating the template?

   1. Change Request
   2. Incident
   3. Problem
   4. Knowledge Article

5. Which activity in the Operations Manager integration pack for Orchestrator do you use to extract data from an Operations Manager alert?

   1. Create Alert
   2. Get Alert
   3. Monitor Alert
   4. Update Alert