Building an automated SharePoint 2010 deployment system

When building a deployment system for SharePoint 2010 system, your only practical option is to use PowerShell. STSADM.EXE could in theory be used if you are only interested in SharePoint Foundation, but the moment you need SharePoint Server, things are only available in PowerShell.

SharePoint is a mature product and has its portion of quirks. Some of these come to surface when working through PowerShell. SharePoint can at times be very fond of caching information. If you keep your PowerShell session running, you may not get the result you were expecting. For instance, if you deploy a custom site definition and try to use this to create a new site using the same PowerShell session, you will be informed that SharePoint cannot find the definition. You will also experience issues when deactivating and activating features; if you have deployed updated assemblies to the GAC, the feature may actually be using an old version.

When using PowerShell and SharePoint, be prepared restart PowerShell on a regular basis. This fact is part of the driving force behind the Mystery Foundation deployment system. The deployment system is driven from a normal Command Shell; it makes it easy for you to construct new cmd files to handle your scenarios.

A copy of the deployment system and all required files can be found in the SharePoint Mystery Package.

All our deployment commands follows this pattern.

ps.cmd <Environment File> <Application> <Command> 

Environment File: the name of the ps1 file that contains environment information
Application: a string that identifies your application, you have control over this value
Command: the command you want to issue on the application, you have control over this value.

In the package you will also find BobTheBuilder.cmd that creates my demo system and ConanTheDestroyer.cmd that tears down the demo system.

File structure

Files starting with MGL are sample files and are meant to be customized by the project. Name them however you want. Files starting with SPM are part of the core deployment system and are not meant to be customized by the project.

TheServer.ps1

This file contains information for a specific environment. It contains information relevant for the farm and the various applications and services we will be using.

In my case it’s named after the server where I do the deployment from. Create one file per environment (development, testing, staging, production)

MGL.Settings.ps1

This file contains information that is relevant across all your environments.

The prefix value that is set at the farm level will be used to create the actual database name. That is why we can specify the name in a common location.

MGL.Commands.ps1

This file is where you map the various commands against the applications.  It translates the Application and Command parameters from ps.cmd into actual action. What commands you want and what you do is up to you. Copy and customize this file based on your own requirements.

The following information that is found in TheServer.cmd is important for forwarding information to this file:

SPM.Types.ps1

This is mainly a container for all the various SPM.Type.*.ps1 files. This is also where the SharePoint PowerShell module gets loaded.

SPM.Functions.ps1

This file contains a set of useful functions that are not bound to a specific type.

SPM.Type.*.ps1

There is one file per custom type. This file contains a method that returns a custom PowerShell object; it’s this object that provides all the properties for setting values and any methods that can be called.

SPM.DefaultTopology.ps1

This file creates all the objects for a default topology; all the service applications and a set of web applications. You can use this or create your own. Even if you only need a subset of this topology, you can use it. Which services and applications actually get created depends on what you do in your command file. If you create your own, stick with the same names; there are dependencies between services and applications. i.e. My Site and User Profile Service

Points of interest

Service Principle Names

The deployment system can automatically configure the SPN’s that are needed for Kerberos Authentication. This is controlled by a flag set on the farm object so that it can be turned on for a development deployment, but not in a staging or production deployment.

Databases

The system can automatically create databases at a specific location if desired.  This allows you to create the databases at a different location than the standard SQL Server location. The database folder creation is only relevant if everything is running on the same machine. i.e. developer.

In general, SharePoint will accept an empty database that it can configure, but there are some services that just refuse to configure an empty database. These will be created in the default SQL Server location.

When creating the database, we impersonate the SharePoint farm account. Some services don’t like it if dbo != SharePoint farm account. I.e. the User Profile Service

Transcript

In the command file we create a transcript file for each command.  When errors occur, the files can be sent to whoever can aid in troubleshooting.

Passwords

No passwords are stored in the deployment files. During the farm configuration process, all the required accounts are configured as Managed Accounts. When needed we read information from the Managed Account system to impersonate users.

Pending

Installing binaries and creating the farm

Currently we don’t support installing the SharePoint binaries or creating the actual farm. This deployment system assumes that the environment has been configured as far as Central Administration, but no further. The initial focus is on developers. Personally I maintain a blank SharePoint 2010 image that I use as a basis for my development; this has Central Administration already configured. There are already other scripts out there that will install the binaries and create the initial farm.

With time I plan to include this functionality as well. How soon will depend on demand.

Other stuff

There is probably a whole bunch of stuff that would be useful to include. If you have suggestions, please let me know.

Automating SharePoint deployments, an introduction

No matter if you are working on a 10 000 user or 100 user project, you will need to have some form of deployment system in place. It doesn’t take long before the number of settings you need configure spiral out of control. Some people like Word documents that detail all the settings with tons of wonderful screen shots. I personally hate these because they take a long time to set up and actually perform. You can also be fairly certain that at some point somebody will forget something and you spend a lot of time debugging the problem. I personally favor an automated approach; at least as much as possible. The downside is that setting up an automated deployment system takes time and money. If you are supporting 10 000 users, you may have the budget for it, but probably not for a ‘cheap’ system handling only 100 users.

The Mystery Foundation provides a lot of building blocks and a sample to help you along the way; it should drastically cut down the time to set up an automated deployment system.

Some people don’t understand the value of an automated deployment system; the guilty shall not be named. The foremost value is that it provides an indisputable documented copy of the collective knowledge that the project has when it comes to deployment. Lengthy prose can open for interpretation and unclear descriptions. A deployment system developed in PowerShell is very clear in what it does. Deployment is also something that often gets delegated to one or two people; if something should happen to them, any ‘magical’ undocumented things they may be doing also gets lost.

Automated deployment systems are also very valuable from a development perspective. If a developer knows that they can rebuild their system in a matter of hours, they are not afraid of scrapping their system and starting afresh. This uncovers any undocumented configuration changes. If a developer keeps their environment for 2 months prior to moving into test, you can be assured that some settings haven’t been document and thus not configured correctly.

Another aspect of automated deployments is that they should work across multiple environments; the information relevant to a specific environment should be separated from the rest. This ensures that you also debug your deployment system when moving between environments. When reaching staging and production, most of the bugs should have been removed and deployment into these environments shouldn’t cause many surprises.

So what are some of the attributes of a good deployment system?

Run again, and again, and again, and again...

It should be possible to run the deployment system multiple times without incurring any errors. If the database or web application already exists, it should use them and just move on.

Scoped configuration values

In any configuration you will have configuration values that are specific to the environment and values that are relevant for all environments. Examples of environment specific values include URL’s, service accounts, and database sizes. Information that can remain the same across multiple environments include timeout values, the service application names probably remain the same, as would the logical architecture.

Having a deployment system that allows you to set values for all environments or for a single environment clearly documents what is relevant for the various scopes. If you need to add a new environment, you know what values you have to come up with.

The Mystery Foundation deployment system

This deployment system is built entirely in PowerShell using the functionality provider by SharePoint 2010. Some custom Cmdlets have been developed to provide additional functionality:

  • Add entries to the host file on all servers in the farm
  • Add SPN’s so that Kerberos works
  • Create directory on all server in the farm
  • Configure Blob cache settings
  • Configure Super user accounts for the Publishing infrastructure.