Visual Studio 2010 and VisualStudio.com TFS Hosting

Yesterday, Stan, the founder of this blog, gave me a link to a project host on the Team Foundation Service (visual studio.com). I tried to connect to it with Visual Studio 2010. it simply refused to work.

Visual Studio Add TFS Server Error

After much annoyance, he asked me to try adding the TFS server to Visual Studio 2012 and it worked (Why didn’t I think of that?). Eventually I figured out that I needed to install the Visual Studio 2010 SP1 Team Foundation Server 2012 Compatibility GDR (KB2662296). Then I was able to add the solution to Visual Studio 2010. It seems there are several updates for Visual Studio 2010 SP1, some specifically dealing with Windows 8 compatibility. Unfortunately Windows update does not prompt me to install them. I will search for and install these tonight to prevent future issues.

Continuous Integration with Windows Azure SDK 1.7, Powershell and TFS Build too!

Environments change and the solutions to support them have to keep up. I was very entertained with my old deployment solution for a good while. However, we eventually moved to Azure, and I needed to scramble to find something new. Tom Hollander’s Automated Build and Deployment with Windows Azure SDK 1.6 filled that void until I upgraded my project type to the 1.7 SDK. At that point, I realized I had to roll up my sleeves and cobble something new together.

From an automated deployment standpoint, the crippling change between the 1.6 to the 1.7 SDK, is the lack of an “ImportAfter” folder, which allowed us to include legacy msbuild files to attach to the build process. This is what Tom used to attach a Powershell deployment script to the SDK’s Publish build target. However, with the Azure 1.7 SDK, I had to figure out how to execute that PowerShell script myself.

Microsoft: Downloads for Managing Services in Windows Azure

GitHub: StanleyGoldman/AzurePublishHelpers

Creating a Management Certificate & Publish Settings file

Visual Studio has a link which allows you to download a publish settings file, without completely explaining what the side effects are. I myself didn’t understand the problem when I encountered the first symptom, you have reached the maximum number of management certificates. I was forced to understand the situation when I tried to get publish settings files for the 2nd and 3rd Azure Subscriptions my account was associated with. The link creates a management certificate, uploads it to your azure account and provides you with a .publishSettings file to install onto your machine. Life is actually easier when we start taking control of our management certificates.

It is easy enough to create a management certificate. Gaurav Mantri’s blog post really helped on this.

makecert -sky exchange -r -n "CN=JustAProgrammer" -pe -a sha1 -len 2048 -ss My "JustAProgrammer.cer"

We can take this certificate and upload it to the Management Certificates console in Azure. Take note of your subscription id and the thumbprint of the certificate. As you will need it to create your publish Settings file.

Using the PublishSettingsCreator utility, we can create a publish settings file to carry our management information.

PublishSettingsCreator.exe "AzureExample" "ecd7cc1d-12ec-8cf6-a60b-0cf14db32020" "99DFFB9D05D0B5B92893FBBA35988DE281E01E9E"

In order to use the Azure Powershell Cmdlets we have to import the Azure Modules into our Powershell session.

PS C:UsersAdministratorDesktop> Import-Module 'C:Program Files (x86)Microsoft SDKsWindows AzurePowerShellAzureAzure.psd1'

And now we can import the publish settings file.

PS C:UsersAdministratorDesktop> Import-AzurePublishSettingsFile .ecd7cc1d-12ec-8cf6-a60b-0cf14db32020.publishsettings
Setting: AzureExample as the default and current subscription. To view other subscriptions use Get-AzureSubscription

Importing the settings file sets the subscription as default. We can get the default subscription as follows.

PS C:UsersAdministratorDesktop> Get-AzureSubscription

SubscriptionName        : AzureExample
SubscriptionId          : ecd7cc1d-12ec-8cf6-a60b-0cf14db32020
Certificate             : [Subject]
                            CN=JustAProgrammer

                          [Issuer]
                            CN=JustAProgrammer

                          [Serial Number]
                            9157E76A714509B54F8B853A1B80555B

                          [Not Before]
                            10/18/2012 8:21:06 AM

                          [Not After]
                            12/31/2039 6:59:59 PM

                          [Thumbprint]
                            99DFFB9D05D0B5B92893FBBA35988DE281E01E9E

ServiceEndpoint         : https://management.core.windows.net/
SqlAzureServiceEndpoint :
CurrentStorageAccount   :
IsDefault               : True

Before we can use this subscription to deploy, we have to set the storage account for the subscription. You can CurrentStorageAccount is not set in the subscription definition above.

PS C:UsersAdministratorDesktop> Set-AzureSubscription -SubscriptionName "AzureExample" -CurrentStorageAccount "azureexample"

With the publish settings file and Import-AzurePublishSettingsFile, Set-AzureSubscription commands we can allow any machine to deploy using said Management Certificate. The certificate and publish settings file should be guarded well, either of these files allow access to your azure subscription.

Customizing the Build

Create a new build definition and configure it to build the solution. Be sure to add the “Publish” target to your build. The argument will cause the Azure 1.7 SDK to create the deployment package during build.

If you are using TFSBuild, you can do this while configuring your build, look for the field ‘MSBuild Arguments’, add set the value ‘/t:Publish’.

Finish configuring the build definition and queue the build. We are going to use the output from the build to test the powershell script.

Testing the Powershell script

Using the output from the build, we should be able to execute the powershell script. Copy the script over to your build server. Execute it with the path to the publish file, location where the output Package and Configuration file can be found and the name of the package file in that location. A tag can optionally be specified to help identify the build. I usually use the version number of the binary here, but the build label works just as fine.

PS C:UsersAdministratorDesktop> C:BuildsAzureDeploy.ps1 
    'C:Builds1AzureExampleAzureExample DeploymentSourcesAzureExample.AzureProfilesAzureExampleProduction.azurePubxml'
    'C:Builds1AzureExampleAzureExample DeploymentSourcesAzureExample.AzurebinDebugapp.publish' 
    'AzureExample.Azure.cspkg' 
    'Test LabelTag'

Note: I’ve had some problems with the Azure Powershell Cmdlets and relative paths.

If your machine is configured correctly, this should deploy without any problems. If you are using TFS the next section is useful for you to wrap this all together. If you are using a build system other than TFS, you already have what you need to continue. Enjoy!

TFS Build Process Template

The Build Process Template included in the package takes a few arguments and handles the execution of the Powrshell script rather nicely. After choosing it as the template for your build, you just have to specify a few arguments.

Deployment Configuration: The build configuration of the Azure project, “Debug” or “Release”
Deployment Profile: The name of the profile file to be used
Deployment Project Name: The name of the project Azure project
Deployment Script: (Optional) In case you dont keep your deployment script in the same spot as mine

The build process template uses the Build Label as a tag for the build. It might be a bit more useful to use something like github: martinbuberl/VersionTasks to tag the builds.

Enjoy Continuous Integration

It never seems worth the effort until after you are done with it.

Continuous Integration with TFS2010, MSDeploy & VSDBCMD

After I created an account at Cytanium, I quickly discovered the glory of One-Click Publish. After setting up my source control in TFS. All I could think of, was getting TFS to do the same thing for me, but even I knew that wouldn’t be enough. I wanted TFS to deploy my database changes as well. Well, there is that Schema Compare utility in Visual Studio, I know it can figure out and deploy changes to a database. I figured if Visual Studio can do all of these things, I should be able to as well… shouldn’t I?

Custom Build Process Templates

Very soon after you start to ask these kinds of questions, you find yourself looking at TFS Build Process Templates. The templates are designed in Windows Workflow and thanks to a few good blogs out there this is a rather approachable topic. Most of my starting information came from this blog sequence: Ewald Hofman’s – Customize Team Build 2010.

Before you start, save yourself the pain, take Ewald’s advice and add the Team Foundation Build Activities to your Toolbox. This will shave 15 seconds off the time to open every template.

Exposing Arguments and Providing Metadata

More lessons from Ewald on adding arguments and adding more complex arguments. We will use them and see how the values are supplied a bit later on.

  • Continuous Integration Database Deployment
    • ciVSDBCMDConnectionString: “Database Connection String”
    • ciVSDBCMDManifestFile: “Database Manifest File”
    • ciVSDBCMDTargetDatabase: “Target Database”
  • Continuous Integration Website Deployment
    • ciMSDeployWebProject: “Web Project”
    • ciMSDeployUsername: “MSDeploy Credentials: Username”
    • ciMSDeployPassword: “MSDeploy Credentials: Password”
    • ciMSDeployWebService: “MSDeploy Web Service Url”
    • ciMSDeployWebApplication: “Web Application Name”

Customizing the Template

Under your Team Project, go to the folder BuildProcessTemplates, and branch DefaultTemplate.xaml to ContinuousIntegrationTemplate.xaml.

Open it up and and take a look around. It will help if you take a top down approach, Collapse All, using the controls in the top right. Locate the AgentScope Activity named “Run On Agent” and expand it. Now look for the open space after the “Try Compile, Test, and Associate Work Item” Sequence Activity; this is where we will start adding our custom functionality.

In this space add a If Activity and name it: “If Build and Test Succeeded”. In its condition we are going to add the following VB statement:

BuildDetail.CompilationStatus = BuildPhaseStatus.Succeeded And
    (BuildDetail.TestStatus = BuildPhaseStatus.Succeeded Or
        BuildDetail.TestStatus = BuildPhaseStatus.Unknown)

In the Then clause we drop a sequence named: “Deploy Database and Website”. Inside that sequence, go ahead and create a “Deploy Database” Sequence and a “Deploy Website” Sequence. In the else clause we can create a WriteBuildWarning with the message: “Deployment Skipped”. It should all look like this in the end…

Deploy Database

A large part of my introduction to VSDBCMD (not to mention a large part of this idea) came from this Visual Studio Walkthrough on the topic. If you are willing to work through the various issues with using Database Deployment, it is worth it.

I happen to have Visual Studio installed on my TFS Server, if you dont, copy this application from your Visual Studio install to all your TFS Build Agents.

C:Program Files (x86)Microsoft Visual Studio 10.0VSTSDBDeploy

We are going to use an InvokeProcess Activity to call VSDMCMD.exe with the following code for arguments property. The manifest file will contain the Database name and Connection String that was configured in the project, but chances are you are going to want to override this per environment.

String.Format("/a:Deploy /dd+ /dsp:Sql" _
     & " /manifest:""{0}{1}""" _
     & " /p:TargetDatabase=""{2}"" /cs:""{3}""", _
     BuildDetail.DropLocation, ciVSDBCMDManifestFile, _
     ciVSDBCMDTargetDatabase, ciVSDBCMDConnectionString)

Be sure to properly name the InvokeProcess Activity, as well as provide WriteBuildMessage and WriteBuildError Actvities for the standard and error output.

Deploy Website

Deploying databases was actually a lot easier than I expected, which is fortunate, because deploying websites was a lot more challenging.

Lets add a few variables to the Deploy Website Sequence:

  • Name: websitePublishDirectoryPath
  • Variable type: String
  • Default:
    String.Format("{0}Website Deployment", BinariesDirectory)
  • Name: msDeployManifestFilePath
  • Variable type: String
  • Default:
    String.Format("{0}{1}.manifest", websitePublishDirectoryPath, ciMSDeployWebProject)

As we will continue by defining a Sequence named “Prepare Deployment”. This sequence is responsible for preparing a folder to publish. So we have to create the directory copy the output to it and perform any Web.config transformations. First lets declare a variable in this scope:

  • Name: platformConfiguration
  • Variable type: Microsoft.TeamFoundation.Build.Workflow.Activities.PlatformConfiguration
  • Default:
    BuildSettings.PlatformConfigurations.First
  • Name: websiteSourceDirectory
  • Variable type: String
  • Default:
    String.Format("{0}_PublishedWebsites{1}", BinariesDirectory, ciMSDeployWebProject)
  • Name: websiteDestinationDirectory
  • Variable type: Microsoft.TeamFoundation.Build.Workflow.Activities.PlatformConfiguration
  • Default:
    String.Format("{0}{1}", websitePublishDirectoryPath, ciMSDeployWebProject)

One CreateDirectory Activity to create the folder named in variable `websitePublishDirectory`, one CopyDirectory Activity to copy `websiteSourceDirectory` to `websiteDestinationDirectory` and an additonal Sequence named “Find and Transform Web.config Files”

Click for the full workflow
Yet Another Web.config Transform

There seems to be a lot of places to find good information on the Web.config
transformation topic. So I’m not reinventing anything here. Generally I looked for web.config files and any matching transformation files. An MSBuild Activity is used to perform the transformation. Finally delete all transform files, and continue. It’s important, but rather boring… and this blog post is long enough already.

Web Deploy 2.0

Hopefully you can have someone else configure Web Deploy 2.0 for you, but since you are reading this, I doubt that is the case. On learn.iis.net, I found the best set of tutorials to install, configure, (and most importantly) debug. If you are having problems with the setup, don’t hesitate to enable tracing of failed requests. Once you are able to deploy with Visual Studio you are ready to continue… right? I mean, the MSDeploy command doesn’t seem too intimidating. After a day and change, I was ready to scream uncle. Until I found this StackOverflow Question, I was unable to figure out the arguments of the command that Visual Studio used to initiate the deployment.

First we have to generate a msdeploy manifest file. An InvokeMethod Activity to call `System.IO.File.WriteAllText` with the following expression

String.Format("<?xml version=""1.0"" encoding=""utf-8""?>" _
    & "<sitemanifest>" _
        & "<contentPath path=""{0}{1}"" />" _
        & "<setAcl path=""{0}{1}"" setAclResourceType=""Directory"" />" _
        & "<setAcl path=""{0}{1}"" setAclUser=""anonymousAuthenticationUser"" " _ & "setAclResourceType=""Directory"" />" _
    &"</sitemanifest>", _
    websitePublishDirectoryPath, ciMSDeployWebProject)

Finally we can execute MSDeploy with an InvokeProcess Activity and the following arguments.

String.Format("-source:manifest='{0}'" _
              & " -dest:auto,ComputerName='{1}?site={2}',UserName='{3}',Password='{4}',IncludeAcls='False',AuthType='Basic'" _
              & " -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension" _
              & " -setParam:kind='ProviderPath',scope='contentPath',match='^{5}\{6}$',value='{2}'" _
              & " -setParam:kind='ProviderPath',scope='setAcl',match='^{5}\{6}$',value='{2}' -allowUntrusted", _
              msDeployManifestFilePath, ciMSDeployWebService, ciMSDeployWebApplicationName, _
              ciMSDeployUsername, ciMSDeployPassword, _
              websitePublishDirectoryPath.Replace("", "\").Replace(" ", " ").Replace(".", "."), _
              ciMSDeployWebProject.Replace(" ", " ").Replace(".", "."))
Configure and Queue the Build

We can finally see the light at the end of the tunnel here. Since we provided the metadata for our arguments, we get a rather impressive display of customization.

Queue up your build and hopefully you wont have to do too much debugging to get this to work for yourself.

Conclusion

After seeing this work, it was all worth it. Of course, this is all very custom, I have my xaml attached here as an example, but you will have to be prepared to work with it to make it work for you. Drop me a line, I’ll help you out if I can.

ContinuousIntegrationTemplate.zip