Continuous Integration with Windows Azure SDK 1.7, Powershell and TFS Build too!

Environments change and the solutions to support them have to keep up. I was very entertained with my old deployment solution for a good while. However, we eventually moved to Azure, and I needed to scramble to find something new. Tom Hollander’s Automated Build and Deployment with Windows Azure SDK 1.6 filled that void until I upgraded my project type to the 1.7 SDK. At that point, I realized I had to roll up my sleeves and cobble something new together.

From an automated deployment standpoint, the crippling change between the 1.6 to the 1.7 SDK, is the lack of an “ImportAfter” folder, which allowed us to include legacy msbuild files to attach to the build process. This is what Tom used to attach a Powershell deployment script to the SDK’s Publish build target. However, with the Azure 1.7 SDK, I had to figure out how to execute that PowerShell script myself.

Microsoft: Downloads for Managing Services in Windows Azure

GitHub: StanleyGoldman/AzurePublishHelpers

Creating a Management Certificate & Publish Settings file

Visual Studio has a link which allows you to download a publish settings file, without completely explaining what the side effects are. I myself didn’t understand the problem when I encountered the first symptom, you have reached the maximum number of management certificates. I was forced to understand the situation when I tried to get publish settings files for the 2nd and 3rd Azure Subscriptions my account was associated with. The link creates a management certificate, uploads it to your azure account and provides you with a .publishSettings file to install onto your machine. Life is actually easier when we start taking control of our management certificates.

It is easy enough to create a management certificate. Gaurav Mantri’s blog post really helped on this.

makecert -sky exchange -r -n "CN=JustAProgrammer" -pe -a sha1 -len 2048 -ss My "JustAProgrammer.cer"

We can take this certificate and upload it to the Management Certificates console in Azure. Take note of your subscription id and the thumbprint of the certificate. As you will need it to create your publish Settings file.

Using the PublishSettingsCreator utility, we can create a publish settings file to carry our management information.

PublishSettingsCreator.exe "AzureExample" "ecd7cc1d-12ec-8cf6-a60b-0cf14db32020" "99DFFB9D05D0B5B92893FBBA35988DE281E01E9E"

In order to use the Azure Powershell Cmdlets we have to import the Azure Modules into our Powershell session.

PS C:UsersAdministratorDesktop> Import-Module 'C:Program Files (x86)Microsoft SDKsWindows AzurePowerShellAzureAzure.psd1'

And now we can import the publish settings file.

PS C:UsersAdministratorDesktop> Import-AzurePublishSettingsFile .ecd7cc1d-12ec-8cf6-a60b-0cf14db32020.publishsettings
Setting: AzureExample as the default and current subscription. To view other subscriptions use Get-AzureSubscription

Importing the settings file sets the subscription as default. We can get the default subscription as follows.

PS C:UsersAdministratorDesktop> Get-AzureSubscription

SubscriptionName        : AzureExample
SubscriptionId          : ecd7cc1d-12ec-8cf6-a60b-0cf14db32020
Certificate             : [Subject]
                            CN=JustAProgrammer

                          [Issuer]
                            CN=JustAProgrammer

                          [Serial Number]
                            9157E76A714509B54F8B853A1B80555B

                          [Not Before]
                            10/18/2012 8:21:06 AM

                          [Not After]
                            12/31/2039 6:59:59 PM

                          [Thumbprint]
                            99DFFB9D05D0B5B92893FBBA35988DE281E01E9E

ServiceEndpoint         : https://management.core.windows.net/
SqlAzureServiceEndpoint :
CurrentStorageAccount   :
IsDefault               : True

Before we can use this subscription to deploy, we have to set the storage account for the subscription. You can CurrentStorageAccount is not set in the subscription definition above.

PS C:UsersAdministratorDesktop> Set-AzureSubscription -SubscriptionName "AzureExample" -CurrentStorageAccount "azureexample"

With the publish settings file and Import-AzurePublishSettingsFile, Set-AzureSubscription commands we can allow any machine to deploy using said Management Certificate. The certificate and publish settings file should be guarded well, either of these files allow access to your azure subscription.

Customizing the Build

Create a new build definition and configure it to build the solution. Be sure to add the “Publish” target to your build. The argument will cause the Azure 1.7 SDK to create the deployment package during build.

If you are using TFSBuild, you can do this while configuring your build, look for the field ‘MSBuild Arguments’, add set the value ‘/t:Publish’.

Finish configuring the build definition and queue the build. We are going to use the output from the build to test the powershell script.

Testing the Powershell script

Using the output from the build, we should be able to execute the powershell script. Copy the script over to your build server. Execute it with the path to the publish file, location where the output Package and Configuration file can be found and the name of the package file in that location. A tag can optionally be specified to help identify the build. I usually use the version number of the binary here, but the build label works just as fine.

PS C:UsersAdministratorDesktop> C:BuildsAzureDeploy.ps1 
    'C:Builds1AzureExampleAzureExample DeploymentSourcesAzureExample.AzureProfilesAzureExampleProduction.azurePubxml'
    'C:Builds1AzureExampleAzureExample DeploymentSourcesAzureExample.AzurebinDebugapp.publish' 
    'AzureExample.Azure.cspkg' 
    'Test LabelTag'

Note: I’ve had some problems with the Azure Powershell Cmdlets and relative paths.

If your machine is configured correctly, this should deploy without any problems. If you are using TFS the next section is useful for you to wrap this all together. If you are using a build system other than TFS, you already have what you need to continue. Enjoy!

TFS Build Process Template

The Build Process Template included in the package takes a few arguments and handles the execution of the Powrshell script rather nicely. After choosing it as the template for your build, you just have to specify a few arguments.

Deployment Configuration: The build configuration of the Azure project, “Debug” or “Release”
Deployment Profile: The name of the profile file to be used
Deployment Project Name: The name of the project Azure project
Deployment Script: (Optional) In case you dont keep your deployment script in the same spot as mine

The build process template uses the Build Label as a tag for the build. It might be a bit more useful to use something like github: martinbuberl/VersionTasks to tag the builds.

Enjoy Continuous Integration

It never seems worth the effort until after you are done with it.

Continuous Integration with TFS2010, MSDeploy & VSDBCMD

After I created an account at Cytanium, I quickly discovered the glory of One-Click Publish. After setting up my source control in TFS. All I could think of, was getting TFS to do the same thing for me, but even I knew that wouldn’t be enough. I wanted TFS to deploy my database changes as well. Well, there is that Schema Compare utility in Visual Studio, I know it can figure out and deploy changes to a database. I figured if Visual Studio can do all of these things, I should be able to as well… shouldn’t I?

Custom Build Process Templates

Very soon after you start to ask these kinds of questions, you find yourself looking at TFS Build Process Templates. The templates are designed in Windows Workflow and thanks to a few good blogs out there this is a rather approachable topic. Most of my starting information came from this blog sequence: Ewald Hofman’s – Customize Team Build 2010.

Before you start, save yourself the pain, take Ewald’s advice and add the Team Foundation Build Activities to your Toolbox. This will shave 15 seconds off the time to open every template.

Exposing Arguments and Providing Metadata

More lessons from Ewald on adding arguments and adding more complex arguments. We will use them and see how the values are supplied a bit later on.

  • Continuous Integration Database Deployment
    • ciVSDBCMDConnectionString: “Database Connection String”
    • ciVSDBCMDManifestFile: “Database Manifest File”
    • ciVSDBCMDTargetDatabase: “Target Database”
  • Continuous Integration Website Deployment
    • ciMSDeployWebProject: “Web Project”
    • ciMSDeployUsername: “MSDeploy Credentials: Username”
    • ciMSDeployPassword: “MSDeploy Credentials: Password”
    • ciMSDeployWebService: “MSDeploy Web Service Url”
    • ciMSDeployWebApplication: “Web Application Name”

Customizing the Template

Under your Team Project, go to the folder BuildProcessTemplates, and branch DefaultTemplate.xaml to ContinuousIntegrationTemplate.xaml.

Open it up and and take a look around. It will help if you take a top down approach, Collapse All, using the controls in the top right. Locate the AgentScope Activity named “Run On Agent” and expand it. Now look for the open space after the “Try Compile, Test, and Associate Work Item” Sequence Activity; this is where we will start adding our custom functionality.

In this space add a If Activity and name it: “If Build and Test Succeeded”. In its condition we are going to add the following VB statement:

BuildDetail.CompilationStatus = BuildPhaseStatus.Succeeded And
    (BuildDetail.TestStatus = BuildPhaseStatus.Succeeded Or
        BuildDetail.TestStatus = BuildPhaseStatus.Unknown)

In the Then clause we drop a sequence named: “Deploy Database and Website”. Inside that sequence, go ahead and create a “Deploy Database” Sequence and a “Deploy Website” Sequence. In the else clause we can create a WriteBuildWarning with the message: “Deployment Skipped”. It should all look like this in the end…

Deploy Database

A large part of my introduction to VSDBCMD (not to mention a large part of this idea) came from this Visual Studio Walkthrough on the topic. If you are willing to work through the various issues with using Database Deployment, it is worth it.

I happen to have Visual Studio installed on my TFS Server, if you dont, copy this application from your Visual Studio install to all your TFS Build Agents.

C:Program Files (x86)Microsoft Visual Studio 10.0VSTSDBDeploy

We are going to use an InvokeProcess Activity to call VSDMCMD.exe with the following code for arguments property. The manifest file will contain the Database name and Connection String that was configured in the project, but chances are you are going to want to override this per environment.

String.Format("/a:Deploy /dd+ /dsp:Sql" _
     & " /manifest:""{0}{1}""" _
     & " /p:TargetDatabase=""{2}"" /cs:""{3}""", _
     BuildDetail.DropLocation, ciVSDBCMDManifestFile, _
     ciVSDBCMDTargetDatabase, ciVSDBCMDConnectionString)

Be sure to properly name the InvokeProcess Activity, as well as provide WriteBuildMessage and WriteBuildError Actvities for the standard and error output.

Deploy Website

Deploying databases was actually a lot easier than I expected, which is fortunate, because deploying websites was a lot more challenging.

Lets add a few variables to the Deploy Website Sequence:

  • Name: websitePublishDirectoryPath
  • Variable type: String
  • Default:
    String.Format("{0}Website Deployment", BinariesDirectory)
  • Name: msDeployManifestFilePath
  • Variable type: String
  • Default:
    String.Format("{0}{1}.manifest", websitePublishDirectoryPath, ciMSDeployWebProject)

As we will continue by defining a Sequence named “Prepare Deployment”. This sequence is responsible for preparing a folder to publish. So we have to create the directory copy the output to it and perform any Web.config transformations. First lets declare a variable in this scope:

  • Name: platformConfiguration
  • Variable type: Microsoft.TeamFoundation.Build.Workflow.Activities.PlatformConfiguration
  • Default:
    BuildSettings.PlatformConfigurations.First
  • Name: websiteSourceDirectory
  • Variable type: String
  • Default:
    String.Format("{0}_PublishedWebsites{1}", BinariesDirectory, ciMSDeployWebProject)
  • Name: websiteDestinationDirectory
  • Variable type: Microsoft.TeamFoundation.Build.Workflow.Activities.PlatformConfiguration
  • Default:
    String.Format("{0}{1}", websitePublishDirectoryPath, ciMSDeployWebProject)

One CreateDirectory Activity to create the folder named in variable `websitePublishDirectory`, one CopyDirectory Activity to copy `websiteSourceDirectory` to `websiteDestinationDirectory` and an additonal Sequence named “Find and Transform Web.config Files”

Click for the full workflow
Yet Another Web.config Transform

There seems to be a lot of places to find good information on the Web.config
transformation topic. So I’m not reinventing anything here. Generally I looked for web.config files and any matching transformation files. An MSBuild Activity is used to perform the transformation. Finally delete all transform files, and continue. It’s important, but rather boring… and this blog post is long enough already.

Web Deploy 2.0

Hopefully you can have someone else configure Web Deploy 2.0 for you, but since you are reading this, I doubt that is the case. On learn.iis.net, I found the best set of tutorials to install, configure, (and most importantly) debug. If you are having problems with the setup, don’t hesitate to enable tracing of failed requests. Once you are able to deploy with Visual Studio you are ready to continue… right? I mean, the MSDeploy command doesn’t seem too intimidating. After a day and change, I was ready to scream uncle. Until I found this StackOverflow Question, I was unable to figure out the arguments of the command that Visual Studio used to initiate the deployment.

First we have to generate a msdeploy manifest file. An InvokeMethod Activity to call `System.IO.File.WriteAllText` with the following expression

String.Format("<?xml version=""1.0"" encoding=""utf-8""?>" _
    & "<sitemanifest>" _
        & "<contentPath path=""{0}{1}"" />" _
        & "<setAcl path=""{0}{1}"" setAclResourceType=""Directory"" />" _
        & "<setAcl path=""{0}{1}"" setAclUser=""anonymousAuthenticationUser"" " _ & "setAclResourceType=""Directory"" />" _
    &"</sitemanifest>", _
    websitePublishDirectoryPath, ciMSDeployWebProject)

Finally we can execute MSDeploy with an InvokeProcess Activity and the following arguments.

String.Format("-source:manifest='{0}'" _
              & " -dest:auto,ComputerName='{1}?site={2}',UserName='{3}',Password='{4}',IncludeAcls='False',AuthType='Basic'" _
              & " -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension" _
              & " -setParam:kind='ProviderPath',scope='contentPath',match='^{5}\{6}$',value='{2}'" _
              & " -setParam:kind='ProviderPath',scope='setAcl',match='^{5}\{6}$',value='{2}' -allowUntrusted", _
              msDeployManifestFilePath, ciMSDeployWebService, ciMSDeployWebApplicationName, _
              ciMSDeployUsername, ciMSDeployPassword, _
              websitePublishDirectoryPath.Replace("", "\").Replace(" ", " ").Replace(".", "."), _
              ciMSDeployWebProject.Replace(" ", " ").Replace(".", "."))
Configure and Queue the Build

We can finally see the light at the end of the tunnel here. Since we provided the metadata for our arguments, we get a rather impressive display of customization.

Queue up your build and hopefully you wont have to do too much debugging to get this to work for yourself.

Conclusion

After seeing this work, it was all worth it. Of course, this is all very custom, I have my xaml attached here as an example, but you will have to be prepared to work with it to make it work for you. Drop me a line, I’ll help you out if I can.

ContinuousIntegrationTemplate.zip

Synchronously Await an Async Workflow Using Observable.guard

The other day, I couldn’t figure out how to Synchronously wait for an Async workflow to complete. I was pretty sure I needed to use Observable.guard, I just couldn’t wrap my head around how to use it. This morning I was on a mission to figure it out.

open System
open FSharp.Control

let event = new Event<_>()

let createObservableByExecutingAsync asyncToExecute =
    event.Publish
    |> Observable.guard(fun _ -> 

        let asyncOperation = async {
            asyncToExecute
            |> Async.RunSynchronously

            event.Trigger(())
        }
        
        printfn "Start Executing Async Operation"

        Async.Start asyncOperation

        printfn "End Executing Async Operation"
    )

let someAsyncOperation = async {
        printfn "Async Operation"
    }

let asyncWorker = createObservableByExecutingAsync someAsyncOperation

printfn "Start Waiting for Async Result"

Async.AwaitObservable asyncWorker
|> Async.RunSynchronously

printfn "End Waiting for Async Result"

Console.ReadLine()
|> ignore

This bit of code produces the most beautiful output of…

Start Waiting for Async Result 
Start Executing Async Operation
End Executing Async Operation
Async Operation
End Waiting for Async Result

Figuring that out makes me feel better, but the bigger realization was that I didn’t need Observable.guard at all. I needed Async.AwaitObservable.

open System
open FSharp.Control
open RateLimiting
 
type Messages =
    | Message of string
 
type Responses =
    | Response of string
 
let operationFunction (message: Messages) =
    match message with
    | Message(input) -> Response(sprintf "Response: %s" input)
 
let agent =
    RateLimitedAgent(operationFunction, 4, 1000)
 
let event = new Event<_>()

let someAsyncOperation (response:Responses) = async {
        printfn "Async Response"
        event.Trigger(())
    }

agent.QueueItem({payload = Message("Awaited Payload") ; response = Some(someAsyncOperation)})
|> Async.Start

printfn "Start Waiting for Async Result"

Async.AwaitObservable event.Publish
|> Async.RunSynchronously

printfn "End Waiting for Async Result"

Console.ReadLine()
|> ignore
Start Waiting for Async Result
Async Response
End Waiting for Async Result

I love watching this stuff work.

Async Wait Operation

Rate Limiting a Function with Async Workflows

So I am trying to deal with an API that says I cannot make more than 4 requests per second. This is a really good use for F# since it has such great support for Asynchronous Workflows. I took a little inspiration from this post by Luca Bolognese.

module RateLimiting

open System
open FSharp.Control

type RateLimitedMessage<'a,'b> = {payload: 'a ; response: ('b -> Async<unit>) option}

type RateLimitedAgent<'a,'b>(
                                operation: 'a -> 'b,
                                workerCount, 
                                ?minExecutionTime: int,
                                ?errorHandler: Exception -> 'a -> Async<unit>
                            ) = class
    let blockingQueue = new BlockingQueueAgent<RateLimitedMessage<'a, 'b>>(workerCount)
    let semaphore = new System.Threading.Semaphore(workerCount, workerCount)

    let errorHandler = defaultArg errorHandler (fun _ _ -> async{()})
    let minExecutionTime = defaultArg minExecutionTime -1

    let createWorker () =
        async {
            while true do
                semaphore.WaitOne()
                |> ignore

                let! message = blockingQueue.AsyncGet()

                let messageProcessor =
                    async {
                            try
                                let response = operation message.payload
                                if message.response.IsSome then
                                    do! message.response.Value response
                            with 
                            | ex ->
                                errorHandler ex message.payload
                                |> Async.Start
                        }

                   
                seq {
                        yield messageProcessor
                        yield async {
                                do! Async.Sleep(minExecutionTime)
                                semaphore.Release()
                                    |> ignore
                            }
                    }
                |> Async.Parallel
                |> Async.Ignore
                |> Async.Start
            }

    do
        (fun _ -> createWorker ())
        |> Seq.init workerCount
        |> Async.Parallel
        |> Async.Ignore
        |> Async.Start

    member x.QueueItem item =
        blockingQueue.AsyncAdd item
end

RateLimitedAgent

Messages are passed in through a BlockingQueueAgent from fsharpx’s Agent Library. This allows users to choose whether or not they wait until messages are picked up for processing.

Here is an example of where we don’t wait at all. Although, we throw in some random exceptions for some fun.

open System
open System.Diagnostics
open System.Threading
open FSharp.Control
open RateLimiting

let counterCreationData name ctype = new CounterCreationData(name, "", ctype)

let counterCategoryName = "Test Rate Limiter"
let requestsProcessedPerSecondCounterItem = "Requests Processed per second"
let requestsQueuedPerSecondCounterItem = "Requests Queued per second"
let requestsQueuedCounterItem = "Number of Items Queued"

let performanceCounters = [|
        counterCreationData requestsProcessedPerSecondCounterItem PerformanceCounterType.RateOfCountsPerSecond32 
        counterCreationData requestsQueuedPerSecondCounterItem PerformanceCounterType.RateOfCountsPerSecond32 ;
        counterCreationData requestsQueuedCounterItem PerformanceCounterType.NumberOfItems32
    |]

let removeCounters () = PerformanceCounterCategory.Delete(counterCategoryName)

let installCounters () =
    if PerformanceCounterCategory.Exists(counterCategoryName)
    then
        removeCounters ()

    PerformanceCounterCategory.Create(counterCategoryName, "", PerformanceCounterCategoryType.SingleInstance, new CounterCreationDataCollection(performanceCounters)) 
    |> ignore 

installCounters()
    
let createCounter name =
    let curProc = Process.GetCurrentProcess()
    new PerformanceCounter(counterCategoryName, name, false)

let requestProcessedSpeedCounter = createCounter requestsProcessedPerSecondCounterItem
let requestQueuedSpeedCounter = createCounter requestsQueuedPerSecondCounterItem

let queueCounter = createCounter requestsQueuedCounterItem
queueCounter.RawValue <- 0L

let rndSeed = new Random()

type Messages =
    | Message of string

type Responses =
    | Response of string

let operationFunction (message: Messages) =
    match rndSeed.Next(10) with
    | x when x = 1 -> failwith "Error"
    | _ ->
        match message with
        | Message(input) -> Response(sprintf "Response: %s" input)

let errorHandler (ex:Exception) message = async {
                                    requestProcessedSpeedCounter.Increment()
                                    |> ignore

                                    queueCounter.Decrement()
                                    |> ignore

                                    printfn "Error"
                                }

let agent =
    RateLimitedAgent(operationFunction, 4, 1000, errorHandler)

let responseFunction (someArg:Responses) = async{
    match someArg with
    | Response(response) ->
        requestProcessedSpeedCounter.Increment()
        |> ignore

        queueCounter.Decrement()
        |> ignore

        printfn "Responded: %s" response
    }

let createDisposable name =
    let x = ref 1
    let rnd = new Random(rndSeed.Next())

    async {
        while true do               
            agent.QueueItem({payload = Message(sprintf "%s Request %i" name (!x)) ; response = Some(responseFunction)})
            |> Async.Start
            
            requestQueuedSpeedCounter.Increment()
            |> ignore

            queueCounter.Increment()
            |> ignore

            x := !x + 1

            do! Async.Sleep(550)
        }
    |> Async.StartDisposable

let disposable1 = createDisposable "Automatic 1"
let disposable2 = createDisposable "Automatic 2"
let disposable3 = createDisposable "Automatic 3"
let disposable4 = createDisposable "Automatic 4"

Console.ReadLine()
|> ignore

disposable1.Dispose()
disposable2.Dispose()
disposable3.Dispose()
disposable4.Dispose()

Console.ReadLine()
|> ignore

So this is just about what I was looking for. I say just about because apparently I can get spikes where I go over limit. I have a strong belief this is a calculation error within Performance Monitor, but if we were really concerned about it, we could increase the worker cooldown time to 1.1s.

Because it’s fun, I’ve toyed some with the operation function to include a random delay, just to see the results.

let operationFunction (message: Messages) =
    Thread.Sleep(rndSeed.Next(50,2500))
    match rndSeed.Next(10) with
    | x when x = 1 -> failwith "Error"
    | _ ->
        match message with
        | Message(input) -> Response(sprintf "Response: %s" input)

Not too shabby..

Messages are bundled with an optional Async response. This allows the caller to provide an Async function to handle the response. If a user were interested, I’m pretty sure this is a more likely and useful target to block your Async Workflow for.

I’m sure I have to use Observable.guard like Thomas Petricek reffered to here and here.

So, create an Async Workflow, make it throw an event, add the message to the agent, then wait until the event is observed. Sounds simple enough, but I just can’t wrap my head around the mechanics of it at this point. I think I’m going to ask Thomas to help me out.

Edited 10/17/2011
As per comments below

YUI’s CompressorTask with MSBuild and Visual Studio

While trying to integrate YUI Compressor as a build task in my current project…

http://yuicompressor.codeplex.com/documentation

Post Build Event:

$(MSBuildBinPath)msbuild.exe "$(SolutionDir)MinificationSettings.xml" /property:SourceLocation=$(ProjectDir) /p:JavaScriptOutputFile=$(ProjectDir)ContentMinScriptsExample.min.js /p:CssOutputFile=$(ProjectDir)ContentMinCSSSite.min.css

I got the error:

MSBUILD : error MSB1008: Only one project can be specified.

C:WindowsMicrosoft.NETFrameworkv4.0.30319msbuild.exe "c:usersspadedocumentsvisual studio 2010ProjectsExampleYUICompressorTaskMinificationSettings.xml"
    /p:SourceLocation=c:usersspadedocumentsvisual studio 2010ProjectsExampleYUICompressorTaskMVCExample
    /p:JavaScriptOutputFile=c:usersspadedocumentsvisual studio 2010ProjectsExampleYUICompressorTaskMVCExampleContentMinScriptsExample.min.js
    /p:CssOutputFile=c:usersspadedocumentsvisual studio 2010ProjectsExampleYUICompressorTaskMVCExampleContentMinCSSSite.min.css

It seems that the spaces in the folder path are preventing me from enjoying YUI Compressor. Quite the same problem this person is having: MSDN Forums: Unable to correctly pass parameters to MS build. Although I suspect he gave up a bit too early.

I tried adding quotes around the value of the parameters, you would think this is going to work, it doesn’t.

Looking for a way to replace the spaces, I came across MS Build Property Functions.

With Build Property Functions, it becomes possible to run the simple replace command in all the spots where I need to, my first guess was to replace space with ‘%20’. That had mixed results, so I was left with choosing an arbitrary token for replacement.

Post Build Event:

$(MSBuildBinPath)msbuild.exe "$(SolutionDir)MinificationSettings.xml"
    /p:SourceLocation=$(ProjectDir.Replace(" ", "[[REPLACESPACE]])
    /p:JavaScriptOutputFile=$(ProjectDir.Replace(" ", "[[REPLACESPACE]])ContentMinScriptsExample.min.js
    /p:CssOutputFile=$(ProjectDir.Replace(" ", "[[REPLACESPACE]])ContentMinCSSSite.min.css

MinificationSettings.Xml:

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/MsBuild/2003">
  <UsingTask
      TaskName="CompressorTask"
      AssemblyFile="packagesYUICompressor.NET-MsBuild-Task.1.6.0.2libNET35Yahoo.Yui.Compressor.MsBuildTask.dll" />

  <PropertyGroup>
    <CssOutputFile Condition=" '$(CssOutputFile)'=='' ">SylesSheetFinal.css</CssOutputFile>
    <JavaScriptOutputFile Condition=" '$(JavaScriptOutputFile)'=='' ">JavaScriptFinal.js</JavaScriptOutputFile>
  </PropertyGroup>

  <PropertyGroup>
    <CssOutputFileFix>$(CssOutputFile.Replace('[[REPLACESPACE]]', ' '))</CssOutputFileFix>
    <JavaScriptOutputFileFix>$(JavaScriptOutputFile.Replace('[[REPLACESPACE]]', ' '))</JavaScriptOutputFileFix>
  </PropertyGroup>

  <Target Name="MyTaskTarget">
    <ItemGroup>
      
      <JavaScriptFiles Include="$(SourceLocation.Replace('[[REPLACESPACE]]', ' '))ContentScriptsExample.js"/>
	  <CssFiles Include="$(SourceLocation.Replace('[[REPLACESPACE]]', ' '))ContentCSSSite.css"/>

    </ItemGroup>
    <CompressorTask
        CssFiles="@(CssFiles)"
        DeleteCssFiles="false"
        CssOutputFile="$(CssOutputFileFix)"
        CssCompressionType="YuiStockCompression"
        JavaScriptFiles="@(JavaScriptFiles)"
        ObfuscateJavaScript="True"
        PreserveAllSemicolons="False"
        DisableOptimizations="Nope"
        EncodingType="Default"
        DeleteJavaScriptFiles="false"
        LineBreakPosition="-1"
        JavaScriptOutputFile="$(JavaScriptOutputFileFix)"
        LoggingType="ALittleBit"
        ThreadCulture="en-au"
        IsEvalIgnored="false"
            />
  </Target>
</Project>

Source Code

Project Euler in F# – Problems 4 – 6

Problem 4

Find the largest palindrome that results from the multiplication of two numbers under 1000

I feel like I have a strange obsession with the Seq.unfold function. That function does most of the heavy lifting here.

Although the wise Euler pointed out to me that I can avoid duplicate multiplication factors, and keep track of the largest palindrome so far to avoid unnecessary `isPalindrome` checks.

let problem4 () =
    let isPalindrome (value: string) =
        match value.Length with
        | x when x % 2 = 1 -> (x-1)/2, (x-1)/2 + 1
        | x -> (x/2), (x/2)
        |> fun (pos1, pos2) ->
                    value.Substring(0, pos1), value.Substring(pos2)
        |> (fun (firstHalf, secondHalf) ->

            Array.rev (secondHalf.ToCharArray())
            |> (fun reversedcharArray -> new string(reversedcharArray))
            |> (fun reversedSecondHalf ->
                    firstHalf.Equals(reversedSecondHalf)))

    let productSeq (maximumInt:int) =
        Seq.unfold (fun state ->
            let mult1, mult2 = state
            let result = mult1 * mult2

            match state with
            | 100, _ -> None
            | _, 100 -> Some((result), ( (mult1 - 1) , (mult1 - 1)))
            | _ -> Some((result), ( mult1, (mult2 - 1)))
        ) (maximumInt, maximumInt)

    let largestPalindromeSoFar = ref None

    productSeq 999
    |> Seq.filter(fun (result) ->

        match !largestPalindromeSoFar with
        | Some(largestResult) when largestResult > result -> false
        | _ -> match isPalindrome (result.ToString()) with
                  | true ->
                        largestPalindromeSoFar := Some(result)
                        true
                  | _ -> false)
    |> Seq.max

Problem 5

What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20?

I was so happy with my brute force solution, it looks so pretty, unfortunately it takes 3 1/2 minutes on my machine.

let problem5 () =
    let evenlyDivisibleByAllUpTo limit number =
        {limit .. -1 .. 1}
        |> Seq.forall(fun testNumber -> number % testNumber = 0)

    Seq.initInfinite(fun i -> (i + 1) * 20)
    |> Seq.find(fun testIndex ->evenlyDivisibleByAllUpTo 20 testIndex)

Project Euler’s math explains the problem a bit differently. I cant explain the formula in ten words or less, but after understanding it and applying a bit of functional style, results in this, which runs in under 1ms.

let problem5_correct () =
    
    let primesWithLimit limit =
        let testPrime (possiblePrime:float) =
            let sqrRootOfPrime = sqrt(possiblePrime)
            
            {2.0 .. sqrRootOfPrime}
            |> Seq.forall(fun divisor -> 
                match divisor with
                | 1.0 -> true
                | x when divisor = possiblePrime -> true
                | _ -> possiblePrime % divisor > 0.0)

        {2.0 .. limit}
        |> Seq.filter(fun index -> testPrime index)

    let computeLimit = sqrt(20.0)

    primesWithLimit 20.0
    |> Seq.fold(fun state prime ->
        
        let exponent =
            match prime < computeLimit with
            | true -> floor( (log(20.0) / log(prime)) )
            | _ -> 1.0

        state * (float(prime) ** exponent)) 1.0
    |> int64

Problem 6

Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.

The brute force of this is rather innocent looking, and runs in < 1ms. [fsharp] let problem6 () = let sumOfSquares maximum = {1L .. maximum} |> Seq.map(fun number -> number * number) |> Seq.sum let squareOfSums maximum = {1L .. maximum} |> Seq.sum |> fun result -> result * result (squareOfSums 100L) - (sumOfSquares 100L) [/fsharp] Following the algorithm from Euler, there isn't much to it at all. [fsharp] let problem6_correct () = let limit = 100L let sum = limit * (limit + 1L) / 2L let sum_sq = ((2L * limit) + 1L) * (limit + 1L) * limit / 6L int64(pown sum 2) - sum_sq [/fsharp]

Project Euler in F# – Problems 1 – 3

Problem 1

Find the sum of all the multiples of 3 or 5 below 1000.

Brute force is the first thing that comes to mind

let problem1 () =
    {1 .. 999}
    |> Seq.filter (fun x -> (x % 3 = 0) || (x % 5 = 0))
    |> Seq.sum

But the wise Euler shows Math to be a better tactic, as his answer is unquestionably faster.

let problem1_correct () =
    let target = 999

    let sumDivisibleBy value =
        let p = target / value
        value * (p * (p + 1)) / 2

    (sumDivisibleBy 3) + (sumDivisibleBy 5) - (sumDivisibleBy 15)

Problem 2

Sum of all even Fibonacci numbers under 4 million

No real curveball here, unfold with a limit was the first Fibonacci method that came to mind

let problem2 () =
    let fibonacciSeq withLimit =
        Seq.unfold (fun previousState ->
            let nMinusTwo, nMinusOne = previousState
            let nextValue = nMinusTwo + nMinusOne
            let nextState = (nMinusOne, nextValue)

            match nextValue < withLimit with
            | true -> Some(nextValue, nextState)
            | false -> None
        ) (0, 1)

    fibonacciSeq 4000000
    |> Seq.filter(fun x -> x % 2 = 0)
    |> Seq.sum

Problem 3

What is the largest prime factor of the number 600851475143?

This one might take a few tries to get correct. I actually went and completed a few other Euler problems before coming back to this one.

The sample number, 13195, is the red herring here because it is so easy to get the answer if you aren’t thinking.

let problem3 () =
    let testPrime (possiblePrime:int64) =
        let sqrRootOfPrime = int64(System.Math.Sqrt(float(possiblePrime)))
            
        {1L .. sqrRootOfPrime}
        |> Seq.forall(fun divisor -> 
            match divisor with
            | 1L -> true
            | x when divisor = possiblePrime -> true
            | _ -> possiblePrime % divisor > 0L)

    let factorSeq ofNumber =
        {2L .. (ofNumber/2L)}
        |> Seq.filter(fun testNumber -> ofNumber % testNumber = 0L)
        |> Seq.map(fun factor1 -> (factor1, (ofNumber / factor1)))
        |> Seq.takeWhile(fun (factor1, factor2) -> factor1 <= factor2)
        |> Seq.fold (fun acc (factor1, factor2) -> factor1 :: factor2 :: acc) []
        |> Seq.sort
        |> Seq.toList
        |> List.rev

    factorSeq 600851475143L
    |> Seq.find (fun factor -> testPrime factor)

That was unofficially my fourth attempt.

Project Euler in F#

While job hunting this time around, a recruiter pointed me to a website called Project Euler, as a great place to practice.

Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve. Although mathematics will help you arrive at elegant and efficient methods, the use of a computer and programming skills will be required to solve most problems.

The problems always seem overly complicated, and somehow ask you to find the sum of this operation or said sequence. Most of the time, you are even provided with a sample input and output. An input I’m sure which is carefully chosen to tempt you to write a brute force algorithm.

I immediately went to do them in F#, because I find it so fun. 25 problems later… Why stop there…
After talking to Justin about it, I decided to blog my progress.

So this is a test to see how far I can get.
This might take a while…

Current Level:

Just an Idea

Things always start out with ideas. Webbloging is a new idea to me.
As programmers we are all surrounded by ideas.

But how do you know an idea is good? How do you know if your idea is good?
From what I’ve learned the only way to find out is to share.

So I rounded some friends and decided to start a place to let our ideas be shared. No matter how eccentric.
Welcome to the site.