TeamCity conditional build configurations - master vs other branches

Friday, 21 June 2019

TeamCity doesn’t currently support conditional build steps (Vote on this issue). So how can you have different steps for a master builds vs. other branches?

This is not an uncommon scenario - you might want to perform similar build steps for all builds, but for a master build there may be some extra steps (eg. publishing NuGet/NPM packages to a repository, triggering an external activity) that you don’t want to run for a branch build (eg. a build for a pull request).

It is possible to work around this limitation by creating multiple build configurations - one for master and another for non-master branches. But how can you do that efficiently? One approach is to enable Versioned Settings using Kotlin.

Kotlin is a language created by JetBrains that targets the JVM (and is now the recommended language for Android development). But it’s also one of the language choices when you enable Versioned Settings in TeamCity. (The other is XML, which will look familiar if you’ve ever played with TeamCity’s meta-runners).

I’m not a Kotlin expert, but I managed to figure out that you can use it to generate the different variations, such that when TeamCity parses the Kotlin, it ends up creating multiple build configurations.

One thing to be aware of is that TeamCity parses the Kotlin up-front to create the build configurations. It doesn’t evaluate it as the build runs (so it isn’t possible to have an expression that tests an environment variable or parameter)

The following example shows generating two build configurations - “Build_CI_Master” and “Build_CI_Branches”. The difference is that the Master build configuration enables package indexing. The build configurations use different branch filters to control whether they apply to master or non-master branches.

import jetbrains.buildServer.configs.kotlin.v2018_2.*
import jetbrains.buildServer.configs.kotlin.v2018_2.buildFeatures.nuGetPackagesIndexer
import jetbrains.buildServer.configs.kotlin.v2018_2.buildSteps.PowerShellStep
import jetbrains.buildServer.configs.kotlin.v2018_2.buildSteps.powerShell
import jetbrains.buildServer.configs.kotlin.v2018_2.projectFeatures.nuGetFeed
import jetbrains.buildServer.configs.kotlin.v2018_2.triggers.vcs
import jetbrains.buildServer.configs.kotlin.v2018_2.ui.*

version = "2018.2"

val variations = listOf("Master", "Branches")
project {

    for (type in variations) {
        buildType(KotlinExample(type))
    }

    features {
        nuGetFeed {
            id = "repository-nuget-project_feed"
            name = "project_feed"
            description = ""
        }
    }
}

class KotlinExample(val variant: String) : BuildType({
    id("Build_CI_${variant}".toId())

    name = "CI $variant"

    buildNumberPattern = "1.0.%build.counter%"

    vcs {
        add(DslContext.settingsRoot.id!!)
    }

    triggers {
        add {
            vcs {
                if (variant == "Master") {
                    branchFilter = "+:<default>"
                } else {
                    branchFilter = """
                            +:*
                            -:<default>
                        """.trimIndent()
                }
            }
        }
    }

    steps {

        step {
            /* additional steps */
        }
    }

    features {
        if (variant == "Master") {
            nuGetPackagesIndexer {
                feed = "project_feed/project_feed"
            }
        }
    }
})

Global DevOps Bootcamp 2019

Sunday, 16 June 2019

Yesterday (Saturday 15th) was the 3rd Global DevOps Bootcamp we’ve run in Adelaide.

Selfie Using the great facilities of the University of South Australia, a bunch of people gathered to learning more about DevOps, and in particular this year on the ‘Run’ part of DevOps, including introducing the role of a Site Reliability Engineer (SRE).

The day started off with a recorded introduction from international event organisers, then a keynote from Niall Murphy (Microsoft Ireland Director of Engineering for Azure Cloud Services and Site Reliability Engineering), followed by a local keynote delivered by me.

Keynote

After this people got into teams and worked through different challenges relating to a mythical online car-parts company that was experiencing all kinds of problems in production. Teams were encouraged to follow the pattern of ‘Detect, Respond and Recover’ - identifying the problem, putting in a quick fix to get the website back up and then implementing a long-term solution to prevent the problem from reoccurring.

Team challenges

The teams worked through the rest of the day, doing a great job on the challenges. The light-hearted video introductions to each challenge were a new thing this year and were a nice touch.

All credit to Rene, Marcel, Mathias and their team of helpers who put all the content together and coordinated a massive global event together with Microsoft.

Microsoft provided Subway for lunch and special thanks to my employer RLDatix who picked up the tab for fresh coffees for all attendees.

An Iced Vo Vo surprise

Friday, 14 June 2019

Those who know me really well know that one of my favourite biscuits is Arnott’s Iced Vo Vos.

Iced Vo Vos.jpg By Bilby - Own work, CC BY-SA 3.0, Link

I’m not sure why - I just really enjoy the combination of the sweet biscuit base with pink icing, coconut and jam. A winning combination.

Recently Arnott’s have been looking to diversify their product range and I saw with interest that they now have Iced Vo Vo flavoured milk and Iced Vo Vo flavoured chocolate!

Imagine my surprise when I got home from work this week to discover a block of the latter waiting for me on my bed (together with a clever Origami bow).

Iced Vo Vo Chocolate My youngest daughter was hanging around and asked me to guess who got it for me..

My wife? No

My sister? No

My friend Andrew? No

I was stumped.

It was later that night that my wife tipped me off that actually it had been my daughter all along who’d bought for me with her own money! What a beautiful (and yummy) gift!

Migrating from Blogger to Jekyll and GitHub Pages

Tuesday, 11 June 2019

I’ve been in a bit of a blogging slump the last few months. Partly because I’ve been particularly busy, but also I’ve been a bit frustrated with Blogger. I used Windows Live Writer (later Open Live Writer), but sadly after all the effort to make it open-source, the project has kind of floundered with no one apparently leading the project now.

So what to do? Well maybe it’s time to move to a different platform. I’d see a few people with their blogs on GitHub, and after a bit of research I settled on using Jekyll with GitHub pages.

How I migrated from Blogger

I followed this useful checklist - https://thefriendlytester.co.uk/2017/07/blogger-to-jekyll-migration-timeline. Note that the jekyll-importer does not convert Blogger content to Markdown (that had me confused for a few hours).

I stuck with the same URL format for posts, so that should mean no broken links.

I made use of Windows Subsystem for Linux to run Jekyll locally (as I’d read it wasn’t so easy to get running natively on Windows). See below for more details of getting that all running.

Update Cloudflare to point to flcdrg.github.io - https://blog.cloudflare.com/secure-and-fast-github-pages-with-cloudflare/

Update FeedBurner settings to point to new source RSS feed.

What I forgot

I use dlvr.it to post to Twitter about new blog posts. I should have disabled that as it got confused by the change and thought I’d posted some new articles.

Still to do

The Blogger migration only copied text. Images are still living in their original locations. It would be good to migrate them over too.

My old blog had a nice Archive/history listing. I’ve started looking at Jekyll equivalents, but not found one that’s similar yet.

Getting Jekyll running on WSL

sudo apt update
sudo apt upgrade
sudo apt install ruby
sudo apt install ruby-dev
sudo apt install make gcc
sudo apt install g++
sudo apt install zlibc
sudo gem install pkg-config -v "~> 1.1"
sudo apt install libxml2-dev
sudo apt install libxslt-dev
sudo gem install nokogiri -- --use-system-libraries
sudo gem install jekyll-import
sudo gem install jekyll

With my locally cloned Git repo at C:\dev\git\flcdrg.github.io in Windows, in WSL I could change to the equivalent path at /mnt/c/dev/git/flcdrg.github.io, and then run jekyll server.

Anything else?

Let me know in the comments if something is broken that used to work!

Find where an object is unintentionally being converted to a string

Monday, 11 February 2019

I've been applying the Replace Primitive with Object pattern to a code base - changing what used to be strings into a custom type (which not only makes the code more readable, but now ensures through type safety that you can't accidentally pass in any old random values to methods that used to just take strings.

The code has tests, and after applying the refactoring, I have a failing test - which hints that somewhere there's an implicit conversion from the new strongly typed object back to a string. The test's failing assertion says it received "MyNamespace.TypedThing" (which is what the default implementation of ToString() returns), rather than the wrapped string value that TypedThing encapsulates.

My initial suspicion is that there's probably code similar to this that's causing the problem:

TypedThing thing = new TypedThing("thingy"); 
string s = $"{thing}";

ReSharper has a cool utility - "Structural Search and Replace". Unfortunately it doesn't work for single expressions like "{thing}".

If I was cluey, I might be able to write a Roslyn tool to search the code and find instances like that, but that's going to take a bit more effort than I want.

What about this: temporarily override the ToString() method on my custom type, and make it throw an exception!

It's a bit of a sledgehammer, but it worked!

As it turns out my suspicion was not quite correct. The offending code was actually assigning the custom type to an Object type (which explains the lack of compiler type warnings), which later on must be converted to a string.

Now that I could see an example, I could use ReSharper's SSR to confirm that was the only instance of that kind of assignment (SSR can be used as I'm searching for an assignment statement, not just a single expression). Just for good measure, I'll also re-run the entire test suite to make sure there aren't any other similar problems still hiding.