• Dependabot

    Keeping dependencies up to date is useful. Even more so if the dependency has a security fix.

    I’ve using Dependabot for a while now. Initially with the preview integration, but now that Dependabot is part of GitHub (complete with a name change to ‘GitHub Dependabot’) the integration is even better.

    All you need to do is add a file under .github/dependabot.yml, and Dependabot integration will be enabled for your repository.

    Here’s the dependabot.yml file for Show Missing:

    version: 2
    updates:
    - package-ecosystem: nuget
      directory: "/"
      schedule:
        interval: daily
        time: '19:30'
      open-pull-requests-limit: 10
      assignees:
      - flcdrg
    
    

    It specifies the following:

    • Look for NuGet packages
    • Based in the root directory
    • Check for updates daily at 7.30pm (UTC)
    • Limit to 10 pull requests
    • Assign those pull requests to me (flcdrg)

    Dependabot will create a pull request to update each outdated dependency. If release notes are available, it will populate the pull request with those details, as well as the commit history between the old version and the new one.

    Dependabot-generated pull request

    There’s comprehensive documentation for using Dependabot on the GitHub Docs site, including many more configuration options.

    I let Dependabot create the pull requests but I still decided whether to approve the request (or not). You could even hook up a GitHub Action to auto-merge your Dependabot pull requests!

    Azure Pipelines

    The interesting thing about Dependabot is the core engine is open source and hosted on GitHub as well. Andrew Craven has created an example of using the Dependabot engine with Azure DevOps. Not sure if he’s updating that repo, but you might find some of the pull requests I’ve submitted there useful.

    You don’t get all the @dependabot bot behaviour like you see on GitHub (as that’s built on top of the core). I guess if you were keen you could build that functionality too!

    I’ve used his code to generate pull requests on some repositories hosted in Azure DevOps and then used Service Hooks to trigger some code in an Azure Function to update the pull requests to set auto-complete and assign a work item.

  • GitHub Action caching

    I’m always interested in making builds faster!

    If your builds run on self-hosted runners then you can persist files between builds so caching is of limited value (or may even make builds slower). However when using a GitHub-hosted runner (build agent) every build gets a brand new VM. It can take a while for dependencies to be restored (eg. NuGet, NPM or similar), and this has to happen every time a build runs. Being able to cache these dependencies and restore them quickly can potentially make a big difference.

    I’ve started adding the Cache task to my Azure Pipelines builds where I can. The equivalent for GitHub Actions is the Cache action.

    These both work in a similar way. You indicate a path whose contents you want to cache for future builds, and a key which is used to determine when the cache is stale.

    Here’s the cache action that I’m using for my Show Missing extension.

    
        - uses: actions/[email protected]
          with:
            path: ${{ github.workspace }}/.nuget/packages
            key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }}
            restore-keys: |
              ${{ runner.os }}-nuget-
    
    

    The first time you run a build with caching enabled, it won’t appear to run any faster. In fact it might take slightly longer, as when just before the build completes, the cache action will bundle up all the files underneath the path specified and save them.

    Subsequent builds will then download and restore the dependencies. Because this is done efficiently (one tar.gz file to download and extract, and the cache presumably lives relatively close to the runner VM), it will usually be a lot faster than relying on the normal package restore process.

    For NuGet packages, you need to have key paths that can indicate when the cache should be updated. Whilst Visual Studio extensions don’t yet support the new ‘SDK-style’ project format, you can still make use of PackageReference, and if you use nuget.exe 4.9 or above, then you can create and use packages.lock.json files. If I hadn’t updated to PackageReference, then the old packages.config would probably work just as well.

    Here’s a build with no caching. It took 1m 54s.

    Build without cache

     Committing restore...
    Generating MSBuild file D:\a\VsShowMissing\VsShowMissing\VS2019\obj\VS2019.csproj.nuget.g.props.
    Generating MSBuild file D:\a\VsShowMissing\VsShowMissing\VS2019\obj\VS2019.csproj.nuget.g.targets.
    Writing assets file to disk. Path: D:\a\VsShowMissing\VsShowMissing\VS2019\obj\project.assets.json
    Restored D:\a\VsShowMissing\VsShowMissing\VS2019\VS2019.csproj (in 8.74 sec).
    
    NuGet Config files used:
        D:\a\VsShowMissing\VsShowMissing\NuGet.Config
        C:\Users\runneradmin\AppData\Roaming\NuGet\NuGet.Config
        C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config
        C:\Program Files (x86)\NuGet\Config\Xamarin.Offline.config
    
    Feeds used:
        C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
        https://api.nuget.org/v3/index.json
    
    Installed:
        45 package(s) to D:\a\VsShowMissing\VsShowMissing\Gardiner.VsShowMissing\Gardiner.VsShowMissing.csproj
        109 package(s) to D:\a\VsShowMissing\VsShowMissing\VS2019\VS2019.csproj
    

    The first time we add the cache (2m)

    First build with cache

    The cache task logs that there is currently nothing to restore

    Run actions/[email protected]
    Cache not found for input keys: Windows-nuget2-3881b0e254e4b0c4e40edd9efa8c26dfd9f5c93c42dad979f6c5869b765a72d0, Windows-nuget2-
    

    But notice there’s a second post-build step for the cache. File to be cache are added to a tar file and that is then saved.

    Post Run actions/[email protected]
    Cache saved successfully
    Post job cleanup.
    C:\windows\System32\tar.exe -z -cf cache.tgz -P -C d:/a/VsShowMissing/VsShowMissing --files-from manifest.txt
    Cache saved successfully
    

    And now subsequent builds use the cache.

    Second build with cache

    You can see the cache action does a restore:

    Run actions/[email protected]
    Cache Size: ~116 MB (122108071 B)
    C:\windows\System32\tar.exe -z -xf d:/a/_temp/50db9096-3e6c-4681-8753-3e12e33854f1/cache.tgz -P -C d:/a/VsShowMissing/VsShowMissing
    Cache restored from key: Windows-nuget2-3881b0e254e4b0c4e40edd9efa8c26dfd9f5c93c42dad979f6c5869b765a72d0
    

    and the output from the nuget restore is a bit different:

     MSBuild auto-detection: using msbuild version '16.6.0.22303' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\bin'.
    Restoring packages for D:\a\VsShowMissing\VsShowMissing\VS2019\VS2019.csproj...
    Restoring packages for D:\a\VsShowMissing\VsShowMissing\Gardiner.VsShowMissing\Gardiner.VsShowMissing.csproj...
    Committing restore...
    Committing restore...
    Generating MSBuild file D:\a\VsShowMissing\VsShowMissing\VS2019\obj\VS2019.csproj.nuget.g.props.
    Generating MSBuild file D:\a\VsShowMissing\VsShowMissing\Gardiner.VsShowMissing\obj\Gardiner.VsShowMissing.csproj.nuget.g.props.
    Generating MSBuild file D:\a\VsShowMissing\VsShowMissing\VS2019\obj\VS2019.csproj.nuget.g.targets.
    Generating MSBuild file D:\a\VsShowMissing\VsShowMissing\Gardiner.VsShowMissing\obj\Gardiner.VsShowMissing.csproj.nuget.g.targets.
    Writing assets file to disk. Path: D:\a\VsShowMissing\VsShowMissing\VS2019\obj\project.assets.json
    Writing assets file to disk. Path: D:\a\VsShowMissing\VsShowMissing\Gardiner.VsShowMissing\obj\project.assets.json
    Restored D:\a\VsShowMissing\VsShowMissing\VS2019\VS2019.csproj (in 844 ms).
    Restored D:\a\VsShowMissing\VsShowMissing\Gardiner.VsShowMissing\Gardiner.VsShowMissing.csproj (in 845 ms).
    
    NuGet Config files used:
        D:\a\VsShowMissing\VsShowMissing\NuGet.Config
        C:\Users\runneradmin\AppData\Roaming\NuGet\NuGet.Config
        C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config
        C:\Program Files (x86)\NuGet\Config\Xamarin.Offline.config
    
    Feeds used:
        C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
        https://api.nuget.org/v3/index.json
    

    But wait.. that build took 2m 4s! What gives? That’s slower than the first time!

    Yeah, that is odd. So a couple of thoughts:

    • Do measure if adding a cache actually makes a difference.
    • The speed of the runner VMs does vary a bit. In that last run, notice that the restore was slightly faster but the build step was quite a bit slower.
    • Possibly you might get different better results from SDK projects?

    I have a theory that in my case there aren’t a huge amount of dependencies, so the time saved downloading them separately isn’t dramatically different to the cache restoring them all. But when you have a lot of dependencies (and NPM packages are likely to be a good example), or the the download speed of all those dependencies is limited, then single large download vs lots of smaller separate downloads should give definite advantages.

  • Building with GitHub Actions

    Adding a GitHub Action to a repository is pretty easy. You can use the Web UI and click on the Actions menu and select one of the builtin workflows. If you’ve use Azure Pipelines before, then things will feel kind of similar. Both use YAML as the format for describing build workflows, though there are subtle differences.

    Creating new workflow

    Using the Web UI is a good way to get started, plus you can use the search field to look for suitable actions. It is just a file though, so you can edit it and commit it just like any other file.

    Editing workflow

    Each workflow is stored in a separate file. These files reside in the special .github/workflows directory.

    Here’s a simple workflow:

    name: CI
    
    on:
      push:
        branches: [ master ]
    
    jobs:
      build:
        runs-on: windows-latest
        steps:
        - uses: actions/[email protected]
          with:
            fetch-depth: 0
    
        - name: nuget restore
          run: nuget restore -Verbosity quiet
    
        - name: setup-msbuild
          uses: microsoft/[email protected]
    
        - name: Build
          id: build
          run: |
            msbuild /p:configuration=Release /p:DeployExtension=false /p:ZipPackageCompressionLevel=normal /v:m
    
    

    The on is the trigger which will cause this workflow to run. Common triggers include push or pull_request, and you can filter triggers to only fire on specific branches. The full list of available triggers are listed in Webhook events.

    Under jobs, you can list one or more jobs. Each job can run on a runner (build agent). Runners can be ones provided by GitHub or yourself. The GitHub hosted runners actually have the same installed software as the Azure Pipelines hosted agents.

    A job (eg. build in the example above) also lists actions under the steps section. There’s a number of actions provided by GitHub and lots of 3rd party actions too. It’s pretty easy to create a new action. In a future post I’ll give an overview of the custom action I created.

    The above workflow does a git checkout (with full history), then restores NuGet packages, ensures msbuild is available and then runs msbuild.

    Once your workflow file is committed to the repository, you can view the execution history, or edit the workflow through the Actions tab.

    GitHub Actions workflow history