Automatic updating of Chocolatey packages with .NET

I maintain quite a few Chocolatey packages. The source for these packages lives in https://github.com/flcdrg/au-packages/, and until recently I used the AU PowerShell module to detect and publish updated versions of the packages.

The first issue was that unfortunately, the original maintainer of the AU module archived the project on GitHub. The Chocolatey Community stepped in and is now maintaining a fork here.

The second issue I hit that was causing issues was a compatibility issue with newer versions of PowerShell 7. AU was originally written for Windows PowerShell 5, but I have made extensive use of features of PowerShell 6 and 7 in my update scripts. That didn't seem to cause issues until the GitHub Actions agents were updated from PowerShell 7.2 to 7.4 in January.

The specific problem would reveal itself like this:

Chocolatey had an error occur: System.ArgumentException: File specified is either not found or not a .nupkg file. 'D:\a\au-packages\au-packages\microsoft-teams.install\microsoft-teams.install.1.7.0.3653.nupkg '

For some reason, the AU module was able to generate a new version of a package, but when it called the Chocolatey CLI (choco.exe) and passed the path to the nupkg file, it appeared that there was a trailing space in the filename!

I spent hours trying to debug this to no avail. This was not made any easier by the fact that AU uses PowerShell Jobs to spin up separate processes for each package so they can be processed in parallel. I could not get debugging to work inside a Job when using the Visual Studio Code PowerShell debugger. Even the old-style debugging approach of Write-Host "I got here" didn't work very well as all output of the job is captured isn't easy to extract (let alone being able to inspect the original variables as proper objects rather than serialised strings)

Eventually, I decided I was wasting my time trying to solve this, and maybe if I rewrote the updating logic myself I could mitigate the issue.

There are essentially two parts to the AU module - the bits that support updating an individual package, and then there are the bits that run over all your packages. It's that second part that makes use of PowerShell Jobs and I suspected was the source of the problem.

I figured rewriting that part in C#/.NET would mean I had a much nicer debugging experience (should I need it). I wanted to leave the individual package updating alone - it would be a significant effort to migrate all the custom update.ps1 scripts to something else.

au-dotnet

And so au-dotnet was born.

It is a reasonably simple .NET 8 console application that iterates over all the packages in my au-packages repository, and then calls the PowerShell update.ps1 script in each to see if there is a new version to generate and publish.

Rather than just call out to the operating system to run each update.ps1 script, I decided to embed PowerShell in the application. This gives me a bit more control over how the scripts are run and the ability to capture any script output (and errors) from each run.

Hosting PowerShell

Figuring out how to host PowerShell in a .NET 8 application took a little bit of research. Many of the articles you find (and even some of the official documentation) are still aimed at Windows PowerShell.

The key was to reference these three NuGet packages (and use the same version of each package):

• Microsoft.PowerShell.Commands.Diagnostics
• Microsoft.PowerShell.SDK
• System.Management.Automation

You can then create a PowerShell Class instance like this:

var iss = InitialSessionState.CreateDefault2();
iss.ExecutionPolicy = Microsoft.PowerShell.ExecutionPolicy.RemoteSigned;

var ps = PowerShell.Create(iss);

You can capture any output via the Streams property. eg. Here I am logging any errors from PowerShell as a GitHub Action error:

ps.Streams.Error.DataAdded += (_, args) =>
{
    core.WriteError(ps.Streams.Error[args.Index].ToString());
};

Running specific PowerShell cmdlets can be done via the AddCommand method. eg.

ps.AddCommand("Set-Location").AddParameter("Path", directory).Invoke();

Whereas running arbitrary PowerShell scripts is done via the AddScript method. eg.

ps.AddScript("$ErrorView = 'DetailedView'").Invoke();

If the script is in a separate .ps1 file, the only way I've found so far is to load that file into a string and pass it in. It would be nicer if you could point it at the file (so debugging/errors could include line numbers) but I have yet to find a way to do that.

var output = ps.AddScript(File.ReadAllText(Path.Combine(directory, "update.ps1")))
.Invoke();

One thing to remember is you must call the Invoke method to actually run the scripts or commands you've just added.

GitHub Action logging and summary

Because I know the application will be run in a GitHub Actions workflow, I made use of the excellent GitHub.Actions.Core NuGet package for formatting output, as well as generating a nice build summary that lists all packages that were updated in the current run.

Commit and publish

If a new package is created (eg. a .nupkg file now exists) then we assume this file can be submitted to the Chocolatey Community Repository. choco push is then called to upload the package. Remember this was where we hit that error with the trailing space? Pleasingly the .NET version doesn't exhibit this behaviour, so that problem is solved.

Assuming the package is submitted successfully then we call git` to stage any modified files from this package and add a tag indicating the package that was updated.

After all packages have been processed, we will commit all staged files and push the commit back to the repo, so that we get a version history of all the package changes.

Enhancements

I am currently using my fork of the chocolatey-au module, which has one minor enhancement. It adds a Files collection property to the AUPackage PowerShell class. This collection is populated with the paths of all the files that were downloaded (and had their checksums calculated).

I make use of this for some of my packages to pre-emptively upload the files to VirusTotal. This can help fast-track the packages being approved by Chocolatey as it means the Chocolatey virus scanning step is already completed. Because I use the VirusTotal CLI tool for this, it also means I can upload files up to 650MB (compared to Chocolatey's current 200MB limit due to using an older API).

I have submitted the Files property enhancement to chocolatey-au.

Summary

You can see this in action in the latest workflow runs at https://github.com/flcdrg/au-packages/actions.

Microsoft MVP Summit 2024

I'm back in Redmond for the 2024 Microsoft MVP Summit! I was last here in 2019 and was all booked to return in 2020, but something significant happened around that time that you might be aware of. The event pivoted to virtual in 2020, but since last year has resumed allowing in-person attendance too. I was a virtual participant last year, but this year I've made the big trip over the pond.

I flew to Seattle via Sydney and San Francisco. It is a long way to come, and I didn't sleep very much (hard to get comfortable, plus we had quite a bit of turbulence), though I did end up in a row all to myself (so at least I could stretch out a bit!)

Narelle kindly lent me her Sony WH1000XM4 noise-cancelling headphones for the trip, and they were brilliant. Air travel can be quite noisy, and they did a great job reducing the background noise and were comfortable to wear for extended periods. Do not fear, I will return them when I get home though!

One hiccup on the trip apparently happened while I was recharging my phone at the San Francisco airport while waiting for my connecting flight. The boarding gate opened and I grabbed my phone and charger and jumped in line. It was only later after I'd landed in Seattle and unpacked at the hotel that I discovered that my US to Australian power adapter was missing. It must still be sitting in that power point back in the San Francisco airport lounge. I put a call for help out to my fellow MVPs and not only got offers of spare adapters (huge thanks to Aussie MVP Benjamin Elias), but also some suggestions as to where I might buy a replacement (The Container Store at Bellevue had this adapter, though it is quite a bit more expensive than the $AU6.50 you'd pay for one from Bunnings). Lesson learned: Next time I travel, I'll pack some spare adapters!

I'm staying at The Residence Inn Seattle Bellevue. It's really close to the Microsoft campus at Redmond (a nice walk for me when weather permits, and Seattle does have a reputation for inclement weather). Rather than a big multistory hotel, the rooms here are in one of 15 cute houses. Your space is self-contained with cooking facilities included. In previous years I stayed in downtown Bellevue (right near all the shops and services), but I like the change.

Today (Monday) there were some pre-summit sessions on AI (they weren't under NDA so I can tell you that!), but tomorrow the summit proper begins and by default, all content will be private.

I'm looking forward to learning a heap, catching up with MVP friends and making new connections.

A big thanks to SixPivot for supporting my attendance at the MVP Summit this year.

I'll start the homeward journey on Friday evening. As nice as it is to be here, it will be good to get home again too.

Amazon affiliate links

C# 12 features: Interceptors

Part 8 in a series on new language features in C# 12.

Interceptors

This is an experimental feature that is disabled by default. It is an enhancement that source generators will be able to take advantage of. It allows source generators to modify (replace) existing code, rather than just adding new code.

Being an experimental feature, the implementation details are likely to change over time, and there's no guarantee that it will necessarily ship as a regular feature in the future.

Further reading

https://github.com/dotnet/roslyn/blob/main/docs/features/interceptors.md

Example source

https://github.com/flcdrg/csharp-12/tree/main/08-interceptors