David Gardiner - Blogging

Google, what happened!!?

2026-05-10T17:00:00.000+09:30

I was working again with Aspire this week and hit another issue launching the solution under Visual Studio. It felt familiar, but I wasn't sure. A quick search online threw up some results, but nothing great.

Did I make a note of the error in OneNote? No.

What about my blog? Umm.. yes! And super embarrassing, it was my most recent post!

So yay! I applied the workaround and was unblocked. But that got me thinking.. surely my post should have been somewhere in the search results by now?

And that's how I discovered that starting in mid-April Google suddenly decided to stop indexing pages on my site!

Here's the summary from Google Search Console:

Only 6 pages currently being indexed? What on earth?

Scrolling further down that page and it breaks down why pages aren't being indexed:

Pretty much all of my actual blog posts are under the 'Crawled - currently not indexed' category.

So for some reason (and they don't really say why) Google knows about the pages, but has not included them in the index, so that's why my search didn't show up my blog post.

So what changed in April?

I looked back at the Git history of my blog repo. There were some changes I merged late March. The only thing that was slightly interesting was I did upgrade from Astro 5 to 6 during that time.

Looking at the source of the homepage, I noticed something curious:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link rel="icon" href="/favicon.ico">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="generator" content="Astro v6.3.1">
    <link rel="canonical" href="https://david.gardiner.net.au/index">
    <link rel="sitemap" href="/sitemap-index.xml">
    <title>David Gardiner</title>

That 'canonical' line looks wrong! The href should be set to https://david.gardiner.net.au/, but it has a /index tacked on the end. I checked the canonical values for other pages, and they were all fine. So just the home page is wrong.

Out of interest, I switched back to the revision of the repo before that change and rebuilt the website at that point in time. Sure enough it was correct back then:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link rel="icon" href="/favicon.ico">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="generator" content="Astro v5.18.1">
    <link rel="canonical" href="https://david.gardiner.net.au/">
    <link rel="sitemap" href="/sitemap-index.xml">
    <title>David Gardiner</title>

The code that created the canonical URL looked like this:

    <link
      rel="canonical"
      href={new URL(Astro.url.pathname.replace(".html", ""), Astro.site)}
    />

So it appears the functionality has changed in Astro 6 slightly. I created a new function that returns the correct value, including for the home page:

export function getCanonicalUrl(astroUrl: URL, astroSite: URL | undefined): string | URL | null | undefined {
  const pathname = astroUrl.pathname.replace(".html", "");
  if (pathname === "/index") {
    return new URL("/", astroSite);
  }
  return new URL(pathname, astroSite);
}

My blog site is rendered statically, and as a way of ensuring that no surprising changes to the generated HTML happen from component updates, I previously created a couple of snapshot tests using Verify CLI. In particular these check the contents of a specific blog post and also the RSS feed.

But I realised I didn't have one in place for the home page. While that wouldn't work well for my main blog repository (which is private), as the home page content would change every time I published a new blog post, that isn't true for the public repo where I maintain the blog engine logic. There I have just a few old blog posts that i use for testing.

I've added that into the pipeline now, so that in the future if there are any unexpected changes from upgrades, the snapshot test will allow me to review them and decide if they are acceptable or not.

Now whether this is the actual cause of Google taking a dislike to me, I have no idea. But it's the most obvious thing I can see so far. Time will tell if I get back in Google's good books or not.

On the plus side, I did check with Bing's Webmaster tools, and that all looks fine (and Bing is returning results for my site), so at least it isn't everyone who doesn't like me.

Cross-platform PowerShell commands in npm scripts

2025-09-09T17:00:00.000+09:30

I'm a big fan of Bob the Builder's catchphrase "Use the right tool for the job". Not that he first came up with it, but his song is quite catchy😃

A tool I often reach for when needing to script functionality is PowerShell. What I really like is that often while you can achieve similar results with Bash stringing together various Unix tools, the PowerShell equivalent almost always much more readable.

And that really matters when someone is trying to understand what a script is doing (and that someone is usually me a few days or weeks later!).

My blog uses Markdown (.md) files for posts, and I make use of the VZ File Templates Visual Studio Code extension, with a template file to start new posts.

The template contains a front-matter section that I then customise for the post:

title: Blog post title
date: '$year$-$monthPadded$-$dayPadded$T$hoursPadded$:$minutesPadded$:00.000+09:30'
#image: /assets
#imageAlt: image description
# description: summary of post
tags:
- A Tag

I wanted to add some simple validation to my blog to check that the front matter in post Markdown files was valid - where "valid" means that I had not forgotten to update those placeholder values appropriately.

I came up with the following PowerShell to do this:

if (Get-ChildItem -Path ./src/posts -Filter '*.md' -Recurse | Where-Object { 
    (Get-Content $_.FullName -Raw) -match '^(description: summary of post|image: /assets|imageAlt: image description title: Blog post title)$' 
    }
) { 
    Write-Host 'Front matter has not been updated'; exit 1 
} else { 
    Write-Host 'All front matter is properly updated' 
}

So if the description, image, imageAlt or title lines aren't updated it will fail.

I could just add this to a build pipeline, but I like the idea of being able to run the same logic locally, as well as from a pipeline. I could have put it into a .ps1 script, but given the script wasn't that long, and the project in question is already using Node and has a package.json file, using a npm (or pnpm) script seemed reasonable.

I added the following to package.json:

  "scripts": {
    "validate-frontmatter": "pwsh -noprofile -Command \"if (Get-ChildItem -Path ./src/posts -Filter '*.md' -Recurse | Where-Object { (Get-Content $_.FullName -Raw) -match '^(description: summary of post|image: /assets|imageAlt: image description|title: Blog post title)$' }) { Write-Host 'Front matter has not been updated'; exit 1 } else { Write-Host 'All front matter is properly updated' }\"",
  },

and confirmed it worked locally by running pnpm validate-frontmatter, and it also ran when called in the build pipeline.

It was a while later that I bothered to review the details of the build step and to my surprise there were a whole bunch of errors being logged, though they didn't cause the step to fail:

 Get-Content: Cannot find path '/home/runner/setup-pnpm/node_modules/.bin/pnpm.FullName' because it does not exist.

That is really strange!

Why is it trying to get the FullName property of the pnpm binary?
The Get-ChildItem is supposed to be only searching under ./src/posts for *.md files, so how it it even ending up at that path?

The only thing I could think of was maybe it related to the actual "Get-Content $_.FullName" part?

$_ in PowerShell is supposed to be the current item when you're iterating over items from a pipeline. The pipeline is running on an Ubuntu build agent with Bash as the default shell. So maybe, just maybe is there a chance that the $_ is being interpreted by Bash before it gets seen by PowerShell?

My first attempt was to escape the $ with a backslash, but VS Code threw up a bunch of squigglies warning me that was invalid JSON. What about two backslashes? That seems valid:

Get-Content -Raw \\$_.FullName

BINGO! And that worked in the pipeline!

But turns out it doesn't now when run in Windows.

So how can we make it compatible with both Bash/Ubuntu and Windows? I don't think we can in the one line. The option I went with was to use a helper package run-script-os. Our scripts now become the following:

    "validate-frontmatter": "run-script-os",
    "validate-frontmatter:default": "pwsh -noprofile -Command \"if (Get-ChildItem -Path ./src/posts -Filter '*.md' -Recurse | Where-Object { (Get-Content -Raw \\$_.FullName) -match '^(description: summary of post|image: /assets|imageAlt: image description|title: Blog post title)$' }) { Write-Host 'Front matter has not been updated'; exit 1 } else { Write-Host 'All front matter is properly updated' }\"",
    "validate-frontmatter:windows": "pwsh -noprofile -Command \"if (Get-ChildItem -Path ./src/posts -Filter '*.md' -Recurse | Where-Object { (Get-Content $_.FullName -Raw) -match '^(description: summary of post|image: /assets|imageAlt: image description|title: Blog post title)$' }) { Write-Host 'Front matter has not been updated'; exit 1 } else { Write-Host 'All front matter is properly updated' }\"",

validate-frontmatter:windows is now the original version that works on Windows. validate-frontmatter:default is the updated version that should work on non-Windows platforms. validate-frontmatter now just calls run-script-os which figures out which other script to defer to depending on the current platform.

You can see the final version in my astro-blog-engine GitHub repository

Alternate solution

The alternative would be to not use inline PowerShell but move the logic to a separate .ps1 script file. That way the package.json script line doesn't contain any $_ variables that are at risk of being interpreted by the wrong shell. If the script logic was any longer or complex then that would be a good solution.

Migrating my blog to Astro - Quality

2025-06-10T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

There were a few things I put in place, or updated to improve the quality of the code and content of my blog.

Lychee link checking

I've blogged previously about using Lychee to find and fix broken links on your website. I figured this was a good time to revisit that process and see if it could be updated.

It turns out that Lychee recently changed their output format, so some of my GitHub Actions were no longer working. I've now fixed the Wayback machine query GitHub Action so it is compatible with Lychee 0.18 and above.

Here's an extract from my main GitHub workflow that runs Lychee:

  linkChecker:
    runs-on: ubuntu-latest
    needs: build
    name: Link Checker
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      # Cache lychee results (e.g. to avoid hitting rate limits)
      - name: Restore lychee cache
        uses: actions/cache@v4
        with:
          path: .lycheecache
          key: cache-lychee-${{ github.sha }}
          restore-keys: cache-lychee-

      - name: Download dist artifact
        uses: actions/download-artifact@v4
        with:
          name: dist
          path: dist

      - name: Link Checker
        id: lychee
        uses: lycheeverse/lychee-action@v2.4.1 # 2.4.0 has a bug. Don't upgrade until fixed
        with:
          fail: false
          output: ${{ github.workspace }}/output.json
          format: json
          token: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
          args: --root-dir "$(pwd)/dist/" 'dist/**/*.html' --cache --max-cache-age 7d --max-retries 0 --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:137.0) Gecko/20100101 Firefox/137.0" --fallback-extensions html -v --cache-exclude-status '429' --insecure --accept '200..=204'

      - name: Report results
        run: |
          ./Generate-LycheeReport.ps1 -InputJsonFile output.json -OutputMarkdownFile output.md

          ls -al

          cat output.md

          cat output.md >> $env:GITHUB_STEP_SUMMARY
        shell: pwsh

      - name: Publish results
        uses: actions/upload-artifact@v4
        with:
          name: link-checker-results
          path: output.*

I created a PowerShell script to generate a simple Markdown report that I could then output as a GitHub Actions summary.

The data file is also published as a build artifact.

I converted the links workflow so that it will now use that artifact as the source for creating a pull request with fixes for broken links where there is a match on archive.org.

Empty frontmatter

Sometimes I'd uncomment some of the frontmatter properties in a blog post, but forget to replace the placeholder text.

Whoops!

So I added an additional job to the main GitHub Actions workflow to validate frontmatter properties.

Lighthouse

Lighthouse is a popular tool for evaluating web page performance and accessibility.

I added a job to my GitHub Actions workflow to run a few key pages against Lighthouse.

The job publishes a GitHub step summary to the build, and links are provided if you want to drill into review the analysis of each page tested.

The results are generally very good. The main thing bring down the blog post page score is because it includes the Disqus commenting addin. One day I'll see if there's better alternative to that which plays nicer with the way it interacts with your web pages.

Markdownlint

I make regular use of David Anson's Markdownlint extension for VS Code, and so it made sense that I incorporate checking the entire set of Markdown files with a separate GitHub Actions workflow. I did have to do a bit of work to clean up some residual issues in older posts, but they're all fixed now and with this in place things should stay in good shape.

Migrating my blog to Astro - Sitemap

2025-06-09T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

Having a sitemap is a great way to ensure that search engines can quickly find all the content on your website, and also a way of indicating to them which pages have been updated and may need re-indexing.

Astro has a sitemap package @astrojs/sitemap that you can use to generate sitemap files for your site.

Configuration of the sitemap is done by modifying the astro.config.ts file.

In my case, I wanted to be able to set the lastMod property for each blog post entry based on the Git "last commit" date. This is a little slow to evaluate as it need to do lots of git queries for every blog post file, but that's the best way I could think of doing it.

The only other alternative would be to add a 'lastModified' frontmatter property to every page, and then remember to update that value every time you modified the file. That just didn't seem feasible.

I'd be interested to hear if there are any other ways to achieve this, especially if they are more efficient.

// @ts-check
import { defineConfig } from 'astro/config';
import { existsSync, globSync } from 'node:fs';
import { execSync } from 'child_process';
import { join, resolve } from 'path';

import sitemap, { type SitemapItem } from "@astrojs/sitemap";

import mdx from "@astrojs/mdx";

// https://astro.build/config
export default defineConfig({
  compressHTML: false,
  site: process.env.DEPLOY_PRIME_URL || "https://david.gardiner.net.au",
  integrations: [sitemap({
    serialize(item) {

      // ensure we have no trailing slash for files
      item.url = item.url.replace(/\/$/, '');

      try {
        // if item is a post, get last modified date from Git
        const urlPattern = /https:\/\/.*?\/(\d{4})\/(\d{2})\/(.+)/;
        const match = item.url.match(urlPattern);
        
        if (match && match[1] && match[2] && match[3]) {
          const year = match[1];
          const month = match[2];
          const slug = match[3];
          
          // Create a glob pattern for the file
          const filePattern = `${year}-${month}-*-${slug}.md`;
          const postsDir = resolve(process.cwd(), 'src', 'posts', year);
          
          try {
            // First check if the directory exists
            if (existsSync(postsDir)) {
              // Use Node's built-in fs.globSync to find files matching the pattern
              const files = globSync(filePattern, { cwd: postsDir });
              
              if (files.length > 0 && files[0]) {
                const filePath = join(postsDir, files[0]);
                
                updateLastModifiedFromGit(filePath, item);
              }
            }
          } catch (err) {
            // Handle errors without crashing
            console.error(`Error finding file for ${item.url}: ${err instanceof Error ? err.message : String(err)}`);
          }
        } // or specific root-level pages /about, /speaking
        else if (item.url.match(/\/about$/)) {
          const filePath = join(process.cwd(), 'src', 'pages', 'about.astro');
          updateLastModifiedFromGit(filePath, item);
        }
        else if (item.url.match(/\/speaking$/)) {
          const filePath = join(process.cwd(), 'src', 'pages', 'speaking.md');
          updateLastModifiedFromGit(filePath, item);
        }
      } catch (error) {
        // Catch any errors to prevent build failures
        console.error(`Error processing sitemap item ${item.url}: ${error instanceof Error ? error.message : String(error)}`);
      }
      
      return item;
    }
  }), mdx()],
  experimental: {
  },
  build: {
    format: 'file',
  }
});

function updateLastModifiedFromGit(filePath: string, item: SitemapItem) {
  if (existsSync(filePath)) {
    // Get last modified date from git
    const gitCmd = `git log -1 --pretty="format:%cI" "${filePath}"`;
    const lastModified = execSync(gitCmd, { encoding: 'utf8' }).trim();

    if (lastModified) {
      item.lastmod = new Date(lastModified).toISOString();
    }
  }
}

Here's an example of the output it generates:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
    xmlns:news="http://www.google.com/schemas/sitemap-news/0.9"
    xmlns:xhtml="http://www.w3.org/1999/xhtml"
    xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"
    xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
    <url>
        <loc>https://david.gardiner.net.au</loc>
    </url>
    <url>
        <loc>https://david.gardiner.net.au/2005</loc>
    </url>
    <url>
        <loc>https://david.gardiner.net.au/2005/10/simulating-workplace</loc>
        <lastmod>2025-05-11T09:52:45.000Z</lastmod>
    </url>
    <url>
        <loc>https://david.gardiner.net.au/2024</loc>
    </url>
    <url>
        <loc>https://david.gardiner.net.au/2024/01/cfs-azure-function</loc>
        <lastmod>2025-05-31T05:03:37.000Z</lastmod>
    </url>
    <url>
        <loc>https://david.gardiner.net.au/2024/01/dotnet8-source-link</loc>
        <lastmod>2025-04-23T06:57:26.000Z</lastmod>
    </url>

I'd really like to be able to set the lastmod property for the other pages on the site (like the home page and the year and tag pages). I think that should be possible by inferring the latest date from all the pages that those pages link to, but I haven't implemented that yet.

Migrating my blog to Astro - RSS/Atom syndication

2025-06-08T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

One common sign that someone has changed or rebuilt their blog engine is that I get a flurry of notifications in Feedly (my feed reader of choice) that there's a whole bunch of "new" posts from them. But when I look closer I realised they're old posts that I remember reading before, but Feedly thinks they're new because the underlying RSS feed has changed.

Astro has an RSS package that you can choose to add to your site. But I realised the old Jekyll site (and I think Blogger before it) was actually generating an Atom format feed.

Atom and RSS are two XML different syndication standards. Both are still widely used, and I figured if I wanted to keep my feed stable, I'd need to stick with Atom if possible.

A quick search didn't find any other implementation of Atom for Astro, so I figured I'd see if I could translate the @astrojs/rss package into something that could generate Atom.

I forked the @astrojs/rss source and created https://github.com/flcdrg/astrojs-atom. Well actually it turns out the rss package source lives at https://github.com/withastro/astro/tree/main/packages/astro-rss so I selectively extracted just the history of that folder from that repo.

I then had a bit of fun kicking the tyres of GitHub Copilot Chat's agent mode. It seemed a good subject to try it out on, given I could provide an example of what the generated output should look like, and Copilot tends to be better at working with JavaScript projects.

Within a day I had a working Atom package that integrated nicely with Astro. I decided it was worth publishing it to npmjs.org, and as such it's my first package that I've published there. I should let the Astro folks know about it in case they're interested or there are others who want to use it.

You can find the package at https://www.npmjs.com/package/astrojs-atom

Integrating with Astro

We use the package in much the same way as the @astrojs/rss one. My intention was to make interchangeable at the package level (though obviously there's some different requirements for the data that you pass in).

The feed is implemented by an Astro Static File Endpoint at /src/pages/feed.xml.ts.

import atom from "astrojs-atom";
import type { AtomEntry } from "astrojs-atom";
import { getCollection } from "astro:content";
import sanitizeHtml from "sanitize-html";
import MarkdownIt from "markdown-it";
import type { APIContext } from "astro";
import getExcerpt from "../scripts/getExcerpt";
import onlyCurrent from "../scripts/filters";
import { parse as htmlParser } from "node-html-parser";
import { getImage } from "astro:assets";

// From https://billyle.dev/posts/adding-rss-feed-content-and-fixing-markdown-image-paths-in-astro

// get dynamic import of images as a map collection
const imagesGlob = import.meta.glob<{ default: ImageMetadata }>(
  "/src/assets/**/*.{jpeg,jpg,png,gif}" // add more image formats if needed
);

const parser = new MarkdownIt();

export async function GET(context: APIContext) {
  if (!context.site) {
    throw Error("site not set");
  }

  const posts = (await getCollection("blog")).filter(onlyCurrent);

  const sortedPosts = posts.sort(
    (a, b) => new Date(b.data.date).getTime() - new Date(a.data.date).getTime()
  );
  const postsToInclude = sortedPosts.filter((post) => post.body).slice(0, 10); // Get the latest 10 posts

  const siteUrl = context.site?.toString();

  if (!siteUrl) {
    throw new Error("Site URL is not defined");
  }

  const feed: AtomEntry[] = [];

  for (const post of postsToInclude) {
    // convert markdown to html string
    const body = parser.render(post.body!);
    // convert html string to DOM-like structure
    const html = htmlParser.parse(body);
    // hold all img tags in variable images
    const images = html.querySelectorAll("img");

    for (const img of images) {
      const src = img.getAttribute("src")!;

      // Relative paths that are optimized by Astro build
      if (src.startsWith("../../assets/")) {
        // remove prefix of `./`
        const prefixRemoved = src.replace("../../assets/", "");
        // create prefix absolute path from root dir
        const imagePathPrefix = `/src/assets/${prefixRemoved}`;

        // call the dynamic import and return the module
        const imagePath = await imagesGlob[imagePathPrefix]?.()?.then(
          (res) => res.default
        );

        if (imagePath) {
          const optimizedImg = await getImage({ src: imagePath });
          const newSrc = context.site + optimizedImg.src.replace("/", "");

          // set the correct path to the optimized image
          img.setAttribute("src", newSrc);
        }
      } else if (src.startsWith("/images")) {
        // images starting with `/images/` is the public dir
        img.setAttribute("src", context.site + src.replace("/", ""));
      } else {
        throw Error(`src unknown: ${src}`);
      }
    }

    const htmlContent = sanitizeHtml(html.toString(), {
      allowedTags: sanitizeHtml.defaults.allowedTags.concat(["img"]),
    });

    feed.push({
      id: `${new URL(post.id, context.site).toString()}`,
      updated: post.data.date,
      published: post.data.date,
      title: post.data.title,
      content: {
        type: "html",
        value: htmlContent,
      },
      summary: {
        type: "html",
        value: post.data.description || getExcerpt(htmlContent, 500),
      },
      category: post.data.tags.map((tag) => ({
        term: tag,
      })),
      link: [
        {
          rel: "alternate",
          href: new URL(post.id, context.site).toString(),
          type: "text/html",
          title: post.data.title,
        },
      ],
      thumbnail: post.data.image
        ? {
            url: `${new URL(post.data.image.src, context.site).toString()}`,
          }
        : undefined,
    });
  }

  const atomFeedUrl = `${siteUrl}feed.xml`;

  return atom({
    id: atomFeedUrl,
    title: {
      value: "David Gardiner",
      type: "html",
    },
    author: [
      {
        name: "David Gardiner",
      },
    ],
    updated: new Date().toISOString(),
    subtitle:
      "A blog of software development, .NET and other interesting things",
    link: [
      {
        rel: "self",
        href: atomFeedUrl,
        type: "application/atom+xml",
      },
      {
        rel: "alternate",
        href: siteUrl,
        type: "text/html",
        hreflang: "en-AU",
      },
    ],
    lang: "en-AU",
    entry: feed,
  });
}

Complexities

There's one thing that makes it slightly more complex than I'd like - including full blog post content in the feed

Unfortunately Astro doesn't provide you access to the rendered content for each page when you're looking at the page data from a call to getCollection in a static file endpoint. Because of this we have to rerender each page ourselves.

This also means we need to deal with fixing links to images, and there's always the risk that what we end up rendering isn't quite the same as how the actual web page is rendered.

It's not ideal and there an existing discussion tracking possible solutions. Ironically there used to be a way in earlier versions of Astro but that approach is no longer possible.

Migrating my blog to Astro - Older posts

2025-06-07T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

The old Jekyll engine would create these continuous /page2, /page3, /page100 pages, that allowed you to navigate further back in time. The problem as I see it with these is that the content of these pages is regenerated every time you add a new post. It's like using the metaphor of a stack of paper, and each time you write a new post you add it to the top of the stack. That then becomes the new page1, and every page underneath that needs to get renumbered. It's the complete opposite of how a book or journal is written.

Now it is true that the consumers of a blog don't need to read it linearly. In fact unless they started reading it when you first started writing, it's unlikely that anyone will ever do that. Instead they'll probably just follow along with your latest posts, or maybe they'll jump directly to a post after finding it through a search engine recommendation or linked from another website.

So I determined to do away with the /pageN metaphor completely, and replace it with a year-based archive indexing approach:

Have an /archive index page
This in turn links to separate per-year index pages
These each have a list of the posts from that year.

Archive page

This is created via the /src/pages/archive.astro file.

---
import { getCollection } from "astro:content";
import BaseLayout from "../layouts/BaseLayout.astro";
import getPostsByGroupCondition from "../scripts/getPostsByGroupCondition";
import { DateTime } from "luxon";
import onlyCurrent from "../scripts/filters";

const pageTitle = "Archive";

const posts = (await getCollection("blog")).filter(onlyCurrent);
---

<BaseLayout pageTitle={pageTitle}>
  <h1>Posts by year</h1>
  <ul>
    {
      Object.entries(
        getPostsByGroupCondition(
          posts,
          (post) => DateTime.fromISO(post.data.date).year
        )
      )
        .sort(([yearA], [yearB]) => Number(yearB) - Number(yearA))
        .map(([year]) => (
          <li>
            <a href={`/${year}`}>{year}</a>
          </li>
        ))
    }
  </ul>
</BaseLayout>

We extract a distinct list of years from the blog content collection and use that to generate a list of links to the per-year pages.

You can see the end result here /archive.

Year pages

Whereas the tag index page is at /tags and each tag page is under that eg. /tags/Azure, for the posts I realised it made more sense to have them appear to reside within the blog post hierarchy. eg. /2005.

To achieve this, we create a file /src/pages/[year]/index.astro. So yes, the subdirectory is named [year].

Again, we leverage Astro's dynamic routes, and infer the year as a parameter from the directory name. Dynamic routes allow you to do that for directories as well as files.

We group all the posts by year, and then ensure that for each year they're sorted chronologically. A link is then made back to the original post.

---
import { getCollection } from "astro:content";
import getPostsByGroupCondition from "../../scripts/getPostsByGroupCondition";
import { DateTime } from "luxon";
import BaseLayout from "../../layouts/BaseLayout.astro";

import type {
  InferGetStaticParamsType,
  InferGetStaticPropsType,
  GetStaticPaths,
} from "astro";
import onlyCurrent from "../../scripts/filters";

export const getStaticPaths = (async () => {
  const posts = (await getCollection("blog")).filter(onlyCurrent);

  const years = Object.entries(
    getPostsByGroupCondition(posts, (post) =>
      DateTime.fromISO(post.data.date).toJSDate().getFullYear()
    )
  )
    .sort(([yearA], [yearB]) => Number(yearB) - Number(yearA))
    .map(([year, yearGroup]) => ({
      params: { year },
      props: {
        posts: yearGroup.sort(
          (a, b) =>
            DateTime.fromISO(a.data.date).toJSDate().getTime() -
            DateTime.fromISO(b.data.date).toJSDate().getTime()
        ),
      },
    }));

  return years;
}) satisfies GetStaticPaths;

type Params = InferGetStaticParamsType<typeof getStaticPaths>;
type Props = InferGetStaticPropsType<typeof getStaticPaths>;

const { year } = Astro.params as Params;
const { posts } = Astro.props as Props;
const pageTitle = `Posts from ${year}`;
const frontmatter = {
  description: `Summary list of posts from the year ${year}`,
};

function formatDateWithSuffix(dateString: string) {
  const date = DateTime.fromISO(dateString);
  const day = date.day;
  const suffix =
    day % 10 === 1 && day !== 11
      ? "st"
      : day % 10 === 2 && day !== 12
        ? "nd"
        : day % 10 === 3 && day !== 13
          ? "rd"
          : "th";
  return `${date.toFormat("d")}${suffix} ${date.toFormat("MMMM")}`;
}
---

<BaseLayout pageTitle={pageTitle} frontmatter={frontmatter}>
  <h1>{year}</h1>

  <ul class="list-none">
    {
      posts.map((post) => (
        <li>
          <time datetime={post.data.date} class="text-gray-500">
            {formatDateWithSuffix(post.data.date)}
          </time>
          - <a href={`/${post.id}`}>{post.data.title}</a>
        </li>
      ))
    }
  </ul>
</BaseLayout>

Here's the live version for /2025.

If I wanted, I could do something similar for months in the year. I think it's fine just at the year level but I don't think it would be too tricky to add later on.

Migrating my blog to Astro - Tags

2025-06-06T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

Tags were something that never really worked properly with Jekyll. This was something I was hoping to implement much better this time around.

My goals:

Use tags as defined in each post's frontmatter.
Highlight tags for each post on the home page and on each blog post page.
Create a single /tags index page that lists all the available tags.
Create a separate file underneath this path for each tag. eg. /tags/Azure, /tags/.NET etc. that include a list of all posts referencing that tag.

1 and 2 are pretty straightforward. As you can see in the MarkdownPostLayout.astro layout component, we can get the tags of the current page from frontmatter, and then loop over them using map to render them out:

---
import DateTimeComponent from "../components/DateTimeComponent.astro";
import BaseLayout from "./BaseLayout.astro";
import { Disqus } from "astro-disqus";

interface Props {
  frontmatter: {
    title: string;
    date: string;
    description?: undefined | string;
    tags: string[];
  };
}
const { frontmatter } = Astro.props;
---

<BaseLayout pageTitle={frontmatter.title}>
  <slot name="head" slot="head" />
  {frontmatter.description && (
    <meta name="description" content={frontmatter.description} slot="head" />
  )}
  
  <h1>{frontmatter.title}</h1>

  <p><DateTimeComponent date={frontmatter.date} /></p>

  <p><em>{frontmatter.description}</em></p>

  <div class="tags">
    {
      frontmatter.tags.map((tag: string) => (
        <p class="tag">
          <a href={`/tags/${tag}`}>{tag}</a>
        </p>
      ))
    }
  </div>

  <slot />

  <Disqus
    embed="https://davidgardiner.disqus.com/embed.js"
    url={Astro.url.toString()}
  />
</BaseLayout>

The /tags page

This is defined by src/pages/tags/index.astro.

The logic here is to make use of the existing blog content collection, and collate all the tags (and calculate totals for each tag).

---
import { getCollection } from "astro:content";
import BaseLayout from "../../layouts/BaseLayout.astro";
import onlyCurrent from "../../scripts/filters";
const pageTitle = "Tags";
const allPosts = (await getCollection("blog")).filter(onlyCurrent);

// Collate all tags from all posts and count posts per tag
const tagCounts = allPosts
  .flatMap((post: any) => post.data.tags)
  .reduce((acc: Record<string, number>, tag: string) => {
    acc[tag] = (acc[tag] || 0) + 1;
    return acc;
  }, {});

const tags = Object.entries(tagCounts).sort(([tagA], [tagB]) =>
  tagA.localeCompare(tagB)
);
---

<BaseLayout pageTitle={pageTitle}>
  <div class="tags">
    {
      tags.map(([tag, count]) => (
        <p class="tag">
          <a href={`/tags/${tag}`}>{tag}</a> ({count})
        </p>
      ))
    }
  </div>
</BaseLayout>

...

You can see the end result at /tags.

Individual tag summary pages

But how do we create individual pages for each tag? It feels similar to what we did with blog posts, but in that case there was a one-to-one relationship between the .md file and the generated .html file.

The solution is again to to use Astro's dynamic routes. By creating a file named src/pages/tags/[tag].astro, we can then use the [tag] placeholder to inject in the tag name.

As we do for posts, we provide an implementation of getStaticPaths to return the list of tags and for each tag the list of posts that have that tag.

---
import BlogPost from "../../components/BlogPost.astro";
import BaseLayout from "../../layouts/BaseLayout.astro";
import { getCollection } from "astro:content";
import DateTimeComponent from "../../components/DateTimeComponent.astro";

import type {
  InferGetStaticParamsType,
  InferGetStaticPropsType,
  GetStaticPaths,
} from "astro";
import onlyCurrent from "../../scripts/filters";

export const getStaticPaths = (async () => {
  const allPosts = (await getCollection("blog")).filter(onlyCurrent);

  const uniqueTags = [
    ...new Set(allPosts.map((post: any) => post.data.tags).flat()),
  ];

  return uniqueTags.map((tag) => {
    const filteredPosts = allPosts
      .filter((post: any) => post.data.tags.includes(tag))
      .sort(
        (a: any, b: any) =>
          new Date(b.data.date).getTime() - new Date(a.data.date).getTime()
      );
    return {
      params: { tag },
      props: { posts: filteredPosts },
    };
  });
}) satisfies GetStaticPaths;

type Params = InferGetStaticParamsType<typeof getStaticPaths>;
type Props = InferGetStaticPropsType<typeof getStaticPaths>;

const { tag } = Astro.params as Params;
const { posts } = Astro.props as Props;
---

<BaseLayout pageTitle={tag}>
  <p>Posts tagged with {tag}</p>
  <ul>
    {
      posts.map((post: any) => (
        <li>
          <BlogPost url={`/${post.id}`} title={post.data.title} /> (
          <DateTimeComponent date={post.data.date} />)
        </li>
      ))
    }
  </ul>
</BaseLayout>

Notice that we get the current tag value from the params property.

And so this is how we get pages like /tags/.NET

Migrating my blog to Astro - Blog posts

2025-06-05T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

In regards to blog posts, remember the original files look like src/posts/2025/2025-05-13-bbs.md, but you will navigate to the corresponding webpage at /2025/05/bbs.

By default Astro will create an output file for each file under /src/pages. So If I had /src/pages/2025/bbs.md then it would generate the desired path. But as I mentioned, I'm storing them in /src/posts/. I could completely change my naming convention, but I was used to the existing one, so would prefer to stick with it if possible.

What I really wanted was for Astro to generate the YYYY/MM output directory structure dynamically.

Eventually I figured out that by leveraging Astro's dynamic routes feature, I could do just that.

We create the curiously named src/pages/[...slug].astro file. The ... prefix is signifiant here. It's a rest parameter.

---
import { getCollection, render } from "astro:content";
import { DateTime } from "luxon";
import MarkdownPostLayout from "../layouts/MarkdownPostLayout.astro";
import type {
  InferGetStaticParamsType,
  InferGetStaticPropsType,
  GetStaticPaths,
} from "astro";
import onlyCurrent from "../scripts/filters";

export const getStaticPaths = (async () => {
  const posts = (await getCollection("blog"))
    .filter(onlyCurrent)
    .sort((a, b) => {
      return DateTime.fromISO(a.data.date).toMillis() -
        DateTime.fromISO(b.data.date).toMillis();
    });

  return posts.map((post) => {

    return ({
      params: {
        slug: post.id
      },
      props: {
        post: post
      }
    });
  });
}) satisfies GetStaticPaths;

type Params = InferGetStaticParamsType<typeof getStaticPaths>;
type Props = InferGetStaticPropsType<typeof getStaticPaths>;

const { post } = Astro.props as Props;
const { } = Astro.params as Params;

if (!post) {
  throw new Error("Post not found");
}

const { Content } = await render(post);
---

{
  post && (
    <MarkdownPostLayout frontmatter={post.data}>
      <Content />
    </MarkdownPostLayout>
  )
}

As the Astro documentation states:

a dynamic route must export a getStaticPaths() that returns an array of objects with a params property. Each of these objects will generate a corresponding route.

The params property sets the slug value to the post.id. This will be the value defined by the generateId function that we implemented in the blogs content collection.

The props property is used to pass through the actual post.

Blog post layout

The blog post page layout MarkdownPostLayout is defined in src/layouts/MarkdownPostLayout.astro

It's not particularly complicated. The actual Markdown content from the post is rendered via the <slot /> element.

One interesting thing is how we can get access to the <head> element to be able to set the description metadata. That's actually set back up in the BaseLayout.astro layout. It defined a named slot, that we can access here by including the slot="head" attribute. That way we can opt in to setting the description to a unique value for each blog post page, but not interfere with the ability of other pages that might use the BaseLayout component and set the description separately.

---
import DateTimeComponent from "../components/DateTimeComponent.astro";
import BaseLayout from "./BaseLayout.astro";
import { Disqus } from "astro-disqus";

interface Props {
  frontmatter: {
    title: string;
    date: string;
    description?: undefined | string;
    tags: string[];
  };
}
const { frontmatter } = Astro.props;
---

<BaseLayout pageTitle={frontmatter.title}>
  <slot name="head" slot="head" />
  {frontmatter.description && (
    <meta name="description" content={frontmatter.description} slot="head" />
  )}
  
  <h1>{frontmatter.title}</h1>

  <p><DateTimeComponent date={frontmatter.date} /></p>

  <p><em>{frontmatter.description}</em></p>

  <div class="tags">
    {
      frontmatter.tags.map((tag: string) => (
        <p class="tag">
          <a href={`/tags/${tag}`}>{tag}</a>
        </p>
      ))
    }
  </div>

  <slot />

  <Disqus
    embed="https://davidgardiner.disqus.com/embed.js"
    url={Astro.url.toString()}
  />
</BaseLayout>

Migrating my blog to Astro - Content collections

2025-06-04T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

The convention I adopted is that all blog posts will live under src/posts. There's a subdirectory for each year, and then each individual post's filename includes the date (which means they end up being sorted in date order).

One of the core concepts of Astro is the idea of content collections. Rather than creating individual custom pages for every post, you can create logical sets of content, and in turn those collections can be queried as part of generating web pages.

Content collections are defined in src/content.config.ts

In my case, I need a content collection which includes all the blog posts. Astro has a glob loader that can match files which is just the ticket.

import { glob } from "astro/loaders";
import { defineCollection, z } from "astro:content";
import { DateTime } from "luxon";

const blog = defineCollection({
  type: "content_layer",
  loader: glob({ 
    pattern: "**/*.{md,mdx}", 
    base: "./src/posts",
    generateId: ({ entry, data }) => {
      const date = DateTime.fromISO(data.date as string, { setZone: true });

      const id = entry.substring(16);
      const slug = `${date.toFormat("yyyy")}/${date.toFormat("MM")}/${id}`;

      return slug.replace(/\.mdx?$/, '');
    },
  }),
  schema: ({ image }) =>
    z.object({
      date: z.string().datetime({ offset: true }),
      title: z.string(),
      draft: z.boolean().optional().default(false),
      tags: z.array(z.string()).default(["others"]),
      image: image().optional(),
      imageAlt: z.string().optional(),
      description: z.string().optional(),
    }),
});

export const collections = { blog };

I also need to take into account that while the source path of a post might be src/posts/2025/2025-05-13-bbs.md, the final URL relative URL for that page should be /2025/05/bbs. That's the reason for the custom generateId function.

The schema allows me to define some additional frontmatter properties, that I'll take advantage of if present when rendering the pages.

Migrating my blog to Astro - Separation of code and data

2025-06-03T08:00:00.000+09:30

This is part of a series of posts on how I migrated my blog to Astro

Since I moved my blog from Blogger to GitHub, all the content and Jekyll files have lived in the same Git repository. Initially the GitHub repository was public, but then I realised that sometimes when you're drafting future posts, you want them to stay private until the appropriate time. True, this is a rare thing, but it's better to be safe.

On the other hand, I really like the concept of open source software. The main way I learned how to use Astro was looking at other people's efforts on GitHub. If I can 'give back' to the community in the same way then I would like to do that.

It took a bit of thinking on best to do this. Git submodules wouldn't work as they assume that it's a specific subdirectory that points to a different repository.

What I ended up with was this:

https://github.com/flcdrg/astro-blog-engine is the public repository for my Astro-based blog engine. It defines the structure and contains all the Astro pages and scripts that are also used by the private repo.
github.com/flcdrg/blog is the private repository. It was based on https://github.com/barryclark/jekyll-now which I forked back in 2019, though there's history there going back to 2014.

However, if you look at https://github.com/flcdrg/astro-blog-engine, the Git history only goes back to 1st February (as part of completing the Astro tutorial). So it's completely unrelated to github.com/flcdrg/blog. So how's that going to work?

What I did was tell git to allow me to merge changes from astro-blog-engine, even though their histories were unrelated by doing this:

git pull --strategy-option=ours astro-blog-engine main --allow-unrelated-histories

Now, every time I have enhancements or updates that I've tested in astro-blog-engine, I follow these steps in the blog repo:

git fetch astro-blog-engine
git checkout -b updates-from-astro-blog-engine
git pull astro-blog-engine main
git push

Then create a pull request and merge the changes in.

As long as I am careful to only make code/structural changes in the public repo and the merge them into the private repo, everything should be fine and I shouldn't encounter any conflicts.

The second advantage of maintaining the core functionality in a different repository is that it is much quicker to test out changes. I have somehow accumulated over 1,000 posts, and that can take a few seconds (or sometimes minutes) to process.

To allow me to try out changes quickly without waiting for my full blog to rebuild, I've copied over just a few of the recent blog posts to the public repo. It's enough to validate that everything works and there's no regressions when updating a package or tweaking the TypeScript or layout.