Azure Pipelines Service Containers and ports

Azure Pipelines

Azure Pipelines provides support for containers in several ways. You can build, run or publish Docker images with the Docker task. Container Jobs allow you to use containers to run all the tasks of a pipeline job, isolated from the host agent by the container. Finally, Service Containers provide a way to run additional containers alongside the pipeline job.

You can see some examples of using Service Containers in the documentation, but one thing that confused me was the port mapping options - particularly with, and without Container Jobs. I ended up creating a pipeline with a number of jobs to determine the correct way to address a service container.

You can see the full pipeline at https://github.com/flcdrg/azure-pipelines-service-containers/blob/main/azure-pipelines.yml

I created two container resources. The first maps the container port 1443 to port 1443 on the host, whereas the second assigns a random mapped port on the host.

resources:
  containers:
    - container: mssql
      image: mcr.microsoft.com/mssql/server:2019-latest
      ports:
        - 1433:1433
      env:
        ACCEPT_EULA: Y
        SA_PASSWORD: yourStrong(!)Password

    - container: mssql2
      image: mcr.microsoft.com/mssql/server:2019-latest
      ports:
        - 1433
      env:
        ACCEPT_EULA: Y
        SA_PASSWORD: yourStrong(!)Password

These are mapped as service containers in each job by one of the following:

  services:
    mssqlsvc: mssql

which maps the container to port 1443 on the host, or

  services:
    mssqlsvc: mssql2

which maps to a random port on the host.

I then figured out what was the correct way to connect to the container. In my case (using a SQL Server image) I’m using the sqlcmd command line utility.

Here are the results of my experiment:

Regular job, local process

A regular (non-container) Azure pipelines job, relying on sqlcmd being preinstalled on the host.

sqlcmd -S localhost

Regular job, inline Docker container

We’re running a separate Docker container in a script task. To allow this container to be able to connect to our service container, we ensure they’re both running on the same network.

network=$(docker network list --filter name=vsts_network -q)
docker run --rm --network $network mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd -S mssqlsvc

Regular job, local process, random port

The host-mapped port has been randomly selected, so we need to use a special variable to obtain the actual port number. The variable name takes the form agent.services.<serviceName>.ports.<port>.

In our case, the <serviceName> corresponds to the left-hand name under the services: section and the <port> corresponds to the port number back up in the - ports: section of the resource container declaration.

sqlcmd -S tcp:localhost,$(agent.services.mssqlsvc.ports.1433)

Regular job, inline Docker container, random port

The random port can be ignored, as we connect directly to the SQL container, not via the host-mapped port.

network=$(docker network list --filter name=vsts_network -q)
docker run --rm --network $network mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd -S mssqlsvc

Container job, local process

A Container job means we’re already running inside a Docker container. (The image used for the container job has sqlcmd installed in /opt/mssql-tools18/bin).

/opt/mssql-tools18/bin/sqlcmd -S mssqlsvc

Container job, inline Docker container

So long as the Docker container created in the script task is on the same network, it’s similar

network=$(docker network list --filter name=vsts_network -q)
docker run --rm --network $network mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd -S mssqlsvc

Container job, local process, random port

Because we’re in the container job, we ignore the random port

/opt/mssql-tools18/bin/sqlcmd -S mssqlsvc

Container job, inline Docker container, random port

And the same for an ‘inline’ container

network=$(docker network list --filter name=vsts_network -q)
docker run --rm --network $network mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd -S mssqlsvc