Getting started with Docker

So, you wanna use docker-compose…

Introduction

Welcome! If you’re looking at this post, I assume that you’re new to Docker and you want to learn to get up and running quickly.

There are several areas of Docker that confuse new users, and I will do my best to address most of the most common pitfalls in this post.

docker run vs docker compose

Don’t use docker run. If the thing you want to run in Docker only gives you the docker run syntax, you take that thing over to Composerize and you convert it.

There’s nothing particularly wrong with docker run, but docker compose is far easier to maintain and troubleshoot.

Tools like yamllint make diagnosing issues with compose file formatting much more tractable.

Docker networking

This seems to be the area that causes the most confusion.

Don’t worry, you don’t need to be a network engineer to get a grasp of this, but having a good understanding of how networking works is very helpful.

Network namespaces

Docker uses Linux kernel network namespaces to achieve isolation. You don’t need to really understand any of this, but click the link if you’re at all interested.

Effectively what this means is that containers by default are isolated from both the host operating system’s networking stack, and from each other.

Whenever you create a new Docker compose project, by default it will create a new Docker network bridge, with its own internal subnet and DNS.

Containers in the same Docker compose project (in the same docker-compose.yml file, or separate files linked together with the -p flag) share the same network bridge, and thereby are able to communicate with each other.

Communicating with containers by name

Docker also does some things under the hood to make it possible for docker containers in the same stack to communicate with each other by service name.

To illustrate this, lets go with a simple example.

services:
  app:
    image: myapp
    restart: unless-stopped
    environment:
      - POSTGRES_CONNECTION_STRING=postgresql://postgres-db1:5432
    volumes:
      - ./config:/config
  postgres-db1:
    image: postgres
    restart: unless-stopped

Above, we have an extremely simple docker compose file, with a single application and a PostgreSQL database.

You can see in the POSTGRES_CONNECTION_STRING environment variable that it is using the name of the service to communicate with the database.

Creating network bridges

But what if I don’t want or can’t have the containers in the same Compose project?

That’s where creating your own bridges comes in.

There are several different adapter types in Docker, the most common of them is the bridge, so that’s the only one we’ll be talking about in this article.

I’m sure as hell not going to get into macvlan or any of that, that’s a whole article on its own.

Here you can create a bridge:

docker network create proxy

Where proxy can be anything you like, it’s just the name you gave the bridge.

This is how you can use external bridges in Docker compose:

services:
  app01:
    networks:
      - proxy

networks:
  proxy:
    external: true

The key here is that you specify that the network is external to the compose project, and that you specify that each container should use that network instead of the default bridge that would have been created.

You can also specify more than one network per container. This can be useful if you have some containers that need external connectivity but others that don’t.

For example, you could have one bridge that’s used by containers to communicate with the database, and another that the frontend uses to talk to the backend.

Example:

services:
  app01:
    networks:
      - proxy
      - database

networks:
  proxy:
    external: true
  database:

Note that the database network is not external here, so it will be created when you run docker compose up.

Exposing ports

Next, lets get into how Docker handles exposing ports.

Lets take the Jellyfin docker compose file for an example:

---
services:
  jellyfin:
    image: lscr.io/linuxserver/jellyfin:latest
    container_name: jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - JELLYFIN_PublishedServerUrl=http://192.168.0.5 #optional
    volumes:
      - /path/to/jellyfin/library:/config
      - /path/to/tvseries:/data/tvshows
      - /path/to/movies:/data/movies
    ports:
      - 8096:8096
      - 8920:8920 #optional
      - 7359:7359/udp #optional
      - 1900:1900/udp #optional
    restart: unless-stopped

You’ll notice the ports key. These are what’s known as published ports.

The number on the left is what is mapped outside the container (on the machine running Docker), and the one on the right is inside the container.

In 99% of cases, you want to leave the number on the right alone.

For example, if I wanted the Jellyfin web interface, which by default runs on port 8096, and expose it on port 9001, I could change the compose file like so:

ports:
  - 9001:8096
  - 8920:8920 #optional
  - 7359:7359/udp #optional
  - 1900:1900/udp #optional

Now, on to another example.

In many cases, you may not want to expose any ports at all!

If users external to the Docker host don’t need to be able to touch that port, then simply don’t publish it.

Take this other example:

services:
  proxy:
    image: "jc21/nginx-proxy-manager:latest"
    restart: unless-stopped
    ports:
      - "80:80" # Public HTTP Port
      - "443:443" # Public HTTPS Port
      - "81:81" # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

  vaultwarden:
    image: vaultwarden/server:latest
    container_name: vaultwarden
    restart: unless-stopped
    environment:
      DOMAIN: "https://vw.domain.tld"
    volumes:
      - ./vw-data/:/data/

You’ll notice that the Vaultwarden container doesn’t have any published ports.

That’s because it doesn’t need them. In this scenario, we would go into Nginx Proxy Manager and configure a proxy host for it.

All communication with Vaultwarden would go through the proxy, so there’s no need to expose the port.

Remember that when containers are talking internally like this, you use the INTERNAL port (the one that would be on the right), instead of the one you would have exposed on the outside.

Nginx can talk to Vaultwarden and vice versa - there is no reason for a user to directly interact with the other container, so why let them?

This is also valid syntax:

ports:
  - 127.0.0.1:8000:80

This is telling the container to only publish port 8000 on localhost. You can substitute any other IP here to make it only listen on that one interface.

I don’t use this a whole lot.

Docker volumes

Docker volumes are simply a piece of your filesystem that you’re giving to the container to store persistent data.

By definition, Docker containers are ephemeral.

Unless you configure a volume, any data produced while the container is running will be lost when the container is destroyed.

The two most popular ways to configure volumes in Docker are named Docker volumes and bind mounts.

This is a named docker volume:

volumes:
  - postgres-data:/var/lib/postgresql/data

And this is a bind mount:

volumes:
  - /apps/myapp/postgres-data:/var/lib/postgresql/data

Honestly as far as I’m concerned it’s mostly personal preference which you use. I like using bind mounts as it makes it easier to keep track of where your data is.

Named docker volumes will store the data in a subdirectory under /var/lib/docker/volumes/.

Whichever one you use, please make sure you do backups… I’m not going to help you try and get your data back, sorry.