Docker Compose: A Practical Guide for System Administrators

Managing individual Docker containers with long docker run commands quickly becomes impractical when your application requires multiple interconnected services. Docker Compose solves this problem by allowing you to define and manage multi-container applications using a single, declarative YAML file. For system administrators, Docker Compose is an indispensable tool that bridges the gap between manual container management and full orchestration platforms like Kubernetes.

This guide covers everything you need to know to use Docker Compose effectively in real-world scenarios, from understanding the YAML syntax to deploying production-ready stacks with health checks, resource limits, and proper restart policies.

Prerequisites

Before you begin, make sure you have:

• Docker Engine installed on your system. If you have not done this yet, follow our guide: How to Install Docker on Ubuntu 22.04 and 24.04
• Basic understanding of Docker concepts (images, containers, volumes, networks)
• A text editor and terminal access with sudo privileges

What Is Docker Compose?

Docker Compose is a tool for defining and running multi-container Docker applications. Instead of running multiple docker run commands with complex flags, you describe your entire application stack in a docker-compose.yml (or compose.yml) file and then start everything with a single command.

A typical use case might include a web application that depends on a database, a cache layer, and a reverse proxy. Docker Compose lets you define all of these services, their configurations, networks, and volumes in one place, making your infrastructure reproducible and version-controllable.

Compose V2 vs. V1: What Changed

Docker Compose has gone through a significant evolution:

Feature	Compose V1 (Legacy)	Compose V2 (Current)
Command	docker-compose (separate binary)	docker compose (CLI plugin)
Language	Python	Go
Performance	Slower startup	Significantly faster
Installation	Separate install required	Included with Docker Engine
Status	Deprecated (EOL June 2023)	Active development

If you installed Docker following our guide, the Compose V2 plugin is already available as docker compose (note the space instead of the hyphen). All examples in this article use V2 syntax.

Important: If you encounter scripts or documentation that reference docker-compose (with a hyphen), replace it with docker compose (with a space). The functionality is the same, but V1 is no longer maintained.

Anatomy of a docker-compose.yml File

A Compose file has three top-level sections: services, networks, and volumes. Here is the basic structure:

# Optional: specify the Compose file version (not required in V2)
services:
  # Define your application services here
  web:
    image: nginx:latest
    ports:
      - "80:80"

  database:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: secretpassword

networks:
  # Define custom networks here (optional)

volumes:
  # Define named volumes here (optional)

Services

Services are the containers that make up your application. Each service definition can include:

• image or build: The Docker image to use or a Dockerfile to build from
• ports: Port mappings between host and container
• environment: Environment variables
• volumes: Mount points for data persistence
• depends_on: Service startup dependencies
• restart: Restart policy
• networks: Which networks to attach to
• command: Override the default container command

Networks

By default, Docker Compose creates a single network for your application and all services can communicate with each other using their service names as hostnames. You can define custom networks for more granular control over service communication:

services:
  web:
    networks:
      - frontend
      - backend

  api:
    networks:
      - backend

  database:
    networks:
      - backend

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true  # No external access

In this example, the web service can talk to both api and database through the backend network, while api and database are isolated from external access by the internal: true flag on the backend network. The frontend network could be used for traffic from a reverse proxy.

Volumes

Volumes persist data beyond the lifecycle of a container. Named volumes are the recommended approach:

services:
  database:
    image: mysql:8.0
    volumes:
      - db_data:/var/lib/mysql       # Named volume
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql  # Bind mount

volumes:
  db_data:
    driver: local

Environment Variables and .env Files

Hardcoding secrets and configuration values in your docker-compose.yml is a bad practice. Docker Compose supports .env files for managing environment variables cleanly.

Create a .env file in the same directory as your docker-compose.yml:

# .env
MYSQL_ROOT_PASSWORD=supersecretpassword
MYSQL_DATABASE=myapp
MYSQL_USER=appuser
MYSQL_PASSWORD=apppassword
NGINX_PORT=8080

Reference these variables in your Compose file:

services:
  database:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}

  web:
    image: nginx:latest
    ports:
      - "${NGINX_PORT}:80"

You can also pass an env_file directive to load environment variables directly into a container:

services:
  api:
    image: myapp:latest
    env_file:
      - ./app.env

Security Tip: Always add .env files to your .gitignore to prevent accidentally committing secrets to version control. Provide a .env.example file with placeholder values for documentation purposes.

Practical Example 1: Nginx + PHP-FPM + MySQL Stack

This is a classic web application stack that many system administrators need to deploy:

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
      - ./app:/var/www/html:ro
      - ./nginx/ssl:/etc/nginx/ssl:ro
    depends_on:
      php:
        condition: service_started
    networks:
      - webnet
    restart: unless-stopped

  php:
    image: php:8.3-fpm-alpine
    volumes:
      - ./app:/var/www/html
      - ./php/php.ini:/usr/local/etc/php/php.ini:ro
    environment:
      DB_HOST: mysql
      DB_NAME: ${MYSQL_DATABASE}
      DB_USER: ${MYSQL_USER}
      DB_PASSWORD: ${MYSQL_PASSWORD}
    depends_on:
      mysql:
        condition: service_healthy
    networks:
      - webnet
    restart: unless-stopped

  mysql:
    image: mysql:8.0
    volumes:
      - mysql_data:/var/lib/mysql
      - ./mysql/init:/docker-entrypoint-initdb.d:ro
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s
    networks:
      - webnet
    restart: unless-stopped

networks:
  webnet:
    driver: bridge

volumes:
  mysql_data:

Key points about this configuration:

• Health checks on MySQL ensure PHP-FPM does not start until the database is actually ready to accept connections, not just when the container starts.
• Read-only mounts (:ro) are used for configuration files and SSL certificates as a security measure.
• Named volumes (mysql_data) persist the database data independently from the container lifecycle.
• restart: unless-stopped ensures services automatically recover from crashes but stay stopped if you manually stop them.

Practical Example 2: Monitoring Stack with Prometheus + Grafana

A monitoring stack is essential for any production environment:

services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    ports:
      - "9090:9090"
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=30d'
      - '--web.console.libraries=/etc/prometheus/console_libraries'
      - '--web.console.templates=/etc/prometheus/consoles'
    networks:
      - monitoring
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning:ro
    ports:
      - "3000:3000"
    environment:
      GF_SECURITY_ADMIN_USER: ${GRAFANA_ADMIN_USER:-admin}
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD}
      GF_USERS_ALLOW_SIGN_UP: "false"
    depends_on:
      - prometheus
    networks:
      - monitoring
    restart: unless-stopped

  node-exporter:
    image: prom/node-exporter:latest
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
    networks:
      - monitoring
    restart: unless-stopped

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    privileged: true
    devices:
      - /dev/kmsg:/dev/kmsg
    networks:
      - monitoring
    restart: unless-stopped

networks:
  monitoring:
    driver: bridge

volumes:
  prometheus_data:
  grafana_data:

This stack provides container-level metrics (cAdvisor), host-level metrics (Node Exporter), metric storage and querying (Prometheus), and visualization dashboards (Grafana).

Common Docker Compose Commands

Here is a comprehensive reference of the commands you will use most frequently:

# Start all services in the background
docker compose up -d

# Start and force rebuild of images
docker compose up -d --build

# Stop all services
docker compose down

# Stop and remove volumes (CAUTION: destroys data)
docker compose down -v

# View running services
docker compose ps

# View logs for all services
docker compose logs

# Follow logs for a specific service
docker compose logs -f nginx

# Restart a specific service
docker compose restart php

# Scale a service (run multiple instances)
docker compose up -d --scale worker=3

# Execute a command in a running service container
docker compose exec mysql mysql -u root -p

# Pull latest images for all services
docker compose pull

# View the resolved Compose configuration
docker compose config

# List all Compose projects running on the system
docker compose ls

Tip: The docker compose config command is extremely useful for debugging. It outputs the fully resolved YAML after variable substitution, showing you exactly what Docker Compose will execute.

Production Considerations

Restart Policies

Always set a restart policy for production services:

services:
  web:
    restart: unless-stopped  # Recommended for most services

Available policies:

• no (default): Never restart
• always: Always restart, even if manually stopped
• on-failure: Restart only if the container exits with a non-zero code
• unless-stopped: Restart unless explicitly stopped by the administrator

Health Checks

Health checks let Docker know if a service is actually functioning, not just running:

services:
  api:
    image: myapi:latest
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Health checks are used by depends_on conditions and by Docker to determine when to restart unhealthy containers.

Resource Limits

Prevent a single service from consuming all available system resources:

services:
  worker:
    image: myworker:latest
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 128M

Note: Resource limits under deploy.resources work with docker compose up in recent versions of Compose V2. In older versions, they only applied to Docker Swarm mode.

Logging Configuration

Configure per-service logging to prevent disk exhaustion:

services:
  web:
    image: nginx:latest
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

Security Best Practices

• Never run containers as root unless absolutely necessary. Use the user directive.
• Set read_only: true on containers that do not need to write to their filesystem.
• Drop unnecessary Linux capabilities using cap_drop: [ALL] and add back only what is needed with cap_add.
• Use secrets management for sensitive data instead of plain environment variables in production.

services:
  api:
    image: myapi:latest
    read_only: true
    user: "1000:1000"
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
    tmpfs:
      - /tmp

Troubleshooting Docker Compose

Service Cannot Connect to Another Service

If one service cannot reach another, verify they are on the same network and use the service name (not the container name) as the hostname:

# Check which networks a service is connected to
docker compose exec web ping database

# Inspect the network
docker network ls
docker network inspect <project_name>_default

Port Conflicts

If you see bind: address already in use, another process is using that port:

# Find what is using port 80
sudo lsof -i :80
# or
sudo ss -tlnp | grep :80

Change the host port mapping in your Compose file, for example from "80:80" to "8080:80".

Containers Keep Restarting

Check the logs to understand why a container is failing:

docker compose logs --tail=50 <service_name>

# Check the container's exit code
docker compose ps -a

Common causes include missing environment variables, incorrect file permissions on mounted volumes, and dependencies not being ready.

Slow Bind Mount Performance on macOS

If you are developing on macOS and experience slow file access with bind mounts, consider using Docker volumes instead of bind mounts for large directories, or use the :cached mount option:

volumes:
  - ./app:/var/www/html:cached

Conclusion

Docker Compose transforms the way system administrators deploy and manage multi-container applications. By defining your infrastructure in a declarative YAML file, you gain reproducibility, version control, and a single-command deployment workflow. The examples and best practices covered in this guide should give you a solid foundation for deploying everything from simple web stacks to comprehensive monitoring solutions.

For your next steps, consider exploring:

• Docker Compose profiles for managing different environments (development, staging, production)
• Docker Compose watch for automatic service updates during development
• Orchestration tools like Docker Swarm or Kubernetes for multi-host deployments

Make sure Docker is installed correctly on your system first. If you have not done so, follow our complete installation guide: How to Install Docker on Ubuntu 22.04 and 24.04.