Skip to content

Containers and Docker

Containers and Docker

Containers and Docker

Containers let you bundle an application plus its runtime dependencies (language runtime, libraries, OS packages) into an immutable image. You can then run that image the same way on a laptop, CI server, or production cluster. Think of an image as a versioned filesystem snapshot + start command.

1. Why Containers? (Problem → Solution)

Problem (Pre-Containers) Impact Container Benefit
"Works on my machine" differences Deployment failures Image has explicit dependency versions
Snowflake servers manually configured Drift over time Immutable rebuildable images
Scaling manual + slow Cannot react to load spikes fast Fast instantiation (seconds)
Mixed dependencies (Python 3.11 vs 3.10) Conflicts / breakages Isolated filesystem namespaces

Core idea: Build once (CI), run many times (dev/staging/prod) without rebuild.

2. Key Vocabulary

Term Definition
Dockerfile Declarative recipe to build an image layer by layer
Image Read-only layered snapshot (immutable)
Container Running instance of an image with writable layer
Registry Remote store for images (Docker Hub, GHCR, ECR)
Layer Cache Reuse of previously built steps to speed builds
Multi-stage Build Multiple FROM stages to discard build tools in final image

3. Layering Mental Model

Each Dockerfile instruction (mostly) creates a new layer. If a line changes, all subsequent layers must rebuild. 

graph TD
    A[FROM python:3.12-slim] --> B[RUN apt-get install build-essentials]
    B --> C[WORKDIR /app]
    C --> D[COPY requirements.txt .]
    D --> E[RUN pip install -r requirements.txt]
    E --> F[COPY src/ ./src]
    F --> G[CMD python -m app]
Place the most frequently changed instructions (like copying source code) later to maximize cache hits.

4. Minimal Python Example

FROM python:3.12-slim AS base
WORKDIR /app

# 1. Install only build deps needed for compilation (if any)
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
        && rm -rf /var/lib/apt/lists/*

# 2. Dependency layer (separate to leverage caching)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# 3. Copy application source last
COPY . .

ENV PORT=8000 PYTHONUNBUFFERED=1
EXPOSE 8000
CMD ["python", "main.py"]

Run locally: 

docker build -t myapp:local .
docker run --rm -p 8000:8000 --env PORT=8000 myapp:local

5. Multi-Stage Build (Node Example)

Keep the final runtime image lean by compiling/building in an earlier stage. 

FROM node:20-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev

FROM node:20-alpine AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build

FROM node:20-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build /app/dist ./dist
COPY package*.json ./
COPY --from=deps /app/node_modules ./node_modules
EXPOSE 3000
CMD ["node", "dist/index.js"]

6. Security & Hygiene

Practice Why Tip
Pin base image versions Avoid silent major upgrades python:3.12-slim not latest
Use slim/alpine base Smaller attack surface Verify needed libc features
Drop root privileges Limit damage if exploited USER app after creating non-root user
Scan images Catch CVEs early Trivy, Grype in CI
Minimal layers Reduce size & complexity Combine related RUN commands
.dockerignore Faster builds + no secrets Exclude .git, tests, local data

Example .dockerignore: 

.git
node_modules
__pycache__
*.log
.env
tests

7. Environment Variables & Config

12-Factor principle: config outside the image. Supply secrets and environment-specific values (DATABASE_URL, API_KEY) at runtime, not baked into the image. For local dev: .env file + --env-file (never commit secrets).

8. Persistent Data & State

Containers themselves are ephemeral. Use volumes for: * Databases (during development only; in prod use managed services or stateful sets) * Caches you don't want to rebuild constantly

Example: docker run -v mydata:/var/lib/postgresql/data postgres:16

9. Networking Basics

  • Default bridge network: containers reachable by IP.
  • Named networks: service discovery via container names.
  • Ports: -p 8000:8000 maps host:container.

docker compose simplifies multi-container dev (app + db + cache) with a YAML file describing services and networks.

10. Observability in Containers

  • Send logs to stdout/stderr (container platform collects).
  • Health checks: HEALTHCHECK CMD curl -f http://localhost:8000/health || exit 1.
  • Add minimal metrics endpoint or integrate OpenTelemetry for richer traces.

11. Common Pitfalls

Pitfall Symptom Fix
Copying entire repo early Rebuilds slow for small changes Copy only manifests (package.json, requirements.txt) first
Using latest tag Unpredictable prod behavior Pin versions + renovate updates
Shipping dev dependencies Bigger images, security risk Multi-stage & prune dev deps
Storing secrets in image Secret leakage Pass via env vars / secret manager
Running everything as root Escalation risk Create user & switch

12. Pushing to a Registry (Flow)

sequenceDiagram
    participant Dev
    participant CI as CI Pipeline
    participant Reg as Container Registry
    participant Prod

    Dev->>CI: Push commit (Dockerfile)
    CI->>CI: Build & tag (myapp:1.2.0)
    CI->>Reg: Push image layers
    Prod->>Reg: Pull myapp:1.2.0
    Prod->>Prod: Run container(s)

Tagging strategy: myapp:<git-sha> for unique builds + myapp:1.2.0 for releases + myapp:latest pointing to stable if desired.

13. Performance Tips

  • Use smaller base (e.g., alpine, distroless for final stage).
  • Leverage build cache: order instructions from least to most frequently changing.
  • Clean build artifacts: rm -rf /var/lib/apt/lists/* after apt installs.
  • Multi-arch builds via docker buildx if you target ARM + x86.

14. Minimal Compose File Example

version: '3.9'
services:
    api:
        build: .
        ports:
            - "8000:8000"
        environment:
            DATABASE_URL: postgres://postgres:postgres@db:5432/app
    db:
        image: postgres:16
        environment:
            POSTGRES_PASSWORD: postgres
        volumes:
            - pgdata:/var/lib/postgresql/data
volumes:
    pgdata:

Run: docker compose up --build.

15. Checklist

  • .dockerignore excludes large/unneeded files
  • Multi-stage build keeps runtime image lean
  • Non-root user configured (if feasible)
  • Base image version pinned
  • Healthcheck defined (or readiness probe in orchestration)
  • Environment config passed at run time
  • Image scanned for CVEs in CI
  • Build cache effective (manifests copied before source)
  • Logs to stdout/stderr (no local file writes)
  • Tagging strategy documented

16. Further Resources

17. Quick Reference (Cheat Table)

Requirement Dockerfile Hint
Fast rebuilds Separate deps & source layers
Smaller image Multi-stage + slim base
Security Non-root, scan, pin versions
Deterministic Avoid apt-get upgrade without version pins
Configurable Use ENV + runtime env vars