Scaling Applications in the Cloud: Kubernetes and GitOps in Practice

Container orchestration transforms how we deploy and manage applications. This guide covers the fundamentals of Docker, Vagrant, K3s, and ArgoCD - from core concepts to production patterns.

Docker Fundamentals

Docker packages applications with their dependencies into portable, isolated units called containers. Unlike virtual machines, containers share the host kernel, making them lightweight and fast.

Containers vs Virtual Machines

CONTAINERS                          VIRTUAL MACHINES

┌─────────┐ ┌─────────┐            ┌─────────┐ ┌─────────┐
│  App A  │ │  App B  │            │  App A  │ │  App B  │
├─────────┤ ├─────────┤            ├─────────┤ ├─────────┤
│  Libs   │ │  Libs   │            │  Libs   │ │  Libs   │
└────┬────┘ └────┬────┘            ├─────────┤ ├─────────┤
     │           │                 │ Guest OS│ │ Guest OS│
┌────┴───────────┴────┐            └────┬────┘ └────┬────┘
│   Container Engine  │                 │           │
├─────────────────────┤            ┌────┴───────────┴────┐
│     Host Kernel     │            │     Hypervisor      │
├─────────────────────┤            ├─────────────────────┤
│      Host OS        │            │      Host OS        │
└─────────────────────┘            └─────────────────────┘

Startup: Milliseconds              Startup: Minutes
Size: MBs                          Size: GBs
Isolation: Process-level           Isolation: Hardware-level

When to use containers: Microservices, CI/CD pipelines, development environments, stateless applications.

When to use VMs: Running different OS kernels, strong security isolation, legacy applications.

Core Concepts

Images are read-only templates containing application code, runtime, libraries, and configuration. They use a layered filesystem where each instruction creates a new layer.

Containers are running instances of images. They add a writable layer on top of the image layers. Multiple containers can share the same image layers, saving disk space.

Volumes persist data beyond container lifecycle. Data in the writable container layer is lost when the container is removed.

Image Layers and Caching

Docker caches each layer. If a layer hasn't changed, Docker reuses it. This is why instruction order matters:

# Bad: Copying code before dependencies invalidates cache on every code change
FROM node:18
COPY . .
RUN npm install

# Good: Dependencies cached separately from code
FROM node:18
COPY package*.json .
RUN npm install
COPY . .
    

The second approach only re-runs npm install when package.json changes.

Dockerfile Instructions

FROM debian:buster          # Base image to build upon

WORKDIR /app                # Set working directory for subsequent instructions

COPY ./src /app/src         # Copy files from build context to image
ADD archive.tar.gz /app/    # Like COPY, but auto-extracts archives

RUN apt-get update && \
    apt-get install -y nginx  # Execute commands during build

ENV NODE_ENV=production     # Set environment variables
ARG VERSION=1.0             # Build-time variables (not in final image)

EXPOSE 80                   # Document which ports the container listens on

VOLUME /data                # Create mount point for persistent data

ENTRYPOINT ["nginx"]        # Command that always runs
CMD ["-g", "daemon off;"]   # Default arguments (can be overridden)
    

ENTRYPOINT vs CMD

Understanding the difference is crucial:

• CMD: Default command, completely replaced if user provides arguments
• ENTRYPOINT: Always runs, user arguments are appended
• Combined: ENTRYPOINT defines the executable, CMD provides default arguments

ENTRYPOINT ["python"]
CMD ["app.py"]
# Runs: python app.py
# User runs: docker run myimage script.py
# Executes: python script.py (CMD replaced, ENTRYPOINT stays)
    

The PID 1 Problem

In containers, your application runs as PID 1 (init process). This creates two problems:

1. Signal handling: PID 1 doesn't receive default signal handlers. SIGTERM won't kill your app unless you explicitly handle it.
2. Zombie reaping: PID 1 must reap orphaned child processes, or they become zombies.

Solutions:

• Use --init flag: docker run --init myimage
• Use dumb-init or tini as entrypoint
• Handle signals in your application

Container Lifecycle

                    docker create
    [Image] ──────────────────────► [Created]
                                        │
                                        │ docker start
                                        ▼
                ┌──────────────────► [Running] ◄──────────────────┐
                │                       │                         │
                │ docker restart        │ docker stop/kill        │ docker start
                │                       ▼                         │
                └─────────────────── [Stopped] ───────────────────┘
                                        │
                                        │ docker rm
                                        ▼
                                    [Removed]

Docker Networking

Docker provides several network drivers:

• bridge (default): Containers on same bridge can communicate. Isolated from host network.
• host: Container shares host's network stack. No isolation, but no NAT overhead.
• none: No networking. Complete isolation.
• overlay: Multi-host networking for Docker Swarm.

Containers on the same user-defined bridge network can resolve each other by container name (Docker's internal DNS):

# Create network
docker network create mynet

# Containers can reach each other by name
docker run --name db --network mynet postgres
docker run --name app --network mynet myapp
# app can connect to "db:5432"
    

Volume Types

• Named volumes: Managed by Docker, persist in Docker's storage area
• Bind mounts: Map host directory to container path
• tmpfs: Stored in memory only, never written to disk

# Named volume
docker run -v mydata:/app/data myimage

# Bind mount
docker run -v /host/path:/container/path myimage

# tmpfs (secrets, sensitive data)
docker run --tmpfs /app/secrets myimage
    

Docker Compose

Docker Compose defines multi-container applications in a single YAML file. It handles networking, volumes, and service dependencies.

Core Structure

version: '3.8'

services:
  web:
    build: ./web
    ports:
      - "80:80"
    depends_on:
      - api
    environment:
      - API_URL=http://api:3000
    networks:
      - frontend
      - backend

  api:
    image: node:18
    volumes:
      - ./api:/app
      - node_modules:/app/node_modules
    networks:
      - backend

  db:
    image: postgres:15
    volumes:
      - db_data:/var/lib/postgresql/data
    networks:
      - backend

networks:
  frontend:
  backend:

volumes:
  db_data:
  node_modules:
    

Service Configuration

Build vs Image:

• build: Build from Dockerfile
• image: Pull from registry
• Both: Build and tag with that name

Dependencies:

• depends_on: Controls startup order only, not readiness
• For readiness, use health checks or wait scripts

services:
  api:
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
    

Restart Policies:

• no: Never restart (default)
• always: Always restart
• on-failure: Restart only on non-zero exit
• unless-stopped: Like always, but respects manual stops

Networking in Compose

Compose creates a default network for your app. All services can reach each other by service name:

services:
  web:
    # Can connect to "db:5432" - service name is the hostname
  db:
    image: postgres
    

Custom networks provide isolation between service groups:

services:
  frontend:
    networks:
      - public
  api:
    networks:
      - public
      - private
  db:
    networks:
      - private  # Not accessible from frontend

networks:
  public:
  private:
    

Environment Variables

Precedence (highest to lowest):

1. Shell environment variables
2. environment: in compose file
3. env_file: contents
4. Dockerfile ENV

services:
  app:
    env_file:
      - .env           # Loaded first
      - .env.local     # Overrides .env
    environment:
      - DEBUG=true     # Overrides env_file
    

Vagrant Fundamentals

Vagrant creates reproducible virtual machine environments. It abstracts away provider differences (VirtualBox, VMware, AWS) behind a simple configuration file.

Core Concepts

• Box: Pre-packaged VM image (like Docker image for VMs)
• Provider: Virtualization backend (VirtualBox, VMware, Hyper-V, cloud)
• Provisioner: Tool to configure VM after boot (shell, Ansible, Puppet)
• Synced Folder: Shared directory between host and guest

Vagrantfile Structure

Vagrant.configure("2") do |config|
  # Base box
  config.vm.box = "ubuntu/focal64"

  # Network configuration
  config.vm.network "private_network", ip: "192.168.56.10"
  config.vm.network "forwarded_port", guest: 80, host: 8080

  # Synced folders
  config.vm.synced_folder "./data", "/vagrant_data"

  # Provider-specific settings
  config.vm.provider "virtualbox" do |vb|
    vb.memory = 2048
    vb.cpus = 2
  end

  # Provisioning
  config.vm.provision "shell", inline: <<-SHELL
    apt-get update
    apt-get install -y nginx
  SHELL
end
    

Networking Modes

• NAT (default): VM can access internet, host can't access VM directly
• Forwarded ports: Map host port to guest port through NAT
• Private network: Host-only network, VMs can communicate with each other and host
• Public network: Bridged to physical network, VM gets IP from router

Multi-Machine Setup

Vagrant excels at creating multi-node environments:

Vagrant.configure("2") do |config|
  config.vm.define "master" do |master|
    master.vm.box = "ubuntu/focal64"
    master.vm.network "private_network", ip: "192.168.56.10"
    master.vm.provision "shell", path: "scripts/master.sh"
  end

  config.vm.define "worker" do |worker|
    worker.vm.box = "ubuntu/focal64"
    worker.vm.network "private_network", ip: "192.168.56.11"
    worker.vm.provision "shell", path: "scripts/worker.sh"
  end
end
    

Synced folders enable simple inter-VM communication (e.g., sharing tokens, certificates).

Essential Commands

vagrant up              # Create and start VMs
vagrant halt            # Stop VMs (preserves state)
vagrant destroy         # Delete VMs completely
vagrant ssh [name]      # SSH into VM
vagrant provision       # Re-run provisioners
vagrant reload          # Restart with new Vagrantfile settings
vagrant status          # Show VM states
    

Kubernetes Fundamentals (K3s)

Kubernetes orchestrates containerized applications across a cluster of machines. K3s is a lightweight, certified Kubernetes distribution packaged as a single binary under 100MB.

Why K3s?

KUBERNETES (K8s)                    K3s

┌─────────────────────┐            ┌─────────────────────┐
│ etcd (separate)     │            │                     │
├─────────────────────┤            │   Single Binary     │
│ API Server          │            │   ┌───────────────┐ │
├─────────────────────┤            │   │ SQLite/etcd   │ │
│ Controller Manager  │     ──►    │   │ API Server    │ │
├─────────────────────┤            │   │ Controller    │ │
│ Scheduler           │            │   │ Scheduler     │ │
├─────────────────────┤            │   └───────────────┘ │
│ Cloud Controller    │            │                     │
└─────────────────────┘            └─────────────────────┘

Memory: 1GB+ per component         Memory: 512MB total
Setup: Complex                     Setup: Single command

K3s includes batteries: containerd, Flannel (CNI), CoreDNS, Traefik (Ingress), ServiceLB.

Core Concepts

Cluster Architecture:

• Control Plane (Master): API Server, Scheduler, Controller Manager, etcd
• Worker Nodes: Run workloads, managed by kubelet

Workload Resources:

• Pod: Smallest deployable unit. One or more containers sharing network/storage.
• Deployment: Manages ReplicaSets, handles rolling updates and rollbacks.
• ReplicaSet: Ensures specified number of pod replicas are running.
• StatefulSet: For stateful apps needing stable network identity and persistent storage.
• DaemonSet: Runs a pod on every node (monitoring, logging agents).

Pod Lifecycle

[Pending] ──► [Running] ──► [Succeeded/Failed]
    │              │
    │              ▼
    │         [CrashLoopBackOff]
    │              │
    └──────────────┘
         restart

• Pending: Accepted but not scheduled (no node available, pulling images)
• Running: Bound to node, at least one container running
• Succeeded: All containers terminated successfully
• Failed: All containers terminated, at least one failed
• CrashLoopBackOff: Container keeps crashing, Kubernetes backs off restarts

Service Types

Services provide stable network endpoints for pods:

• ClusterIP (default): Internal cluster IP only. Pods can reach it, external traffic cannot.
• NodePort: Exposes on each node's IP at a static port (30000-32767).
• LoadBalancer: Provisions external load balancer (cloud providers).
• ExternalName: Maps service to external DNS name.

                   External Traffic
                         │
                         ▼
┌───────────────────────────────────────────┐
│              LoadBalancer                 │
│        (Cloud Provider / MetalLB)         │
└───────────────────────────────────────────┘
                         │
                         ▼
┌───────────────────────────────────────────┐
│             NodePort :30080               │
│         (Every Node in Cluster)           │
└───────────────────────────────────────────┘
                         │
                         ▼
┌───────────────────────────────────────────┐
│          ClusterIP (10.96.x.x)            │
│            (Internal Only)                │
└───────────────────────────────────────────┘
                         │
                         ▼
                   ┌─────────┐
                   │  Pods   │
                   └─────────┘

Labels and Selectors

Labels are key-value pairs attached to objects. Selectors filter objects by labels:

# Deployment creates pods with labels
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: web        # Deployment manages pods with this label
  template:
    metadata:
      labels:
        app: web      # Pods get this label
        tier: frontend
    spec:
      containers:
      - name: nginx
        image: nginx
---
# Service selects pods by label
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web          # Routes traffic to pods with app=web
  ports:
  - port: 80
    

Ingress

Ingress manages external HTTP/HTTPS access to services. It provides:

• Host-based routing (app1.example.com → service1)
• Path-based routing (/api → api-service, /web → web-service)
• TLS termination

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
spec:
  ingressClassName: traefik
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80
  - host: web.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80
    

ConfigMaps and Secrets

Decouple configuration from container images:

# ConfigMap for non-sensitive data
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  DATABASE_HOST: "postgres"
  LOG_LEVEL: "info"
---
# Secret for sensitive data (base64 encoded)
apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
type: Opaque
data:
  DATABASE_PASSWORD: cGFzc3dvcmQxMjM=  # base64 encoded
---
# Using in Pod
spec:
  containers:
  - name: app
    envFrom:
    - configMapRef:
        name: app-config
    - secretRef:
        name: app-secrets
    

Resource Limits

Always set resource requests and limits for production:

spec:
  containers:
  - name: app
    resources:
      requests:        # Scheduler uses for placement
        memory: "128Mi"
        cpu: "250m"    # 0.25 CPU cores
      limits:          # Container cannot exceed
        memory: "256Mi"
        cpu: "500m"
    

Essential kubectl Commands

# Cluster info
kubectl cluster-info
kubectl get nodes

# Workloads
kubectl get pods -A                    # All namespaces
kubectl get deployments
kubectl describe pod             # Detailed info
kubectl logs  -f                  # Follow logs
kubectl exec -it  -- /bin/sh      # Shell into container

# Apply/Delete
kubectl apply -f manifest.yaml
kubectl delete -f manifest.yaml

# Debugging
kubectl get events --sort-by=.lastTimestamp
kubectl top pods                       # Resource usage
    

K3d: K3s in Docker

K3d runs K3s clusters inside Docker containers. Faster than VMs, ideal for local development and CI/CD.

When to Use

• K3s in VMs (Vagrant): Simulates production, tests node failures, network policies
• K3d (Docker): Fast iteration, CI pipelines, limited resources

# Create cluster with port mapping
k3d cluster create dev -p "8080:80@loadbalancer"

# Multi-node cluster
k3d cluster create prod --servers 3 --agents 2

# With local registry
k3d cluster create dev --registry-create dev-registry:5000

# List and delete
k3d cluster list
k3d cluster delete dev
    

GitOps with ArgoCD

GitOps uses Git as the single source of truth for infrastructure and applications. ArgoCD continuously syncs cluster state to match Git.

GitOps Principles

1. Declarative: Entire system described declaratively (YAML manifests)
2. Versioned: Desired state stored in Git with full history
3. Automated: Changes automatically applied to cluster
4. Continuously Reconciled: Agents ensure cluster matches Git

TRADITIONAL                         GITOPS

Developer ──► CI ──► kubectl        Developer ──► Git ──► ArgoCD ──► Cluster
                        │                          │           │
                        ▼                          ▼           ▼
                    Cluster              Audit Trail    Self-Healing

Problems:                           Benefits:
- Who deployed what?                - Full audit trail
- Manual rollbacks                  - Git revert = Rollback
- Drift between envs                - Consistent environments

ArgoCD Architecture

Core Concepts

Sync Strategies

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/repo
    targetRevision: HEAD
    path: k8s/overlays/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true       # Delete removed resources
      selfHeal: true    # Revert manual changes
    syncOptions:
    - CreateNamespace=true
    

Application Health

ArgoCD tracks health status of all resources:

Rollback

GitOps makes rollback trivial:

# Option 1: Git revert
git revert HEAD
git push
# ArgoCD automatically syncs

# Option 2: ArgoCD CLI
argocd app rollback myapp

# Option 3: Sync to specific revision
argocd app sync myapp --revision abc123
    

Putting It All Together

CONTAINER ORCHESTRATION STACK

┌────────────────────────────────────────────────────────────┐
│                          GitOps                            │
│    Git Repo ──► ArgoCD ──► Continuous Reconciliation       │
└────────────────────────────────────────────────────────────┘
                             │
                             ▼
┌────────────────────────────────────────────────────────────┐
│                        Kubernetes                          │
│      Pods ◄── Deployments ◄── Services ◄── Ingress         │
└────────────────────────────────────────────────────────────┘
                             │
                             ▼
┌────────────────────────────────────────────────────────────┐
│                     Container Runtime                      │
│     Docker/containerd ──► Images ──► Containers            │
└────────────────────────────────────────────────────────────┘
                             │
                             ▼
┌────────────────────────────────────────────────────────────┐
│                       Infrastructure                       │
│          VMs (Vagrant) / Cloud / Bare Metal                │
└────────────────────────────────────────────────────────────┘

Key Takeaways

• Docker: Package applications consistently. Understand layers, networking, and volumes.
• Compose: Define multi-container apps. Use health checks and proper dependency management.
• Vagrant: Reproducible VMs for development and testing multi-node setups.
• Kubernetes: Orchestrate at scale. Master Pods, Services, Deployments, and Ingress.
• GitOps: Git as source of truth. Automated, auditable, reversible deployments.

When to Use What

Conclusion

Container orchestration is not a single tool but a layered approach to application deployment. Each layer solves specific problems:

Docker solves the "works on my machine" problem by packaging applications with their dependencies. Understanding image layers, networking modes, and volume types is essential for building efficient, production-ready containers.

Docker Compose extends this to multi-container applications, handling service dependencies, networking, and configuration in a single declarative file. Health checks and proper restart policies make the difference between development setups and production-grade deployments.

Vagrant remains valuable when you need full VM isolation, testing multi-node clusters, or simulating production infrastructure locally. Its provider abstraction means the same Vagrantfile works across VirtualBox, VMware, and cloud providers.

Kubernetes is the industry standard for container orchestration at scale. The learning curve is steep, but understanding the core primitives—Pods, Services, Deployments, and Ingress—unlocks powerful deployment patterns. K3s makes this accessible without the operational overhead of full Kubernetes.

GitOps with ArgoCD completes the picture by making deployments declarative, auditable, and reversible. When Git is your source of truth, rollbacks become git reverts, and your infrastructure history lives in version control.

Start with Docker for single applications. Move to Compose when you need multiple services. Consider Kubernetes when you need scaling, self-healing, and advanced deployment strategies. Add GitOps when you need audit trails and automated reconciliation. Each tool builds on the previous, and understanding the fundamentals makes adopting the next layer straightforward.