Docker Compose to Kubernetes Migration

This comprehensive guide covers migrating applications from Docker Compose to Kubernetes, including both developer workflows and operational procedures.

Overview

Docker Compose is excellent for local development and simple deployments, but Kubernetes provides enterprise-grade orchestration, scaling, and management capabilities. This guide helps you migrate your Docker Compose applications to Kubernetes.

Core Concepts and Mapping

Docker Compose → Kubernetes Equivalents

Docker Compose	Kubernetes	Purpose
`services`	`Deployment` + `Service`	Application containers
`networks`	`Service` + `NetworkPolicy`	Service communication
`volumes`	`PersistentVolumeClaim`	Persistent storage
`environment`	`ConfigMap` + `Secret`	Configuration
`ports`	`Service` + `Ingress`	External access
`depends_on`	`initContainers`	Startup order

Example Mapping

Docker Compose:

version: '3.8'
services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
    environment:
      - NGINX_HOST=localhost
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - db
  
  db:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_PASSWORD=secret
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Kubernetes Equivalent:

# Database Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        env:
        - name: POSTGRES_DB
          value: "myapp"
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
      volumes:
      - name: postgres-storage
        persistentVolumeClaim:
          claimName: postgres-pvc
---
# Web Application Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
        env:
        - name: NGINX_HOST
          value: "localhost"
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-config
---
# Services
apiVersion: v1
kind: Service
metadata:
  name: postgres-service
spec:
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
  - port: 80
    targetPort: 80
  type: LoadBalancer
---
# Storage
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
# Configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  nginx.conf: |
    events {
      worker_connections 1024;
    }
    http {
      server {
        listen 80;
        location / {
          root /usr/share/nginx/html;
        }
      }
    }
---
# Secrets
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
type: Opaque
data:
  password: <base64-encoded-password>

Manual Conversion Process

Step 1: Analyze Your Docker Compose File

Identify Components:

# Analyze your compose file
docker-compose config

# List all services
docker-compose ps

# Check service dependencies
docker-compose config --services

Step 2: Create Kubernetes Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: myapp
  labels:
    name: myapp

Step 3: Convert Services to Deployments

For each service in your compose file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: service-name
  namespace: myapp
spec:
  replicas: 3  # Production-ready replication
  selector:
    matchLabels:
      app: service-name
  template:
    metadata:
      labels:
        app: service-name
    spec:
      containers:
      - name: service-name
        image: your-image:tag
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

Step 4: Create Services for Networking

apiVersion: v1
kind: Service
metadata:
  name: service-name
  namespace: myapp
spec:
  selector:
    app: service-name
  ports:
  - name: http
    port: 80
    targetPort: 8080
  type: ClusterIP  # Internal communication

Step 5: Handle External Access

LoadBalancer Service:

apiVersion: v1
kind: Service
metadata:
  name: web-service
  namespace: myapp
spec:
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
  type: LoadBalancer

Ingress Controller:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  namespace: myapp
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Step 6: Manage Configuration

ConfigMaps for Environment Variables:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: myapp
data:
  DATABASE_URL: "postgresql://postgres:5432/myapp"
  REDIS_URL: "redis://redis:6379"
  LOG_LEVEL: "INFO"

Secrets for Sensitive Data:

apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
  namespace: myapp
type: Opaque
data:
  DATABASE_PASSWORD: <base64-encoded>
  API_KEY: <base64-encoded>

Step 7: Handle Persistent Storage

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-storage
  namespace: myapp
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: rook-ceph-block

Automated Conversion Tools

1. Kompose Tool

Installation:

# Download Kompose
curl -L https://github.com/kubernetes/kompose/releases/latest/download/kompose-linux-amd64 -o kompose
chmod +x kompose
sudo mv ./kompose /usr/local/bin/kompose

Basic Conversion:

# Convert Docker Compose to Kubernetes
kompose convert

# Convert with specific output format
kompose convert -f docker-compose.yml -o k8s/

# Convert with custom namespace
kompose convert --namespace myapp

Advanced Kompose Options:

# Convert with specific controller type
kompose convert --controller deployment

# Convert with custom labels
kompose convert --labels app=myapp,env=production

# Convert with custom annotations
kompose convert --annotations ingress.kubernetes.io/rewrite-target=/

2. Katenary Tool

Installation:

# Install Katenary
npm install -g katenary

# Or use Docker
docker run --rm -v $(pwd):/workspace katenary/katenary

Usage:

# Convert Docker Compose
katenary convert docker-compose.yml

# Convert with custom output
katenary convert docker-compose.yml --output k8s/

3. Custom Conversion Scripts

Python Script Example:

#!/usr/bin/env python3
import yaml
import sys

def convert_compose_to_k8s(compose_file):
    with open(compose_file, 'r') as f:
        compose = yaml.safe_load(f)
    
    k8s_resources = []
    
    for service_name, service_config in compose['services'].items():
        # Create Deployment
        deployment = {
            'apiVersion': 'apps/v1',
            'kind': 'Deployment',
            'metadata': {'name': service_name},
            'spec': {
                'replicas': 3,
                'selector': {'matchLabels': {'app': service_name}},
                'template': {
                    'metadata': {'labels': {'app': service_name}},
                    'spec': {
                        'containers': [{
                            'name': service_name,
                            'image': service_config['image'],
                            'ports': [{'containerPort': p} for p in service_config.get('ports', [])]
                        }]
                    }
                }
            }
        }
        k8s_resources.append(deployment)
        
        # Create Service
        service = {
            'apiVersion': 'v1',
            'kind': 'Service',
            'metadata': {'name': service_name},
            'spec': {
                'selector': {'app': service_name},
                'ports': [{'port': 80, 'targetPort': 8080}],
                'type': 'ClusterIP'
            }
        }
        k8s_resources.append(service)
    
    return k8s_resources

if __name__ == '__main__':
    resources = convert_compose_to_k8s(sys.argv[1])
    for resource in resources:
        print('---')
        print(yaml.dump(resource, default_flow_style=False))

Practical Examples and Best Practices

Example 1: Multi-Tier Web Application

Docker Compose:

version: '3.8'
services:
  frontend:
    image: nginx:alpine
    ports:
      - "80:80"
    depends_on:
      - backend
  
  backend:
    image: node:16-alpine
    environment:
      - DATABASE_URL=postgresql://postgres:5432/myapp
    depends_on:
      - postgres
  
  postgres:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_PASSWORD=secret
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Kubernetes Equivalent:

# Namespace
apiVersion: v1
kind: Namespace
metadata:
  name: webapp
---
# Database
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: webapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        env:
        - name: POSTGRES_DB
          value: "myapp"
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
      volumes:
      - name: postgres-storage
        persistentVolumeClaim:
          claimName: postgres-pvc
---
# Backend API
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
  namespace: webapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: backend
  template:
    metadata:
      labels:
        app: backend
    spec:
      containers:
      - name: backend
        image: node:16-alpine
        env:
        - name: DATABASE_URL
          value: "postgresql://postgres-service:5432/myapp"
        ports:
        - containerPort: 3000
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
        readinessProbe:
          httpGet:
            path: /ready
            port: 3000
---
# Frontend
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: webapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      containers:
      - name: frontend
        image: nginx:alpine
        ports:
        - containerPort: 80
---
# Services
apiVersion: v1
kind: Service
metadata:
  name: postgres-service
  namespace: webapp
spec:
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: backend-service
  namespace: webapp
spec:
  selector:
    app: backend
  ports:
  - port: 80
    targetPort: 3000
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: frontend-service
  namespace: webapp
spec:
  selector:
    app: frontend
  ports:
  - port: 80
    targetPort: 80
  type: LoadBalancer
---
# Storage
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  namespace: webapp
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
# Secrets
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
  namespace: webapp
type: Opaque
data:
  password: <base64-encoded-password>

Example 2: Microservices with Message Queue

Docker Compose:

version: '3.8'
services:
  producer:
    image: producer:latest
    environment:
      - REDIS_URL=redis://redis:6379
  
  consumer:
    image: consumer:latest
    environment:
      - REDIS_URL=redis://redis:6379
    depends_on:
      - redis
  
  redis:
    image: redis:alpine
    ports:
      - "6379:6379"

Kubernetes Equivalent:

# Redis Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:alpine
        ports:
        - containerPort: 6379
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
---
# Producer Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: producer
spec:
  replicas: 2
  selector:
    matchLabels:
      app: producer
  template:
    metadata:
      labels:
        app: producer
    spec:
      containers:
      - name: producer
        image: producer:latest
        env:
        - name: REDIS_URL
          value: "redis://redis-service:6379"
---
# Consumer Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: consumer
spec:
  replicas: 3
  selector:
    matchLabels:
      app: consumer
  template:
    metadata:
      labels:
        app: consumer
    spec:
      containers:
      - name: consumer
        image: consumer:latest
        env:
        - name: REDIS_URL
          value: "redis://redis-service:6379"
---
# Services
apiVersion: v1
kind: Service
metadata:
  name: redis-service
spec:
  selector:
    app: redis
  ports:
  - port: 6379
    targetPort: 6379
  type: ClusterIP

Operational Runbook

Prerequisites and Setup

1. Install kubectl:

# Download kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/

2. Configure kubectl:

# Set cluster context
kubectl config set-cluster my-cluster --server=https://your-cluster-api

# Set credentials
kubectl config set-credentials my-user --token=your-token

# Set context
kubectl config set-context my-context --cluster=my-cluster --user=my-user

# Use context
kubectl config use-context my-context

3. Verify Setup:

# Test connection
kubectl cluster-info

# Check nodes
kubectl get nodes

# Check namespaces
kubectl get namespaces

Converting with Kompose

1. Install Kompose:

# Download and install
curl -L https://github.com/kubernetes/kompose/releases/latest/download/kompose-linux-amd64 -o kompose
chmod +x kompose
sudo mv ./kompose /usr/local/bin/kompose

2. Basic Conversion:

# Navigate to your project directory
cd /path/to/your/project

# Convert Docker Compose
kompose convert

# This creates multiple YAML files
ls *.yaml

3. Customize Conversion:

# Convert with specific namespace
kompose convert --namespace production

# Convert with custom labels
kompose convert --labels app=myapp,env=prod

# Convert with specific controller type
kompose convert --controller deployment

4. Apply to Cluster:

# Create namespace first
kubectl create namespace myapp

# Apply all generated files
kubectl apply -f .

# Or apply specific files
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml

Manual Kubernetes Resource Creation

1. Create Namespace:

kubectl create namespace myapp

2. Create ConfigMaps and Secrets:

# Create ConfigMap from file
kubectl create configmap app-config --from-file=config.properties -n myapp

# Create Secret
kubectl create secret generic app-secrets \
  --from-literal=db-password=secret123 \
  --from-literal=api-key=abc123 \
  -n myapp

3. Create Storage:

# Create PVC
kubectl apply -f pvc.yaml -n myapp

# Verify PVC
kubectl get pvc -n myapp

4. Deploy Applications:

# Deploy database first
kubectl apply -f postgres-deployment.yaml -n myapp

# Wait for database to be ready
kubectl wait --for=condition=ready pod -l app=postgres -n myapp

# Deploy application
kubectl apply -f app-deployment.yaml -n myapp

5. Create Services:

# Create internal services
kubectl apply -f internal-services.yaml -n myapp

# Create external service
kubectl apply -f external-service.yaml -n myapp

6. Configure Ingress:

# Apply ingress rules
kubectl apply -f ingress.yaml -n myapp

# Verify ingress
kubectl get ingress -n myapp

Advanced Topics and Cleanup

1. Horizontal Pod Autoscaling:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa
  namespace: myapp
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

2. Network Policies:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: app-network-policy
  namespace: myapp
spec:
  podSelector:
    matchLabels:
      app: myapp
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: frontend
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: database
    ports:
    - protocol: TCP
      port: 5432

3. Resource Quotas:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: app-quota
  namespace: myapp
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    persistentvolumeclaims: "5"

4. Cleanup Procedures:

# Delete all resources in namespace
kubectl delete namespace myapp

# Or delete specific resources
kubectl delete deployment app-deployment -n myapp
kubectl delete service app-service -n myapp
kubectl delete pvc app-storage -n myapp

# Clean up persistent volumes
kubectl delete pv --all

# Remove namespace
kubectl delete namespace myapp

Troubleshooting

Common Issues

1. Pod Startup Issues:

# Check pod status
kubectl get pods -n myapp

# Check pod logs
kubectl logs pod-name -n myapp

# Describe pod for details
kubectl describe pod pod-name -n myapp

2. Service Communication Issues:

# Check service endpoints
kubectl get endpoints -n myapp

# Test service connectivity
kubectl run test-pod --image=busybox --rm -it --restart=Never -- nslookup service-name

# Check service configuration
kubectl describe service service-name -n myapp

3. Storage Issues:

# Check PVC status
kubectl get pvc -n myapp

# Check PV status
kubectl get pv

# Describe PVC for details
kubectl describe pvc pvc-name -n myapp

4. Configuration Issues:

# Check ConfigMap
kubectl get configmap -n myapp
kubectl describe configmap config-name -n myapp

# Check Secrets
kubectl get secrets -n myapp
kubectl describe secret secret-name -n myapp

Summary

Migrating from Docker Compose to Kubernetes involves:

Understanding the mapping between Compose and Kubernetes concepts
Manual conversion for full control and customization
Automated tools like Kompose for quick conversion
Operational procedures for deployment and management
Troubleshooting skills for common issues

The key is to start simple, understand the differences, and gradually adopt Kubernetes-native features like auto-scaling, rolling updates, and advanced networking.

Overview​

Core Concepts and Mapping​

Docker Compose → Kubernetes Equivalents​

Example Mapping​

Manual Conversion Process​

Step 1: Analyze Your Docker Compose File​

Step 2: Create Kubernetes Namespace​

Step 3: Convert Services to Deployments​

Step 4: Create Services for Networking​

Step 5: Handle External Access​

Step 6: Manage Configuration​

Step 7: Handle Persistent Storage​

Automated Conversion Tools​

1. Kompose Tool​

2. Katenary Tool​

3. Custom Conversion Scripts​

Practical Examples and Best Practices​

Example 1: Multi-Tier Web Application​

Example 2: Microservices with Message Queue​

Operational Runbook​

Prerequisites and Setup​

Converting with Kompose​

Manual Kubernetes Resource Creation​

Advanced Topics and Cleanup​

Troubleshooting​

Common Issues​

Summary​

Overview

Core Concepts and Mapping

Docker Compose → Kubernetes Equivalents

Example Mapping

Manual Conversion Process

Step 1: Analyze Your Docker Compose File

Step 2: Create Kubernetes Namespace

Step 3: Convert Services to Deployments

Step 4: Create Services for Networking

Step 5: Handle External Access

Step 6: Manage Configuration

Step 7: Handle Persistent Storage

Automated Conversion Tools

1. Kompose Tool

2. Katenary Tool

3. Custom Conversion Scripts

Practical Examples and Best Practices

Example 1: Multi-Tier Web Application

Example 2: Microservices with Message Queue

Operational Runbook

Prerequisites and Setup

Converting with Kompose

Manual Kubernetes Resource Creation

Advanced Topics and Cleanup

Troubleshooting

Common Issues

Summary