Docker Compose to Virtual Machines Migration

This guide covers migrating Docker Compose applications to virtual machines using KubeVirt, including golden image preparation, VM resource definition, and deployment strategies.

Introduction and Migration Strategy

Why Virtual Machines?

While containers are excellent for most workloads, some applications require:

Legacy applications that can't be containerized
OS-level dependencies that don't work in containers
Hardware access requirements (GPUs, specific drivers)
Security isolation requirements
Compliance requirements for VM-based deployments

KubeVirt Overview

KubeVirt extends Kubernetes to run virtual machines alongside containers, providing:

Unified management of containers and VMs
Kubernetes-native VM lifecycle management
Resource sharing between containers and VMs
Network integration with Kubernetes services

Migration Strategy

Assessment: Identify applications suitable for VM migration
Preparation: Create golden images and VM templates
Resource Definition: Define VM resources and configurations
Deployment: Deploy VMs using KubeVirt
Integration: Connect VMs with Kubernetes services

Preparing Golden Images

Creating Base Images

Using Packer for Image Creation:

# packer.json
{
  "builders": [
    {
      "type": "qemu",
      "iso_url": "https://releases.ubuntu.com/22.04/ubuntu-22.04.3-live-server-amd64.iso",
      "iso_checksum": "sha256:5e38b55d57d94ff029719342357325ed3bda38fa80054f9330dc789cd2d43931",
      "output_directory": "output",
      "vm_name": "ubuntu-22.04-base",
      "disk_size": "20G",
      "memory": "2048",
      "cpus": "2",
      "ssh_username": "ubuntu",
      "ssh_password": "ubuntu",
      "ssh_timeout": "30m",
      "shutdown_command": "sudo shutdown -P now"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "sudo apt-get update",
        "sudo apt-get install -y docker.io",
        "sudo systemctl enable docker",
        "sudo usermod -aG docker ubuntu"
      ]
    },
    {
      "type": "shell",
      "script": "scripts/install-app.sh"
    }
  ]
}

Installation Script:

#!/bin/bash
# scripts/install-app.sh

# Install application dependencies
sudo apt-get install -y python3 python3-pip nginx

# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Create application directory
sudo mkdir -p /opt/myapp
sudo chown ubuntu:ubuntu /opt/myapp

# Copy application files
cp -r /tmp/app/* /opt/myapp/

# Create systemd service
sudo tee /etc/systemd/system/myapp.service > /dev/null <<EOF
[Unit]
Description=MyApp Service
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/myapp
ExecStart=/usr/local/bin/docker-compose up -d
ExecStop=/usr/local/bin/docker-compose down
User=ubuntu
Group=ubuntu

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable myapp.service

Building Images with Packer

# Install Packer
wget https://releases.hashicorp.com/packer/1.9.4/packer_1.9.4_linux_amd64.zip
unzip packer_1.9.4_linux_amd64.zip
sudo mv packer /usr/local/bin/

# Build image
packer build packer.json

# Verify image
ls -la output/

Converting to KubeVirt Format

Convert QCOW2 to KubeVirt format:

# Install virtctl
VERSION=$(kubectl get kubevirt.kubevirt.io/kubevirt -n kubevirt -o=jsonpath="{.status.observedKubeVirtVersion}")
echo $VERSION
VERSION=v0.59.0
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml

# Create DataVolume
cat <<EOF | kubectl apply -f -
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: ubuntu-base-image
spec:
  source:
    http:
      url: "https://your-storage.com/ubuntu-22.04-base.qcow2"
  pvc:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 20Gi
EOF

Defining Virtual Machine Resources

Basic VM Definition

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: myapp-vm
  namespace: default
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/vm: myapp-vm
    spec:
      domain:
        devices:
          disks:
          - name: containerdisk
            disk:
              bus: virtio
          - name: cloudinitdisk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
        resources:
          requests:
            memory: 2Gi
            cpu: 2
          limits:
            memory: 4Gi
            cpu: 4
        features:
          acpi: {}
          apic: {}
          hyperv:
            relaxed: {}
            vapic: {}
            spinlocks:
              spinlocks: 8191
      networks:
      - name: default
        pod: {}
      volumes:
      - name: containerdisk
        persistentVolumeClaim:
          claimName: ubuntu-base-image
      - name: cloudinitdisk
        cloudInitNoCloud:
          userData: |
            #cloud-config
            password: ubuntu
            chpasswd: { expire: False }
            ssh_pwauth: True
            users:
            - name: ubuntu
              sudo: ALL=(ALL) NOPASSWD:ALL
              shell: /bin/bash
            packages:
            - docker.io
            - docker-compose
            runcmd:
            - systemctl start docker
            - systemctl enable docker
            - usermod -aG docker ubuntu

Advanced VM Configuration

VM with Multiple Disks:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: myapp-vm-advanced
spec:
  running: true
  template:
    spec:
      domain:
        devices:
          disks:
          - name: os-disk
            disk:
              bus: virtio
          - name: data-disk
            disk:
              bus: virtio
          - name: config-disk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
          - name: secondary
            bridge: {}
        resources:
          requests:
            memory: 4Gi
            cpu: 4
          limits:
            memory: 8Gi
            cpu: 8
        features:
          acpi: {}
          apic: {}
      networks:
      - name: default
        pod: {}
      - name: secondary
        multus:
          networkName: secondary-network
      volumes:
      - name: os-disk
        persistentVolumeClaim:
          claimName: ubuntu-os-disk
      - name: data-disk
        persistentVolumeClaim:
          claimName: app-data-disk
      - name: config-disk
        configMap:
          name: app-config

VM with GPU Support:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: gpu-vm
spec:
  running: true
  template:
    spec:
      domain:
        devices:
          disks:
          - name: containerdisk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
          gpus:
          - deviceName: nvidia.com/gpu
            name: gpu1
        resources:
          requests:
            memory: 8Gi
            cpu: 4
            nvidia.com/gpu: 1
          limits:
            memory: 16Gi
            cpu: 8
            nvidia.com/gpu: 1
      networks:
      - name: default
        pod: {}
      volumes:
      - name: containerdisk
        persistentVolumeClaim:
          claimName: gpu-vm-disk

KubeVirt Deployment and Best Practices

Installing KubeVirt

Prerequisites:

# Check Kubernetes version (1.20+ required)
kubectl version --short

# Check if virtualization is enabled
kubectl get nodes -o jsonpath='{.items[*].status.allocatable}' | grep -o '"devices\.kubevirt\.io/kvm":[^,]*'

# Install KubeVirt operator
export VERSION=v0.59.0
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml

# Install KubeVirt
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml

# Wait for installation
kubectl -n kubevirt wait kv kubevirt --for condition=Available --timeout=300s

Install virtctl:

# Download virtctl
VERSION=$(kubectl get kubevirt.kubevirt.io/kubevirt -n kubevirt -o=jsonpath="{.status.observedKubeVirtVersion}")
echo $VERSION
curl -L -o virtctl https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/virtctl-${VERSION}-linux-amd64
chmod +x virtctl
sudo mv virtctl /usr/local/bin/

Deploying VMs

Create VM:

# Apply VM definition
kubectl apply -f vm-definition.yaml

# Check VM status
kubectl get vmi

# Access VM console
virtctl console myapp-vm

# Access VM via VNC
virtctl vnc myapp-vm

VM Lifecycle Management:

# Start VM
virtctl start myapp-vm

# Stop VM
virtctl stop myapp-vm

# Restart VM
virtctl restart myapp-vm

# Delete VM
kubectl delete vm myapp-vm

Network Integration

Service Integration:

apiVersion: v1
kind: Service
metadata:
  name: myapp-vm-service
spec:
  selector:
    kubevirt.io/vm: myapp-vm
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: ssh
    port: 22
    targetPort: 22
  type: LoadBalancer

Ingress Integration:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-vm-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp-vm-service
            port:
              number: 80

Storage Management

Persistent Storage:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: vm-storage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: rook-ceph-block

DataVolume for VM:

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: myapp-vm-dv
spec:
  source:
    http:
      url: "https://example.com/myapp-image.qcow2"
  pvc:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 50Gi

Monitoring and Logging

VM Metrics:

# Enable VM metrics
kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/vm-metrics.yaml

# Check metrics
kubectl get --raw /apis/metrics.k8s.io/v1beta1/pods | jq '.'

# VM-specific metrics
kubectl get --raw /apis/metrics.kubevirt.io/v1alpha1/namespaces/default/virtualmachines/myapp-vm

Logging Configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: vm-logging-config
data:
  fluentd.conf: |
    <source>
      @type tail
      path /var/log/vm/*.log
      pos_file /var/log/vm.log.pos
      tag vm.logs
      <parse>
        @type json
      </parse>
    </source>
    
    <match vm.logs>
      @type elasticsearch
      host elasticsearch-service
      port 9200
      index_name vm-logs
    </match>

Migration Examples

Example 1: Legacy Application Migration

Original Docker Compose:

version: '3.8'
services:
  legacy-app:
    image: legacy-app:v1.0
    ports:
      - "8080:8080"
    environment:
      - DB_HOST=postgres
      - DB_PORT=5432
    volumes:
      - app-data:/app/data
    depends_on:
      - postgres
  
  postgres:
    image: postgres:12
    environment:
      - POSTGRES_DB=legacy
      - POSTGRES_PASSWORD=secret
    volumes:
      - postgres-data:/var/lib/postgresql/data

volumes:
  app-data:
  postgres-data:

KubeVirt VM Definition:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: legacy-app-vm
spec:
  running: true
  template:
    spec:
      domain:
        devices:
          disks:
          - name: os-disk
            disk:
              bus: virtio
          - name: app-data
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
        resources:
          requests:
            memory: 4Gi
            cpu: 2
          limits:
            memory: 8Gi
            cpu: 4
      networks:
      - name: default
        pod: {}
      volumes:
      - name: os-disk
        persistentVolumeClaim:
          claimName: legacy-app-os
      - name: app-data
        persistentVolumeClaim:
          claimName: legacy-app-data
      - name: cloudinitdisk
        cloudInitNoCloud:
          userData: |
            #cloud-config
            packages:
            - docker.io
            - docker-compose
            runcmd:
            - systemctl start docker
            - systemctl enable docker
            - mkdir -p /app
            - mount /dev/vdb /app
            - cd /app && docker-compose up -d

Example 2: Multi-Tier Application

VM Cluster Definition:

# Web Tier VM
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: web-tier-vm
spec:
  running: true
  template:
    spec:
      domain:
        devices:
          disks:
          - name: containerdisk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
        resources:
          requests:
            memory: 2Gi
            cpu: 2
      networks:
      - name: default
        pod: {}
      volumes:
      - name: containerdisk
        persistentVolumeClaim:
          claimName: web-tier-disk
---
# Application Tier VM
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: app-tier-vm
spec:
  running: true
  template:
    spec:
      domain:
        devices:
          disks:
          - name: containerdisk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
        resources:
          requests:
            memory: 4Gi
            cpu: 4
      networks:
      - name: default
        pod: {}
      volumes:
      - name: containerdisk
        persistentVolumeClaim:
          claimName: app-tier-disk
---
# Database Tier VM
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: db-tier-vm
spec:
  running: true
  template:
    spec:
      domain:
        devices:
          disks:
          - name: containerdisk
            disk:
              bus: virtio
          - name: data-disk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
        resources:
          requests:
            memory: 8Gi
            cpu: 4
      networks:
      - name: default
        pod: {}
      volumes:
      - name: containerdisk
        persistentVolumeClaim:
          claimName: db-tier-disk
      - name: data-disk
        persistentVolumeClaim:
          claimName: db-data-disk

Best Practices

Performance Optimization

Resource Allocation:

spec:
  template:
    spec:
      domain:
        resources:
          requests:
            memory: 2Gi
            cpu: 2
          limits:
            memory: 4Gi
            cpu: 4
        cpu:
          cores: 2
          sockets: 1
          threads: 1
        memory:
          hugepages:
            pageSize: 2Mi

Storage Optimization:

spec:
  template:
    spec:
      domain:
        devices:
          disks:
          - name: containerdisk
            disk:
              bus: virtio
              cache: writeback
              io: threads

Security Considerations

VM Security:

spec:
  template:
    spec:
      domain:
        features:
          acpi: {}
          apic: {}
          hyperv:
            relaxed: {}
            vapic: {}
        devices:
          interfaces:
          - name: default
            bridge: {}
            model: virtio
        securityContext:
          runAsNonRoot: true
          runAsUser: 1000

Network Security:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: vm-network-policy
spec:
  podSelector:
    matchLabels:
      kubevirt.io/vm: myapp-vm
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: frontend
    ports:
    - protocol: TCP
      port: 80
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: database
    ports:
    - protocol: TCP
      port: 5432

Backup and Recovery

VM Backup Strategy:

# Create VM snapshot
virtctl snapshot create myapp-vm --name backup-$(date +%Y%m%d)

# Export VM
virtctl export vm myapp-vm --output myapp-vm-backup.yaml

# Restore VM
kubectl apply -f myapp-vm-backup.yaml

Automated Backup:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: vm-backup
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: kubevirt/virtctl:latest
            command:
            - /bin/bash
            - -c
            - |
              virtctl snapshot create myapp-vm --name backup-$(date +%Y%m%d)
              virtctl export vm myapp-vm --output /backup/myapp-vm-$(date +%Y%m%d).yaml
          volumes:
          - name: backup-storage
            persistentVolumeClaim:
              claimName: backup-pvc
          restartPolicy: OnFailure

Summary

Migrating from Docker Compose to Virtual Machines using KubeVirt provides:

Legacy application support for non-containerizable workloads
Unified management of containers and VMs in Kubernetes
Flexible resource allocation for performance-critical applications
Network integration with Kubernetes services
Storage management with persistent volumes

The key is to assess your application requirements, prepare appropriate golden images, and leverage KubeVirt's capabilities for seamless VM management within your Kubernetes environment.

Introduction and Migration Strategy​

Why Virtual Machines?​

KubeVirt Overview​

Migration Strategy​

Preparing Golden Images​

Creating Base Images​

Building Images with Packer​

Converting to KubeVirt Format​

Defining Virtual Machine Resources​

Basic VM Definition​

Advanced VM Configuration​

KubeVirt Deployment and Best Practices​

Installing KubeVirt​

Deploying VMs​

Network Integration​

Storage Management​

Monitoring and Logging​

Migration Examples​

Example 1: Legacy Application Migration​

Example 2: Multi-Tier Application​

Best Practices​

Performance Optimization​

Security Considerations​

Backup and Recovery​

Summary​