09. NoSQL Key-Value Stores

Kubernetes (K8s) is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. This lecture covers the fundamentals of Kubernetes and its architecture.

What is Container Orchestration?

The Challenge

As applications grow, managing containers manually becomes impossible:

Orchestration Solution

An orchestrator manages and organizes both hosts and containers running on a cluster:

┌─────────────────────────────────────────────┐
│  Orchestrator (Kubernetes)                  │
│  ├── Resource allocation                    │
│  ├── Container scheduling                   │
│  ├── Health monitoring                      │
│  ├── Auto-scaling                           │
│  ├── Load balancing                         │
│  ├── Service discovery                      │
│  └── Rolling updates                        │
└─────────────────────────────────────────────┘

Key Orchestrator Tasks

Orchestrator Options

Kubernetes

Docker Swarm

Apache Mesos

Winner: Kubernetes has become the industry standard

What is Kubernetes?

Origin and Meaning

Core Philosophy: Self-Healing

Kubernetes always tries to steer the cluster to its desired state:

You: "I want 3 healthy instances of redis to always be running."

Kubernetes: "Okay, I'll ensure there are always 3 instances up and running."

[One instance dies]

Kubernetes: "Oh look, one has died. I'm going to spin up a new one."

This is called the reconciliation loop:

  1. Desired state: What you want (3 redis instances)
  2. Current state: What actually exists (2 redis instances)
  3. Action: Kubernetes takes action to match desired state (create 1 instance)
  4. Repeat: Continuously monitor and adjust

Kubernetes Architecture

Kubernetes uses a master-worker architecture:

┌─────────────────────────────────────────────────────────┐
│  Control Plane (Master Node)                            │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ API Server   │  │  Scheduler   │  │  Controller  │  │
│  │              │  │              │  │   Manager    │  │
│  └──────────────┘  └──────────────┘  └──────────────┘  │
│  ┌──────────────────────────────────────────────────┐  │
│  │  etcd (Distributed Key-Value Store)              │  │
│  └──────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────┘
                          │
        ┌─────────────────┼─────────────────┐
        │                 │                 │
┌───────▼────────┐ ┌──────▼───────┐ ┌──────▼───────┐
│  Worker Node 1 │ │ Worker Node 2│ │ Worker Node 3│
│  ┌──────────┐  │ │  ┌──────────┐│ │  ┌──────────┐│
│  │ kubelet  │  │ │  │ kubelet  ││ │  │ kubelet  ││
│  ├──────────┤  │ │  ├──────────┤│ │  ├──────────┤│
│  │kube-proxy│  │ │  │kube-proxy││ │  │kube-proxy││
│  ├──────────┤  │ │  ├──────────┤│ │  ├──────────┤│
│  │Container │  │ │  │Container ││ │  │Container ││
│  │ Runtime  │  │ │  │ Runtime  ││ │  │ Runtime  ││
│  └──────────┘  │ │  └──────────┘│ │  └──────────┘│
│  [Pods...]     │ │  [Pods...]   │ │  [Pods...]   │
└────────────────┘ └──────────────┘ └──────────────┘

Control Plane Components

1. API Server

2. etcd

3. Scheduler

4. Controller Manager

Worker Node Components

1. kubelet

2. kube-proxy

3. Container Runtime

Core Concepts

Desired State Management

Kubernetes operates on a declarative model:

# You declare what you want
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3  # I want 3 instances
  template:
    spec:
      containers:
      - name: nginx
        image: nginx:1.21

Kubernetes continuously works to achieve and maintain this state:

Reconciliation Loop

┌─────────────────────────────────────────┐
│  1. Observe current state               │
│     (What is actually running?)         │
└────────────┬────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────┐
│  2. Compare with desired state          │
│     (What should be running?)           │
└────────────┬────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────┐
│  3. Take action to reconcile            │
│     (Create, update, or delete)         │
└────────────┬────────────────────────────┘
             │
             ▼
        [Repeat continuously]

This creates a goal-driven, self-healing system.

Kubernetes Objects

Everything in Kubernetes is represented as an object:

Object Structure

All Kubernetes objects have:

apiVersion: v1  # API version
kind: Pod       # Object type
metadata:       # Object metadata
  name: my-pod
  labels:
    app: web
spec:           # Desired state
  containers:
  - name: nginx
    image: nginx

Common Object Types

Object Purpose
Pod Smallest deployable unit, runs containers
Service Stable network endpoint for pods
Deployment Manages pod replicas and updates
ReplicaSet Ensures desired number of pod replicas
ConfigMap Configuration data
Secret Sensitive data (passwords, tokens)
Namespace Virtual cluster for isolation
PersistentVolume Storage resource

Working with Kubernetes

kubectl - The Kubernetes CLI

kubectl is the command-line tool for Kubernetes:

# Get cluster information
kubectl cluster-info

# List nodes
kubectl get nodes

# List all pods
kubectl get pods --all-namespaces

# Create resources from YAML
kubectl apply -f deployment.yaml

# Get detailed information
kubectl describe pod my-pod

# View logs
kubectl logs my-pod

# Execute command in pod
kubectl exec -it my-pod -- /bin/bash

# Delete resources
kubectl delete pod my-pod

Imperative vs Declarative

Imperative (tell Kubernetes what to do):

kubectl create deployment nginx --image=nginx
kubectl scale deployment nginx --replicas=3
kubectl expose deployment nginx --port=80

Declarative (tell Kubernetes what you want):

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  # ... rest of spec
kubectl apply -f deployment.yaml

Best Practice: Use declarative approach for production (version control, reproducibility)

Kubernetes Distributions

Kubernetes can run in many environments:

Cloud-Managed Kubernetes

Advantages: Managed control plane, automatic updates, integrated with cloud services

Self-Managed Kubernetes

Advantages: Full control, can run anywhere

Local Development

Benefits of Kubernetes

Portability: Run anywhere (cloud, on-prem, hybrid)
Scalability: Handle massive scale (Google runs billions of containers)
High Availability: Automatic failover and recovery
Resource Efficiency: Optimal bin-packing of containers
Declarative Configuration: Infrastructure as code
Extensibility: Plugin architecture, custom resources
Ecosystem: Huge community, tools, and integrations

Challenges

Complexity: Steep learning curve
Overhead: Requires resources for control plane
Overkill: May be too complex for simple applications
Networking: Can be complex to configure
Storage: Stateful applications require careful planning

Summary

Kubernetes is a powerful container orchestration platform that:

In the next lecture, we’ll dive deep into Pods - the fundamental building block of Kubernetes applications.

Further Reading