The foundation everything sits on.

This guide uses AWS as the canonical because it has the largest ecosystem and the most patterns to learn from. The concepts transfer to other clouds — VPCs, IAM, load balancers, object storage, managed databases all have direct equivalents.

Most teams pick one cloud and stay there. "Multi-cloud" usually means "using one cloud's services and another for one specific feature" (e.g., AWS for compute + Cloudflare for CDN). True multi-cloud (the same workload running on multiple clouds) is expensive, complex, and rarely worth it.

AWS is geographically distributed:

• Region — a geographic area (us-east-1, eu-west-1). ~33 regions globally.
• Availability Zone (AZ) — an isolated data center within a region. Each region has 3-6 AZs.
• Edge location — CloudFront PoPs for caching. ~600 globally.

Production systems use multiple AZs for high availability. Multi-region is for disaster recovery or geographic latency.

Object storage with HTTP API. Each object has a key (path-like string), value (bytes), and metadata.

# key concepts
- Buckets: top-level containers, globally unique names
- Objects: files inside buckets
- Versioning: keep multiple versions, never lose data
- Lifecycle policies: auto-archive to cheaper storage classes
- Storage classes: Standard, IA, Glacier (cheaper, slower)
- Encryption: SSE-S3, SSE-KMS, SSE-C
- Access: public-by-default OFF, presigned URLs for sharing

S3 is durable to 11 nines (99.999999999%) — you'd lose one object every billion years per object stored. The downside: it's eventually consistent in some operations (was, until 2020 — now strongly consistent).

IAM controls who can do what. Three main concepts:

• Users — humans with API keys / login. Avoid for production code.
• Roles — permissions assumed by services or users. The right way for code to access AWS.
• Policies — JSON documents describing allowed actions.

// Example policy: allow reading from one S3 bucket
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

The principle: least privilege. Every role gets the minimum permissions needed. Never use the root account for normal work; never share credentials between services.

AWS bills are confusing because services have many pricing dimensions. The big ones:

• EC2/Fargate — per second of compute time.
• S3 — storage + requests + data transfer out.
• Data transfer — out of AWS to internet ($0.05-0.09/GB), between AZs ($0.01/GB), within an AZ (free).
• NAT gateway — $0.045/hour per gateway + $0.045/GB processed. Often the silent cost killer.
• RDS — instance hours + storage + I/O + backup storage.

The 80/20 of cost optimization:

1. Right-size instances (most are oversized).
2. Use Savings Plans / Reserved Instances for steady workloads (~30% savings).
3. Watch data transfer costs (they hide).
4. Lifecycle old S3 data to Glacier.
5. Shut down dev/staging environments outside business hours.

A Virtual Private Cloud is a private network within AWS. You define the IP range (CIDR block), subnets, and routing.

VPC: 10.0.0.0/16  (65,536 IPs)
├── Subnet (public, AZ-a): 10.0.1.0/24  (256 IPs)
├── Subnet (public, AZ-b): 10.0.2.0/24
├── Subnet (private, AZ-a): 10.0.10.0/24
├── Subnet (private, AZ-b): 10.0.11.0/24
└── Route tables: where traffic goes

• Public subnet — route to the internet via Internet Gateway. Resources here have public IPs. Load balancers, bastion hosts.
• Private subnet — no direct internet route. Resources have only private IPs. Application servers, databases.

Why this matters: databases and app servers should never be reachable from the public internet. They live in private subnets. Only the load balancer is public.

Private resources sometimes need to reach the internet (download packages, call third-party APIs). NAT Gateway provides outbound access without exposing them to inbound traffic.

private subnet → NAT Gateway → Internet Gateway → internet

NAT gateways cost money — both per hour AND per GB of data processed. They're often a silent budget hole. Workarounds for common cases:

• VPC Endpoints for AWS services — traffic stays inside AWS, no NAT needed.
• VPC Peering or Transit Gateway for connecting VPCs without internet.

Security groups control what traffic is allowed in/out of resources. Rules are stateful — if traffic is allowed in, the response is automatically allowed out.

// example: web tier security group
Inbound:
  - 443 from 0.0.0.0/0 (HTTPS from anywhere)
  - 80 from 0.0.0.0/0 (redirect to HTTPS)
Outbound:
  - All to 0.0.0.0/0 (default; lock down for sensitive systems)

// example: DB security group
Inbound:
  - 5432 from web-tier-sg (Postgres only from web tier)
Outbound:
  - none needed

The pattern: reference security groups by ID, not by IP. web-tier-sg can talk to db-tier-sg regardless of which specific instances are running.

NACLs are subnet-level rules. Stateless (must explicitly allow both directions). Most production systems use security groups primarily and leave NACLs at defaults.

Default to ALB. Use NLB only when you have specific reasons (websocket-heavy, non-HTTP, regulatory IP allowlisting).

Caches content at AWS edge locations near users. Reduces origin load, improves latency.

What CloudFront caches by default:

• Anything matching the cache behavior (path patterns).
• Subject to Cache-Control headers from the origin.
• Per-edge-location caching with TTL.

What it doesn't cache:

• Requests with cookies (by default).
• POST/PUT/DELETE.
• Anything with Cache-Control: no-store.

User → CloudFront (edge cache) → ALB → ECS containers

AWS-managed DNS. Routes domain names to resources.

Record types worth knowing:

• A — domain to IPv4 address.
• AAAA — domain to IPv6.
• CNAME — domain to another domain.
• ALIAS — Route 53 specific; like CNAME but works at the apex (example.com, not just www).
• MX — mail server.
• TXT — verification, SPF/DKIM/DMARC.

Routing policies:

• Simple — one record, one answer.
• Weighted — split traffic by percentage (canary deploys).
• Latency-based — route to lowest-latency region.
• Failover — primary + secondary, automatic switch on health check failure.
• Geolocation — route by user's country.

When traffic isn't flowing, check in this order:

1. DNS — does nslookup/dig resolve?
2. Reachability — does curl/ping get through?
3. Security groups — is the source allowed?
4. NACLs — subnet-level allow?
5. Route table — does the subnet have a route?
6. Application — is it actually listening on the port?

VPC Reachability Analyzer (built-in tool) walks this for you for any source-destination pair.

You write a function. AWS runs it on demand. No server, no scaling, no deploy. Triggered by events: HTTP requests, S3 uploads, queue messages, schedules.

// Lambda handler (Node.js)
export async function handler(event) {
  const userId = event.pathParameters.id;
  const user = await fetchUser(userId);
  return {
    statusCode: 200,
    body: JSON.stringify(user),
  };
}

Strengths:

• Pay only when running. Idle = $0.
• Auto-scales from 0 to thousands instantly.
• No server management.
• Great for event-driven workloads (S3 triggers, scheduled jobs, webhooks).

Weaknesses:

• Cold starts (~100ms-2s for first invocation after idle).
• 15-minute max execution time.
• Stateless — each invocation is fresh.
• Local development is awkward (need SAM/serverless framework).
• Connection pooling is hard (each invocation may need a fresh DB connection).

You write a Docker container; ECS runs it. Fargate means AWS manages the underlying VMs. You see only the containers.

# task definition (simplified)
{
  "family": "my-app",
  "containerDefinitions": [{
    "name": "app",
    "image": "123456.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3",
    "portMappings": [{ "containerPort": 3000 }],
    "memory": 512,
    "cpu": 256
  }],
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"]
}

Strengths:

• Same container model as Kubernetes, simpler to operate.
• Long-running processes work fine.
• No 15-minute limit.
• Direct VPC networking — talks to private resources naturally.
• Auto-scaling based on CPU/memory/custom metrics.

Weaknesses:

• More expensive per-CPU than EC2 if you can keep instances utilized.
• Slower scale-up than Lambda (containers take 30s+ to start).
• No GPU support directly (use ECS on EC2).

AWS-managed Kubernetes control plane; you bring worker nodes (or use Fargate for workers).

Strengths:

• Standard Kubernetes — works the same as on-prem or other clouds.
• Massive ecosystem (Helm charts, operators, service mesh).
• Multi-region, multi-cluster, hybrid possible.
• Best for complex microservice architectures.

Weaknesses:

• Steepest learning curve.
• $72/month per cluster control plane (before workers).
• More YAML, more concepts (pods, deployments, services, ingress, namespaces).
• Requires more ops investment.

Just a VM. Use directly when:

• You need GPUs.
• You need very specific OS tuning (kernel parameters, network stack).
• You're running a stateful service that doesn't fit container patterns (database, message broker).
• You need spot instances for cheap batch work.

For some apps, AWS itself is too complex. Higher-level platforms wrap cloud infrastructure with developer experience focus:

Use these when developer time is more expensive than infrastructure cost. Move to AWS directly when you outgrow them.

A container is a process running with isolation features the Linux kernel provides:

• Namespaces — isolated view of processes, network, filesystem, users.
• cgroups — resource limits (CPU, memory).
• Layered filesystem — image is built up of layers (base OS + app dependencies + your code).

The container shares the host kernel. Lighter than a VM (no separate OS), but less isolated.

# Multi-stage build for a Node.js app
# Stage 1: build
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: runtime
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./

# run as non-root user
USER node

EXPOSE 3000
CMD ["node", "dist/server.js"]

Build dependencies (compilers, dev packages) shouldn't ship to production. Multi-stage builds use one image to build, copy artifacts to a smaller runtime image. Final image: small, no build tools, only runtime needs.

Without multi-stage: Node app might be 1.2GB. With multi-stage: 150MB.

• Use alpine variants for small images — node:20-alpine not node:20.
• Order layers by change frequency — copy package.json before app code, so dependency installs cache.
• Use .dockerignore — exclude node_modules, .git, tests, secrets.
• Pin versions — node:20.10.0-alpine not node:latest. Reproducible builds.
• Run as non-root — USER node or specific UID. Security.
• Use distroless or scratch images for compiled languages (Go, Rust). Tiny, no shell.

Containers work best when apps follow these patterns:

1. Codebase — one repo per service.
2. Dependencies — explicitly declared, vendored or locked.
3. Config — in environment variables, not code.
4. Backing services — DBs, queues are attached resources, swappable.
5. Build, release, run — three separate stages.
6. Processes — stateless. State goes to backing services.
7. Port binding — app exports HTTP on a port; ingress routes to it.
8. Concurrency — scale by adding processes, not threads.
9. Disposability — fast startup, graceful shutdown.
10. Dev/prod parity — keep environments similar.
11. Logs — stdout/stderr, captured by infrastructure.
12. Admin processes — one-off jobs run in same environment.

Where built images live before deployment.

• ECR (AWS) — integrates with ECS/EKS, IAM-based access.
• GHCR (GitHub) — free for public, paid for private. Integrates with GitHub Actions.
• Docker Hub — original. Rate limits on pulls.
• Self-hosted (Harbor, Nexus) — for air-gapped or strict-control environments.

Containers contain dependencies; dependencies have CVEs. Scan images for vulnerabilities:

• Trivy — open-source, fast, runs in CI.
• Snyk — commercial, deeper analysis.
• ECR scanning — built-in if you use ECR.
• GitHub dependabot — alerts on Dockerfile base image issues.

# Trivy in GitHub Actions
- name: Scan image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'my-app:${{ github.sha }}'
    format: 'sarif'
    severity: 'CRITICAL,HIGH'
    exit-code: '1'

# docker-compose.yml
services:
  app:
    build: .
    ports: ['3000:3000']
    environment:
      DATABASE_URL: postgres://postgres:secret@db:5432/myapp
    depends_on: [db, redis]

  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    volumes:
      - pgdata:/var/lib/postgresql/data

  redis:
    image: redis:7-alpine

volumes:
  pgdata:

One command (docker compose up) brings up the entire local environment. Used for development; production uses orchestrators (ECS, Kubernetes), not Compose.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: app
          image: my-app:1.2.3
          ports:
            - containerPort: 3000
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: 500m
              memory: 512Mi
          livenessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /ready
              port: 3000
            periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 3000

This: 3 pods running the image, with health checks, resource limits, and a stable network endpoint.

• Liveness — "is this container still alive?" Failure = restart pod.
• Readiness — "is this container ready to receive traffic?" Failure = remove from service routing.
• Startup — for slow-starting apps. Liveness/readiness don't run until startup passes.

Common mistake: same probe for both. The right pattern: liveness checks "process is alive and not stuck" (lightweight), readiness checks "I can serve requests" (heavier — DB connection, dependencies).

• Requests — what the pod is guaranteed. Kubernetes uses this for scheduling.
• Limits — the maximum it can use. Exceeding CPU = throttle. Exceeding memory = OOM kill.

Typical pattern:

requests: { cpu: 100m, memory: 256Mi }
limits: { cpu: 500m, memory: 512Mi }

Request what you typically need; limit at peak. Setting requests too high wastes capacity. Setting them too low causes scheduling on busy nodes.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Scales pod count based on CPU utilization (or custom metrics). Combined with cluster autoscaling, the cluster grows and shrinks automatically.

Helm packages Kubernetes manifests as templated "charts." Install with one command, parameterize per environment.

# install Postgres via Helm
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-postgres bitnami/postgresql \
  --set auth.postgresPassword=secretpass \
  --set primary.persistence.size=20Gi

Most production apps either use Helm or Kustomize (built into kubectl, simpler).

A layer between services that handles cross-cutting concerns: mTLS between services, traffic routing, retries, circuit breaking, observability.

Adds significant complexity. Use only when you have specific problems it solves (regulatory mTLS, advanced traffic shaping, deep service-to-service observability). Most apps don't need this.

# terraform/main.tf
terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.0" }
  }
  backend "s3" {
    bucket = "my-tf-state"
    key    = "prod/terraform.tfstate"
    region = "us-east-1"
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "uploads" {
  bucket = "my-app-uploads-prod"
}

resource "aws_s3_bucket_versioning" "uploads" {
  bucket = aws_s3_bucket.uploads.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_public_access_block" "uploads" {
  bucket = aws_s3_bucket.uploads.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

output "uploads_bucket" {
  value = aws_s3_bucket.uploads.id
}

# commands
terraform init      # download providers, set up backend
terraform plan      # show what will change
terraform apply     # execute the changes
terraform destroy   # tear it all down

// infra/index.ts
import * as aws from "@pulumi/aws";

const uploadsBucket = new aws.s3.Bucket("uploads", {
  bucket: "my-app-uploads-prod",
  versioning: { enabled: true },
});

new aws.s3.BucketPublicAccessBlock("uploads-block", {
  bucket: uploadsBucket.id,
  blockPublicAcls: true,
  blockPublicPolicy: true,
  ignorePublicAcls: true,
  restrictPublicBuckets: true,
});

export const uploadsBucketName = uploadsBucket.id;

Same result, but with TypeScript: real loops, conditions, functions, modules, type checking, and IDE autocomplete. The infra is just code.

IaC tools track the resources they've created in a "state file." This is critical:

• State maps logical names to real cloud resources.
• Without it, the tool can't know what to update or delete.
• State must be shared (team) and locked (prevent concurrent applies).

For Terraform: store state in S3 with DynamoDB locking, or use Terraform Cloud / Spacelift / Atlantis.

For Pulumi: Pulumi Cloud (free for individuals, paid for teams), or self-host with S3.

• Everything in code. No clicking in the console for production. Drift is the enemy.
• One state per environment. Dev, staging, prod separate.
• Plan in CI, apply via PR. Code review for infra changes.
• Modules / components for reuse. Common patterns extracted, parameterized.
• Tagging. Every resource tagged with environment, owner, cost-center.
• Drift detection. Run plan regularly to find resources that changed outside IaC.

Once you've written one VPC, you don't want to write another. Extract patterns:

# Terraform module usage
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.5.0"

  name = "prod-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.10.0/24", "10.0.11.0/24", "10.0.12.0/24"]
  public_subnets  = ["10.0.1.0/24",  "10.0.2.0/24",  "10.0.3.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = false  # one per AZ for HA
}

Public modules from the Terraform Registry cover most common patterns. Avoid reinventing.

1. Trigger — push to branch, PR opened, schedule, manual.
2. Checkout — get the source code.
3. Setup — install dependencies, configure cache.
4. Lint and typecheck — fast checks that fail early.
5. Test — unit, integration, possibly e2e.
6. Build — compile, bundle, build Docker image.
7. Push — image to registry, artifacts to storage.
8. Deploy — to staging, then production (with approval gates).
9. Verify — smoke tests, health checks.
10. Notify — Slack, email on success/failure.

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - run: npm ci

      - run: npm run lint
      - run: npm run typecheck
      - run: npm run test

      - run: npm run build

      - uses: actions/upload-artifact@v4
        with:
          name: build
          path: dist/

  deploy:
    needs: test
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    permissions:
      id-token: write  # for OIDC
      contents: read
    steps:
      - uses: actions/checkout@v4

      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789:role/github-deploy
          aws-region: us-east-1

      - run: ./deploy.sh

Most pipeline time is spent installing dependencies. Caching them between runs is the single biggest speedup.

# cache npm
- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'

# cache Docker layers
- uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

# cache custom paths
- uses: actions/cache@v4
  with:
    path: ~/.cargo
    key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

Run the same job across multiple configurations.

strategy:
  matrix:
    node: ['18', '20', '21']
    os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}

Six jobs run in parallel, testing every combination.

The old pattern: store AWS access keys in GitHub Secrets. Rotation hell, leak risk.

The 2026 pattern: OIDC federation. GitHub Actions identifies itself to AWS as a workflow run; AWS assumes a role on the fly. No long-lived credentials.

permissions:
  id-token: write  # request OIDC token
  contents: read

- uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789:role/github-deploy
    aws-region: us-east-1

The AWS IAM role has a trust policy that only accepts tokens from your specific GitHub repo and branches. No secret to leak.

Different ways to ship code:

• Rolling — replace instances one at a time. Default in ECS/Kubernetes.
• Blue/green — bring up new version alongside old, switch traffic, keep old around for instant rollback.
• Canary — send small % of traffic to new version, ramp up if metrics look good.
• Feature flag — code is deployed but disabled until flag is flipped.

• Fast feedback. CI should finish in under 10 minutes for normal changes. Optimize ruthlessly.
• Fail fast. Run cheap checks (lint, typecheck) before expensive ones (e2e tests).
• Reproducible. Same code → same artifact, every time. Pin versions, lock files.
• Idempotent deploys. Re-running a deploy yields the same result.
• Roll back, don't roll forward. When something breaks, the safest action is to deploy the previous version.
• Trunk-based with PRs. Short-lived feature branches, merged into main daily. Long-lived branches accumulate conflicts.

1. Never commit secrets to git. Once committed, assume compromised. Even private repos.
2. Never put secrets in container images. Anyone with the image has them.
3. Never log secrets. Filter them in logging libraries.
4. Use short-lived credentials when possible. Rotation reduces blast radius.
5. Audit who has access to what. Log every secret access.

How secrets get into your running app:

Secrets should rotate regularly. Static API keys are time bombs.

• Database passwords — Secrets Manager can rotate via Lambda. New password generated, applied to RDS, app re-fetches at next connection.
• API keys (Stripe, OpenAI) — manually create new key, update secret, deploy, deactivate old.
• JWT signing keys — rotate by issuing new tokens with new key while old ones remain valid for transition period.
• TLS certificates — automated via Let's Encrypt + cert-manager (Kubernetes) or AWS ACM (auto-renewal).

The modern shift: don't have long-lived credentials at all.

• IAM Roles for Service Accounts (IRSA) in EKS — pods assume IAM roles directly, no AWS keys.
• ECS task roles — same idea for ECS.
• OIDC federation for CI/CD — GitHub Actions assumes IAM role per workflow.
• Workload Identity in GCP — same pattern.

Code doesn't see credentials at all. The platform handles authentication; your code just makes API calls.

• Secrets in a manager, never in code.
• Pre-commit hooks catch accidents.
• OIDC for CI/CD — no long-lived keys.
• IAM roles for runtime — no API keys.
• Rotation automated where possible, scheduled where not.
• Audit logs on secret access. Alert on anomalies.
• Treat any leaked secret as compromised — rotate, don't try to "clean up."

Logs are best for debugging a specific incident. Metrics are best for monitoring trends and alerting. Traces are best for understanding distributed system performance.

The right way to log: structured JSON with fields, not freeform text.

// bad — unsearchable
console.log(`User ${userId} bought ${productId} for $${amount}`);

// good — structured
logger.info('purchase_completed', {
  user_id: userId,
  product_id: productId,
  amount,
  currency: 'USD',
  payment_method: 'stripe',
});

Now you can query: "all purchases over $100 in the last hour." With freeform logs, you can't.

• DEBUG — detailed info for debugging. Off in production.
• INFO — normal events worth recording (request received, user signed up).
• WARN — unexpected but recoverable (retrying, fallback used).
• ERROR — something failed (request couldn't complete, exception thrown).
• FATAL — service is going down.

Production usually runs at INFO. Tools query and aggregate; humans rarely scroll through logs.

Numbers over time, queryable by tags.

// counter — monotonically increasing
metrics.counter('http.requests', { method: 'POST', route: '/api/login', status: 200 });

// gauge — current value
metrics.gauge('queue.depth', 42, { queue: 'emails' });

// histogram — distribution
metrics.histogram('http.duration_ms', 145, { route: '/api/search' });

Metric types matter:

• Counters — only go up. Track totals (requests, errors). Calculate rate as rate(counter[5m]).
• Gauges — go up and down. Track current values (queue depth, active connections, memory).
• Histograms / summaries — distributions. Track latency percentiles, request sizes.

Google SRE's framework for monitoring any service:

1. Latency — how long requests take (p50, p95, p99).
2. Traffic — how many requests (RPS).
3. Errors — how many fail (error rate).
4. Saturation — how full the system is (CPU, memory, disk, queue depth).

Dashboards for any service start with these four.

One request flows through many services. A trace shows the full call graph with timing.

POST /api/checkout                                            450ms
├── auth.verify                                                15ms
├── inventory.check                                            22ms
├── payment.charge                                            380ms  ← bottleneck
│   ├── stripe.api                                            350ms
│   └── ledger.record                                          25ms
├── email.send (async)                                          5ms
└── response                                                   28ms

Without tracing, you'd see "checkout is slow." With tracing, you see Stripe is the bottleneck.

OpenTelemetry is the standard now — vendor-neutral instrumentation. Most languages have auto-instrumentation libraries that capture HTTP, DB, and RPC calls automatically.

Single platform for logs, metrics, traces, infra monitoring, APM, RUM (real user monitoring), synthetics. Expensive but comprehensive.

// Node.js APM auto-instrumentation
import tracer from 'dd-trace';
tracer.init();

// custom span
const span = tracer.startSpan('process.payment');
try {
  await processPayment(order);
  span.setTag('order_id', order.id);
} catch (err) {
  span.setTag('error', err);
  throw err;
} finally {
  span.finish();
}

Alerts wake humans up. Bad alerts waste sleep and trust.

Good alerts are:

• Symptomatic — alert on user-facing symptoms (latency, error rate), not on causes (high CPU). Causes change; symptoms don't.
• Actionable — there's something the on-call can do. If not, why are you waking them?
• Worth waking up for — reserve pages for real impact. Email or Slack for warnings.

The alert hierarchy:

• Pages (call/SMS) — user-impacting outage. Site is down, payments are failing.
• Slack alerts — concerning trends, errors above baseline. Check during business hours.
• Email digests — daily summaries, low-priority items.
• Dashboards — for proactive monitoring, not alerts.

An SLO (Service Level Objective) is a measurable promise about reliability:

SLO: 99.9% of requests succeed in < 500ms over a 30-day window
Error budget: 0.1% of requests = ~43 minutes of downtime per month

The team can spend the error budget on risky deploys, experimentation, etc. When the budget runs out, freeze deploys until reliability improves.

SLOs replace "uptime" as the metric that matters — they're focused on user-experienced reliability, not on whether servers are technically running.

1. Vertical scaling (bigger machines) — usually the cheapest first move.
2. Read replicas — most apps are read-heavy.
3. Caching — Redis in front of expensive queries.
4. Horizontal scaling — more app instances behind a load balancer.
5. CDN — push static and cacheable content to edge.
6. Async processing — move slow work to background queues.
7. Sharding / partitioning — split data across DBs when one isn't enough.
8. Multi-region — geographic distribution for global latency or DR.

Boring but effective. A db.r6g.xlarge handles dramatically more than a db.t3.micro. Most teams' first scaling problem is "we should have used bigger instances from the start."

Limits:

• Eventually you hit the largest available instance.
• Single point of failure.
• Diminishing returns past a certain size.

But: vertical scaling is the right answer until you actually outgrow it. Don't preemptively shard at 1k users.

Stateless services scale horizontally by adding copies behind a load balancer.

// ECS service with auto-scaling
{
  "desiredCount": 3,
  "minCapacity": 3,
  "maxCapacity": 30,
  "scaling": {
    "targetCpuUtilization": 70
  }
}

The hard part: making your app actually stateless. Sessions in Redis, not memory. Files in S3, not disk. WebSocket connections through a pub/sub layer.

• CPU utilization — most common. Add instances when avg CPU > 70%.
• Request count — scale by traffic.
• Queue depth — scale background workers when queue backs up.
• Custom metrics — anything you can graph (response time, error rate).
• Schedule — predictable traffic patterns (scale up at 9 AM, down at 6 PM).

Tune the cooldown carefully. Scale up fast (don't drop traffic), scale down slow (don't churn).

Most apps are 80%+ reads. Read replicas multiply read capacity:

writes → primary
reads → primary OR replicas

RDS Multi-AZ + read replicas give you HA + read scaling. Configure your app to route reads to replicas, writes to primary.

Caveat: replica lag. Async replicas are slightly stale. For "read your writes" consistency, route a user's reads to primary for N seconds after their write, or use a causal consistency token.

The cheapest performance gain you can buy.

The patterns:

• Cache-aside — app checks cache, falls back to DB on miss, populates cache.
• Write-through — every write goes to cache and DB simultaneously.
• Write-behind — writes go to cache; cache writes to DB asynchronously.

Cache-aside is the default. Other patterns when you have specific consistency requirements.

Slow work shouldn't block user-facing requests. Move it to background workers.

// synchronous (slow)
async function signUp(email, password) {
  const user = await createUser(email, password);
  await sendWelcomeEmail(user);  // 800ms
  await indexUserInSearch(user);  // 200ms
  await provisionTenant(user);    // 1500ms
  return user;
  // total: 2.5+ seconds
}

// async (fast)
async function signUp(email, password) {
  const user = await createUser(email, password);
  await queue.enqueue('user.signed_up', { userId: user.id });
  return user;
  // total: 100ms; the rest happens in workers
}

// worker
queue.consume('user.signed_up', async (msg) => {
  await sendWelcomeEmail(...);
  await indexUserInSearch(...);
  await provisionTenant(...);
});

The tradeoff: eventual consistency. The user is created instantly, but the welcome email may be 5 seconds late. For most flows, this is fine. The fast response matters more.

When one database can't hold all your data or handle all the writes:

• Vertical partitioning — split tables across DBs by domain (users in one, orders in another).
• Horizontal sharding — split rows of one table across DBs by some key (user ID hash, geographic region).

Sharding is a one-way door. Cross-shard queries are expensive or impossible. Joins across shards are pain. Plan extensively before sharding.

Many teams that thought they needed sharding just needed: better indexes, read replicas, caching, or moving to a bigger instance.

Code rolls back fast. Schemas don't. The mature pattern is expand-contract:

1. Expand — add new column/table, code reads old AND new.
2. Backfill — populate new column for old rows.
3. Migrate reads — code reads new only.
4. Stop writing old — code writes new only.
5. Contract — drop old column/table.

Each step is independently rollback-safe. Slower (5 deploys instead of 1) but never breaks production.

-- Step 1 (deploy): add column, code reads username, writes both
ALTER TABLE users ADD COLUMN handle VARCHAR(50);

-- Step 2 (background job): backfill
UPDATE users SET handle = username WHERE handle IS NULL;

-- Step 3 (deploy): code reads handle, falls back to username; writes both
-- Step 4 (deploy): code reads handle only, still writes both
-- Step 5 (deploy): code writes handle only

-- Step 6 (deploy): drop column
ALTER TABLE users DROP COLUMN username;

ALTER TABLE accounts ADD COLUMN preferences JSONB DEFAULT '{}';

-- step 1: add column without default (instant in modern Postgres)
ALTER TABLE accounts ADD COLUMN preferences JSONB;

-- step 2: backfill in batches
UPDATE accounts SET preferences = '{}' WHERE id BETWEEN 1 AND 100000;
-- repeat in chunks; vacuum between

-- step 3: add default for new rows
ALTER TABLE accounts ALTER COLUMN preferences SET DEFAULT '{}';

-- step 4: NOT NULL after backfill complete
ALTER TABLE accounts ALTER COLUMN preferences SET NOT NULL;

Don't trust your deploy was successful just because the pipeline turned green. Run smoke tests against the deployed environment:

// after deploy completes
- name: Smoke test
  run: |
    curl -f https://api.example.com/health || exit 1
    curl -f https://api.example.com/api/version | grep "${{ github.sha }}" || exit 1
    npm run test:smoke -- --baseUrl=https://api.example.com

If smoke tests fail, automated rollback. The pipeline that deploys must own the verification.

• Every deploy must be rollback-able to the previous version with one command.
• Database migrations must be backwards-compatible during the deploy window.
• Roll back first, investigate second. Don't let users suffer while you debug.
• Automate rollback triggers — error rate > X for > Y minutes = automatic rollback.

Counterintuitive but proven: deploy more often, not less. Small frequent deploys mean:

• Each deploy has fewer changes — easier to debug if something breaks.
• Rollbacks are smaller — less to undo.
• Confidence builds — engineers trust the pipeline.
• Issues surface fast — caught while context is fresh.

Top-tier teams deploy multiple times per day per service. Big batched releases (weekly, monthly) accumulate risk and turn each deploy into an event.

Managed (RDS, Cloud SQL, Aurora, PlanetScale): cloud provider handles backups, upgrades, replication, failover.

Self-hosted (Postgres on EC2, on Kubernetes): you handle everything.

For 95% of teams in 2026, use managed. The premium pays for itself in operational time saved. Self-host only when you have specific reasons:

• Regulatory requirement (data residency, audit access).
• Specific extensions not supported by managed offerings.
• Cost at very large scale (managed becomes expensive over ~$100k/month DB spend).
• You have a dedicated DBA team.

Postgres backs each connection with a process. Too many connections = high memory, context-switching overhead, performance collapse.

• db.t3.medium: ~80 connections.
• db.r6g.large: ~200 connections.
• db.r6g.4xlarge: ~5000 connections.

Most apps don't need 5000 connections. Most apps need 50, achieved through connection pooling.

A connection pooler sits between your app and Postgres. Apps open lots of "virtual" connections; the pooler maps a small number of real connections. Especially important for serverless (Lambda) where each invocation might want a fresh connection.

5000 Lambda invocations
  ↓ (each has its own connection pool of, say, 5)
[ RDS Proxy ]
  ↓ (multiplexes to a small real pool)
20 actual connections to Postgres

RDS Proxy is AWS-managed; PgBouncer is self-hosted. Both work.

Two metrics:

• RPO (Recovery Point Objective) — how much data are you willing to lose? "We're OK losing 5 minutes" → RPO 5 minutes.
• RTO (Recovery Time Objective) — how long can you be down? "We must be back in 1 hour" → RTO 1 hour.

Cheaper RPO/RTO = more expensive infrastructure. Set them based on actual business impact, not "as low as possible."

• Automated daily snapshots (35-day retention).
• Continuous WAL archive for point-in-time recovery.
• Cross-region snapshots for disaster recovery — encrypted with separate keys.
• Quarterly restore drill — actually restore from backup. Untested backups don't exist.

-- enable slow query logging
ALTER SYSTEM SET log_min_duration_statement = 1000;  -- log queries > 1s
SELECT pg_reload_conf();

-- find current long-running queries
SELECT pid, now() - query_start AS duration, query
FROM pg_stat_activity
WHERE state = 'active' AND now() - query_start > interval '30 seconds';

-- find queries with most total time
SELECT query, calls, total_exec_time, mean_exec_time
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;

The pg_stat_statements extension tracks query stats. Essential for debugging what's actually slow vs occasionally slow.

For most apps in 2026, the answer is still Postgres + Redis cache. NoSQL when you have specific reasons. "We might need to scale" isn't a specific reason.

In-memory key-value store. Sub-millisecond latency. Used for caching, sessions, rate limiting, pub/sub, locks, queues, leaderboards.

# basic operations
SET user:123 '{"name":"Alex"}' EX 300       # set with 5min TTL
GET user:123
DEL user:123

# atomic counters (rate limiting)
INCR rate:user:123:requests
EXPIRE rate:user:123:requests 60

# hashes for partial updates
HSET user:123 name "Alex" email "a@b.com"
HGET user:123 name

# sorted sets (leaderboards)
ZADD scores 100 "user:1" 200 "user:2"
ZREVRANGE scores 0 9              # top 10

# pub/sub
PUBLISH channel "message"
SUBSCRIBE channel

Two flavors:

• Cluster mode disabled — single primary + read replicas. Simpler, scales to ~100GB.
• Cluster mode enabled — sharded across nodes. Scales horizontally, but client must support clustering.

For most apps, cluster mode disabled with a few replicas is plenty.

Two famous quotes apply: "There are only two hard things in computer science: cache invalidation and naming things." (Karlton)

Strategies:

• TTL only — accept staleness up to TTL. Simplest.
• Invalidate on write — delete the cached value when the source changes.
• Versioned keys — bump a version number on write; reads include version. Old versions age out.
• Pub/sub invalidation — emit invalidation messages for distributed cache layers.

Cache expires; thousands of requests hit the DB simultaneously trying to refresh it. The DB collapses.

Solutions:

• Locks — only one process refreshes the cache; others wait.
• Probabilistic early expiration — refresh slightly before actual expiry, randomized.
• Stale-while-revalidate — serve stale value, refresh in background.

Layer 1 — CDN (CloudFront)
  - Cache-Control: public, max-age=60, stale-while-revalidate=300
  - Most requests served from edge. Origin sees a fraction.

Layer 2 — Application cache (Redis)
  - Key: product:{id}, TTL 5 minutes
  - Origin DB sees only cache misses

Layer 3 — Application in-process cache
  - Local LRU, 10s TTL
  - Avoid even Redis trips for hot products in the same instance

Layer 4 — Database (Postgres)
  - With proper indexing on products.id

// producer
import { SQSClient, SendMessageCommand } from '@aws-sdk/client-sqs';
const sqs = new SQSClient({});
await sqs.send(new SendMessageCommand({
  QueueUrl: 'https://sqs.us-east-1.amazonaws.com/123/my-queue',
  MessageBody: JSON.stringify({ userId: 123, action: 'send_email' }),
}));

// consumer (polls in a loop, or use Lambda trigger)
const { Messages } = await sqs.send(new ReceiveMessageCommand({
  QueueUrl: '...',
  MaxNumberOfMessages: 10,
  WaitTimeSeconds: 20,  // long polling
}));

for (const msg of Messages || []) {
  try {
    await processMessage(JSON.parse(msg.Body));
    await sqs.send(new DeleteMessageCommand({
      QueueUrl: '...',
      ReceiptHandle: msg.ReceiptHandle,
    }));
  } catch (err) {
    // don't delete; will be redelivered after visibility timeout
    // SQS auto-moves to DLQ after N failed attempts
  }
}

Messages that fail repeatedly should go to a DLQ — a separate queue for failed messages. Manual investigation, optional reprocessing.

# SQS configuration
{
  "RedrivePolicy": {
    "deadLetterTargetArn": "arn:aws:sqs:us-east-1:123:my-queue-dlq",
    "maxReceiveCount": 5
  }
}

After 5 failures, the message moves to the DLQ. Alarms watch DLQ depth — non-zero = bug.

Queues guarantee at-least-once delivery, sometimes exactly-once. Most are at-least-once. Consumers must be idempotent — processing the same message twice should be safe.

async function processMessage(msg) {
  const { idempotencyKey, action, data } = msg;

  // check if we've already processed this
  if (await processed.has(idempotencyKey)) {
    return;  // skip
  }

  await doTheWork(action, data);
  await processed.set(idempotencyKey, true);
}

• Queue (SQS) — point-to-point, one consumer per message. Work distribution.
• Pub/sub (SNS) — one publisher, many subscribers, all see every message. Event broadcasting.
• Stream (Kafka) — durable log, multiple consumers replay history independently. Event sourcing.

1. You point your domain at the CDN (CNAME or anycast IP).
2. User's DNS lookup returns an IP near them.
3. Request hits the closest CDN edge node.
4. If cached, edge serves directly (10-50ms).
5. If not, edge fetches from origin and caches per the response headers.

By default:

• Static assets (JS, CSS, images) with long TTLs.
• Anything with Cache-Control: public, max-age=....
• Responses to GET (and sometimes HEAD) requests.

By default, NOT:

• Anything with cookies (varies by configuration).
• Anything with Cache-Control: private or no-store.
• POST/PUT/DELETE requests.

Cache-Control: public, max-age=31536000, immutable
# Static asset that never changes (hashed filename). Cache forever.

Cache-Control: public, max-age=60, stale-while-revalidate=600
# API response. Fresh for 60s; if requested 60-660s later, serve stale
# while fetching fresh in background. Best of both worlds.

Cache-Control: private, max-age=300
# User-specific. Browser cache only, not shared (CDN won't cache).

Cache-Control: no-store
# Don't cache anywhere. For sensitive data.

Cache-Control: public, max-age=0, must-revalidate
# Always check with server before using. Negotiated by ETag.

ETag: "v3-abc123"
# Validation. Browser sends If-None-Match: "v3-abc123" on next request.
# Server compares; if match, returns 304 Not Modified (no body).

Vary: Accept-Encoding, Authorization
# Cache key includes these headers. Different cached versions per value.

The combination max-age + stale-while-revalidate is magic for APIs. Users get fast cached responses; refreshes happen in background.

For AWS-hosted apps: CloudFront is fine. For websites with worldwide audience: Cloudflare is hard to beat. For complex CDN logic with instant cache purges: Fastly.

Run code at edge nodes near users. Use cases:

• Authentication checks — reject unauth'd requests before they hit your origin.
• A/B test routing — split traffic at the edge.
• Header manipulation — rewrite URLs, add security headers.
• Geolocation routing — different content per country.
• Personalization — small dynamic pieces without full origin fetch.

// Cloudflare Worker
export default {
  async fetch(request) {
    const country = request.cf.country;
    const url = new URL(request.url);
    if (country === 'GB') {
      url.hostname = 'uk.example.com';
    }
    return fetch(url, request);
  },
};

The CDN has many edge locations. Without shielding, a cache miss in each location separately fetches from origin. With shielding, edges fetch through one or two "shield" locations that aggregate misses.

Practical effect: origin sees one cache miss per shield, not one per edge location.

• Time-based — wait for TTL to expire. Simple but slow.
• URL-based purge — invalidate specific paths. Available in all major CDNs.
• Tag-based purge (Fastly surrogate keys, Cloudflare cache tags) — tag responses, purge by tag. Best for invalidating "all pages featuring product X."

Typical cloud spend distribution for a web app:

• Compute (EC2/Fargate/Lambda) — 30-50%
• Database (RDS) — 15-30%
• Data transfer — 10-20% (often hidden until you look)
• Storage (S3, EBS) — 5-15%
• NAT gateways — surprisingly large for many setups
• Other services — 5-15%

For steady workloads, commit to spending and save 30-72%.

• Savings Plans — commit to $X/hour of compute spending. Flexible across instance types/regions/services. 1 or 3 year terms.
• Reserved Instances — commit to specific instance types. Less flexible, slightly higher discount.

Most teams should use Savings Plans for ECS/Fargate/EC2/Lambda baseline. Even 1-year no-upfront saves ~30%.

EC2 capacity sold at 60-90% discount, with 2-minute notice if AWS reclaims them. Right for:

• Batch processing.
• Stateless web servers (with autoscaling backup).
• CI/CD runners.
• Dev/test environments.

Wrong for: stateful workloads, anything that can't tolerate sudden termination.

Lifecycle policies automate transitions: "logs > 30 days → IA, > 90 days → Glacier."

tags:
  Environment: production
  Team: platform
  Cost-Center: engineering
  Application: api
  ManagedBy: terraform

Cost Explorer can break down spend by tag. Now you can see "Team X spent $50k this month, of which $30k was on dev environments."

• Tags on everything. Cost allocation reports monthly.
• Budgets with alerts. AWS Budgets emails when projected spend exceeds threshold.
• Quarterly rightsizing review. Run Compute Optimizer; act on findings.
• Savings Plans for baseline workloads.
• Auto-shutdown non-production environments.
• Regular cleanup sweeps — old snapshots, detached volumes, unused IPs.
• One person owns cost — typically a senior infra engineer or finance partner.

AWS handles security of the cloud (physical, hypervisor, default service security). You handle security in the cloud (your configuration, code, data, access).

Don't assume "AWS handles security." Misconfigured S3 buckets, leaked IAM keys, open security groups — these are all your responsibility.

Most-publicized AWS breaches involve public S3 buckets. By 2026, AWS has block-public-access ON by default, but legacy buckets and misconfigurations still happen.

The defense:

• Account-level public access block (overrides any bucket settings).
• Bucket-level public access block.
• S3 Storage Lens flags public buckets.
• For "public" content (CDN-served images), use CloudFront with Origin Access Identity. Bucket stays private.

Pre-commit hooks, GitHub push protection, and runtime scanning catch secrets in code. Detailed in the Secrets section.

• Scan images for CVEs (Trivy, ECR scanning).
• Run as non-root.
• Read-only root filesystem where possible.
• No latest tags — pin specific versions.
• Minimal base images (alpine, distroless).
• Sign images (Cosign / Notary).

If you're subject to SOC 2, HIPAA, PCI-DSS, GDPR, etc., security work overlaps heavily with compliance:

• Encryption at rest and in transit (all of them).
• Access logs and audit trails (SOC 2, HIPAA, SOX).
• Access reviews quarterly (SOC 2).
• Vulnerability management (SOC 2, PCI).
• Backup and DR plans (SOC 2, HIPAA).
• Vendor management (SOC 2, GDPR DPAs).

Build the security controls; compliance evidence comes from them.

These stories share a pattern: a small problem cascades because the system wasn't designed to fail gracefully. The slow query, the long timeout, the missing index, the forgotten cert — none of these were the actual cause. The actual cause was that the system amplified the small problem instead of containing it.

The defenses are well-known: short timeouts with bulkheads, retry budgets, circuit breakers, expand-contract migrations, automated cert renewal, statement timeouts, stale-while-revalidate, auto-shutdown of non-prod, specific IAM policies, billing alarms, region tagging.