Architecture Atlas

system · deep · draft

Compute: VM / container / serverless / batch

Compute abstractions range from full guest VMs to containers under cluster managers, event-driven serverless, and batch frameworks — with a hard line between landmark papers (Borg/Omega/K8s lineage) and customer-facing cloud products.

In one sentence. Collapsing “servers” into one concept hides isolation, packing, scheduling, and scaling semantics that decide cost, blast radius, and operability.

Why it exists

Workloads differ: long-lived services need stable packing and isolation; bursty events fit serverless invocation models; analytics fits batch. Google’s Borg paper states nearly all production workloads ran in containers managed by Borg; Kubernetes incorporates Borg/Omega lessons with a REST control plane. Cloud products (EC2, ECS/EKS-class, Lambda, Batch, GKE, AKS, Functions, Cloud Run) are **public product surfaces**, not disclosures of internal Borg topology.

Visual walkthrough

Walkthrough

VM host packing containers

Illustrative: control-plane API places container tasks on a VM/machine. Borg paper: production workloads in containers; EC2/ECS docs are separate product surfaces.

Step 1 / 3
Operator / CIControl plane APIVM / machineVM / machineContainer AContainer B

Start from a VM/machine abstraction (Amazon EC2 product docs). The guest OS is shared by collocated workloads when you pack containers.

Walkthrough

Serverless invocation burst

Event-driven workers scale per invocation as product docs describe. Account concurrency limits still exist — do not treat marketing “infinite scale” as a primary claim.

Step 1 / 3
InvokeEvent sourceInvoke front doorFunction workerFunction workerDependency

An event source invokes the serverless front door. AWS Lambda docs describe the programming model and scaling behavior of the product.

Walkthrough

Batch and services on one cluster theme

Borg mixes latency-sensitive services and batch; MapReduce-style controllers run as jobs. Teaching contrast — not a 2026 Google topology claim.

Step 1 / 3
SchedulerLatency servicelatency-sensitiveBatch controllersbatch jobsWorker pool

Borg describes mixing latency-sensitive services and batch on shared machines with packing and isolation. Treat paper statements as era-labeled.

Control vs data plane

Control plane

Schedulers, admission control, APIs, and desired-state controllers: Borg/Omega papers; Kubernetes control plane via REST; Service Fabric as Microsoft’s microservice platform paper; managed product control planes (ECS, GKE, AKS, Lambda event sources). Do not equate paper-era cell sizes with 2026 live deployments.

Data plane

Where user code runs: VM guests, container tasks/pods, serverless workers, batch workers. Netflix chaos/Open Connect materials discuss control-plane services on AWS and bytes on OCAs — compute posture for playback control is AWS-shaped as published, not a Titus topology dump on this page.

Request / packet path

North–south

User/event → load balancer or API → service instances (VM/container) or serverless function → dependencies. Product docs describe scaling behavior of Lambda/Functions/Cloud Run — not each vendor’s private allocator.

East–west

Service-to-service RPC inside the cluster; mesh/proxy optional (see apis-edge). Batch shuffles (MapReduce lineage) move intermediate data across workers on Borg-era clusters as published.

Scaling & math

Teaching headroom: productiveSlots = floor(capacity × targetUtilization); reject or queue when concurrentDemand exceeds productive slots. Inspired by admission-control / packing themes in Borg — **illustrative**, not a Borg cell calculator or cloud quota API. Over-commit and priority are paper topics; product autoscaling knobs are separate cites.

Compute capacity headroom

Illustrative admission/packing headroom. Inspired by cluster-manager themes — not Borg telemetry or a cloud quota API.

Capacity headroom (illustrative)

productive = floor(capacity × targetUtil); headroom = capacity − productive; fits = demand ≤ productive

Inspired by admission/packing themes — not Borg cell telemetry or a cloud quota API.

productive = floor(100 × 70%) = 70; headroom = 30; demand 60 (60% of capacity) fits policy

Productive slots
70
Headroom slots
30
Fits policy?
yes

When it breaks

  • Pods/tasks reschedule; local disks/state may be lost.

    Cause. Worker machine failure under cluster manager.

    Mitigation. Replicate services across failure domains; treat local disk as ephemeral unless product says otherwise.

  • New jobs pending forever or rejected at admission.

    Cause. Cluster at capacity relative to admission/packing policy.

    Mitigation. Scale workers, reduce requests, or lower utilization target — teaching headroom calculator is illustrative only.

  • Function invocations throttled or queued despite “infinite scale” marketing language.

    Cause. Account/region concurrency limits in the serverless product.

    Mitigation. Read product concurrency quotas; design backpressure. Do not invent vendor hidden caps.

  • Job runtime dominated by slow tasks.

    Cause. Stragglers in large batch (MapReduce-class) workloads.

    Mitigation. Speculative execution / re-exec patterns as discussed in MapReduce lineage; monitor skew.

  • Operators apply Borg paper numbers to GKE/EKS capacity tickets.

    Cause. Conflating landmark papers with managed Kubernetes product SLOs.

    Mitigation. Cite Borg for lineage; cite GKE/AKS/ECS docs for product behavior.

Misconceptions

  • “Kubernetes is Borg with a different logo.” — Papers describe lineage and lessons; K8s uses a REST control plane rather than Borg’s store-access model.
  • “Serverless has no capacity limits.” — Product docs describe scaling and account concurrency; treat marketing “infinite” as non-authoritative.
  • “Everything at Google still matches the 2015 Borg cell sizes.” — Paper figures are era-labeled; live topology is a known unknown here.

Reference expression

Primary sources: Borg 2015, Borg/Omega/K8s 2016, Omega 2013, K8s Borg predecessor post, MapReduce 2004, Service Fabric 2018, AWS EC2/ECS/Lambda/Batch docs, GKE/Cloud Run docs, AKS/Functions docs. review: draft. Separate internal cluster OS from managed products.

Standards & sources

Known unknowns

  • Current Borg cell sizes, GPU/TPU scheduler details, and live Borg↔Kubernetes feature parity inside Google.
  • Exact Netflix container orchestration stack today beyond chaos/Open Connect primaries cited elsewhere.
  • Anthropic/OpenAI private training cluster topologies beyond MRC/Managed Agents materials (not deep-dived on this page).

Check yourself

  1. Borg’s paper states that Google production workloads largely ran…

    • Only on bare-metal without isolation
    • In containers managed by Borg
    • Only on AWS Lambda
    • Only as client-side WASM

    Answer: In containers managed by Borg. Borg 2015: virtually all production workloads in Borg-managed containers.

  2. Kubernetes relative to Borg/Omega is best described as…

    • Unrelated marketing
    • Open-source successor incorporating lessons; REST control plane
    • Identical store-access internals
    • A VXLAN underlay

    Answer: Open-source successor incorporating lessons; REST control plane. Borg/Omega/K8s 2016 + predecessor posts describe lineage.

  3. Amazon EC2 primarily provides…

    • Object put/get
    • On-demand virtual machines
    • EVPN Type-5 routes
    • OAuth AS

    Answer: On-demand virtual machines. EC2 docs: VM instances.

  4. AWS Lambda is best categorized as…

    • Event-driven serverless compute product
    • A Clos spine switch
    • A block volume type
    • An RFC for BGP

    Answer: Event-driven serverless compute product. Lambda docs: invocation-scaled compute.

  5. MapReduce is primarily associated with…

    • Interactive single-thread UIs
    • Large-scale batch over commodity clusters
    • Token-bucket edge quotas only
    • Satellite gateway RF

    Answer: Large-scale batch over commodity clusters. MapReduce 2004 formalized large-scale batch.

  6. The capacity headroom calculator on this page is…

    • Live Borg cell telemetry
    • Illustrative teaching math distinguishing paper themes from cloud quotas
    • GKE’s private autoscaler source
    • Netflix Titus internals

    Answer: Illustrative teaching math distinguishing paper themes from cloud quotas. Illustrative; label paper lineage vs product docs.

  7. Service Fabric (Microsoft paper) underpins…

    • Only Windows XP
    • Several Azure microservice platforms as listed in the paper
    • Only Starlink user terminals
    • Only OSPF areas

    Answer: Several Azure microservice platforms as listed in the paper. Service Fabric 2018 lists Azure services built on it.

  8. GKE / AKS / ECS docs should be read as…

    • Disclosures of Borg cell wiring
    • Customer-facing managed product surfaces
    • Proof of unpublished GPU counts
    • Unlabeled inference

    Answer: Customer-facing managed product surfaces. Managed products ≠ internal cluster OS topology.