Cloud Run

Concept

Cloud Run is a fully managed serverless platform that runs stateless containers. You hand it a container image; it handles provisioning, TLS, autoscaling (including scale-to-zero), and request routing. It comes in two shapes: Services (request-driven, respond to HTTP/gRPC/events) and Jobs (run-to-completion tasks that don't serve traffic).

Problem it solves. Cloud Run removes almost all infrastructure operations for containerized workloads. You don't manage clusters, nodes, OS patching, or load balancers. You get per-request billing with scale-to-zero, so idle services cost nothing, while bursts scale out automatically to a configurable ceiling.

Limitations

• Request timeout ceiling (currently up to 60 minutes for services) — heavy work belongs in Jobs/Batch
• Instances are stateless; local disk is an in-memory/ephemeral filesystem
• Cold starts add latency unless you set minimum instances
• Per-instance concurrency and CPU/memory ceilings constrain very large single requests

Interactive architecture

flowchart TD
  U[User] --> DNS[Cloud DNS]
  DNS --> ARM[Cloud Armor]
  ARM --> XLB[External Application LB]
  XLB --> CR[Cloud Run Service - FastAPI]
  CR -->|Direct VPC egress| PSC[Private Service Connect]
  PSC --> DB[(AlloyDB - private IP)]
  CR --> SM[Secret Manager]
  CR --> LOG[Cloud Logging]
  CR --> TR[Cloud Trace]
  PUB[Pub/Sub topic] -->|OIDC push| CR2[Cloud Run - worker]
  CR2 -.nack.-> DLQ((Dead-letter topic))
  classDef gcp fill:#e8f0fe,stroke:#4285f4,color:#174ea6;
  classDef sec fill:#fce8e6,stroke:#ea4335,color:#a50e0e;
  class CR,CR2,DB,SM,LOG,TR,PSC,XLB,DNS gcp;
  class ARM,DLQ,PUB sec;

flowchart TD
  U[User] --> DNS[Cloud DNS]
  DNS --> ARM[Cloud Armor]
  ARM --> XLB[External Application LB]
  XLB --> CR[Cloud Run Service - FastAPI]
  CR -->|Direct VPC egress| PSC[Private Service Connect]
  PSC --> DB[(AlloyDB - private IP)]
  CR --> SM[Secret Manager]
  CR --> LOG[Cloud Logging]
  CR --> TR[Cloud Trace]
  PUB[Pub/Sub topic] -->|OIDC push| CR2[Cloud Run - worker]
  CR2 -.nack.-> DLQ((Dead-letter topic))
  classDef gcp fill:#e8f0fe,stroke:#4285f4,color:#174ea6;
  classDef sec fill:#fce8e6,stroke:#ea4335,color:#a50e0e;
  class CR,CR2,DB,SM,LOG,TR,PSC,XLB,DNS gcp;
  class ARM,DLQ,PUB sec;

How it works

A request hits Cloud Run's front end, which routes it to a revision. If no warm instance exists, Cloud Run cold-starts a container. Each instance handles up to `concurrency` requests at once. IAM decides whether the caller may invoke the service before your code ever runs.

1. 1 · Request arrivesCloud Run front end / load balancer

   TLS is terminated and the request is routed to the target revision (or split across revisions for traffic splitting).

   ⚠ 503 if no capacity and max-instances is hit; 429 on overload 📈 request_count, request_latencies, container_instance_count 💰 Requests are billed; ingress is free
2. 2 · Invocation authorization (IAM)IAM

   For private services the caller must present an ID token and hold roles/run.invoker. Public services skip this. This happens before your container runs.

   ⚠ 403 if the caller lacks run.invoker 📈 Cloud Audit Logs
3. 3 · Instance selection / cold startCloud Run autoscaler

   A warm instance handles the request if one is free (up to `concurrency`). Otherwise a new instance cold-starts: pull image, start container, pass health check.

   ⚠ Cold-start latency spikes; startup probe failures crash-loop the revision 📈 startup_latencies, instance count; set min-instances to avoid cold starts 💰 CPU is billed during startup; min-instances bill even when idle
4. 4 · Application handles the requestYour container (FastAPI + Uvicorn)

   Reads secrets from env/Secret Manager, connects to the database over a pooled connection, does work, emits structured JSON logs and trace spans with a correlation ID.

   ⚠ Unhandled exceptions → 500; exhausted DB pool → timeouts 📈 Structured logs, Cloud Trace spans, custom metrics 💰 vCPU-seconds and memory-seconds billed while the request is in flight (and always-on if CPU-always-allocated)
5. 5 · Egress to backendsDirect VPC egress / connector

   Outbound calls to AlloyDB/Cloud SQL/Memorystore route through the VPC to private IPs; external calls can go through Cloud NAT.

   ⚠ Connection refused if firewall/PSC/authorized-networks misconfigured 📈 VPC Flow Logs, DB connection metrics 💰 Cloud NAT and cross-region egress are billed
6. 6 · Response & scale-downCloud Run autoscaler

   The response is returned; idle instances are scaled down (to zero if min-instances=0) after a cool-down.

   📈 SLO dashboards, error budgets 💰 Scale-to-zero means no charge while idle

Hands-on example

Cloud Console

1. Cloud Console → Cloud Run → Deploy container
2. Choose an image from Artifact Registry (or 'Continuously deploy from a repository')
3. Set region, CPU/memory, concurrency, min/max instances
4. Under 'Authentication', choose 'Require authentication' for a private service
5. Under 'Connections', enable Direct VPC egress and select your VPC/subnet
6. Deploy, then copy the service URL

Code

PROJECT=my-project
REGION=us-central1
REPO=apps
IMAGE="$REGION-docker.pkg.dev/$PROJECT/$REPO/rag-api:latest"

gcloud config set project "$PROJECT"
gcloud services enable run.googleapis.com artifactregistry.googleapis.com

# Build & push with Cloud Build (no local Docker needed)
gcloud builds submit --tag "$IMAGE" .

# Deploy: private (no unauthenticated access), private DB egress, secrets, sane scaling
gcloud run deploy rag-api \
  --image "$IMAGE" \
  --region "$REGION" \
  --no-allow-unauthenticated \
  --service-account rag-api-sa@$PROJECT.iam.gserviceaccount.com \
  --concurrency 40 \
  --cpu 1 --memory 512Mi \
  --min-instances 1 --max-instances 20 \
  --timeout 120 \
  --network projects/$PROJECT/global/networks/prod-vpc \
  --subnet projects/$PROJECT/regions/$REGION/subnetworks/run-subnet \
  --vpc-egress private-ranges-only \
  --set-secrets DB_PASSWORD=alloydb-password:latest

# Grant a specific caller permission to invoke it
gcloud run services add-iam-policy-binding rag-api \
  --region "$REGION" \
  --member "serviceAccount:gateway-sa@$PROJECT.iam.gserviceaccount.com" \
  --role roles/run.invoker

# ---------- app/main.py : the FastAPI service ----------
import logging, json, os
from fastapi import FastAPI, Request
from pydantic import BaseModel

app = FastAPI()
logging.basicConfig(level=logging.INFO)

class Query(BaseModel):
    question: str

@app.get("/healthz")
def health() -> dict:
    return {"status": "ok"}

@app.post("/ask")
def ask(q: Query, request: Request) -> dict:
    # Structured log with a correlation id (Cloud Run injects trace headers)
    trace = request.headers.get("X-Cloud-Trace-Context", "")
    logging.info(json.dumps({"event": "ask", "trace": trace, "q": q.question}))
    return {"answer": f"You asked: {q.question}", "model": os.getenv("MODEL", "gemini")}

# ---------- client: invoking a PRIVATE service with an ID token ----------
import google.auth.transport.requests
import google.oauth2.id_token
import requests

def call_private(url: str, payload: dict) -> dict:
    # Mint an OIDC ID token whose audience is the target URL (ADC identity)
    auth_req = google.auth.transport.requests.Request()
    token = google.oauth2.id_token.fetch_id_token(auth_req, url)
    resp = requests.post(url + "/ask", json=payload,
                         headers={"Authorization": f"Bearer {token}"}, timeout=30)
    resp.raise_for_status()
    return resp.json()

URL="https://rag-api-xxxx-uc.a.run.app"
TOKEN="$(gcloud auth print-identity-token --audiences=$URL)"
curl -s -X POST "$URL/ask" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"question":"How does Cloud Run scaling work?"}'

resource "google_service_account" "rag_api" {
  account_id   = "rag-api-sa"
  display_name = "RAG API runtime SA"
}

resource "google_cloud_run_v2_service" "rag_api" {
  name     = "rag-api"
  location = var.region
  ingress  = "INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER" # private

  template {
    service_account = google_service_account.rag_api.email
    scaling { min_instance_count = 1, max_instance_count = 20 }

    vpc_access {
      network_interfaces {
        network    = google_compute_network.vpc.id
        subnetwork = google_compute_subnetwork.run_subnet.id
      }
      egress = "PRIVATE_RANGES_ONLY"
    }

    containers {
      image = "${var.region}-docker.pkg.dev/${var.project_id}/apps/rag-api:latest"
      resources { limits = { cpu = "1", memory = "512Mi" } }
      env {
        name = "DB_PASSWORD"
        value_source { secret_key_ref { secret = "alloydb-password", version = "latest" } }
      }
    }
  }
}

# Only the gateway SA may invoke it
resource "google_cloud_run_v2_service_iam_member" "invoker" {
  name     = google_cloud_run_v2_service.rag_api.name
  location = var.region
  role     = "roles/run.invoker"
  member   = "serviceAccount:gateway-sa@${var.project_id}.iam.gserviceaccount.com"
}

Troubleshooting

• 503 on deploy → container didn't listen on $PORT or failed the startup probe
• 403 invoking → caller lacks roles/run.invoker, or ID-token audience ≠ service URL
• Cold-start latency → raise min-instances; trim image size and startup work
• Can't reach AlloyDB → check Direct VPC egress, firewall, and PSC/authorized networks
• Timeouts on long work → move it to a Cloud Run Job or Batch; don't extend request timeouts indefinitely

Production considerations

What interviewers expect you to know

Common mistakes

• Deploying with --allow-unauthenticated by accident, exposing an internal API
• Using the default compute service account instead of a dedicated least-privilege SA
• Not bounding max-instances → connection storms that exhaust the database
• Trying to run long jobs inside a Service instead of a Cloud Run Job / Batch
• Baking secrets into the image or env instead of Secret Manager references
• Ignoring cold starts for latency-sensitive endpoints (no min-instances)
• Forgetting idempotency on Pub/Sub push consumers (at-least-once delivery)
• No dead-letter topic, so poison messages retry forever

Comparisons

Knowledge check