Plan: Blue/Green Deployments for Azure Container Apps¶

Status: Planning Priority: Next up (roadmap item: blue-green) Scope: Frontend, Backend, UIGen Container Apps on prod

Goal¶

Zero-downtime deployments with instant rollback. New code goes to a staging revision ("green"), smoke tests run against it, then traffic shifts 100% green. If anything fails, traffic stays on blue and green is deactivated.

Approach¶

Azure Container Apps supports multiple revisions with weighted traffic splitting natively. We use this instead of a second Container App Environment (cheaper, simpler).

Key concepts¶

activeRevisionsMode: Multiple — allows two revisions alive simultaneously
traffic array in ingress — split by revision name or label (blue, green)
Revision labels — stable names (blue/green) we assign to latest revisions
Smoke test gate — run Playwright/curl against the new revision's FQDN before shifting traffic

Phases¶

Phase 1 — Bicep: switch to Multiple revisions mode¶

Change activeRevisionsMode: Single → Multiple for frontend, backend, UIGen
Add trafficWeight to ingress: default 100% to label blue
Add revisionSuffix param so Bicep stamps each deploy with a unique suffix
No CI changes yet — existing deploys still work (new revision gets 0% traffic, blue stays at 100%)

Phase 2 — Workflow: staged traffic shift¶

In deploy-application.yml, after images are pushed:

Deploy new revision (0% traffic) — Bicep deploy as today, new revisionSuffix
Assign label green to new revision via az containerapp revision label add
Run smoke tests against green's direct FQDN (bypass main ingress)
If smoke tests pass → shift traffic: az containerapp ingress traffic set → green=100%, blue=0%
Assign label blue to green revision (it becomes the new stable blue)
Deactivate old revision
If smoke tests fail → deactivate green, keep blue, fail the workflow

Phase 3 — Rollback command¶

Add workflow_dispatch input rollback: true to deploy-application.yml
When set, find the previous blue-labelled revision and restore it to 100% traffic

Bicep Changes¶

infra/modules/container_apps.bicep:

bicep

// Add param
param revision_suffix string = ''

// Change per container app
activeRevisionsMode: 'Multiple'

// In template block
revisionSuffix: !empty(revision_suffix) ? revision_suffix : ''

// In ingress
traffic: [
  {
    label: 'blue'
    weight: 100
    latestRevision: false
  }
]

infra/main.bicep — pass revision_suffix down from workflow.

Workflow Changes¶

deploy-application.yml additions after image push:

yaml

- name: Deploy green revision (0% traffic)
  # az deployment sub create ... --parameters revision_suffix=green-<sha>

- name: Label new revision as green
  # az containerapp revision label add --name <app> --label green --revision <name>

- name: Smoke test green revision
  # curl https://<green-fqdn>/api/health

- name: Shift traffic to green
  # az containerapp ingress traffic set ... --label green=100 blue=0

- name: Promote green → blue
  # az containerapp revision label add --label blue --revision <green-revision>

- name: Deactivate old blue revision
  # az containerapp revision deactivate ...

Risks & Mitigations¶

Risk	Mitigation
First deploy has no `blue` label → traffic set fails	Bootstrap step: if no `blue` label exists, skip traffic shift and label current as `blue`
Postgres schema migrations run before traffic shifts	Alembic runs on startup — green revision runs migrations before receiving traffic. Ensure migrations are backward-compatible (expand/contract pattern)
Revision limit (100 per app)	Deactivate old revisions after each deploy
UIGen stateless?	Yes — no session state, safe for traffic shifting

Success Criteria¶

[ ] New revision receives 0% traffic until smoke tests pass
[ ] Traffic shift completes in < 30s after smoke tests
[ ] Failed smoke test leaves prod on the previous revision
[ ] Manual rollback via workflow_dispatch works within 2 minutes
[ ] No downtime measured during a deploy (Playwright keeps polling /api/health)

Open Questions¶

Should we run full Playwright E2E against green, or just /api/health + /api/feature-flags? (E2E adds ~3 min to deploy time)
Do we need blue/green on dev, or only prod?
Revision suffix strategy: green-<short-sha> or timestamp?