Multi-CDN Failover Architecture
A resilience architecture that removes a single CDN/edge provider as a single point of failure, using DNS-based traffic management to automatically fail over between two independent CDN providers.
Project Brief
- Role
- Architecture and implementation owner
- Scope
- A business-critical web application portfolio spanning 2 independent CDN providers and a direct-origin fallback
- Collaboration
- Cloud, security, application, and on-call stakeholders
- Result
- Automatic health-driven failover with a one-tap manual override, while preserving TLS, WAF, routing, and authentication behavior
Evidence Included
Sanitized architecture topology
The production design is reconstructed below with provider and control-plane boundaries intact while internal identifiers stay private.
Failure-path methodology
Normal operation, automatic failover, and the manual incident-control path are documented step by step.
Component reference
A layer-by-layer table connects each cloud service to its operational role in the design.
2 independent (active + failover)
CDN providers
4 distinct root causes
Edge-case failure modes diagnosed
Seconds (1-click trigger)
Manual failover time
The Problem
A set of business-critical web applications sat entirely behind one CDN/edge provider. If that provider degraded or had an outage, every one of those applications went down at once — there was no fallback path. The goal was to put a second, independent CDN in place as an automatic failover target, without disrupting existing TLS, routing, or authentication behavior.
My Approach
- Selected a DNS-based traffic manager configured with priority routing — primary CDN → secondary CDN → origin — so failover is automatic based on active health probes rather than manual intervention.
- Stood up a second, independent CDN distribution set for every application, mirrored 1:1 against the existing primary CDN configuration (routing rules, custom domains, TLS).
- Replicated the existing edge security policy (WAF managed rule sets + custom allow/deny rules) on the secondary CDN so failover doesn't also mean a drop in protection.
- Wrote the entire stack as Infrastructure-as-Code so the topology is reproducible, diffable, and auditable rather than click-ops.
- Built a one-click manual failover trigger (serverless webhook + mobile shortcut) so an on-call engineer can force traffic off the primary CDN in seconds during a live incident or planned maintenance, without CLI access.
- Ran a cost/tradeoff analysis on the secondary CDN's security add-ons (bot management, managed rule groups) to right-size protection vs. spend.
Key Engineering Challenges Solved
Health-probe blind spots
The traffic manager's health checks didn't always reflect a real outage (e.g., an intentionally disabled edge endpoint reported as “healthy”), so I identified the correct trigger point for planned failover drills vs. relying on automatic detection alone.
Security policy parity
A managed WAF rule blocked legitimate server-to-server traffic that lacked a User-Agent header — diagnosed via sampled request logs and fixed with a scoped rule override rather than a blanket disable.
Identity/session continuity
Authentication callback URLs depended on which header-forwarding policy the CDN used, which could silently break single sign-on after a failover if misconfigured.
Client-side connection caching
Documented a browser-level HTTP/2 connection-reuse behavior that could make a failover look “stuck” from a single client's point of view, distinguishing it from a real infrastructure problem.
Architecture Diagram
Multi-CDN Failover Topology
Client
Client / Browser
DNS
Route 53
CNAME record
Traffic Steering
Traffic Manager
Priority routing + health probes
Edge (priority order)
Front Door
P1 · WAF + TLS
CloudFront
P2 · Function + WAFv2
Direct Origin
P3 · Last resort
Origin
Origin Compute
App Service / Container Apps / Static Web Apps
Manual Failover Control Plane → feeds into “Traffic Manager”
iOS Shortcut
HTTPS webhook trigger
Function App
PowerShell 7.4 + Managed Identity
ARM REST API
Toggle P1 endpoint
Entire stack provisioned as code via Terraform (azurerm + aws providers).
Traffic Flow & Methodology
Normal Operation
- 1Client resolves the application's DNS name, which points to an Azure Traffic Manager profile configured for priority routing.
- 2Traffic Manager's active health probes confirm Azure Front Door (P1) is healthy, and DNS resolution returns the Front Door endpoint.
- 3Front Door terminates TLS, applies its WAF policy, and forwards the request to the origin compute layer.
Failover Event
- 1Traffic Manager's health probes detect Front Door is unhealthy — or an on-call engineer manually triggers failover.
- 2DNS resolution shifts to Priority 2: the AWS CloudFront distribution.
- 3A CloudFront Function runs at viewer-request to apply geo-filtering and path-rewrite logic, with an explicit bypass for health-check paths.
- 4Requests pass through the AWS WAFv2 WebACL — managed rule groups plus custom rules mirroring the primary CDN's policy.
- 5CloudFront forwards to the same origin compute layer, preserving correct Host header behavior for authentication callback continuity.
- 6If both P1 and P2 are unavailable, Traffic Manager falls back to Priority 3: direct-to-origin.
Manual Failover Control Plane
- 1An iOS Shortcut sends an HTTPS request to an Azure Function App (PowerShell 7.4, Consumption plan).
- 2The function authenticates using its System-Assigned Managed Identity and calls the Traffic Manager ARM REST API directly — deliberately avoiding the Az.TrafficManager PowerShell module to eliminate cold-start dependency issues.
- 3This gives an on-call engineer a one-tap failover trigger from a phone, with no CLI or portal access required.
Component & Technology Reference
| Layer | Technology | Role |
|---|---|---|
| DNS / Traffic Steering | Azure Traffic Manager (Priority routing) | Health-probe-driven automatic failover between CDNs and origin |
| DNS Records | AWS Route 53 | CNAME records pointing application domains at the Traffic Manager profile |
| Primary Edge/CDN | Azure Front Door (Standard/Premium) | TLS termination, edge routing, WAF policy enforcement (P1) |
| Secondary Edge/CDN | AWS CloudFront | Independent failover CDN distribution (P2) |
| Edge Compute Logic | CloudFront Functions | Viewer-request geo-filtering, path rewriting, health-check bypass |
| Edge Security | AWS WAFv2 (Managed Rule Groups + custom rules) | Threat protection parity with the primary CDN's WAF policy |
| Origin Compute | Azure App Service, Container Apps, Static Web Apps | Application runtime behind both CDNs |
| Failover Automation | Azure Functions (PowerShell 7.4, Consumption plan) | Serverless control-plane for toggling failover state |
| Automation Identity | Azure Managed Identity (System-Assigned) | Passwordless auth from the Function App to Azure Resource Manager |
| Automation API | Azure Resource Manager (ARM) REST API | Direct endpoint enable/disable calls, bypassing SDK module dependencies |
| Incident Trigger | iOS Shortcuts | Mobile, one-tap manual failover trigger |
| Infrastructure as Code | Terraform (azurerm + aws providers) | Reproducible, version-controlled provisioning of the entire stack |
Stack
Cloud & Networking
Security
Infrastructure as Code
Automation
Practices
Skills Demonstrated
- ▸Multi-cloud network architecture and disaster-recovery design
- ▸Infrastructure as Code at production scale
- ▸Security-conscious edge configuration (WAF tuning without sacrificing functionality)
- ▸Rigorous, methodical debugging of distributed-systems failure modes
- ▸Building operational tooling that shortens incident response time
- ▸Clear technical documentation of non-obvious system behavior for future engineers