Hybrid Cloud Connectivity

Overview

Overview Hybrid cloud connectivity is the practice of linking on-premises infrastructure with public cloud environments (AWS, Azure, GCP) in a secure, performant, and resilient manner. It is one of the most common architectural challenges faced by enterprise SREs and cloud architects, as most large organizations have existing on-premises systems that must coexist and interoperate with cloud-native workloads.

The connectivity options exist on a spectrum of cost, bandwidth, latency, and reliability. At one end, public internet with TLS provides basic connectivity but with unpredictable latency and no bandwidth guarantees. VPN tunnels over internet add IPsec encryption and are cost-effective for low-to-medium bandwidth needs. SD-WAN solutions improve internet-based connectivity with intelligent path selection and application awareness. At the other end, dedicated circuits (AWS Direct Connect, Azure ExpressRoute, GCP Dedicated Interconnect) provide consistent performance, high bandwidth, and reduced data transfer costs at higher fixed expense and longer provisioning times.

Transit Gateway with Direct Connect Gateway represents the modern AWS reference architecture for large-scale hybrid connectivity: a single TGW acts as the centralized hub, all VPCs attach to it, a Direct Connect Gateway bridges the on-premises Direct Connect circuit to the TGW, and a Site-to-Site VPN provides backup. This architecture centralizes routing policy, enables any-to-any connectivity between on-premises and all VPCs, and provides redundancy.

Latency is a critical consideration in hybrid architectures. Even with Direct Connect, round-trip latency from an on-premises data center to an AWS region can be 5-50ms. Applications with synchronous calls crossing the hybrid boundary — particularly databases and distributed transactions — are highly sensitive to this latency and may require architectural changes.

Key concepts

Hybrid Connectivity Options Spectrum

Public internet (no guarantee, cheapest) to IPsec VPN over internet (encrypted, 1.25Gbps/tunnel, minutes to provision) to SD-WAN with cloud integration (intelligent path selection, multiple internet paths) to Direct Connect Hosted Connection (dedicated bandwidth, partner-provisioned) to Direct Connect Dedicated Connection (physical port, 1G/10G/100G, highest performance).

Transit VIF + Direct Connect Gateway + TGW

The AWS reference architecture for scale. A Transit VIF on the Direct Connect connection terminates on a Direct Connect Gateway (global resource). The DXGW attaches to the TGW in each region. On-premises traffic enters via Direct Connect, routes through DXGW to TGW, and TGW routes to any attached VPC. This provides centralized connectivity without per-VPC VGW configuration.

Latency Considerations

Direct Connect typically provides 1-20ms RTT to AWS regions depending on geographic proximity. Over internet VPN, latency is higher and variable (10-100ms+). Applications using synchronous distributed transactions, database replication, or chatty RPC calls across the hybrid boundary should measure actual latency and consider async patterns or caching to absorb the round-trip overhead.

SD-WAN to Cloud Integration

SD-WAN vendors provide cloud-native integration: Cisco SD-WAN (Viptela) integrates with AWS Transit Gateway Connect using GRE tunnels and BGP for dynamic routing. VMware SD-WAN provides cloud gateways in AWS and Azure. Palo Alto Prisma SD-WAN connects to cloud via IPsec. These integrations allow SD-WAN-managed branches to reach cloud resources through the SD-WAN overlay.

DNS in Hybrid Environments

Hybrid environments require DNS resolution to work bidirectionally: on-premises clients resolving AWS private hostnames, and cloud workloads resolving on-premises hostnames. AWS Route 53 Resolver endpoints (inbound for on-premises queries, outbound for cloud queries to on-premises DNS) are the standard solution. Forwarding rules route queries for specific domains to the appropriate DNS server.

When to use it

• Designing on-premises to cloud connectivity for a multi-account AWS Organization
• Planning failover between Direct Connect and VPN for production hybrid workloads
• Integrating SD-WAN branch offices with cloud-hosted applications
• Analyzing latency impact of hybrid architecture on distributed application performance
• Migrating on-premises workloads to cloud while maintaining connectivity during transition

Interview questions

• Design a resilient hybrid connectivity architecture for an enterprise with a main data center and 20 branch offices needing to access AWS workloads.

More Networking topics

• eBPF Networking & XDP
• Envoy Proxy & xDS API
• gRPC vs REST
• Health Checks and Circuit Breaking
• HTTP/2 & HTTP/3
• HTTP/HTTPS & TLS
• IP Addressing & CIDR
• IPSec & VPN Deep Dive
• iptables and nftables
• IPv6 Addressing & Dual Stack
• Kubernetes Network Policies
• L4 vs L7 Load Balancing