Envoy Proxy & xDS API

Envoy listener-filter-cluster pipeline, the xDS dynamic configuration protocol, circuit breaking, outlier detection, and Envoy as the Istio data plane.

Overview

Overview Envoy is an open-source edge and service proxy written in C++. Originally developed at Lyft and donated to the CNCF, Envoy is the data plane for most production service meshes (Istio, AWS App Mesh, Consul Connect) and is used standalone in projects like Contour and Emissary-ingress.

Traffic flows through a layered pipeline: Listeners bind to ports and define filter chains. Network Filters process the raw byte stream. The HTTP Connection Manager (HCM) is the most important network filter, parsing HTTP. HTTP Filters process individual requests. Clusters define upstream endpoints. Envoy selects an endpoint using the configured load balancing policy (round-robin, least request, ring hash).

The xDS API (discovery services) is Envoy's dynamic configuration protocol. Control planes (Istio's Istiod, Consul) push configuration updates via gRPC streams without restarts. LDS pushes listener configs, CDS pushes cluster definitions, EDS pushes endpoint lists, RDS pushes route tables. An Envoy receiving xDS updates can shift traffic from v1 to v2 within seconds -- zero-downtime canary deployments without pod restarts or DNS changes.

Key concepts

xDS Protocol
xDS is a collection of gRPC streaming APIs between a control plane and Envoy. LDS (Listener DS) pushes listener configurations. CDS (Cluster DS) pushes upstream cluster definitions. EDS (Endpoint DS) pushes IP:port lists per cluster. RDS (Route DS) pushes HTTP route tables. Changes stream immediately to all connected Envoys without restart, enabling live traffic shifting (canary: route 20% to v2) by updating route weights in real time.
Circuit Breaking
Envoy implements circuit breaking per cluster with thresholds: max_connections, max_pending_requests, max_requests, max_retries. When a threshold is exceeded, Envoy returns 503 immediately to new requests instead of queuing. This prevents cascading failures where a slow upstream causes the caller to accumulate threads waiting for connections. Monitor cx_overflow and rq_overflow metrics to detect when the circuit breaker is triggering.
Outlier Detection
Outlier detection automatically ejects unhealthy upstream endpoints from the load balancing pool using real traffic signals. Envoy tracks consecutive 5xx errors and success rate per endpoint. When an endpoint exceeds thresholds, it is ejected for base_ejection_time and periodically retested. Unlike health checks (synthetic probes), outlier detection uses production traffic. Combined with circuit breaking: outlier detection removes bad pods, circuit breaking protects against overloading the remaining ones.

When to use it

  • Debugging Istio traffic management by reading Envoy admin interface (port 15000) config dump and access logs.
  • Configuring circuit breaking and outlier detection for upstream services in a service mesh.
  • Implementing traffic shifting for canary deployments via Istio VirtualService and DestinationRule.
  • Understanding why Envoy xDS enables incremental config updates while static Nginx config requires reload.

Interview questions

  • How does Istio use Envoy xDS to implement canary traffic shifting without pod restarts?

Sign up free to read the full topic

More Networking topics

Start free on Cariara · Coding problems · System design studio