Service Mesh

Envoy sidecar, Istio vs Linkerd, mTLS between services, traffic management, observability, and eBPF alternatives.

Overview

A service mesh is an infrastructure layer that handles service-to-service communication in a microservices architecture. It uses a sidecar proxy pattern: a proxy (Envoy) is injected into every pod. All inbound and outbound traffic from the application flows through the sidecar, transparent to the application code.

Control plane vs data plane

Control plane (Istiod) — distributes configuration to all sidecars: service endpoints, traffic policies (retries, timeouts, circuit breakers), and certificates for mTLS
Data plane (sidecars) — enforces policies on every request

Key capabilities

mTLS — each sidecar gets a short-lived certificate from the mesh CA; all service-to-service traffic is automatically encrypted and mutually authenticated
Traffic management — weighted routing (canary, A/B), retries, timeouts, circuit breaking, fault injection for chaos testing
Observability — L7 metrics for every service pair (request rate, error rate, p99 latency), distributed traces via OpenTelemetry, access logs — without any application instrumentation

Istio vs Linkerd vs Cilium

Istio — Envoy sidecar; powerful, feature-rich, higher resource usage (~50-100 MB RAM per sidecar)
Linkerd — Rust micro-proxy; lighter, simpler, less feature-rich
Cilium — eBPF kernel programs instead of sidecar proxies; near-zero overhead, no proxy containers, fewer features

Sidecar overhead: each Envoy adds ~0.5-2ms latency per hop. For high-throughput latency-sensitive services, eBPF (Cilium) eliminates this at the cost of some feature richness.

Key concepts

Sidecar proxy: Envoy container injected into every pod by a mutating webhook. All pod network traffic flows through the proxy via iptables REDIRECT rules. Application code is unchanged.
mTLS in mesh: Each sidecar gets a short-lived certificate from the mesh CA (SPIFFE/SPIRE standard). Sidecars authenticate each other and encrypt all traffic automatically. No application code changes needed.
VirtualService (Istio): Defines routing rules: weight-based (canary), header-based (A/B), fault injection (delay/abort), retries, and timeouts. Applied on top of existing Kubernetes Services.
DestinationRule (Istio): Configures post-routing behavior: load balancing algorithm, circuit breaker settings (outlier detection), and TLS settings to the destination.
eBPF service mesh (Cilium): Uses eBPF kernel programs instead of sidecar proxies. Near-zero overhead, no proxy containers, policies enforced in the kernel networking stack.

When to use it

Implementing zero-trust networking for microservices without modifying application code
Getting L7 observability (request rate, error rate, latency) across all services automatically
Implementing progressive delivery (canary, A/B) at the traffic level without application changes
Replacing custom retry/timeout code in services with mesh-level policies

Interview questions

How does Istio implement mTLS between two microservices without changing the application code?