BGP Routing

AS numbers, iBGP vs eBGP, path selection attributes (MED/LocalPref/AS_PATH), route aggregation, and BGP hijacking attacks.

Overview

Overview BGP (Border Gateway Protocol) is the routing protocol that holds the internet together. As an exterior gateway protocol standardized in RFC 4271, BGP exchanges reachability information between autonomous systems (AS) — collections of IP prefixes under a single administrative domain identified by a unique AS number. Unlike interior routing protocols that optimize for shortest path within a network, BGP is a path-vector protocol that makes routing decisions based on policies and a rich set of attributes.

BGP sessions operate over TCP port 179, ensuring reliable delivery of routing updates. Two fundamental session types exist: eBGP (external BGP) connects routers in different autonomous systems and is the basis of inter-domain internet routing, while iBGP (internal BGP) connects routers within the same AS to distribute externally learned routes internally.

Understanding BGP is critical for SRE and network engineering roles at large-scale companies. Google, Amazon, and Meta each operate large autonomous systems and peer with hundreds of ISPs globally. BGP misconfigurations have caused some of the internet's most severe outages, including the 2021 Facebook incident where an erroneous BGP withdrawal made all Facebook properties unreachable worldwide for over six hours.

In cloud environments, BGP is used extensively: AWS Direct Connect uses BGP to exchange routes between on-premises networks and AWS, Transit Gateway route tables propagate BGP-learned prefixes, and Kubernetes CNI plugins like Calico use BGP to distribute pod CIDR routes across nodes. Mastery of BGP path selection, attribute manipulation, and troubleshooting is expected at the senior SRE and network engineer level.

Key concepts

Autonomous System (AS)
A collection of IP prefixes under a single administrative domain, identified by a 16-bit (1-65535) or 32-bit AS number. Public AS numbers are assigned by RIRs. Private AS numbers (64512-65535 for 16-bit) are used internally and stripped at eBGP boundaries.
Path Vector Protocol
BGP carries the full AS_PATH attribute listing every AS a route has traversed. This prevents routing loops (a router will not accept a route containing its own AS number) and enables policy-based path selection beyond simple hop count.
LOCAL_PREF
A BGP attribute used within an AS (iBGP only) to influence outbound traffic. Higher LOCAL_PREF is preferred. Routers use this to prefer one exit point over another. Default value is 100. Not advertised to eBGP peers.
MED (Multi-Exit Discriminator)
A BGP attribute advertised to eBGP peers to suggest a preferred entry point into your AS for a given prefix. Lower MED is preferred. Comparison is only done between routes from the same neighboring AS by default.
BGP Path Selection Order
BGP selects the best path using a deterministic algorithm: highest WEIGHT (Cisco-proprietary, local only), highest LOCAL_PREF, locally originated routes, shortest AS_PATH, lowest ORIGIN code (IGP less than EGP less than incomplete), lowest MED, eBGP over iBGP, lowest IGP metric to next-hop, lowest router ID.
Route Reflector
In iBGP, a full mesh of sessions is required between all routers (n*(n-1)/2 sessions). Route reflectors solve this scaling problem by acting as a hub, re-advertising iBGP-learned routes to other iBGP peers (clients). The route reflector does not modify path attributes except adding CLUSTER_LIST and ORIGINATOR_ID to prevent loops.

When to use it

  • Designing multi-homed internet connectivity with multiple ISPs
  • Configuring AWS Direct Connect or Azure ExpressRoute BGP sessions
  • Building large-scale Kubernetes clusters using Calico or similar CNI with BGP
  • Investigating internet routing incidents, prefix hijacks, or route leaks
  • Architecting anycast routing for global load distribution

Interview questions

  • Explain the BGP path selection algorithm. Given multiple paths to the same prefix, how does BGP choose the best one?
  • What is a BGP route leak and a BGP hijack? How would you detect and mitigate each?
  • How does BGP work over AWS Direct Connect? Walk through the configuration and what happens if the BGP session drops.

Sign up free to read the full topic

More Networking topics

Start free on Cariara · Coding problems · System design studio