MTU & IP Fragmentation

Overview

Overview Maximum Transmission Unit (MTU) and IP fragmentation are foundational networking concepts that become critical in cloud and overlay networking environments where encapsulation overhead can silently degrade or break connectivity.

The MTU is the largest IP packet that can be transmitted over a network link without fragmentation. Ethernet's standard MTU is 1500 bytes. This has been the dominant link MTU for decades. However, the effective MTU for application data is smaller: IP headers consume 20 bytes, TCP headers consume 20-60 bytes, leaving a Maximum Segment Size (MSS) of approximately 1460 bytes for TCP payload.

When a packet exceeds the MTU of a link in its path, one of two things happens depending on the IP header's Don't Fragment (DF) bit. If DF is not set, the router fragments the packet into smaller pieces that each fit within the MTU, then reassembles them at the destination (IPv4 only — IPv6 has removed router fragmentation entirely). If DF is set, the router discards the packet and sends an ICMP Type 3 Code 4 "Fragmentation Needed" message back to the sender, which should then reduce its packet size. This mechanism is called Path MTU Discovery (PMTUD).

Fragmentation is generally undesirable: it adds processing overhead at routers and end hosts, increases the number of packets, and any fragment loss requires retransmission of the entire original datagram. VPN and overlay encapsulation aggravate the MTU situation: an IPsec VPN adds 50-100 bytes of overhead, VXLAN adds 50 bytes, and GRE adds 24 bytes. If the outer MTU is 1500 bytes and the encapsulation adds 50 bytes, the inner payload must be 1450 bytes or less — but if inner senders assume 1500 bytes is safe, they will be setting DF and their packets will be dropped with ICMP messages that are often filtered by firewalls.

MSS clamping is the practical fix used in most VPN and tunnel deployments: a firewall or router rewrites the MSS value in TCP SYN packets to a safe value, preventing TCP endpoints from ever sending segments large enough to require fragmentation.

Key concepts

MTU vs MSS

MTU (Maximum Transmission Unit) is the maximum Layer 3 IP packet size for a link, typically 1500 bytes for Ethernet. MSS (Maximum Segment Size) is the maximum TCP payload in a single segment, negotiated during the 3-way handshake. MSS = MTU - IP header (20 bytes) - TCP header (20 bytes) = 1460 bytes for standard Ethernet. MSS ensures TCP does not create IP packets that exceed the MTU.

DF Bit (Don't Fragment)

A flag in the IPv4 header that instructs routers not to fragment the packet. When set, a router that cannot forward the packet without fragmentation must drop it and send ICMP Type 3 Code 4 (Fragmentation Needed) back to the source with the next-hop MTU. Most TCP stacks set DF by default to enable PMTUD. UDP applications often do not set DF.

Path MTU Discovery (PMTUD)

The mechanism by which an IP sender discovers the smallest MTU along the path to a destination. The sender sends packets with DF set. If a router must fragment but cannot (due to DF), it returns ICMP Fragmentation Needed with the max segment size it can handle. The sender reduces its packet size accordingly. PMTUD breaks when ICMP is filtered, causing MTU black holes.

Jumbo Frames

Ethernet frames larger than the standard 1500-byte MTU. Jumbo frames typically carry 9000-byte MTU payloads. Require all devices in the path (NICs, switches, routers) to support the larger frame size. Common in storage networks (iSCSI, NFS), high-performance computing, and cloud provider backbone networks. AWS VPC supports 9001-byte MTU within a VPC.

MSS Clamping

A technique where a router or firewall rewrites the MSS value in TCP SYN and SYN-ACK packets to a safe value that accounts for encapsulation overhead. Prevents TCP endpoints from negotiating an MSS that would result in IP packets too large for the tunnel. Implemented with iptables TCPMSS target on Linux. The recommended MSS for a 1500-byte outer MTU VPN with 50-byte overhead is 1410 bytes (1500 - 20 IP - 20 TCP - 50 tunnel).

When to use it

• Configuring VPN or VXLAN tunnels and ensuring inner traffic is not silently dropped
• Diagnosing mysterious TCP session hangs where handshake succeeds but data transfer fails
• Enabling jumbo frames in AWS VPCs or on-premises Ethernet for high-throughput workloads
• Configuring MSS clamping on VPN gateway iptables rules
• Explaining why IPv6 behaves differently from IPv4 regarding fragmentation

Interview questions

• Explain Path MTU Discovery and how it breaks. Give a concrete example with VPN encapsulation.
• When should you enable jumbo frames in a cloud environment, and what must you verify before doing so?

More Networking topics

• Kubernetes Network Policies
• L4 vs L7 Load Balancing
• Latency Diagnosis
• Load Balancing Algorithms
• MPLS
• mTLS & SPIFFE/SPIRE Workload Identity
• NAT & PAT
• Network Chaos Engineering
• Network Debugging Toolkit
• Network Observability & Flow Analysis
• nginx and HAProxy Configuration
• OSI & TCP/IP Models