Cross-Compilation and Sysroots

Build, host, and target machines; GNU triples; what a sysroot actually contains; CMake toolchain files and the find_root_path knobs; and testing cross-built binaries under QEMU.

Overview

Cross-compilation is the routine case

Embedded Linux, mobile, WebAssembly, Apple Silicon universal binaries and any multi-architecture CI matrix all produce binaries on a machine that cannot run them. It feels hard because build systems assume the compiler, the libraries it links and the machine running the result are the same box, and every layer where that assumption leaks must be corrected by hand.

Hurdle 1: vocabulary

Build — Where the compiler runs — CMAKE_HOST_SYSTEM_NAME
Host — Where the result runs — CMAKE_SYSTEM_NAME
Target — Only when the result is itself a compiler — n/a

> [!TIP] Interviewers use this as a filter. Candidates who explain it cleanly have done it; those who say "host is where I am compiling" have read about it.

Hurdle 2: the sysroot

A cross compiler alone knows how to emit aarch64 instructions, but not what struct stat looks like on your target glibc, where libssl.so.3 lives, or what the target kernel headers say. The sysroot supplies that.

> [!WARNING] A sysroot that does not exactly match the deployment image is the largest source of production-only cross-build failures: glibc symbol version mismatches, a libstdc++ newer than the device's, headers describing a different structure layout than the running kernel.

Hurdle 3: silent reach for host artifacts

PROGRAM — NEVER — Host tools must run on the build machine
LIBRARY, INCLUDE — ONLY — Target artifacts must come from the sysroot

A cross build needs host tools and target libraries at the same time; the mode split expresses exactly that.

Hurdle 4: feature detection

Any configure check that compiles and runs a probe is broken under cross-compilation by construction.

Autotools — AC_RUN_IFELSE with a mandatory cross fallback value
CMake — try_run stops and demands a hand-populated cache variable
Both — Escape hatch: an emulator; CMAKE_CROSSCOMPILING_EMULATOR set to a qemu-user binary is the modern answer

Where interviewers push: validation

A successful build proves almost nothing. How do you know the artifact is correct?

Machine type and interpreter — file, readelf
NEEDED entries and RPATH — readelf -d
Float ABI tags — objdump
Test suite — qemu-user in CI
What emulation cannot reproduce — A smoke test on real hardware

Key concepts

Build / host / target: Autotools terminology for three machines: build runs the compiler, host runs the produced binary, target is the architecture a produced compiler will emit for. Only Canadian crosses have a meaningful target. CMake inverts the vocabulary: CMAKE_SYSTEM_NAME is the autotools host, CMAKE_HOST_SYSTEM_NAME is the autotools build.
GNU target triple: The arch-vendor-os-abi identifier that names a toolchain configuration, for example aarch64-unknown-linux-gnu or arm-none-eabi. The ABI field is load-bearing: gnueabi and gnueabihf differ in how floating point arguments are passed, and mixing them produces wrong results rather than link errors. gcc -dumpmachine prints the triple a compiler was built for.
Sysroot: A directory tree mirroring the target filesystem for compile and link purposes: usr/include with target and kernel headers, usr/lib and lib with target libraries and their symlinks, and pkgconfig data with sysroot-relative prefixes. Passing --sysroot re-roots the compiler built-in search paths rather than merely appending to them.
Toolchain file: A CMake script loaded before compiler detection via --toolchain or -DCMAKE_TOOLCHAIN_FILE. It sets CMAKE_SYSTEM_NAME, CMAKE_SYSTEM_PROCESSOR, the CMAKE__COMPILER paths, CMAKE_SYSROOT, and the find-root-path modes. Setting CMAKE_SYSTEM_NAME is what makes CMAKE_CROSSCOMPILING true.
CMAKE_FIND_ROOT_PATH_MODE_*: Four variables (PROGRAM, LIBRARY, INCLUDE, PACKAGE) controlling per-category search behaviour. ONLY searches only the re-rooted paths, NEVER ignores them and uses the host root, BOTH searches both. The standard cross configuration is PROGRAM NEVER plus LIBRARY, INCLUDE and PACKAGE set to ONLY.
CMAKE_STAGING_PREFIX: Where the install tree lands on the build machine, as distinct from CMAKE_INSTALL_PREFIX, which stays the runtime location on the target and is what gets baked into rpaths and generated config files. Omitting it means either installing over your own system or corrupting every prefix-derived path.
CMAKE_CROSSCOMPILING_EMULATOR: A command CMake prefixes when it needs to execute a target binary during a cross build. It is used for try_run generated executables, avoiding manual population of TryRunResults.cmake, and it seeds the CROSSCOMPILING_EMULATOR target property so tests run under the emulator too. Accepts a semicolon-separated command plus arguments since CMake 3.15.
Multiarch: Debian and Ubuntu scheme for co-installing libraries from different architectures by relocating them to /usr/lib/. Enabled with dpkg --add-architecture arm64 and used with package suffixes like libssl-dev:arm64. Distinct from multilib, which is several ABIs of one architecture under lib and lib64.

When to use it

Building for embedded Linux boards, set-top boxes, or automotive ECUs where the device has no compiler and no room for one
CI matrices that publish arm64 and amd64 artifacts from a single runner architecture without paying for emulated compile time
Consuming a Yocto or Buildroot SDK, where the vendor hands you a matched compiler plus sysroot plus environment script
Producing Windows binaries from Linux CI via mingw-w64, or macOS universal binaries covering both arm64 and x86_64 slices
Any WebAssembly or bare-metal target, where there is no possibility of a native build at all

Interview questions

Write a CMake toolchain file for an aarch64 Linux target and explain every line.
A colleague says the cross build works but the binary segfaults on the device. Walk through your diagnosis.
How do you handle configure-time checks that need to run a program, when you cannot run target binaries?
Compare the realistic ways to obtain a cross toolchain, and when you would pick each.
Design a CI pipeline that builds and tests for three architectures from x86-64 runners.