Kernel & Driver Build Environments

How to build the kernel driver and out-of-tree modules like amdgpu across kernel versions in CI, covering Kbuild, vermagic/ABI, DKMS, cross-compiling, and Secure Boot signing.

Overview

Building Against the Kernel, Not Against libc

An out-of-tree kernel module is not a normal C program. It has no libc, no dynamic linker, and no stable ABI. It is compiled against one specific prepared kernel tree and is only guaranteed to load into a kernel built from that exact tree, with that exact .config and compiler. This is why amdgpu built for 6.8.0-31-generic refuses to load on 6.8.0-35-generic with invalid module format, even though the two version strings look almost identical.

The Core Build Invocation

Every out-of-tree build delegates to the kernel's own build system, Kbuild. The canonical command is make -C /lib/modules/$(uname -r)/build M=$PWD modules. The -C switches into the prepared kernel build directory so the kernel's top-level Makefile and .config drive the compile; M=$PWD tells Kbuild to build the external module in the current directory using its small Kbuild/Makefile with obj-m += mydriver.o.

Why This Matters in CI

In an AMD GPU-stack CI role you never build just once. You build the same driver across a matrix of kernel versions and architectures, on runners where the headers arrive over NFS, where Secure Boot demands signed modules, and where a kernel upgrade on a test host must not silently break the driver. That is exactly what kernel-devel packages, vermagic, DKMS, CROSS_COMPILE, and module signing collectively solve. This topic walks the practical mechanics of each.

Key concepts

Kbuild and obj-m: The kernel's recursive make-based build system. An external module ships a Kbuild or Makefile that lists obj-m += foo.o (build foo.ko as a loadable module) and, for multi-file modules, foo-y := a.o b.o. You invoke it via the prepared kernel tree with make -C M=$PWD modules, so the kernel's flags, .config, and compiler settings govern the build rather than your own.
kernel-devel / linux-headers vs full source: You do not need full kernel source to build a module. The kernel-devel (Fedora/RHEL) or linux-headers-$(uname -r) (Debian/Ubuntu) package provides the prepared build tree: headers, Kconfig/.config, Module.symvers, and scripts/ (including modpost). It is exposed at /lib/modules/$(uname -r)/build, a symlink into that package. Full source is only needed to build the kernel itself.
vermagic: A version-magic string baked into every .ko by modpost, combining kernel release, SMP, preemption model, module-versioning state, and other ABI markers (for example 6.8.0-35-generic SMP mod_unload modversions). At load time the kernel compares the module's vermagic against its own; any mismatch yields invalid module format. It is the coarse gate that stops an ABI-incompatible module from loading.
Module versioning (modversions / Module.symvers): With CONFIG_MODVERSIONS=y, each exported symbol gets a CRC derived from its type signature, recorded in Module.symvers. A module stores the CRCs it was built against; if the running kernel's CRC for a symbol differs, load fails with disagrees about version of symbol. This catches ABI drift finer than vermagic (for example a changed struct that keeps the same kernel release).
DKMS (Dynamic Kernel Module Support): A framework that stores driver source under /usr/src/-/ with a dkms.conf, then rebuilds and installs the module automatically whenever a new kernel is installed. Driven by dkms add, dkms build, dkms install; kernel package post-install hooks trigger rebuilds so the driver survives upgrades. This is how AMD ships amdgpu-dkms.
ARCH and CROSS_COMPILE: Two make variables controlling cross-compilation. ARCH selects the target architecture subtree (for example arm64), and CROSS_COMPILE is the toolchain prefix (for example aarch64-linux-gnu-) prepended to gcc, ld, objcopy. You must build against a kernel tree prepared for that same ARCH; mixing a host x86_64 tree with ARCH=arm64 will not work.
Module signing and Secure Boot: Under Secure Boot with kernel lockdown, unsigned modules are rejected (or the kernel is tainted and load blocked). Modules are signed with scripts/sign-file using a private key whose matching certificate is enrolled in the platform (a MOK via mokutil, or a build-time key in CONFIG_MODULE_SIG_KEY). The signature is appended to the .ko; modinfo then shows sig_id/signer.

When to use it

You maintain an out-of-tree driver (for example amdgpu-dkms or a custom module) that must build and load cleanly across many kernel versions in CI.
A module compiled successfully but fails at modprobe/insmod with invalid module format or disagrees about version of symbol, and you must trace it to an ABI/vermagic mismatch.
You are packaging a driver so it automatically rebuilds when a test host receives a kernel upgrade, rather than silently losing the device after reboot.
You must cross-compile a driver for a different architecture (for example aarch64) from x86_64 build runners.
Target hosts enforce Secure Boot, so every module must be signed with an enrolled key or the kernel refuses to load it (or taints and blocks under lockdown).

Interview questions

How do you build an out-of-tree module in CI across a matrix of kernel versions?
How do you package a driver with DKMS so it survives kernel upgrades?
How do you diagnose a vermagic/ABI mismatch at load time?
How do you cross-compile a driver for aarch64 from an x86_64 runner?
How do you sign modules so they load on a Secure Boot host?
How do you handle NFS-mounted kernel headers and toolchains on the runner, and keep builds reproducible?