C++ Test Automation and Coverage

GoogleTest and GoogleMock, Catch2 and doctest, CTest as the runner, and coverage with gcov/lcov and llvm-cov. Covers test tiering, flake hunting, and coverage gates that are not trivially gamed.

Overview

Failure modes other ecosystems mostly lack

Tests have a build time as well as a run time — Tests are compiled artifacts; in a large repo the build often dominates
Global state leaks between tests — Tests share a process
A test can crash the runner — It takes every other test in the binary with it
Coverage numbers are subtly wrong — Instrumentation interacts with optimization

The framework layer is largely settled

GoogleTest — Default for anything large: per-test fixtures, value-parameterized and typed tests, death tests, matchers that turn a failure into a readable diff
GoogleMock — MOCK_METHOD and EXPECT_CALL for interaction testing
Catch2, doctest — Alternatives, chosen mostly for compile time

> [!NOTE] Catch2 v3 abandoned the single-header model for a linked library: the clearest evidence that header-only test frameworks are a build-time liability at scale.

CTest: deliberately dumb

CTest runs commands and reads exit codes, which is why it works with any framework and why the engineering is in registration and properties.

gtest_discover_tests, not one test per binary — Per-test timeouts, per-test parallelism, failures naming the test
Labels — Tiers
RESOURCE_LOCK — Serializes the minimum necessary, not the whole run
--output-junit — CI reporting

Coverage: where confident wrong answers live

GCC — --coverage, gcov, lcov, genhtml
LLVM — -fprofile-instr-generate -fcoverage-mapping, llvm-profdata, llvm-cov

They are not interchangeable within one build.

Line — A line executed
Branch — Each decision outcome was taken
MC/DC (required by DO-178C for avionics) — Each condition in a compound decision independently affected the outcome

> [!WARNING] In C++ the line-vs-branch gap is unusually wide: short-circuit operators, ternaries, and implicit destructor and unwinding paths pack several decisions onto one line.

Tiering and gates

A forty-minute suite stops being run before merge, and its value collapses regardless of coverage percentage.

> [!TIP] A coverage gate is useful only if it cannot be satisfied without adding verification. A global threshold is gamed by executing without asserting and punishes whoever touches a badly covered file next. Diff coverage on changed lines, branch rather than line, and mutation testing survive contact with a team under deadline.

Key concepts

TEST_F fixture lifecycle: GoogleTest constructs a fresh fixture instance for each test, calls SetUp(), runs the body, calls TearDown(), then destroys it. Per test, not per suite. Anything shared across tests must be a static or a SetUpTestSuite member, and that sharing is where cross-test contamination comes from.
ASSERT_ vs EXPECT_: ASSERT_ produces a fatal failure and returns from the current function immediately; EXPECT_ produces a nonfatal failure and continues. Default to EXPECT_ so one run surfaces every problem. Use ASSERT_ only when continuing would dereference a null pointer or index out of range.
TEST_P and INSTANTIATE_TEST_SUITE_P: Value-parameterized tests: one body, a set of inputs, and each input registered as its own named test. Unlike a for loop inside a single TEST, a failing case is individually named, individually reportable, individually rerunnable with --gtest_filter, and does not stop the remaining cases.
gtest_discover_tests: CMake GoogleTest module function that runs the built test binary to enumerate its tests and registers each one with CTest. Yields per-test parallelism, timeouts, labels and reporting. gtest_add_tests is the older variant that scrapes source files and misses anything generated by a macro.
RESOURCE_LOCK: A CTest test property naming resources this test locks. Tests declaring the same lock never run concurrently; everything else still runs in parallel. The targeted alternative to RUN_SERIAL, which stops all other tests. Orthogonal to RESOURCE_GROUPS, which allocates counted instances rather than providing mutual exclusion.
ctest --repeat until-fail:N: Runs each selected test up to N times and passes only if every iteration passes. The flake-hunting mode. until-pass:N is the opposite — accept the first success — and is a retry policy that hides flakes rather than finding them.
gcov vs source-based coverage: GCC: --coverage emits .gcno at compile time and .gcda at process exit, consumed by gcov and aggregated by lcov and genhtml. LLVM: -fprofile-instr-generate -fcoverage-mapping writes .profraw controlled by LLVM_PROFILE_FILE, indexed by llvm-profdata merge -sparse, reported by llvm-cov. Do not mix them in one build.
Line vs branch vs MC/DC coverage: Line: this line executed. Branch: each outcome of each decision was taken. MC/DC: each condition in a compound decision was independently shown to change the outcome, requiring roughly N+1 test cases for N conditions. llvm-cov exposes all three plus function, instantiation and region metrics; -fcoverage-mcdc enables the last.

When to use it

Standing up a test suite in a large C++ repo: GoogleTest plus GoogleMock plus gtest_discover_tests, because per-test CTest registration is what makes everything downstream possible
Choosing a lighter framework: doctest when test binary compile time is the binding constraint, Catch2 v3 when you want its expression decomposition and are willing to link a library
Tiering a suite that has grown past the point where developers run it: LABELS unit and integration, ctest -L unit pre-merge, ctest -L integration nightly
Tests that contend on a fixed port, a shared database, or a GPU: RESOURCE_LOCK to serialize exactly those, keeping -j parallelism for everything else
Chasing a flake: ctest --repeat until-fail:100 -R name, --schedule-random, and a TSan build if threads are involved
Introducing a coverage gate: llvm-cov or lcov with branch coverage on, enforced as diff coverage on changed lines rather than a global percentage

Interview questions

Design the CMake and CTest layout for a repo with 4000 fast unit tests and 60 integration tests that need a real database and a fixed port.
Explain line, branch and MC/DC coverage, and why branch coverage matters more in C++ specifically.
A test passes when run alone and fails under ctest -j16. Walk me through diagnosing it.
When would you use TEST_P versus TYPED_TEST versus a plain loop inside one TEST, and what do you give up with the loop?
How do you enforce a coverage gate that actually improves quality instead of getting gamed?