Kubescape and Kubernetes Runtime Security

Overview

Overview

Kubernetes security spans two time horizons. Static posture assessment catches misconfigurations before workloads are admitted; runtime security detects malicious behavior in workloads that already passed every gate.

A mature program needs both. No admission policy anticipates every zero-day or insider threat, and no runtime detector makes up for an attacker already holding an overprivileged RBAC role.

The tools and where they act

• Kubescape — Posture — Implements the NSA and CISA Kubernetes Hardening Guidance and MITRE ATT&CK for Kubernetes as executable controls
• Falco — Runtime — Watches syscalls through a kernel module or eBPF probe
• OPA Gatekeeper — Admission — Rego constraints wrapped in Kubernetes CRDs
• Kyverno — Admission — Kubernetes-native rules written as YAML patterns
• Pod Security Standards — Admission — Built-in controller enforcing Privileged, Baseline or Restricted per namespace
• seccomp, AppArmor — Kernel LSM — Restrict syscalls and filesystem paths below Kubernetes

Kubescape

Kubescape was created by ARMO and donated to the CNCF as the first open-source tool to implement the NSA and CISA guidance as machine-executable controls. MITRE ATT&CK maps techniques such as container escape, lateral movement via the service account token, and DNS-tunneling exfiltration to specific checks.

> [!TIP] The ATT&CK mapping gives security teams vocabulary aligned with threat intelligence reports rather than arbitrary checklist items.

Falco

Falco, originally built at Sysdig and now a CNCF graduated project, attaches to the kernel. Its rules are conditions in Sysdig filter syntax that can reference syscall arguments, the process tree, container metadata and pod fields at once.

• No kernel headers at deploy time
• Loads without a reboot
• Checked by the kernel's BPF verifier, so a probe bug is less likely to panic the node

Gatekeeper versus Kyverno

• Language — Rego — YAML patterns
• Audience — Teams fluent in Rego — Platform engineers fluent in manifests
• Webhooks — Validating and mutating — Validating and mutating, with more ergonomic mutation for tasks like image tag pinning or seccomp injection

Pod Security Standards

PSS replaced the deprecated PodSecurityPolicy API in Kubernetes 1.25, enforced by a built-in admission controller instead of a webhook. A namespace label picks the profile and the mode: enforce, audit or warn.

> [!IMPORTANT] Roll out Restricted in audit mode first, see what would be blocked, harden those workloads, then enforce.

seccomp and AppArmor

• RuntimeDefault seccomp — Maintained by containerd or CRI-O; blocks the most dangerous syscalls; a safe default for nearly all workloads
• Custom seccomp — Only the syscalls an app needs, found by profiling it under realistic load and generating a minimal allowlist

Key concepts

Kubescape Frameworks and Risk Scoring

Kubescape evaluates cluster resources against control frameworks defined as JSON configuration files. The NSA-CISA framework contains controls covering RBAC hardening, pod security, network policies, and secrets management. The MITRE ATT&CK for Kubernetes framework maps each control to a tactic and technique identifier such as TA0004 (Privilege Escalation) or T1525 (Implant Container Image). Each control has a weight that reflects its severity, and Kubescape computes a weighted risk score from zero to one hundred where higher values indicate better posture. Running a scan in CI mode against a directory of manifests uses the kubescape scan framework command. The exit code is nonzero when findings exceed a configurable threshold, making it easy to integrate with GitHub Actions or GitLab CI. The in-cluster operator creates WorkloadConfigurationScan and WorkloadConfigurationScanSummary custom resources for every namespace. A Prometheus ServiceMonitor scrapes the kubescape_resources_by_severity gauge, allowing Grafana alerting rules to fire when the critical finding count increases. The operator also integrates with the ARMO cloud platform for centralized multi-cluster posture management, though the open-source standalone mode does not require cloud connectivity.

Falco Rules: Condition, Output, Priority, and Tags

A Falco rule is a YAML document with four mandatory fields and several optional ones. The condition field is a Sysdig filter expression that can reference syscall arguments, process metadata, container information, and Kubernetes pod fields. The output field is a Go-style format string that embeds variable references for the alert message. The priority field is one of EMERGENCY, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFORMATIONAL, or DEBUG. The tags list helps downstream consumers such as Falco Sidekick apply routing or suppression rules. Macros are named boolean sub-expressions that can be composed into larger conditions, and lists are named YAML arrays of strings or patterns. Both reduce duplication across rules. Falco ships with a default ruleset that covers the most common attack patterns: reading sensitive files like /etc/shadow, writing to /etc/crontab, modifying binary directories, spawning shells inside containers, and making outbound network connections from containers that normally have no egress. Overriding a default rule is done by redefining it with the same name and append: true or by setting enabled: false to disable it entirely.

Falco eBPF Probe vs Kernel Module

Falco can instrument the kernel using two mechanisms. The classic kernel module (falco.ko) is compiled against the running kernel headers and loaded with insmod. It can intercept all syscalls but requires kernel headers at build time and can cause a kernel panic if it contains a bug, because kernel modules run in ring 0 with no sandboxing. The eBPF probe uses the Linux extended Berkeley Packet Filter subsystem. eBPF programs are verified by the kernel's built-in verifier before loading, which guarantees they cannot crash the kernel, loop infinitely, or access memory out of bounds. The probe is loaded into the kernel using the bpf() syscall and attaches to tracepoints. The modern eBPF driver (also called the Falco libs driver) is preferred because it requires only kernel version 4.14 or later with CONFIG_BPF_SYSCALL enabled, which covers all current distributions. GKE, EKS, and AKS all support the eBPF probe without additional node configuration. The eBPF probe does have slightly higher CPU overhead than the kernel module on very high syscall rate workloads because verified programs cannot use certain kernel optimization paths. For most production workloads the difference is below one percent of node CPU and the safety and portability benefits outweigh the cost.

Seccomp Profiles: RuntimeDefault and Custom

Seccomp (secure computing mode) is a Linux kernel feature that restricts which syscalls a process may invoke. In Kubernetes, a seccomp profile is set in the pod's securityContext at either the pod or container level. The RuntimeDefault value delegates profile selection to the container runtime (containerd or cri-o), which ships a hardened default profile that blocks roughly 44 of the most dangerous syscalls including ptrace, mount, create_module, and kexec_load while permitting the hundreds of syscalls that typical workloads need. A Localhost profile references a JSON file placed on each node at a path relative to the kubelet's seccompProfileRoot directory, which defaults to /var/lib/kubelet/seccomp. The JSON file lists syscalls and actions using the seccomp filter format. Generating a minimal custom profile is done by running the application under load with seccomp set to SCMP_ACT_LOG, capturing the logged syscall numbers from the kernel audit log or from strace, and then constructing the allowlist. Tools such as oci-seccomp-bpf-hook or inspektor-gadget automate this profiling step.

Pod Security Standards: Privileged, Baseline, and Restricted

Pod Security Standards are enforced by a built-in admission controller called PodSecurity. Each namespace receives zero or one PSS label per enforcement mode. The three modes are enforce (reject non-conforming pods), audit (allow but record in the audit log), and warn (allow but return a warning header to the client). A namespace can simultaneously have different profiles for different modes, for example Baseline enforcement with Restricted audit and Restricted warn, which lets teams see what would break under a stricter profile before enabling it. The Privileged profile applies no restrictions and is intended for cluster system components that need full host access. The Baseline profile prevents the most dangerous configurations: hostNetwork, hostPID, hostIPC, privileged containers, host path mounts, and dangerous capabilities such as NET_RAW or SYS_ADMIN. The Restricted profile adds all Baseline restrictions plus requirements for a seccomp profile (RuntimeDefault or Localhost), dropping all capabilities with capabilities.drop containing ALL, setting allowPrivilegeEscalation to false, and requiring runAsNonRoot or a nonzero runAsUser. The version label pins the policy evaluation to a specific Kubernetes version so that upgrading the cluster does not silently change which fields are checked. Using latest as the version tracks the current Kubernetes version's policy, which may add new checks that break existing workloads on upgrade.

OPA Gatekeeper vs Kyverno

Both tools implement Kubernetes admission webhooks but differ in their policy language and Kubernetes integration philosophy. OPA Gatekeeper uses a two-resource model: a ConstraintTemplate defines the Rego logic and a corresponding Constraint resource (generated as a CRD from the template) configures the scope and parameters. This separation lets cluster administrators install templates from a library and then allow namespace owners to instantiate constraints with their own parameters. Kyverno uses a single ClusterPolicy or Policy resource where rules are expressed as YAML patterns. A mutate rule can add fields, patch existing fields, or inject strategic merge patches. A validate rule can enforce field values, required fields, or patterns using the match and deny blocks. Kyverno also supports generate rules that create ConfigMaps, NetworkPolicies, or other resources in response to namespace or pod creation events. Gatekeeper is the better choice for organizations that already use OPA in other infrastructure contexts (Terraform, Envoy, Kafka) because Rego is reusable across policy domains. Kyverno is more accessible for Kubernetes-native teams and excels at mutation and generation use cases. Both tools support audit mode that generates PolicyReport resources listing all existing cluster objects that violate policy without blocking new admissions.

AppArmor Profiles in Kubernetes

AppArmor is a Linux Mandatory Access Control module that restricts what files, network operations, and capabilities a process is allowed to use, based on profiles loaded into the kernel at node startup. Kubernetes supports AppArmor profiles through annotations on pod metadata and, starting in Kubernetes 1.30, through the securityContext.appArmorProfile field at both the pod and container level. An AppArmor profile is a text file with three sections: the profile path (usually the container binary path or a wildcard), file rules, and capability rules. The complain mode generates audit log entries for violations without enforcing, which is used during profiling. The enforce mode blocks violations and optionally signals the process with SIGKILL. AppArmor profiles must be pre-loaded on each node where the pod can schedule. In managed Kubernetes, this typically means a DaemonSet that copies profiles and runs apparmor_parser to load them at startup, or a node configuration tool like node-feature-discovery combined with a node setup script baked into the AMI or machine image.

When to use it

• Running a Kubernetes security audit before a compliance review such as SOC 2, PCI-DSS, or FedRAMP, where documented evidence of cluster hardening controls is required.
• Setting up a new cluster and wanting to gate pull requests so that manifests failing NSA-CISA or CIS Benchmark controls are blocked before merge.
• Investigating a suspected container escape or credential theft incident where kernel-level syscall telemetry is needed to reconstruct the attack timeline.
• Enforcing organizational policies such as required resource limits, prohibited images from untrusted registries, or mandatory seccomp profiles across all namespaces without burdening each development team.
• Detecting cryptomining or reverse shell activity in workloads that passed static analysis but contain a runtime vulnerability or depend on a compromised dependency.
• Measuring posture regression over time after infrastructure changes, using Kubescape's Prometheus metrics and Grafana dashboards to track the weekly critical finding count.
• Hardening a multi-tenant cluster where different tenants require different Pod Security Standard profiles and the default namespace isolation is insufficient.
• Building a CI pipeline for Helm chart or Kustomize overlay validation where Kubescape scan failures should break the build before any artifact is published.

Interview questions

• How does Falco syscall detection differ from Kubernetes audit log analysis, and what threats does each approach miss?
• What syscalls does RuntimeDefault seccomp actually block, and when should you use a custom profile instead?
• How does Kubescape calculate its risk score and which controls carry the most weight?
• What does the PSS Restricted profile require, and how do you migrate a workload from Baseline to Restricted without downtime?
• When should you choose Kyverno over OPA Gatekeeper, and what are the operational tradeoffs?

More DevOps topics

• Kubernetes Runtime Classes
• Kubernetes Secrets Management
• Kubernetes Security: Pod Security and Runtime Hardening
• Kubernetes Services
• Kubernetes Storage
• Kubernetes the Hard Way
• Kustomization and Reconciliation
• Lease and Reservation Control Planes
• libstdc++, libc++, glibc and Symbol Versioning
• Lifecycle Hooks and Tests
• Linking Deep Dive
• Linux Cron & Task Scheduling