HashiCorp Vault

Vault as a system: seal and unseal, Raft storage and HA, the secret engines and auth methods that matter on Kubernetes, policy design, leases, and what happens to your applications when Vault is unavailable.

Overview

Vault as a system you operate

Vault appears inside other secret-management topics as one option among several. This topic treats it as the system it is: on a platform team Vault is rarely a component you consume, it is a component you operate, and the interview questions are operational.

Short-lived, identity-bound secrets

Unbounded lifetime — Minted on the spot with a lease and a TTL
base64 in etcd — Encrypted; the database engine stores nothing
Visible to anyone with get on the Secret — Read only through a policy-bearing token
No audit trail tying use to a caller — Every credential traces to an authenticated identity

The database secrets engine is the clearest expression: the application never knows a password that outlives its session.

Architecture that explains the surprises

Storage is encrypted; the key is not in storage — A restarted Vault is sealed until a Shamir quorum or a KMS call unseals it
Integrated Storage (Raft) replicates data with one leader for writes — This is where HA comes from
Audit devices are a hard dependency — If Vault cannot write an audit record it refuses the request

> [!WARNING] A full audit disk is a Vault outage.

Kubernetes: secret zero solved

A pod already has a verifiable identity: its projected ServiceAccount token. Vault validates it against the cluster's TokenReview API and maps service account and namespace onto a Vault role with policies. No bootstrap secret is needed, so the secret-zero problem is solved rather than moved.

Vault Agent Injector — No
Vault Secrets Operator — Yes
External Secrets Operator — Yes

Where interviewers push

Seal model — What does auto-unseal cost you?
KV v1 to v2 — The path change that catches everyone
Kubernetes auth — How a pod is verified, and what Vault needs permission to do
Lease lifecycle — What breaks when a credential expires under a running app
Multi-tenancy — Namespaces (Enterprise) or path conventions (open source)
Blast radius — Vault is down; what still works?

Key concepts

Seal and unseal: Vault starts sealed and cannot decrypt its storage. Unsealing reconstructs the unseal key that decrypts the root key that decrypts the keyring. Shamir mode splits the unseal key into N shares with threshold K, requiring a manual quorum after every restart. Auto-unseal delegates the unwrap to a cloud KMS or HSM so a restarted node unseals itself; recovery keys replace unseal shares for quorum operations.
Integrated Storage (Raft): The default storage backend. Every node holds a full replica; the Raft leader serves writes and standbys redirect or forward. Configured with a node_id and retry_join stanzas. Autopilot handles dead-server cleanup and stabilization. Backups are vault operator raft snapshot save. Losing quorum halts writes until nodes return or a snapshot is restored.
KV v2 path split: A secret at kv/myapp/config is read and written at kv/data/myapp/config, while versions, soft-delete state, and custom metadata live at kv/metadata/myapp/config. Delete is a recoverable soft delete; destroy is permanent. check-and-set (cas) prevents lost updates. Policies must target the data/ and metadata/ paths, not the logical path.
Dynamic secrets and leases: Reading database/creds/ or aws/creds/ makes Vault create a credential on demand and attach a lease with a lease_id and TTL. Clients renew before expiry with an advisory increment the backend may cap at max_ttl. On expiry or explicit revocation, Vault runs the revocation path and the credential ceases to exist. Revoking a token revokes every lease created with it.
Kubernetes auth method: Exchanges a pod ServiceAccount JWT for a Vault token. Vault validates the JWT through the cluster TokenReview API, then matches the resulting service account name and namespace against a role bound by bound_service_account_names and bound_service_account_namespaces. Configured with kubernetes_host, kubernetes_ca_cert, and either a token_reviewer_jwt or reliance on the client JWT.
Policies: HCL granting capabilities on paths. Deny by default, so an empty policy grants nothing. Capabilities are create, read, update, patch, delete, list, sudo, and deny; deny always wins regardless of any other grant, including sudo. Trailing * globs match everything below a point, + matches a single path segment, and templated policies interpolate identity values such as the entity id.
Transit engine: Encryption as a service. The application posts base64 plaintext to transit/encrypt/ and receives ciphertext prefixed with the key version; Vault never stores the data. Key rotation creates a new version, min_decryption_version retires old ones, and rewrap re-encrypts existing ciphertext to the current version without exposing plaintext.
Audit devices: file, syslog, or socket sinks recording every request and response, with most string values written as an HMAC-SHA256 rather than plaintext. Vault guarantees the record reaches at least one enabled device and refuses the API request if it cannot, so a full disk on the only audit device is a Vault outage. Enable at least two devices.

When to use it

Any workload needing database, cloud, or API credentials that should be short-lived and per-instance rather than a shared static password
Centralizing secret access with an audit trail that ties every read to an authenticated identity, which a Kubernetes Secret cannot provide
Encryption as a service, where applications must encrypt data but must not hold or manage the key — the transit engine
Internal PKI at scale: issuing short-lived service certificates from a CA with per-role domain and key-usage constraints, without a manual CSR workflow
Multi-cluster or hybrid estates where secrets must be reachable from Kubernetes, VMs, and CI with one policy model instead of three

Interview questions

Walk me through what happens from Vault process start to a pod reading a database credential.
Compare KV v1 and KV v2, and explain the policy and migration consequences.
Design the auth and policy model for a multi-team Kubernetes platform on Vault. How do you get tenant isolation?
Compare the Vault Agent Injector, the Vault Secrets Operator, and External Secrets Operator. When do you pick each?
Vault becomes unavailable. Walk me through the blast radius and how you design so it is survivable.
Explain dynamic database credentials end to end, and compare them with static roles and with plain KV.