The KV (Key-Value) secrets engine is the most used one in Vault. A secrets engine is a component that Vault mounts on a path to store or generate secrets; KV simply stores key/value pairs such as passwords, API keys or tokens. KV v2, the second version of the engine, adds versioning: every change creates a new version, so you can read or restore earlier values.
This lesson covers every operation on KV v2 secrets, from basic storage to concurrency control with Check-and-Set. It is the first of the three lessons of the module: it fixes the paths under which your secrets live, and that authentication and policies will reuse.
What you will learn
- Enable the KV v2 engine and store a secret without writing it in clear text in the shell history
- Read a value, a precise version or a metadata history, including as JSON
- Avoid concurrent overwrites with Check-and-Set
- Tell apart
delete,undelete,destroyandmetadata delete - Organise paths so that a policy can scope them
- SOCLE anchoring: this lesson implements SOCLE-SEC-STO-1 (in French), secrets in a vault encrypted at rest, and SOCLE-SEC-STO-2 (in French), no clear-text secret in the persistent environment, the shell history or project files
Prerequisites
You need a test Vault server and a terminal. Dev mode (vault server -dev) is enough to follow this lesson, because it enables KV v2 by itself;
installing a persistent server is covered in the optional lesson
Install and configure Vault.
Read the module introduction first: it places the KV engine in Vault's access chain.
Three commands are enough to get that test server. In a first terminal, start the server, which stays in the foreground:
vault server -dev -dev-root-token-id="root"In a second terminal, point the CLI (command-line interface) at this server, then check that it answers:
export VAULT_ADDR='http://127.0.0.1:8200'export VAULT_TOKEN='root'vault status # Initialized true, Sealed falseDev mode keeps everything in memory and starts unsealed (ready to serve, without the unlock step of a real server): it disappears when the process stops, which is exactly what you want while learning. Never use it anywhere other than on your own workstation.
What is the difference between KV v1 and KV v2?
KV v2 is the default choice: Vault offers two versions of the KV secrets
engine, and they do not differ only by a few options. They are two distinct
engines, with different API paths and incompatible policies. A policy
(the rule that says which paths a token may reach, and with which actions)
written for a v1 mount does not work on a v2 mount. A mount is the place
where an engine is attached in Vault's path tree, for example secret/.
The decisive row of the table is Versioning: it drives everything else,
the soft delete as well as Check-and-Set. Keep the last row as your
rule of choice: KV v2 is the default, and KV v1 is only justified on an inherited mount or
on a write volume where the storage cost of versions has become a measured
problem.
| Aspect | KV v1 | KV v2 |
|---|---|---|
| Versioning | No | Yes (keeps the versions) |
| Soft delete | No | Yes (recoverable) |
| Check-and-Set | No | Yes (prevents overwrites) |
| Metadata | Basic | Complete (timestamps, custom) |
| Use case | Legacy, very high performance | Standard, recommended |
How do you enable KV v2 in production?
Run vault secrets enable -path=secret kv-v2. The mount point you choose
here is final in practice: it appears in all your policies, in the API URLs
and in the configuration of your applications. Renaming it later means
rewriting all of that. So pick an explicit name (secret, kv, or one
name per business domain) rather than the dev-mode default out of habit.
The check command confirms two things: that the engine is mounted in
the right place, and that the Options column of the detailed output does
contain version:2. Without -detailed, the list only shows the type kv
and you cannot tell a v1 from a v2.
# Enable KV v2 on the path "secret/"vault secrets enable -path=secret kv-v2
# Check the activationvault secrets list -detailed | grep '^secret/'How do you store a secret?
vault kv put serves both creation and update: the command is the
same, and Vault decides depending on whether the path exists. One point is
worth stating right away because it often surprises people: a write replaces
the whole secret, it does not merge. Rewriting a single field therefore
erases all the others. To change one key without touching the others, use
vault kv patch.
Which syntax should you use?
Use -mount=. Two ways of naming a secret coexist, and they are not equal. The
built-in help of vault kv put calls the first one a deprecated syntax
("deprecated path-like syntax"), precisely because it looks like an API path
without being one: secret/path/to/secret does not exist on the server side,
the real path is secret/data/path/to/secret. The second form explicitly
separates the mount point from the relative path and removes that
ambiguity.
# Deprecated form for KV v2: looks like an API path without being onevault kv put secret/path/to/secret key1="value1" key2="value2"
# Recommended form with -mountvault kv put -mount=secret path/to/secret key1="value1" key2="value2"What does a concrete example look like?
Imagine a web application that needs to connect to a PostgreSQL database:
vault kv put -mount=secret apps/webapp/database \ host="db.example.com" \ port="5432" \ username="webapp_user" \ password="SuperSecret123!"Output:
========== Secret Path ==========secret/data/apps/webapp/database
======= Metadata =======Key Value--- -----created_time 2026-03-16T13:32:40.176016777Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 1This example writes the password directly in the command: it therefore stays in the shell history and, while it runs, in the process list. That is acceptable with a fake value on a test server, but it is exactly what SOCLE-SEC-STO-2 (in French) forbids for a real secret. The section "How do you write values from a file?" below shows three ways to avoid it.
Why does the real path differ from the logical path?
You write to secret/apps/webapp/database, but Vault stores at
secret/data/apps/webapp/database. KV v2 uses API sub-paths to separate
operations, and the CLI inserts the right one for you.
| Operation | API path | Example |
|---|---|---|
| Data (CRUD) | secret/data/... | secret/data/apps/webapp/database |
| Metadata | secret/metadata/... | secret/metadata/apps/webapp/database |
| Delete (versions) | secret/delete/... | secret/delete/apps/webapp/database |
| Restore | secret/undelete/... | secret/undelete/apps/webapp/database |
| Destroy | secret/destroy/... | secret/destroy/apps/webapp/database |
The CLI hides this complexity, but you will meet it in:
- Vault policies (capabilities on
secret/data/*) - the direct HTTP API
- third-party integrations (Terraform, Kubernetes operator...)
How do you write values from a file?
Vault accepts several input formats. For complex secrets, you can read from a file or from stdin, which keeps the values out of your command line.
Option 1: key/value pairs from stdin
# The lone argument "-" tells the CLI to read a JSON object on stdinecho '{"api_key":"sk-abc123","api_secret":"secret-xyz789"}' | \ vault kv put -mount=secret apps/webapp/api-config -The JSON content is interpreted as several key/value pairs. Vault will
store two fields: api_key and api_secret. The benefit goes beyond comfort:
the values no longer appear in the arguments of the vault process, so no
longer in the output of ps that any user of the machine can read. The echo
of the demonstration is still written in the shell history; in real use,
make the JSON arrive from a file with 0600 permissions or from the output of
another tool.
Option 2: store a whole blob
If you want to store a whole file (certificate, private key) as a single value:
vault kv put -mount=secret apps/webapp/tls-cert \ cert="$(cat server.crt)" \ key="$(cat server.key)"Option 3: JSON file with @
# The file must contain a JSON object with the key/value pairsinstall -m 0600 /dev/null creds.jsoncat > creds.json << 'EOF'{ "username": "admin", "password": "secret123"}EOF
vault kv put -mount=secret apps/webapp/creds @creds.json
# Wipe the content before removing the fileshred -u creds.jsonThis command creates a secret with two fields: username and password.
install -m 0600 creates the empty file with the right permissions before
anything is written to it: creating then running chmod leaves a window during
which the secret is readable by everyone. shred -u overwrites the content
before removing the directory entry, which rm alone does not do. Its own help
page warns, however, that it assumes in-place rewriting: on a journaling or
copy-on-write file system, or on an SSD with wear leveling, the guarantee is
not total. The real reflex remains to write this kind of file in a tmpfs (a
memory-backed file system such as /dev/shm, never persisted to disk).
How do you read a secret?
Use vault kv get. A read returns data, metadata, or a single
field depending on the options, and the output is where you diagnose a
secret that "does not work".
How do you do a simple read?
Without options, vault kv get returns the latest version and displays the
values in clear text in the terminal. Two blocks follow each other in the
output: Metadata, which describes the state of the version (number, dates,
destroyed flag), and Data, which holds the fields. The first reflex with a
secret that "does not work" is to read version and deletion_time: a
soft-deleted secret still displays its metadata but no data at all.
vault kv get -mount=secret apps/webapp/databaseOutput:
========== Secret Path ==========secret/data/apps/webapp/database
======= Metadata =======Key Value--- -----created_time 2026-03-16T13:32:40.176016777Zcustom_metadata <nil>deletion_time n/adestroyed falseversion 1
====== Data ======Key Value--- -----host db.example.compassword SuperSecret123!port 5432username webapp_userHow do you extract a specific value?
For use in scripts, extract a single value:
# Extract the passwordvault kv get -mount=secret -field=password apps/webapp/database# SuperSecret123!How do you get JSON output?
For automated parsing, use the JSON format:
vault kv get -mount=secret -format=json apps/webapp/database | \ jq -r '.data.data.password'# SuperSecret123!How do you avoid overwrites with Check-and-Set?
Use the -cas=<version> option. When several processes write to the same
secret, there is a risk of silent overwrite. Check-and-Set (CAS) solves
this problem: the write only succeeds if the version you name is still the
current one.
What is the problem?
Without CAS, if two processes modify the same secret in parallel:
- Process A reads version 5
- Process B reads version 5
- Process A writes, which creates version 6
- Process B writes, which creates version 7 and overwrites A's changes
How does -cas solve it?
The -cas=<version> option requires that the write succeeds only if the
current version matches:
# Write only if the current version is 2vault kv put -mount=secret -cas=2 apps/webapp/database \ password="NewSecurePassword!"If another process modified the secret in the meantime:
Error writing data to secret/data/apps/webapp/database: Error making API request.
Code: 400. Errors:* check-and-set parameter did not match the current versionHow do you create a secret only if it does not exist?
Use -cas=0. This special case means "expected current version: none". It
turns the write into an exclusive creation and fails if the path already
exists. It is the provisioning tool: a bootstrap script that sets an
initial password must refuse to overwrite the one of an installation already in
service, rather than discovering the damage at the first application restart.
# Fails if the secret already existsvault kv put -mount=secret -cas=0 apps/webapp/new-secret \ api_key="sk-newkey123"How do you force CAS at the metadata level?
You can require CAS for all writers by setting cas-required on the
secret's metadata. After that, any write without -cas is refused.
# Any write without -cas will failvault kv metadata put -mount=secret -cas-required=true apps/webapp/databaseAttempt to write without CAS:
Error writing data to secret/data/apps/webapp/database: Error making API request.
Code: 400. Errors:* check-and-set parameter required for this callHow does versioning work?
Every change to a secret creates a new version. This is one of the major advantages of KV v2, and it is what makes rollback and audit possible.
How do you update a secret?
When you modify a secret, the version number increments automatically:
# Update the password (creates version 2)vault kv put -mount=secret apps/webapp/database \ host="db.example.com" \ port="5432" \ username="webapp_user" \ password="NewPassword456!"Output:
...version 2How do you read a specific version?
Use -version=N. All versions remain readable as long as they have not
been destroyed or pushed out by max-versions. That is what lets you
answer "what value did this secret have before Tuesday's deployment" in a few
seconds, without restoring a backup. Mind the downside: a revoked password
stays readable in the history by anyone who has read on the path, which
makes max-versions and delete-version-after necessary on sensitive
secrets.
# Read version 1 (old password)vault kv get -mount=secret -version=1 apps/webapp/databaseHow do you see the version history?
vault kv metadata get lists all versions without ever displaying their
values. It is the command to give to someone who must audit a secret
without being allowed to read it: it only requires the read capability on
secret/metadata/.... In the output, current_version is what a get without
options returns, oldest_version is the oldest version still recoverable, and
each Version N block carries its deletion_time and its destroyed flag. A
destroyed version stays listed: it is the metadata that remains, not the
data.
vault kv metadata get -mount=secret apps/webapp/databaseOutput (excerpt):
======= Metadata =======Key Value--- -----cas_required falsecreated_time 2026-03-16T13:32:40.176016777Zcurrent_version 2delete_version_after 0smax_versions 0oldest_version 0updated_time 2026-03-16T13:45:12.234567890Z
====== Version 1 ======Key Value--- -----created_time 2026-03-16T13:32:40.176016777Zdeletion_time n/adestroyed false
====== Version 2 ======Key Value--- -----created_time 2026-03-16T13:45:12.234567890Zdeletion_time n/adestroyed falseHow do you go back to an earlier version?
If an update causes a problem, you can "roll back":
# Restore version 1 as a new version (creates version 3)vault kv rollback -mount=secret -version=1 apps/webapp/databaseHow do you delete secrets?
KV v2 distinguishes several levels of deletion. The mental model:
| Action | Effect | Reversible? |
|---|---|---|
delete | Hides the version (soft delete) | Yes (undelete) |
undelete | Makes the version visible again | Not applicable |
destroy | Removes the data of the version | No |
metadata delete | Removes everything: versions + metadata | No |
What does delete (soft delete) do?
vault kv delete marks the version as deleted but keeps the data. A
soft delete is a deletion that can be undone, because the value is still
stored:
vault kv delete -mount=secret apps/webapp/databaseThe data is no longer accessible through a normal get:
vault kv get -mount=secret apps/webapp/database# Displays only the metadata, not the dataHow do you restore with undelete?
vault kv undelete clears the deletion_time of a version and makes it
readable again. Two differences with delete deserve attention. The restore
requires the explicit list of versions through -versions=; there is no
default behaviour on the latest one. And it only has an effect on a version
that was simply deleted: on a destroyed version, the command succeeds
without bringing anything back, since the data no longer exists.
# Restore version 2vault kv undelete -mount=secret -versions=2 apps/webapp/databaseThe data is accessible again.
What does destroy do?
vault kv destroy permanently removes the data of one or more versions:
# Permanently destroy versions 1 and 2vault kv destroy -mount=secret -versions=1,2 apps/webapp/databaseHow do you delete the whole secret object?
To remove a secret completely (all versions + metadata):
vault kv metadata delete -mount=secret apps/webapp/databaseThis command removes everything: the secret no longer exists in Vault.
How do you organise your secrets for policies?
Organise paths so that each branch matches one policy. A good organisation
makes policies and maintenance easier. The structure of your paths directly
determines your security policies. The lesson
Policies
writes those rules; here, we prepare the ground for wildcards (the *
that matches everything below a path).
What structure is recommended?
The tree below is not a Vault convention: the engine accepts any path. It is
built so that each branch matches a policy. Look at the first level
(apps/, infra/, teams/): it is the one that will carry your
wildcards. A secret filed in the wrong place forces you either to widen an
existing policy or to write one for a single key, and that is how permissions
drift.
secret/├── apps/ # Application secrets│ ├── webapp/│ │ ├── database # DB credentials│ │ ├── api-keys # Third-party API keys│ │ └── config # Sensitive config│ └── mobile-app/│ └── firebase├── infra/ # Infrastructure secrets│ ├── aws/│ │ └── credentials│ └── docker-registry/│ └── credentials└── teams/ # Per-team secrets ├── dev/ └── ops/Why this structure?
This hierarchy lets you write precise policies without overly broad wildcards:
# Policy for the webapp application (read only)path "secret/data/apps/webapp/*" { capabilities = ["read"]}
# Policy for the ops team (read/write on infra)path "secret/data/infra/*" { capabilities = ["create", "update", "read", "delete"]}
path "secret/metadata/infra/*" { capabilities = ["list", "read"]}Which criteria should drive the organisation?
Four axes of division are possible, and you must choose one as the top level of the tree, then combine the others below it. The criterion to place first is the one on which you want the strictest isolation boundary. On most installations it is the environment: a production policy must never be able to slip onto a development path because of a misplaced wildcard.
| Criterion | Example path | Advantage |
|---|---|---|
| By application | secret/apps/webapp/... | Per-service isolation |
| By environment | secret/prod/..., secret/dev/... | Prod/dev separation |
| By team | secret/teams/backend/... | Clear ownership |
| By type | secret/databases/..., secret/api-keys/... | Cross-cutting management |
Combine these criteria according to your context:
secret/prod/apps/webapp/database or secret/apps/webapp/prod/database.
How do you list secrets?
vault kv list explores one level of the tree at a time: it does not
descend recursively. Entries ending in / are intermediate directories
that you must list in turn; those without a slash are secrets. This command
hits the secret/metadata/... path and not secret/data/...: an application
that is allowed to read its secrets can therefore very well get a permission denied on a list, which is the desired behaviour.
# List the secrets under a pathvault kv list -mount=secret apps/Output:
Keys----webapp/mobile-app/What are custom metadata for?
custom_metadata attaches key/value pairs to the whole secret, not to a
version. This data is readable by anyone who has read on
secret/metadata/..., hence without access to the values: it is exactly what
an operations team needs to know who owns a secret and since when it has
not been renewed, without giving them the secret. Never put sensitive data
in it; it gets no extra protection.
vault kv metadata put -mount=secret \ -custom-metadata="owner=team-backend" \ -custom-metadata="environment=production" \ -custom-metadata="rotation=90days" \ apps/webapp/databaseHow do you limit the versions kept?
Use -max-versions. By default, Vault keeps all versions. This can be a
problem:
- Storage growth
- Old sensitive values kept longer than necessary
Limit it with -max-versions:
# Keep only the last 10 versionsvault kv metadata put -mount=secret -max-versions=10 apps/webapp/databaseThe same limit can be set once for the whole engine, through its
secret/config configuration endpoint. Keys that do not set their own
max_versions then inherit this value:
vault write secret/config max_versions=10vault read secret/configHow do you delete versions automatically after a soft delete?
Use -delete-version-after. max-versions bounds the number of versions,
delete-version-after bounds their lifetime. They answer different needs:
the first contains the volume, the second meets a retention requirement ("no
revoked password must stay recoverable for more than 24 hours"). The delay
applies to new versions written after the setting: Vault stamps them with a
deletion_time computed at write time. Mind the trap: what the deadline
triggers is a soft delete, not a destroy. The data stays present in the
storage and an undelete brings it back. For a genuine erasure requirement,
combine this setting with max-versions or an explicit destroy.
# Mark new versions for a soft delete 24h after they are writtenvault kv metadata put -mount=secret -delete-version-after=24h apps/webapp/databaseHow do applications consume KV secrets?
This lesson shows management through the CLI. In production, your applications consume secrets through other methods:
| Method | Typical use |
|---|---|
| HTTP API | Direct integration in code |
| Agent + templates | Injection into configuration files |
| CSI driver | Mount as a Kubernetes volume |
| Vault Secrets Operator | Synchronisation into Kubernetes Secrets |
These four methods change nothing about the KV engine; they change who authenticates. None of them should depend on a token written in a configuration file: the application proves its identity (Kubernetes service account, IAM role, certificate) and Vault hands it a short-lived token. The Vault authentication lesson details the methods available for a human and for a machine; the guide Vault as an identity broker describes the mechanism for Kubernetes and the cloud.
What does KV v2 not do?
KV v2 greatly improves the management of static secrets, but it has structural limits:
| Capability | KV v2 | Vault alternative |
|---|---|---|
| Generate dynamic secrets | No | Database, AWS, Azure... |
| Automatic renewal on the target side | No | Database secrets engine |
| Encryption of application data | No | Transit |
| X.509 certificates | No | PKI |
| Signed SSH keys | No | SSH secrets engine |
Simple rule: if you can avoid a static secret, do so.
- Database: Database secrets engine (ephemeral credentials)
- Cloud provider: AWS/Azure/GCP secrets engine
- Certificates: PKI secrets engine
- SSH: SSH secrets engine with certificates
KV v2 remains indispensable for:
- Third-party API keys (imposed by the provider)
- Legacy secrets (systems that cannot be integrated)
- Shared tokens (Slack webhooks, API tokens...)
- Bootstrap secrets (the first credentials needed to reach the rest)
How do you troubleshoot KV v2 errors?
Two families of errors dominate on KV v2, and you tell them apart by the
message. Everything that talks about a path (no value found,
permission denied) almost always comes from a missing data/ or metadata/
segment in a policy: the CLI adds it for you, the policy engine does not.
Everything that talks about check-and-set comes from a concurrent write or
from a cas-required set on the key; in both cases the answer is the same,
re-read the current version with vault kv get then replay the write with the
right -cas.
| Symptom | Probable cause | Solution |
|---|---|---|
no value found at secret/data/... | Secret missing or deleted | Check the path, use undelete |
permission denied | Insufficient policy | Check the read, list capabilities on secret/data/... |
check-and-set parameter did not match | Concurrent write | Re-read the current version, use -cas=<version> |
check-and-set parameter required | cas-required=true on the key | Add -cas=<version> to the command |
| Version not found | Version destroyed | Check with metadata get |
| Old versions inaccessible | max-versions reached | Raise the limit or accept the loss |
Key points
- KV v2 is the standard for static secrets; prefer dynamic secrets when possible
- Check-and-Set (
-cas) prevents concurrent overwrites; enable-cas-requiredon critical secrets - The CLI hides the API paths; your policies must target
secret/data/*andsecret/metadata/* delete= recoverable soft delete,destroy= permanent,metadata delete= total deletion- Organise your secrets to make policies easier: by application/environment/team
- Custom metadata document but trigger nothing
- Limit versions with
-max-versionsto avoid endless growth and excessive retention - Proof expected by SOCLE: secrets live in the vault, encrypted at rest (SOCLE-SEC-STO-1, in French), and an inspection finds no clear-text secret in the persistent environment, the shell history or project files (SOCLE-SEC-STO-2, in French)
Knowledge check
Check that the essentials of this lesson are in place. The questions only cover what has just been explained here.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
Continue the course: Vault authentication: userpass and AppRole
A new full quiz with random questions
Work again only on the questions you missed