The secret zero problem: who hands out the first secret?

The secret zero problem describes a loop: to read its passwords in a secrets vault, an application must first authenticate, so it must hold a first secret. That secret must be protected in turn, and putting it in a second vault calls for yet another one. The modern answer breaks the loop: you no longer give the application a secret, you give it a provable identity, attested by the platform that runs it.

This lesson is for the DevOps engineer who is moving secrets out of the code and hits this question. Three measurements illustrate it: a SPIRE 1.15.3 join token is rejected the second time it is used, a Kubernetes ServiceAccount token is invalidated when the pod is deleted, and an AppRole SecretID wrapped by Vault 2.1.1 can be unwrapped only once.

What you will learn

The word secret here means any sensitive data an application uses to connect to a service: a database password, an API key, a token. In concrete terms, you will be able to:

• Describe the secret zero problem and the recursive loop it creates
• Tell apart the bad answers (hardcoded secret, environment variable) from the good ones
• Explain how workload identity and attestation break the loop
• Reduce the risk when a bootstrap secret is unavoidable: single use, short duration, wrapping
• Define the key terms of the field: attestation, OIDC, SPIFFE, SVID
• Link this choice to requirement SOCLE-SEC-GEN-5 (in French), which favours short-lived identities (OIDC)

What is the secret zero problem

The secret zero problem (also called the bootstrap trust problem) starts from a simple observation. Good practice forbids writing a password in the code: secrets go into a secrets vault, software that keeps them encrypted and only hands them out to authorised requesters. The application queries the vault at startup.

A question arises straight away: how does the application prove to the vault that it is entitled to access the secrets? The vault hands out nothing to a stranger. You therefore have to present it with a first token or a first password: that is the secret zero. The problem has moved without being solved, there is still a secret to put somewhere.

A recursive difficulty

The difficulty repeats endlessly, as the lower part of the diagram shows. If the secret zero is in clear in the code, you are back to the starting problem. If it is stored in a second vault, that vault also needs a secret to open, a secret zero number two. Each added layer moves the first secret, none removes it.

Source · SVG

Esc to close, + and − to zoom, scroll to move around. 100 % − Fit + Close

The upper part of the diagram, in red, shows the way out: the platform (cloud, Kubernetes) attests an identity that the vault checks, and the vault hands short-lived secrets to the application. There is no longer a fixed first key to hide, only a proof that the platform renews.

Why this problem concentrates the risk

As long as a first secret is stored somewhere, an attacker who finds it gains access to everything else. Protecting a thousand secrets well is pointless if the one that opens the vault lies around in a configuration file.

The possible answers

Faced with this loop, there are convenient but dangerous answers, and solid answers that really break it. The difference comes down to one idea: stop passing on a secret, and start proving an identity.

The bad answers

The first bad answer is the hardcoded secret, written directly in the source code. The secret zero ends up in a versioned file, often pushed to a Git repository, sometimes a public one. Once in the history, it stays there even after the file is deleted, as long as the history is not rewritten, and above all as long as it is not revoked.

The second is the environment variable, a value injected into the process environment at launch. The secret no longer lives in the repository, but it still has to be defined somewhere: a deployment script, a configuration file, an orchestrator. And environment variables leak easily: logs, error reports, diagnostic tools that list a process's environment.

What these two approaches have in common: the secret zero remains a static secret, a fixed value that has to be put somewhere and stays valid for a long time. The lesson on static secrets details why this model accumulates risk.

The good answer: workload identity through attestation

The modern answer turns the reasoning around. Instead of a secret, the application is given a workload identity (the identity of a running program, distinct from a human's). The application no longer says "here is my password", it says "here is who I am, check it with the platform that runs me".

The mechanism that makes this possible is attestation: having a fact confirmed by a trusted third party. The execution platform (the cloud, Kubernetes) knows which application it runs, and can vouch for its identity to the vault. The vault no longer trusts a secret presented by the application, it trusts the platform's signature.

Two standards shape this field. OIDC (OpenID Connect) is a very widespread authentication protocol: the platform issues a signed, short-lived OIDC token that the vault checks. This token is not a secret to hide for the long term: it expires quickly and carries a precise audience, so it is worthless elsewhere or later.

The second is SPIFFE (Secure Production Identity Framework For Everyone), an open framework that gives each workload a verifiable identity, independent of the cloud provider. This identity takes the form of an SVID (SPIFFE Verifiable Identity Document), issued and renewed automatically. Measured on SPIRE 1.15.3, the reference implementation: an X509-SVID lasts 1 hour and a JWT-SVID 5 minutes by default.

What attestation changes: measurements

Attestation does not remove every initial proof, it makes it short-lived, bound to a context and often single-use. The two measurements below show what this changes in practice compared with a password stored in a file.

The SPIRE join token works only once

To register a node without hardware proof, SPIRE offers a join token: a token generated by the server, here with a lifetime of 600 seconds. The agent uses it for its node attestation, then the server invalidates it. A second agent that replays the same token is rejected.

$ spire-server token generate -spiffeID spiffe://example.org/node/verif -ttl 600
Token: f5c790ed-beb5-4a48-af8e-3d407d42629c

# First agent
level=info msg="Node attestation was successful" ...

# Second agent, same token
level=error msg="Agent crashed" error="... failed to attest:
  join token does not exist or has already been used"

The join token remains a secret zero, but a disposable one: stolen after use, it is worth nothing. In production, SPIRE prefers attestation by the platform (cloud instance identity document, TPM, Kubernetes ServiceAccount token), which requires no secret placed by hand.

The Kubernetes token is bound to its pod

In Kubernetes, a pod's identity is a JWT signed by the cluster API, projected into the container. Measured on Kubernetes 1.37: a token projected with expirationSeconds: 3600 lasts one hour, carries the name of the pod and its UID, and becomes invalid as soon as the pod is deleted.

# Token review (TokenReview) after the pod is deleted
"error": "[invalid bearer token, service account token has been invalidated]"

An attacker who copies this token does not get a permanent key: they get a proof that expires within the hour and dies with the pod. The lesson on machine identity shows how a cloud or a vault consumes this token.

When a bootstrap secret is unavoidable

Some environments offer no attestation: a physical server without a TPM, a platform that cannot issue an OIDC token. You then have to deliver a bootstrap secret, but you can make it unusable once stolen. This is secure introduction, which Vault implements with the AppRole method.

AppRole splits the credential into two halves: a RoleID, not very sensitive, placed in the image, and a SecretID, delivered at the last moment. The SecretID can be limited to one use and to a duration, and delivered wrapped (response wrapping): the vault returns a single-use wrapping token that contains only the address of the real secret. Measured on Vault 2.1.1:

$ vault write auth/approle/role/app secret_id_ttl=10m secret_id_num_uses=1 \
    token_ttl=15m token_max_ttl=1h token_policies=default
$ vault write -wrap-ttl=60s -f auth/approle/role/app/secret-id
wrapping_token_ttl:              1m

# First unwrap: the SecretID is retrieved
# Second unwrap (an attacker who intercepted the token):
* wrapping token is not valid or does not exist

# Login with RoleID + SecretID: 15-minute Vault token
token_duration          15m
# Second login with the same SecretID (num_uses=1):
* invalid role or secret ID

The benefit is twofold. Interception is useless after the fact, and it is detectable: if the legitimate application fails to unwrap, someone did it before it, and the alert must go off. The bootstrap secret is no longer an everlasting password, it is a single-use ticket valid for one minute.

The idea to remember

You do not remove the need to prove your identity to the vault. You replace a static secret placed by hand with an ephemeral proof, issued by the platform, or failing that with a single-use bootstrap secret. The loop breaks because there is no longer a durable first key to hide.

Putting it into practice

This concept is implemented with concrete tools. The lesson on machine identity and workload identity details how an application obtains a verifiable identity, through SPIFFE/SPIRE for internal exchanges or through the cloud providers' OIDC federation, then uses it to authenticate without handling a secret zero.

On the vault side, the HashiCorp Vault guide (in French) shows how a vault accepts an attested identity rather than a fixed password: Kubernetes auth method, JWT/OIDC, or AppRole when no attestation exists. Combined with workload identity, it closes the loop end to end.

Finally, least privilege remains essential: an identity, however solidly attested, must only open the secrets it needs. A reliable identity that opens the whole vault is still a risk, just better traced.

Common pitfalls

These four pitfalls come up in first deployments. The first is the most common: it gives an impression of security while the secret zero is still there, simply moved to another configuration.

The first pitfall is believing a vault alone solves the problem. Installing a vault is useful, but if the application accesses it with a fixed password stored in its configuration, the secret zero is hidden, not removed. The vault centralises secrets, it does not solve bootstrapping.

The second is treating an ephemeral token like a durable secret. An OIDC token or an SVID gets its security from its short lifetime and its automatic renewal. Copying it into a file or freezing it in a variable recreates exactly the static secret you wanted to get rid of.

The third is neglecting the root of trust. Workload identity relies on the platform that attests. If any container can claim any identity, or if the SPIRE registration entries are too broad, attestation is worthless. Security then concentrates on this root, which becomes the new critical point.

The last is forgetting revocation and audit. Moving to workload identity does not exempt you from knowing who accessed what, or from being able to cut off a compromised workload. A verifiable identity that is poorly traced remains a blind spot during an incident.

Key points

• The secret zero is the first secret, the one that lets an application open the vault where the others are stored.
• The problem is recursive: protecting it in another vault creates a new secret zero.
• The hardcoded secret and the environment variable move the static secret without breaking the loop.
• Attestation turns the problem around: the platform proves the application's identity, which no longer presents a secret.
• OIDC and SPIFFE provide ephemeral identities: measured on SPIRE 1.15.3, 1 hour for an X509-SVID, 5 minutes for a JWT-SVID.
• When a bootstrap secret is unavoidable, you make it single-use: SPIRE join token, wrapped AppRole SecretID, both rejected on second use.
• The root of trust becomes the critical point: the platform that attests must be protected.
• The evidence expected by SOCLE-SEC-GEN-5 is the use of short-lived or ephemeral identities (OIDC) rather than long-lived secrets.

Knowledge check

Check that you have the essentials of this lesson. The questions only cover what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →