Container security: images, clusters, runtime

Securing containers means placing a control at each of the three stages of their life: the build of the image, its admission into the cluster and its execution. This lesson is for a DevOps engineer who already runs containers and wants to know which control goes where. Everything is measured on Kubernetes 1.37 and Docker 29.1.3: the python:3.13 image carries 4,122 CVEs against 155 for the same application in distroless, a restricted namespace rejects a default nginx pod with four violations, and a Falco alert comes out a few seconds after a cat /etc/shadow.

Recommended reading

To master container security (namespaces, cgroups, isolation, runtime), Container Security, Liz Rice is the absolute reference. The author dives into the Linux kernel to explain why containers are what they are.

Amazon affiliate link. Buying through this link supports the site at no extra cost to you.

What you will learn

• Identify the five attack surfaces specific to containers
• Shrink an image and measure the CVE gain with a scanner
• Harden a namespace with the Pod Security Standards and NetworkPolicies
• Place each tool, Trivy, Cosign, Kyverno, Falco, at the right phase
• Avoid the pitfalls that silently disable a control
• Locate these controls in the Deployment and admission (in French) and Runtime and operations (in French) families of the SOCLE framework

Why are containers a target?

A container is not a virtual machine: it is an isolated process that shares the kernel of its host. A VM ships its own kernel, a container calls the node's kernel directly. If an attacker exploits a kernel flaw from a container, they can compromise the whole node, and with it every other container running there. The protections in this lesson all serve to narrow down what this shared kernel exposes.

The risk is not theoretical. In February 2018, the company RedLock revealed that attackers had taken over a Kubernetes cluster belonging to Tesla through an administration console left without a password. They were running a cryptocurrency miner on it, and the cluster gave access to cloud credentials and to telemetry data.

The 5 main attack surfaces

Understanding where the risks lie is the first step to reducing them. Each row of the table matches a control covered below: the scan for images, the Pod Security Standards for configurations, NetworkPolicies for the network.

Attack surface	Risk	Concrete example
Vulnerable images	CVEs in dependencies, outdated base image	Log4Shell present in an unpatched Java image
Permissive configurations	Root containers, excessive capabilities	A container with CAP_SYS_ADMIN can modify the host system
Exposed secrets	Plain-text environment variables, readable ConfigMaps	Database password visible through kubectl describe pod
Open network	No NetworkPolicy, free east-west traffic	A compromised pod can scan the whole cluster
Supply chain	Unverified third-party images, public registries	Docker Hub image hiding a miner

Swipe to see more

The last row is documented. In 2022, the Sysdig threat research team analysed 250,000 public Linux images on Docker Hub and identified 1,652 malicious ones: cryptocurrency miners, embedded secrets used as backdoors, DNS hijacking. An image pulled from a public registry is audited by nobody before it lands on your nodes.

Who should secure what?

Container security is a shared responsibility: the developer builds the image, the operator hardens the cluster, and the DevSecOps team automates the controls that link the two. The table says who acts, and the right-hand column gives the concrete action expected from each, not an intention.

Role	Main responsibility	Concrete actions
Developer	Build secure images	Minimal base images, no embedded secret, non-root USER
Ops / SRE	Harden the infrastructure	Pod Security Standards, NetworkPolicies, updates
DevSecOps	Automate the controls	Scan in CI, blocking admission, runtime alerts
CISO / Compliance	Ensure compliance	CIS audit, documented exceptions, evidence for audits

Swipe to see more

Detecting early costs less, and it can be measured. On this machine, Trivy 0.74.0 scans the application image in 9 seconds, including the download of its CVE database, then in under a second once the database is cached. Fixing the same flaw after a production incident is counted in days.

The defence-in-depth strategy

Container security does not rest on a single tool. It is organised around three phases that complement each other: Build, Deploy and Runtime. If one layer fails, the others limit the impact.

The diagram follows an image from its build to its execution. Look at the two side exits: admission rejects what is not compliant or not signed before anything runs, while runtime only alerts on what is already running.

Source · SVG

Esc to close, + and − to zoom, scroll to move around. 100 % − Fit + Close

Phase 1: Build, securing the images

This is the most effective moment to act: a vulnerable image should never reach production. Four actions are enough, detailed in the lessons on Dockerfile best practices and on the Trivy scanner.

• Minimal base images: fewer packages, fewer CVEs to handle.
• Scan systematically: a scanner such as Trivy or Grype in CI, blocking on critical CVEs.
• Never run as root: a non-root USER in the Dockerfile.
• Sign the images: Cosign proves origin and integrity, and the cluster can reject an unsigned image.

The minimal image is the action that pays off the most, and it can be measured. The multi-stage build below installs the dependencies in a full image, then copies only the result into a distroless image, which has neither a shell nor pip. Both stages use Python 3.13, the version of distroless/python3-debian13: a dependency compiled for another Python version would not load.

# Stage 1: install the dependencies with pip, in a full image
FROM python:3.13-slim@sha256:7c61056e61ac89e852de05f3dc6fa51a6dd2181797bceed46aa725dd7cb2cd3b AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --target=/app/deps -r requirements.txt

# Stage 2: keep only the result, without a shell or pip
FROM gcr.io/distroless/python3-debian13:nonroot@sha256:774595d652a294b54c9bd575b2d9fdd1a4b47547dc17b8bfa4c0e953c64855b3
WORKDIR /app
ENV PYTHONPATH=/app/deps
COPY --from=builder /app/deps /app/deps
COPY app.py .
USER 65532
CMD ["app.py"]

The point that makes most attempts fail: pip install writes to site-packages, not to /app. Without --target and without PYTHONPATH, the image builds without error but the application dies at start-up on a ModuleNotFoundError. So always check the image by running it: a docker build that succeeds proves nothing about the presence of the dependencies.

$ docker run --rm mon-app:1.0.0
requests 2.32.5 yaml 6.0.3 uid 65532

The gain, measured with Trivy 0.74.0 on the same application (requests and PyYAML):

Image	Size on disk	CVEs detected	of which CRITICAL
python:3.13	1.62 GB	4,122	19
application on distroless/python3-debian13	99.5 MB	155	0

Swipe to see more

Twenty-six times fewer CVEs to triage, and no critical one: everything that disappears, shell, package manager, compilers, is that much less attack surface. User 65532 is the one the nonroot variant of distroless already provides.

Phase 2: Deploy, hardening the configurations

Even a safe image can be deployed dangerously. Kubernetes offers the protections but enables none of them by default: without a label, a namespace accepts a privileged pod. Three controls are added here, and all of them sit on the namespace or the cluster, not on the image.

The Pod Security Standards (PSS) are three restriction levels that Kubernetes applies through a simple namespace label. The restricted level forbids running as root and requires dropping capabilities and a seccomp profile. The next lesson covers them in detail.

apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted

A typo does not behave the same way depending on where it lands. Measured on Kubernetes 1.37: a misspelled value (restrictd) is rejected by the API server, which lists the allowed values. A misspelled key (pod-security.kubernetes.io/enforse) is accepted like any other label, and the namespace remains without any constraint. So check the effect, not just the label: a non-compliant pod must be rejected.

The Namespace "pss-typo" is invalid: metadata.labels[pod-security.kubernetes.io/enforce]:
Invalid value: "restrictd": must be one of privileged, baseline, restricted

NetworkPolicies close the network. By default, all the pods of a cluster can reach each other: the rule below rejects all incoming traffic into the namespace until another rule allows it. It only has an effect if the cluster's network plugin (CNI) enforces NetworkPolicies, which each CNI's documentation states.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress

Two settings complete the picture. A ResourceQuota and a LimitRange stop a compromised pod from consuming all the node's resources, which would be a denial of service. Least-privilege RBAC limits what a stolen service account can do.

Phase 3: Runtime, detecting and blocking

The Build and Deploy phases are not enough: new CVEs come out every day, and a zero-day flaw is known to no scanner. Runtime protection watches the running containers, most often by observing their system calls.

Signal	What it detects	Tool
Abnormal system calls	Shell in a container, reading of sensitive files	Falco
Network connections	Traffic to suspicious IPs, port scan	NeuVector, Cilium
Unexpected processes	Cryptocurrency miner, reverse shell	Falco, Sysdig
Configuration drift	Files modified compared with the original image	NeuVector

Swipe to see more

Runtime detection comes after the intrusion: it limits the damage but does not prevent it. Measured with Falco 0.45.0 on a kind cluster: a cat /etc/shadow in a pod produces the Read sensitive file untrusted alert within a few seconds, but the file has indeed been read. The Build and Deploy phases remain the first line of defence.

Scenario: building security into your pipeline

Here is how a team chains the three phases so that no vulnerable or misconfigured image reaches production. Each step uses a tool seen in the course, with the pinned version used in the measurements, and each command fails when the control fails.

1. Build: scan the image

   CI runs Trivy in its official image pinned by digest and fails if a CRITICAL CVE is found. Calling the image directly removes the intermediary of a third-party action, as the Trivy lesson shows.

   docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
     aquasec/trivy:0.74.0@sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969 \
     image --scanners vuln --severity CRITICAL --exit-code 1 mon-app:1.0.0

2. Registry: sign the image by its digest

   The pushed image is signed with Cosign, pointing to its digest and not its tag: a tag can be moved, a digest cannot. The Cosign lesson covers keyless signing, which avoids managing cosign.key.

   cosign sign --key cosign.key registry.example.com/mon-app@sha256:<digest>

3. Admission: validate the manifests

   At deployment, an admission controller checks every pod before it is stored. This Kyverno policy of type ValidatingPolicy rejects any container without runAsNonRoot: true. Measured on Kyverno 1.19.1: an nginx pod without a securityContext is rejected with the policy message.

   apiVersion: policies.kyverno.io/v1
   kind: ValidatingPolicy
   metadata:
     name: require-non-root
   spec:
     validationActions: [Deny]
     matchConstraints:
       resourceRules:
         - apiGroups: [""]
           apiVersions: ["v1"]
           operations: ["CREATE", "UPDATE"]
           resources: ["pods"]
     validations:
       - expression: >-
           object.spec.containers.all(c,
             has(c.securityContext) &&
             has(c.securityContext.runAsNonRoot) &&
             c.securityContext.runAsNonRoot == true
           )
         message: "Les conteneurs doivent tourner en non-root (runAsNonRoot: true)."

4. Runtime: monitor in production

   Falco runs on every node and alerts on an interactive shell, the reading of a sensitive file or the execution of a binary missing from the image. The alerts go to the team's on-call tool.

The result can be measured at each step: the python:3.13 image fails the scan with 19 critical CVEs, the distroless image passes; a pod without runAsNonRoot is not stored; an interactive shell in a pod triggers Terminal shell in container.

Common pitfalls to avoid

Even with the right tools, five mistakes keep coming back and each one neutralises a control without a sound. Most come from a control set up once and never checked again, or relaxed "temporarily" to unblock a deployment.

• Scanning only at build time. CVEs are published continuously: an image that is clean today can be vulnerable tomorrow. Also scan the images already deployed.
• Disabling the Pod Security Standards "to make it work". Fix the manifest instead, or pick an image designed to run as non-root.
• Trusting the latest tag. A tag can point to anything tomorrow. Deploy by digest (image@sha256:…).
• Ignoring NetworkPolicies. Without them, a compromised pod reaches the whole cluster: east-west traffic, between pods, is often more dangerous than incoming traffic.
• Putting secrets in environment variables. They leak into logs, crash reports and kubectl describe. Mount them as files, or better, serve them from an external manager, as in the secrets module.

Recommended tools

Each phase has its family of tools, and a Build phase tool never replaces a Runtime phase tool. The three tables are read through the When to use it column: it gives the triggering context, not a quality preference.

Vulnerability scanner

A vulnerability scanner compares the packages present in an image with the public CVE databases. Its result depends on how fresh its database is: a scan run on an unchanged image can report ten more vulnerabilities than yesterday. Look at the Strengths column to see whether the tool also covers repositories and IaC.

Tool	Strengths	When to use it
Trivy	Fast, free, multi-target (images, repositories, IaC)	First choice for CI/CD
Grype	Lightweight, Anchore ecosystem	Alternative to Trivy
Snyk	Enriched CVE database, guided remediation	If you have a security budget

Swipe to see more

Compliance and audit

These tools do not look for CVEs: they compare the cluster's actual configuration with a benchmark (CIS Benchmark, NSA recommendations). They answer "is my configuration compliant", not "are my images vulnerable".

Tool	Strengths	When to use it
Kubescape	CIS Benchmarks, NSA recommendations, free	Initial and continuous audit
kube-bench	Official CIS Kubernetes Benchmark	Formal compliance audit
Polaris	Kubernetes best practices, simple	First steps in Kubernetes security

Swipe to see more

Runtime protection

Runtime protection observes the system calls and traffic of containers that have already started. It produces continuous alerts rather than a one-off report: plan who receives them and who responds before you deploy it.

Tool	Strengths	When to use it
Falco	Open source, system call capture through eBPF	Intrusion detection
NeuVector	Network and runtime protection, L7 firewall	Enterprise security
Sysdig Secure	Integrated observability and security	If you already use Sysdig

Swipe to see more

Detailed guides

The pages below cover these tools in practice. If you are starting from scratch, begin with Trivy, which gives a result in one command, then Kubescape to measure the gap between your cluster and the CIS Benchmarks. NeuVector assumes a cluster already in place.

• Kubescape (in French): audit a cluster against the CIS Benchmarks and the NSA recommendations.
• NeuVector (in French): runtime protection, L7 network firewall and continuous compliance.
• Securing Docker in production: capabilities, seccomp, rootless, measured on Docker 29.1.3.

Key points

• A container shares the host kernel: an exploited flaw can compromise the whole node.
• Three phases, three families of tools: build (minimal image, scan, signing), admission (PSS, Kyverno), runtime (Falco).
• The minimal image pays off the most: 4,122 CVEs for python:3.13, 155 for the same application in distroless.
• A namespace without a label accepts everything; a misspelled PSS label key is ignored without an error.
• NetworkPolicies only protect if the network plugin enforces them.
• Runtime detection alerts after the fact: it replaces neither scanning nor admission.
• These controls are required by the SOCLE families Deployment and admission (DPL) and Runtime and operations (RUN).

Knowledge check

Check that the essentials of this lesson have sunk in. The questions only cover what has just been explained here: the three phases, the lesson's measurements and the pitfalls that disable a control.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

Container security checklist

This checklist is a review before going to production, not a shopping list to tick in one go. Work through it in the Build, Deploy, Runtime order: each block assumes the previous one is in place. An empty box must be a documented exception, with a name against it.

Build

This block happens entirely in the repository and in CI, before any image reaches a registry. It is the cheapest moment: a fix here is a commit, the same fix in production is an incident. The most often forgotten point remains image signing.

• Minimal base images (distroless, alpine)
• Scanner integrated into CI (Trivy, Grype)
• Pipeline blocked when a critical CVE is detected
• No secrets in the images
• Dockerfile with a non-root USER
• Images signed (Cosign) and deployed by digest

Deploy

These controls sit on the namespace and on the cluster, not on the image. They protect even against an image you did not build. Check each box on a real namespace: a misspelled Pod Security label key is ignored without the slightest error message.

• Pod Security Standards applied (baseline or restricted)
• NetworkPolicies defined (deny by default)
• Least-privilege RBAC
• ResourceQuota and LimitRange configured
• Secrets mounted as files, not as environment variables

Runtime

The runtime block is the one that degrades the fastest: an agent deployed once and never reviewed ends up alerting into the void. The two boxes that matter are the alert destination and the response procedure: a detector without a recipient changes nothing in the reaction time.

• Detection tool deployed (Falco, NeuVector)
• Alerts configured for abnormal behaviour
• Regular scan of the images in production
• Documented incident response procedure

×

See in the glossary →