Persist the data of a recreated stack

You change one line of the compose.yaml, you run docker compose up -d again, and Compose replaces the database container. Does the database data survive? Yes, if it lives in a named volume declared by the stack. No, if you then run docker compose down -v, which deletes that volume.

This lesson is for a beginner who has already written a Compose stack. You build a small notes database, you recreate it in three different ways and each time you look at what happens to the data. You also see why an init script does not run again on a volume that is already filled, how to replay a migration without losing anything, and how to refuse a write with a read-only mount.

What you will learn

• Predict what stop, up -d --force-recreate, down and down -v do to the containers, the network and the volume of a project.
• Explain why the init scripts of the postgres image run only on an empty volume.
• Read who owns the data of a volume and prove that another identity is refused.
• Replay a migration written to be repeatable, without losing a single row.
• Mount a folder read-only with :ro and see the write refused.
• Recognize the anonymous volume created by a wrong mount path, and clean up the stack.

Prerequisites

You need a Linux host with Docker Engine 29.9.2 and its Compose 5.6.1 plugin, installed from the official repository as shown in the lesson on installing Docker. The images are pinned by their digest.

You wrote a first stack with the lesson Describe an application with Compose, which explains the project name and the difference between down and down -v. This lesson comes after the ones on interpolation and environment files and on organizing the networks of a stack, but it reuses nothing from them.

This page assumes you know what two lessons of the "Keep your data" module teach. The lesson on Docker volumes explains what a named volume, a bind mount and an anonymous volume are. The lesson on volume permissions explains how a file is owned by a UID and a GID. Here, you see what Compose adds to them.

You work in a ~/carnet folder, which the lesson has you create. The folder name becomes the project name: the containers are called carnet-... and the volume carnet_....

What happens to the data when Compose recreates a service?

The data of a named volume survives any container recreation, and even docker compose down. Among the Compose commands, only docker compose down -v destroys it. Compose replaces a container whenever it needs to, but it deletes a volume only if you ask for it with -v.

The table gives, for four commands, what stays and what disappears among the three objects of a project: the container, the network, and the volume with its data. Read it line by line. Only one cell makes you lose data: the last one.

Command	Container	Network	Volume and data
docker compose stop	stopped	kept	kept
docker compose up -d --force-recreate	removed then recreated	kept	kept, reused
docker compose down	removed	removed	kept
docker compose down -v	removed	removed	removed, data lost

Swipe to see more

Between down and down -v there is only one letter, and the second one erases your data. Type -v only to start from scratch, never to "clean up" a stack that is in use. The next sections check this table on a real database.

How do you declare the stack with its volume and its init script?

Declare the volume in the top-level volumes: block, mount it on the folder where the image stores its data, and mount the init script read-only. The file below describes a notes database, with a migrations folder that you fill in further down.

First create the folder with its two subfolders.

mkdir -p ~/carnet/init ~/carnet/migrations && cd ~/carnet

On first start, the postgres image runs the .sql files it finds in /docker-entrypoint-initdb.d. This file creates the table and writes a note into it.

-- ~/carnet/init/01-schema.sql
CREATE TABLE notes (
  id serial PRIMARY KEY,
  texte text NOT NULL
);
INSERT INTO notes (texte) VALUES ('première note');

Here is the compose.yaml.

services:
  base:
    image: postgres:18.6-alpine3.24@sha256:10a5c69a83035ab5239f5620d708520e8b28f5aa1f2b20143e6024532edadfb5
    environment:
      POSTGRES_USER: carnet
      POSTGRES_PASSWORD: demo-seulement
      POSTGRES_DB: carnet
    volumes:
      - donnees-base:/var/lib/postgresql
      - ./init:/docker-entrypoint-initdb.d:ro
      - ./migrations:/migrations:ro

volumes:
  donnees-base:

Read the base service line by line. The donnees-base:/var/lib/postgresql mount sends the database files into the named volume donnees-base, which the bottom block declares: the data lives in the volume, not in the container. The ./init:/docker-entrypoint-initdb.d:ro mount exposes your init folder to the container, and the final :ro forbids it from modifying it. The ./migrations:/migrations:ro mount does the same for the migrations, which you will run from the database container.

Start the stack and read the contents of the table.

docker compose up -d
docker compose exec base psql -U carnet -d carnet -tAc 'SELECT id, texte FROM notes ORDER BY id'

On first start, the database is not ready: the first attempts fail with the message in the first block below. Run the read again until you see the note, as in the second block.

psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
  Is the server running locally and accepting connections on that socket?

1|première note

The database takes a few seconds to answer, because the image first starts a temporary server to run your scripts, stops it, then starts the real server. In the meantime, psql is refused. The message varies with the moment (No such file or directory, the database system is starting up or the database system is shutting down): just run the command again. The log confirms the sequence: docker compose logs base contains PostgreSQL init process complete; ready for start up., then database system is ready to accept connections for the real server.

Compose also created the volume carnet_donnees-base, that is, the project name followed by the key in the file. docker volume ls lists it.

Why does the init script run only once?

Because the postgres image runs the files in /docker-entrypoint-initdb.d only if the data folder is empty. On a volume that is already filled, the container starts on the existing database and ignores the script. The container is new, but the volume is not.

Predict what will happen. You add two notes by hand, you modify the script so that it writes one more note, then you recreate the database container. Will the table contain the note from the modified script?

docker compose exec base psql -U carnet -d carnet -c "INSERT INTO notes (texte) VALUES ('note ajoutée à la main'), ('seconde note à la main')"
echo "INSERT INTO notes (texte) VALUES ('note du script modifié');" >> init/01-schema.sql
docker compose up -d --force-recreate base
docker compose exec base psql -U carnet -d carnet -tAc 'SELECT id, texte FROM notes ORDER BY id'

INSERT 0 2
1|première note
2|note ajoutée à la main
3|seconde note à la main

The table contains exactly the three notes from before, and the note from the modified script is missing. The container was indeed replaced, but the volume kept the database, and the image skipped the initialization. If psql refuses the connection right after the recreation, run the query again. The logs confirm the skip.

docker compose logs base | grep -i "skipping initialization"

base-1  | PostgreSQL Database directory appears to contain a database; Skipping initialization

This line is the proof to look for when a modified script "does nothing". It says that the image found a database and did not touch it.

To see the modified script run, you must start from an empty volume. Here only, down -v is the right command, since you accept losing these demonstration notes. The volume starts empty: run the query again as long as psql refuses the connection, as on first start.

docker compose down -v
docker compose up -d
docker compose exec base psql -U carnet -d carnet -tAc 'SELECT id, texte FROM notes ORDER BY id'

1|première note
2|note du script modifié

The two notes added by hand are gone, and the note from the modified script is there: the volume was new, so the image replayed the whole script. On a database that is in use, this route is closed. To change its schema without emptying it, you need a migration, which the section on migrations writes.

Insert a note by hand again, to have data to protect. Then check the down line of the table: a docker compose down followed by a docker compose up -d must keep this note.

docker compose exec base psql -U carnet -d carnet -c "INSERT INTO notes (texte) VALUES ('note à protéger')"
docker compose down
docker compose up -d
docker compose exec base psql -U carnet -d carnet -tAc 'SELECT id, texte FROM notes ORDER BY id'

INSERT 0 1
1|première note
2|note du script modifié
3|note à protéger

The down removed the container and the network, and the up recreated them. The volume did not move: the note added by hand is still there, and the other two with it.

Who owns the data, and who is refused?

The database files belong to the identity under which PostgreSQL runs, and they are closed to everyone else. This identity is a UID/GID pair, which you read with id in the container, then on the volume itself. As long as the recreated container keeps the same identity, it gets its permissions back.

First read the identity of PostgreSQL, then the owner and the mode of the data folder in the volume.

docker compose exec base id postgres
docker compose exec base stat -c '%u:%g %a %n' /var/lib/postgresql/18/docker

uid=70(postgres) gid=70(postgres) groups=70(postgres),70(postgres)
70:0 700 /var/lib/postgresql/18/docker

The PostgreSQL 18.6 image stores the data in a version subfolder, /var/lib/postgresql/18/docker, under the folder you mounted. This folder belongs to UID 70, the one of the postgres account, and its 700 mode lets in only its owner. Its group is 0 (root) and has no effect, since the mode gives nothing to the group. The volume therefore carries numbers, not an account name. The lesson on volume permissions details this point.

Now prove the refusal. Run a throwaway container under another identity, which mounts the same volume and tries to read the data folder. The volume name is the one Compose created: carnet_donnees-base.

docker run --rm --user 1234:1234 -v carnet_donnees-base:/v alpine:3.24@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a ls /v/18/docker

ls: can't open '/v/18/docker': Permission denied

The refusal comes from the file permissions, not from the mount: the volume is mounted read-write, and the 1234:1234 identity has no right on a folder in 700 mode. This is the refusal you want to keep after a recreation. Check it: recreate the database, then read the owner again.

docker compose up -d --force-recreate base
docker compose exec base stat -c '%u:%g %a' /var/lib/postgresql/18/docker

70:0 700

The owner and the mode are identical to those before the recreation. The new container did not receive more permissions, and nobody had to give it any.

In a stack, the user: "UID:GID" key sets the identity of a service. If you change this identity without changing the owner of the volume, the service is refused on its own data. Fixing it with chmod 777 or by running the service as root widens the permissions instead of settling them: the lesson on volume permissions explains why these two answers hold up poorly, and how to align a group instead.

How do you replay a migration without losing data?

Write the migration so that it is repeatable, that is, so that it does nothing if its work is already done, then run it with psql from the database container. That way, replaying it after each deployment costs no data. A non-repeatable migration fails on the second run or, if it recreates the table, erases the rows.

The file adds an auteur column to the table. The IF NOT EXISTS keyword makes the addition repeatable.

-- ~/carnet/migrations/02-ajout-auteur.sql
ALTER TABLE notes ADD COLUMN IF NOT EXISTS auteur text NOT NULL DEFAULT 'inconnu';

The migrations folder is mounted in the container under /migrations. Run the file with the -f option of psql, twice in a row, then read the table again. The -v ON_ERROR_STOP=1 option stops psql at the first real error.

docker compose exec base psql -U carnet -d carnet -v ON_ERROR_STOP=1 -f /migrations/02-ajout-auteur.sql
docker compose exec base psql -U carnet -d carnet -v ON_ERROR_STOP=1 -f /migrations/02-ajout-auteur.sql
docker compose exec base psql -U carnet -d carnet -tAc 'SELECT id, texte, auteur FROM notes ORDER BY id'

ALTER TABLE
psql:/migrations/02-ajout-auteur.sql:1: NOTICE:  column "auteur" of relation "notes" already exists, skipping
ALTER TABLE
1|première note|inconnu
2|note du script modifié|inconnu
3|note à protéger|inconnu

Read the first two lines. On the first run, ALTER TABLE adds the column. On the second, PostgreSQL reports with a NOTICE that the column already exists and moves on, without an error. The three notes are still there, each with the default value of the new column.

The migrations folder is mounted :ro for the same reason as the init script: the container reads the file, and it has no reason to change it.

How do you forbid writing with a read-only mount?

Add :ro at the end of the volume in the short form. The container then reads the mounted content and is refused any write, whatever its identity. The lesson on Docker volumes showed this option with docker run; Compose takes it over as is.

Try to write into the database init folder, which your file mounted :ro.

docker compose exec base sh -c 'echo "DROP TABLE notes;" >> /docker-entrypoint-initdb.d/01-schema.sql'

sh: can't create /docker-entrypoint-initdb.d/01-schema.sql: Read-only file system

The Read-only file system message shows that the mount itself refuses the write. docker compose exec opens a root session here: docker compose exec base id prints uid=0(root) gid=0(root). The refusal therefore does not come from a lack of permissions, it comes from the mount, which root does not bypass.

Check the same thing by inspection, which reads what Docker actually mounted. The RW field is false for a read-only mount.

docker inspect -f '{{range .Mounts}}{{.Destination}} rw={{.RW}}{{"\n"}}{{end}}' carnet-base-1

/docker-entrypoint-initdb.d rw=false
/migrations rw=false
/var/lib/postgresql rw=true

The data volume is read-write, since the database must write to it. The two file folders are read-only. This is the setting to aim for: each mount gets only the access the service needs.

What should you do when the data is not kept as expected?

Start from what you observe, then find the mount or the command responsible. The table links the cases seen in this lesson to their cause.

What you observe	Probable cause	What you do
The modified init script changes nothing	The volume already contains a database, the image skips the initialization	Look for Skipping initialization in docker compose logs base. To change the schema, write a migration. To start from scratch, docker compose down -v, which erases the data
The data disappeared after down then up	The data folder is not on the named volume, or a down -v went through	Read docker inspect -f '{{range .Mounts}}{{.Destination}} {{.Name}}{{"\n"}}{{end}}' carnet-base-1: the named volume must be mounted on /var/lib/postgresql
The database does not start, the log begins with Error: in 18+, these Docker images are configured to store database data, and a volume with a 64-character hexadecimal name appears in docker volume ls	Your named volume is mounted on /var/lib/postgresql/data, the old path from before PostgreSQL 18: Docker then creates an anonymous volume on /var/lib/postgresql, and the image refuses to start	Mount the named volume on /var/lib/postgresql and run docker compose up -d again. The anonymous volume already created stays: after docker compose down -v, spot it with docker volume ls -f dangling=true and remove it with docker volume rm
Permission denied when the service accesses its volume	The identity of the service no longer matches the owner of the files	Compare id in the container and stat -c '%u:%g' on the folder. Align the identity, without chmod 777
Read-only file system when writing	The path is mounted :ro	Check the RW field with docker inspect. If the write is intended, remove :ro; otherwise, write to another folder
psql answers No such file or directory, starting up or shutting down right after a start	The database is not ready yet: on an empty volume, the image first starts a temporary server for your scripts, then the real server	Run the command again after a few seconds, or read docker compose logs base until the last line database system is ready to accept connections

Swipe to see more

When in doubt, come back to two commands: docker compose config, which shows the mounts as Compose applies them, and docker volume ls --filter label=com.docker.compose.project=carnet, which lists the volumes of the project.

How do you clean everything up after the demonstration?

Delete the stack with its volumes, check that no volume of the project lingers, then remove the images and the folder. The demonstration leaves one container, one network, one named volume, two images and the ~/carnet folder.

The down -v command removes the named volume of the project. It leaves in place an anonymous volume that the stack created then abandoned, for example after a mount on the wrong path fixed by recreating the container: no container uses it anymore, but Docker keeps it. If you did not make this mistake, the volume list is empty. Otherwise, it shows a 64-character hexadecimal name, which docker volume ls -f dangling=true isolates and docker volume rm removes. Avoid docker volume prune on a machine that hosts work other than yours.

cd ~/carnet
docker compose down -v
docker volume ls

DRIVER    VOLUME NAME

If an anonymous volume appears, remove it by its name.

docker volume ls -f dangling=true -q
docker volume rm <nom-du-volume>

Then remove the two images and the folder.

docker rmi postgres:18.6-alpine3.24@sha256:10a5c69a83035ab5239f5620d708520e8b28f5aa1f2b20143e6024532edadfb5 \
  alpine:3.24@sha256:be2656061b536c1883c4034790e21d0a09c5aaaf51968b89368a3c0371ea2c9a
cd ~ && rm -r carnet

After that, docker ps -a, docker volume ls and docker network ls show nothing left of the lesson.

Practice

The lab gives a prepared image and no running stack. You write the compose.yaml of a single service with a named volume for its data. The service runs under a non-root identity, with the user: key cited in this lesson. After the first start, you set up once the owner and the mode of the volume that Compose created, as in the lesson on volume permissions: a volume created outside Compose would not carry the labels of the project. You check that the permissions stay the same after each recreation.

The validation looks at the state of the machine, not at the commands typed. It temporarily replaces your data with random values, force-recreates the container, replays a down then up cycle, and checks that the values survive. It also checks that another identity is refused. Reproducing the initial data in each new container proves nothing: only real persistence counts.

Guided lab Practice

Learn how to do it

Keep application data across Compose recreation

$ dsoxlab run compose-persistance

Environment: VM·Duration: ~30 min

How it is checked: the state of the system is checked after a reboot, not the command you typed.

First time? What DSOXLab is and how to install it

DSOXLab is the command line that prepares a disposable environment (an isolated shell or a VM), runs the lab scenario, then checks the real state of the system and computes a score. Nothing is installed or broken on your workstation.

It needs Python 3.11 or later, uv or pipx, and git. The installation guide details these prerequisites; on Windows, or on a Mac without Python, use the appliance instead.

This lab starts virtual machines: it also needs ansible-playbook, a hypervisor (KVM or Incus), Terraform and hardware virtualization (/dev/kvm), which inside a VirtualBox VM means nested virtualization. The first time, dsoxlab start compose-persistance checks all of it before creating the machines. Prepare your machine for vm labs.

Install the CLI and add the catalogue, once:

uv tool install dsoxlab   # or: pipx install dsoxlab
dsoxlab catalog add https://github.com/stephrobert/containers-dsoxlab-training

Before you run the lab, check:

• DSOXLab installed
• catalogue available
• compatible environment (isolated shell or VM)

To check, nothing is probed from this page: run dsoxlab doctor. Read the diagnosis

Run this lab, then validate your work:

dsoxlab run compose-persistance
dsoxlab check compose-persistance

See every lab: dsoxlab list-labs. Track your progress: dsoxlab progress. The full dsoxlab guide covers scoring and the pitfalls.

Key points

1. A named volume declared in the volumes: block survives up -d --force-recreate and docker compose down. Among the Compose commands, only docker compose down -v removes it, and it is typed only to start from scratch.

2. The scripts in /docker-entrypoint-initdb.d run only on an empty data folder. On a filled volume, the log prints Skipping initialization and the modified script is not replayed.

3. To change the schema of a database that is in use, write a repeatable migration, with IF NOT EXISTS, and run it with docker compose exec base psql -f. Replayed, it loses no data.

4. A volume stores UID/GID numbers, not account names. After a recreation, the container gets its permissions back as long as its identity is the same. A refusal is proved by a throwaway container under another identity, and it is settled neither with chmod 777 nor by switching the service to root.

5. :ro closes a mount to writes for every identity, root included, and the error is Read-only file system. The RW field of docker inspect confirms it.

6. On first start, the database takes a few seconds to answer: run the query again, and read docker compose logs base until database system is ready to accept connections.

7. With PostgreSQL 18, mount the named volume on /var/lib/postgresql. On /var/lib/postgresql/data, the image refuses to start and Docker creates an anonymous volume. Once the path is fixed, down -v does not remove it: spot it with docker volume ls -f dangling=true and remove it with docker volume rm.

Knowledge check

Check that the essentials of this lesson have stuck. The questions cover only what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

8 questions

8 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →