C3: Industrializing GitLab CI/CD pipelines

Goal of level C3: industrialize

Do you have a working pipeline? Now make it grow. Level C3 "Industrialize" teaches you to build pipelines that are maintainable, reusable and fast, which becomes essential as soon as you manage several projects or teams.

Prerequisites

This level assumes you have mastered levels C1 "Write a pipeline and read its failures" and C2 "Deliver", grouped under the Fundamentals. In particular:

The .gitlab-ci.yml structure (stages, jobs, script)
Variables and rules
Artifacts and cache

The modules

Phase 1: Reuse and factoring

This phase follows an increasing scope: extends and YAML anchors factor out code inside a single .gitlab-ci.yml, include shares it between several projects, and Components publish a versioned building block in the GitLab catalog. Take them in that order: include without extends produces shared files that nobody dares to modify, and a component written before you have factored locally freezes duplications at the scale of the whole organization.

Module	Goal	Practice
extends and anchors	Factor out within one file	lab
include and templates	Share between projects	lab
Components & Catalog	Create reusable components	quiz

Phase 2: Performance and orchestration

Here the goal changes: you no longer try to write less YAML, you try to shorten the pipeline and cut its consumption of CI minutes. needs breaks the sequential execution by stages in favor of a dependency graph, matrices generate several runs of the same job, and CI services start a database next to the job instead of inside the script. The three modules combine, since a matrix job remains subject to the same needs as the others.

Module	Goal	Practice
DAG and parallelism	Optimize with `needs`	lab
Job matrices	Multiply runs	lab
CI services and advanced cache	Databases, Redis in CI	quiz

Phase 3: Advanced orchestration

The three modules answer the same limit: a single .gitlab-ci.yml does not scale to a monorepo. Parent-child pipelines delegate part of the work to a separate pipeline started by trigger, dynamic pipelines generate the child YAML at runtime according to the folders that really changed, and multi-project triggers cross the repository boundary. Reserve this phase for repositories that host several applications: on a single project, it adds indirection with no gain.

Module	Goal	Practice
Parent-child pipelines	Orchestrate monorepos	lab, which also covers dynamic generation
Dynamic pipelines	Generate by context	quiz
Multi-project and downstream	Trigger other projects	quiz

Phase 4: Workflows and reliability

This last phase decides when a pipeline starts and how it behaves when something fails. workflow:rules avoid the classic duplicate of a branch pipeline started at the same time as a merge request pipeline; retry, timeout and job idempotence tell a real regression apart from a transient network failure. The capstone gathers the eleven previous modules into a single pipeline, and it closes level C3.

Module	Goal	Practice
CI/CD workflows	MR, branch and release patterns	lab
Pipeline reliability	Retry, timeouts, idempotence	lab
Industrialization capstone	Complete industrial pipeline	capstone

Recommended path

You are not required to read the twelve modules in order. Choose the tab that matches the symptom you see today on your pipelines: YAML duplicated between repositories, a run time that keeps drifting, or a repository that hosts several applications. Each path fits in three modules and gives a measurable result, lines of YAML removed or CI minutes saved, before you tackle the rest of the level.

Are you copy-pasting YAML between projects?

Where to start

If none of the three paths matches your situation, open extends and anchors: it is the only module the others really depend on, and it applies to any existing .gitlab-ci.yml without breaking anything. The other entry points each answer a precise need: a gain in run time with needs in DAG and parallelism, splitting a monorepo with parent-child pipelines, or sharing between repositories with include and templates.

How is the practice organised?

There is no exercise series sitting next to the course: every lesson that teaches a hands-on move carries its lab at the bottom of the page, in a "Hands-on lab" section. That is where the pipeline really runs, on a GitLab CE forge of your own on a virtual machine, and the mark is about the state read from the API rather than about the file you wrote.

Four lessons in this level have no lab of their own and are assessed by their quiz: Components and Catalog, CI services and advanced cache, Multi-project and downstream and Dynamic pipelines. The last one is barely an exception: generating a child pipeline is graded by the Parent-child pipelines lab, then asked for again by the capstone that closes the level.

This level assumes the fundamentals are in place: validating YAML with the CI Lint, and unblocking a job stuck on pending. Both lessons that teach them carry their own lab, to be played before you tackle industrialization.