Debug: a job stuck on pending

Does your job show "pending" for several minutes without ever starting? This guide helps you find the cause quickly and unblock your pipeline. A job stays pending when no runner (the agent that executes jobs) can or will pick it up, so the diagnosis always comes down to the same question: what stands between this job and a runner?

Quick diagnosis

1. "This job is stuck..." → No runner online, protected runner, or tags that do not match
2. Pending without a "stuck" message → Runners busy or quotas reached
3. Pending for a long time → Check the queue and the quotas

What you will learn

By the end of this guide, you will be able to find out why no runner picks up your job, and to unblock it:

• Recognize the six causes of a pending job and check them in order
• Read the message "This job is stuck" to point the diagnosis
• Compare the tags of a job and of a runner, then fix them in the interface or through the REST API
• Diagnose an offline runner with systemctl status gitlab-runner, gitlab-runner verify and journalctl
• Spot an exhausted quota, the GitLab.com identity verification or busy runners (concurrent)
• Check the "Run untagged jobs" option and the protection of branches and runners

The 6 causes of "pending"

A job stays in pending when no runner can or will take it. The table lists the six causes in the order of checking followed by this page, and says where to check each one, from the job's tags to the branch or runner protections.

Order	Cause	Check
1	Incompatible tags	CI/CD > Runners
2	Runners offline	Check the runners' status
3	Quotas exhausted / queue saturated	Settings > Usage Quotas
4	Runners busy	Wait or add runners
5	Run untagged jobs = false	CI/CD > Runners > Options
6	Protected branch/runner	Check the protections

Swipe to see more

The tree below covers the first four causes in three diagnostic questions to ask in order: each "yes" exits to the right toward its solution, each "no" goes down to the next question. With no "yes" at all, the runners are busy. The two remaining causes (untagged jobs, protected runner) are checked with the checklist at the end of the page.

Source · SVG

Esc to close, + and − to zoom, scroll to move around. 100 % − Fit + Close

Step-by-step diagnosis

1. Open the pending job

   Click the job. The message shown tells you the cause.

2. Read the error message

   • This job is stuck because of one of the following problems. There are no active runners online, no runners for the protected branch, or no runners that match all of the job's tags → Runners offline, protected branch or incompatible tags
   • This job is stuck because the project doesn't have any runners online assigned to it. → No runner online for this project
   • This job is stuck because you don't have any active runners that can run this job. → No active runner able to run the job
   • No "stuck" message, just a pending status → Runners busy or quotas reached

   If the instance runners are turned off, the Settings > CI/CD > Runners page shows Instance runners are disabled. (or Instance runners are disabled in the group settings. when the group disables them).

3. Check the available runners

   Settings > CI/CD > Runners shows:

   • Active runners (green dot)
   • The tags of each runner
   • The last contact

4. Compare the tags

   Does your job ask for tags that the runner does not have?

Cause #1: Incompatible tags

Symptom: This job is stuck because you don't have any active runners that can run this job. or the variant that lists the possible causes ("... or no runners that match all of the job's tags")

The job asks for tags that the runners do not have. A runner only takes a job whose tags it all carries.

Diagnosis:

# Your job asks for these tags
deploy:
  tags:
    - docker
    - production

But your runner only has docker, staging.

Solutions:

Quick fix

Change the job's tags to match the available runners:

deploy:
  tags:
    - docker    # Remove 'production' if no runner has it

Or add the missing tags to the runner:

Option A (recommended): in GitLab, go to CI/CD > Runners, click the runner, edit the tags and save.

Option B: if you automate, go through GitLab's REST API. The tags live on the server side, not in the runner's config.toml: you set them when the runner is created (POST /user/runners, parameter tag_list) and then edit them on the existing runner:

# Replace the tag list of runner 42 (API token in GITLAB_TOKEN)
curl --request PUT --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  --form "tag_list=docker,production,staging" \
  "https://gitlab.com/api/v4/runners/42"

Do not re-register to change the tags

Avoid gitlab-runner unregister followed by register just to change the tags. With a glrt- authentication token, the --tag-list option of gitlab-runner register is ignored anyway: it only applies to the old registration token, which is deprecated and scheduled for removal in GitLab 20.0 (GitLab Runner documentation).

Cause #2: Runners offline

Symptom: Runners with a red or grey dot in Settings > CI/CD > Runners

A runner that no longer contacts GitLab cannot take any job, and the cause is almost always on the runner's own machine: stopped service, lost network or expired token.

Diagnosis:

On the runner's server:

# Check the service (most reliable method)
systemctl status gitlab-runner

# Check connectivity with GitLab
gitlab-runner verify

# See recent logs
journalctl -u gitlab-runner -n 200 --no-pager

# Real-time logs
journalctl -u gitlab-runner -f

Solutions:

# Restart the service
sudo systemctl restart gitlab-runner

# Check that the runner reconnects
gitlab-runner verify

Invalid token

If verify fails with a token error, the runner must be re-registered from the GitLab interface (new glrt-... token).

Cause #3: Quotas exhausted

Symptom: On GitLab.com, jobs waiting even with green runners

GitLab.com limits CI/CD minutes according to your plan: the Free plan gives 400 per month on the runners hosted by GitLab. A job also does not start until the identity verification required for these runners is completed: GitLab then displays "Identity verification is required in order to run CI jobs". Depending on a risk score, the verification takes the form of an email, a phone number or adding a payment method. Complete it from the link in the message, or use a project, group or self-managed runner.

Diagnosis:

Settings > Usage Quotas (or group > Settings > Usage Quotas)

Saturated queue

Even without an exhausted quota, the instance runners (formerly shared runners) of GitLab.com can be saturated (queue). The job stays pending until a runner frees up.

Solutions:

1. Wait for the start of the month (quota reset)
2. Buy extra minutes
3. Complete the identity verification if GitLab asks for it
4. Install your own runner (no minutes quota, but limited by your resources and the concurrent setting)

Cause #4: Runners busy

Symptom: Pending for 1-5 minutes, then it starts

All the runners are running other jobs.

Diagnosis:

Look at the runner's concurrent setting:

/etc/gitlab-runner/config.toml

concurrent = 4  # Maximum number of simultaneous jobs

Solutions:

• Increase concurrent (if the machine can handle it)
• Add runners
• Optimize the pipelines so they are shorter

Cause #5: The runner refuses untagged jobs

Symptom: The job has no tags, but no runner takes it

The runner is configured with "Run untagged jobs" = false. It only accepts the jobs that explicitly ask for its tags.

Diagnosis:

In CI/CD > Runners, click the runner and check the "Run untagged jobs" option.

Solutions:

• Tick "Run untagged jobs" on the runner
• Or add tags: to your job to match the runner's tags

Cause #6: Protected branch/runner

Symptom: Works on some branches, not on others

The runner is marked "protected" and only runs on protected branches.

Diagnosis:

In CI/CD > Runners, check whether the runner has the "protected" badge.

Solutions:

• Untick "Protected" on the runner
• Protect your branch in Settings > Repository > Protected branches

Debug checklist

□ Does the job have tags?
  → Yes: Check that a runner has ALL these tags
  → No: Check that a runner has "Run untagged jobs" enabled

□ Are the runners online?
  → Settings > CI/CD > Runners (green dot)

□ Do you have quotas or a saturation?
  → GitLab.com: Settings > Usage Quotas
  → Self-managed: no minutes quota, but limited by `concurrent` and resources

□ Is the runner protected?
  → Is the branch protected too?

□ Are the instance runners (formerly shared runners) enabled?
  → CI/CD > Runners > "Turn on instance runners for this project"
  → If disabled, only your project/group runners can run

□ Does the runner accept jobs without tags?
  → CI/CD > Runners > "Run untagged jobs"

Useful commands

# On the runner's server

# Check the service
systemctl status gitlab-runner

# List the registered runners
gitlab-runner list

# Check the connection to GitLab
gitlab-runner verify

# Real-time logs
journalctl -u gitlab-runner -f

Advanced debugging: run-single

To reproduce a problem on a test machine:

gitlab-runner run-single \
  --url https://gitlab.com/ \
  --token TOKEN \
  --executor shell \
  --builds-dir /tmp/builds

Use this only in a test environment. This command bypasses the normal operation of the service.

Hands-on lab

The lab "Two silent failures: nothing runs, nothing complains" replays this guide on a GitLab CE forge of your own, set up on a virtual machine with its runner, rather than on gitlab.com: no minute quota, no identity verification, and a project you can break as often as you need.

You diagnose two failures that never turn red and raise no error, then prove the pipeline runs:

1. You push, and nothing happens: not a failed or skipped pipeline, but no pipeline at all, because the workflow:rules of the file accepts no source for your push. This failure is not covered by this page: it is diagnosed in Debug: pipeline skipped, and the lab suggests checking it with the CI Lint API, whose dry_run option is mentioned in Validate a .gitlab-ci.yml file.
2. Once a pipeline is born, a job hangs forever without failing. This is the case of this page: the job asks for a tag that no runner carries (cause #1). Compare the tags of the job with those of the runner in the interface, as in the diagnosis above.

Grading is taken on the pipeline left on the forge: a pipeline exists on the default branch, no job is stuck in pending, no job failed, and the linter job really ran. Removing that job from the pipeline would unblock nothing: it has to run. A job in manual status waits for a human click: it is healthy, do not mistake it for the stuck one.

The lab starts with dsoxlab run gl-11-diagnostiquer-un-job-bloque, after dsoxlab provision prepares the virtual machine and its runner. Installing DSOXLab and its prerequisites are described in the DSOXLab documentation.

Guided lab Practice

Learn how to do it

Two silent failures: nothing runs, nothing complains

$ dsoxlab run gl-11-diagnostiquer-un-job-bloque

Environment: VM·Duration: ~35 min

How it is checked: the state of the system is checked after a reboot, not the command you typed.

First time? What DSOXLab is and how to install it

DSOXLab is the command line that prepares a disposable environment (an isolated shell or a VM), runs the lab scenario, then checks the real state of the system and computes a score. Nothing is installed or broken on your workstation.

It needs Python 3.11 or later, uv or pipx, and git. The installation guide details these prerequisites; on Windows, or on a Mac without Python, use the appliance instead.

This lab starts virtual machines: it also needs ansible-playbook, a hypervisor (KVM or Incus), Terraform and hardware virtualization (/dev/kvm), which inside a VirtualBox VM means nested virtualization. The first time, dsoxlab start gl-11-diagnostiquer-un-job-bloque checks all of it before creating the machines. Prepare your machine for vm labs.

Install the CLI and add the catalogue, once:

uv tool install dsoxlab   # or: pipx install dsoxlab
dsoxlab catalog add https://github.com/stephrobert/gitlab-ci-dsoxlab-training

Before you run the lab, check:

• DSOXLab installed
• catalogue available
• compatible environment (isolated shell or VM)

To check, nothing is probed from this page: run dsoxlab doctor. Read the diagnosis

Run this lab, then validate your work:

dsoxlab run gl-11-diagnostiquer-un-job-bloque
dsoxlab check gl-11-diagnostiquer-un-job-bloque

See every lab: dsoxlab list-labs. Track your progress: dsoxlab progress. The full dsoxlab guide covers scoring and the pitfalls.

Key takeaways

1. "This job is stuck" = no runner can take the job: offline, protected, tags that do not match or "Run untagged jobs" disabled
2. Tags: the job must ask for tags that the runner has
3. Run untagged jobs: if your job has no tags, the runner must accept untagged jobs
4. Runner offline: systemctl status gitlab-runner and verify to diagnose
5. GitLab.com quotas: 400 minutes per month on the Free plan, identity verification to complete, then Usage Quotas (or saturated queue)
6. Changing tags: through the GitLab interface, not by re-registering the runner

Test your knowledge

Knowledge check

Check what you have learned with this interactive quiz

10 questions

5 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →