wget: download from the command line

wget fetches a file from an HTTP, HTTPS or FTP server without any interaction. You give it a URL, it writes the file to disk, and it can resume an interrupted transfer, limit its bandwidth and mirror a whole site. It is the tool you call in an installation script, a cron job or a build container: it runs without a terminal, without a keyboard and without supervision.

This page is a reference: each option is presented with the moment it is really useful in administration. If you would rather query a REST API, read headers or send JSON, curl is what you need. The rule fits in one sentence: wget downloads, curl talks.

The options described here are those of GNU Wget 1.21.4, the version shipped with Ubuntu 24.04, and the examples were replayed with Wget 1.25.0 on Debian 13 and 1.24.5 on AlmaLinux 10. An option missing on an older machine is checked with wget --version.

What you will learn

• Download a file, rename it and resume an interrupted transfer.
• Script wget cleanly: silence, log, timeouts, retries, return codes.
• Fetch several files from a list or a numbered series.
• Mirror a site while controlling depth, domains and file types.
• Authenticate with a password, a token or a cookie, and go through a proxy.
• Avoid the classic pitfalls: overwriting, no globbing in HTTP, -q hiding errors.

Installing wget

wget is present by default on most server distributions, but not in every minimal container image, where sometimes only curl is installed. The reflex before building a script on it: check that it is there with wget --version, or watch the build break in production.

Debian/Ubuntu

sudo apt install wget

RHEL/Fedora

sudo dnf install wget

Syntax and essential options

The command always takes the same form: options, then one or more URLs. Options can go before or after the URL, but grouping them before remains much more readable in a script.

wget [options] URL...

Here are the ten options you will really use day to day. Each one is detailed below with its context of use: a reference page is useless if it does not say when to take the option out of the drawer.

Option	Role	When it is useful
-O fichier	Writes to the given file	Force a name, or write to /dev/null
-c	Resumes a partial transfer	ISO or dump cut off midway
-nc	Does not rewrite an existing file	Script replayed several times
-q / -nv	Total silence / condensed output	Cron, CI, clean logs
-i liste.txt	Reads the URLs from a file	Batch of packages or artifacts
-P dossier	Puts the files in a directory	Avoid cluttering the current directory
--limit-rate=RATE	Caps the bandwidth	Do not saturate a shared link
--tries=N / --timeout=S	Retries and timeouts	Make a script predictable
--spider	Tests without downloading	Check that a URL still answers
-m	Mirror of a site	Backup of a documentation

Swipe to see more

wget reads its options from three sources

An option can come from the command line, from the .wgetrc file, or from the -e option, which injects a .wgetrc directive on the fly. The command line always wins. Remember it: a forgotten .wgetrc in your $HOME will explain many surprising behaviours.

Downloading a file

Without any option, wget saves the file under the last segment of the URL, in the current directory, and shows a progress bar. It is by far the most frequent use case: fetching an ISO image, a package or a release binary.

wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/SHA256SUMS

The SHA256SUMS file appears in the current directory. Always check with ls -l that the size is not zero: a server that returns an HTML error page produces a file that is indeed present, but unusable.

Choosing the file name with -O

The -O option (capital letter O, for output document) forces the local name. It is essential when the URL ends with an unreadable identifier, and when you want to throw the content away to /dev/null and keep only the return code.

wget -O jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64
chmod +x jq

Two useful variants avoid guessing the name. --content-disposition honours the name announced by the server in the Content-Disposition header, which is what GitHub releases do behind their redirect URLs:

wget --content-disposition https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64
# does write: jq-linux-amd64

--trust-server-names names the file after the final URL after redirection, and not after the one you typed. On a short link that redirects to installeur-2.14.tar.gz, the difference is very concrete: without the option, wget writes a file carrying the starting name, often unreadable.

Not overwriting an existing file

Contrary to a widespread intuition, wget does not overwrite a file with the same name: it creates SHA256SUMS.1, then SHA256SUMS.2. In a script replayed every day, you therefore pile up numbered duplicates that fill the disk without any message.

The -nc option (no-clobber) cuts this short: if the file is already present, wget downloads nothing, renumbers nothing and moves on to the next one.

wget -nc https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/SHA256SUMS
# File ‘SHA256SUMS’ already there; not retrieving.

It is the right reflex for a local cache of artifacts. Be careful, though: -nc checks neither the date nor the content. If the remote file changed, you keep the old one. To refresh based on the date, use -N (timestamping), which downloads again only if the remote copy is newer than the local one.

Resuming an interrupted transfer with -c

On a 4 GB ISO cut off at 80%, restarting wget from zero is a waste of time. The -c option (continue) sends a Range header to the server, which answers 206 Partial Content and resumes at the exact byte where the transfer had stopped.

wget -c https://releases.ubuntu.com/24.04/ubuntu-24.04-live-server-amd64.iso

Resuming is only possible if the server accepts partial requests. If it ignores them, wget starts again from the beginning: that is the expected behaviour, not a bug. And if the local file is larger than the remote one, wget considers that there is nothing to resume.

Scripting wget cleanly

A wget run by hand can afford to be chatty. In a cron job, a CI pipeline or a Dockerfile, it must be silent, bounded in time and verifiable. These four settings make the difference between a reliable script and a script that hangs forever on a Friday evening.

Silence, log and background

-q (quiet) removes all output, errors included. -nv (no-verbose) removes the progress bar but keeps the error messages and one summary line per file: it is the best compromise for a CI log.

wget -nv -O /tmp/SHA256SUMS https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/SHA256SUMS

For a very large transfer started from an SSH session, -b detaches wget into the background and -o writes the progress into a log file. The session can then close without interrupting the download.

wget -b -o /var/log/iso-ubuntu.log https://releases.ubuntu.com/24.04/ubuntu-24.04-live-server-amd64.iso

Checking the return code

It is the most often forgotten point. With -q, a 404 error produces no message: the script carries on as if all was well, and fails three steps later on an empty file. Only the return code tells the truth.

wget -q -O paquet.deb https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb
echo "code de retour : $?"

Code	Observed meaning	Typical cause
0	Download succeeded	Nothing to do
4	Network failure	DNS not found, host unreachable
8	The server answered with an error	404, 403, 500

Swipe to see more

Hence the reflex to anchor in every script: test the return code, or let set -e stop the execution. An unchecked download is a silent failure waiting to happen.

if ! wget -nv -O paquet.deb https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb; then
  echo "Téléchargement du paquet impossible, on arrête là." >&2
  exit 1
fi

Bounding timeouts and retries

By default, wget retries 20 times, which can make a script hang for a very long time on a server that does not answer. Two options restore order: --tries sets the number of attempts, --timeout sets the timeouts.

wget --tries=2 --timeout=10 -O agent.deb https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb

Remember the exact scope of --timeout: it sets the three timeouts at once, namely DNS resolution, connection establishment and data reading. If you need finer control, the three options exist separately:

• --dns-timeout=SEC: name resolution timeout.
• --connect-timeout=SEC: TCP connection establishment timeout.
• --read-timeout=SEC: idle timeout during the transfer.

On an unreachable host, --tries=2 --timeout=3 gives control back in a few seconds (7 s measured) instead of several minutes. It is exactly what you want in a healthcheck or a scheduled task.

Capping the bandwidth

A wget started on a large ISO readily saturates the site's shared link and makes everyone suffer. --limit-rate caps the bandwidth, in k (kilobytes) or m (megabytes) per second.

wget --limit-rate=500k https://releases.ubuntu.com/24.04/ubuntu-24.04-live-server-amd64.iso

The effect is immediate and measurable: 5 MiB capped at 500 k/s take 10 seconds instead of being swallowed instantly. It is the right way to fetch a large image during business hours without degrading the shared line.

Testing a URL without downloading it

--spider sends the request, reads the response and writes nothing to disk. Combined with the return code, it gives a one-line URL probe, perfect to check that an internal mirror still answers or that a documentation link is not dead.

wget --spider -q https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb
echo $?   # 0 : l'URL répond ; 8 : le serveur renvoie une erreur

Add -S (server response) to print the HTTP headers returned and diagnose a looping redirect or a misconfigured cache.

Downloading several files

Three approaches, depending on the shape of your need: a known list of URLs, a numbered series, or a richer logic that calls for a shell loop.

From a list of URLs

The -i option reads a text file containing one URL per line. It is the most robust method to fetch a batch of packages or artifacts, because the list is versionable and can be reviewed before execution.

cat > paquets.txt <<'EOF'
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/SHA256SUMS
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/SHA512SUMS
EOF

wget -nv -i paquets.txt -P /srv/miroir/

-P puts everything in the given destination directory, without cluttering the current directory. Add -c to replay the list after an interruption without starting from zero.

Never combine -i and -O

With -i liste.txt -O tout.dat, wget concatenates the content of every URL into a single file. It is not a rename: it is a merge, and the result is unusable for binaries. To rename, use -O on a single URL, or a loop.

A series of numbered URLs

It is a frequent confusion: wget does not generate URLs from a pattern in HTTP. Writing wget https://serveur/rapport-0[1-5].csv literally asks the server for the file named rapport-0[1-5].csv, which answers with a 404 error. The bracket pattern is curl syntax, not wget's.

wget globbing only concerns FTP

The wget manual does describe a globbing mechanism, but it only applies to FTP URLs (ftp://), where wget queries the server to list the directory. In HTTP, no expansion takes place: the URL leaves as it is. The --no-glob option is actually there to turn off this behaviour on the FTP side.

The solution with wget is to let the shell generate the URLs, thanks to Bash brace expansion. The braces are expanded before the call, and wget therefore receives several distinct URLs:

wget https://depot.interne.lan/rapports/rapport-0{1..5}.csv

Bash also handles leading zeros and increment steps, which covers almost every real need, such as a numbered series of dumps:

wget https://depot.interne.lan/sauvegardes/dump-{01..10}.sql.gz

With curl, the bracket pattern works natively, and -O keeps the remote name of each file:

curl -O "https://depot.interne.lan/rapports/rapport-0[1-5].csv"

With a shell loop

As soon as you need to rename, filter or test between two downloads, the loop takes the lead again. It also makes the script more readable than a wget command overloaded with options.

while read -r url; do
  wget -nc -nv -P /srv/miroir/ "$url" || echo "Échec : $url" >> /tmp/echecs.txt
done < paquets.txt

The quotes around $url are not decorative: without them, a URL containing & or a space breaks the command. wget is not parallel: to speed things up, start several instances in the background with & then wait, keeping in mind that you multiply the load on the remote server.

Mirroring a site in recursive mode

Recursive mode follows the links contained in HTML pages and downloads what it finds. It is the function that really sets wget apart from curl: it serves to archive a documentation before a service is shut down, or to keep an offline copy of an internal site.

Turning on recursion

-r turns on link following, -m (mirror) is the ready-to-use shortcut. The manual is explicit: -m is equivalent to -r -N -l inf --no-remove-listing, so infinite depth, update by date and keeping FTP listings.

wget -m -k -p -np -nH https://docs.interne.lan/runbooks/

This combination is the one actually used for archiving, and each of its four options fixes a precise weakness of a raw mirror:

• -k (--convert-links) rewrites the links so that they point to the local files, without which the copy cannot be browsed offline.
• -p (--page-requisites) fetches the images, CSS and scripts needed for display.
• -np (--no-parent) forbids going up above the starting directory.
• -nH (--no-host-directories) avoids creating a directory named after the server.

A clarification that avoids much misunderstanding: wget only discovers the files really linked from the pages walked. A file present on the server but that no link references will never be fetched: recursive mode is not a directory browser.

Controlling depth and domains

By default, the recursion depth is 5 levels. -l changes it, -l inf makes it infinite. A poorly bounded mirror can download gigabytes: always start small, then widen.

wget -r -l 2 -np https://docs.interne.lan/runbooks/

For safety, wget stays on the starting host. To widen, -H (--span-hosts) allows other hosts, and is then bounded with -D (--domains) or --exclude-domains. The trio reads as an allow list and a deny list of domains.

wget -r -l 2 -H -D docs.interne.lan,cdn.interne.lan https://docs.interne.lan/runbooks/

Filtering the files fetched

-A (accept) and -R (reject) take a list of comma-separated extensions, which is enough in most cases. To target a URL pattern, --accept-regex and --reject-regex accept a regular expression.

# Ne récupérer que les PDF et les archives d'un dépôt interne
wget -r -l 2 -np -A pdf,tar.gz https://docs.interne.lan/livrables/

# Ignorer les URL contenant un identifiant de session
wget -r -np --reject-regex ".*\?sessionid=.*" https://docs.interne.lan/runbooks/

Add -Q (quota) to cap the total volume downloaded, for example -Q500m. It is the safeguard that avoids filling a disk overnight, and it combines with -l to bound the mirror on two axes.

Mirroring a site is not neutral

Seen from the server, aggressive recursion looks like a denial-of-service attack. On a site you do not own: respect robots.txt (wget does it by default), cap with --limit-rate, and space out the requests with -w 1, that is one second of waiting between two files. On an internal site, warn the team that runs it.

Authenticating

Three mechanisms cover most of the protected resources you will meet: the HTTP password, the token in a header, and the session cookie. In all three cases, the security rule is the same: never write a secret on the command line, because it is visible to everyone through ps aux and ends up in the shell history.

HTTP password

--http-user and --http-password provide the credentials of a Basic or Digest authentication: wget picks the method announced by the server, you have nothing to force.

wget --http-user=deploy --ask-password https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb

--ask-password asks for the password interactively, without leaving it in clear text on the command line: it is the form to prefer for manual use. For a script, prefer a ~/.netrc file in chmod 600, which wget reads automatically. The --auth-no-challenge option sends the credentials from the first request, without waiting for the server challenge; only turn it on if the server requires it, because it exposes the password even when it was not necessary.

Token in a header

Artifact repositories and APIs most often expect a Bearer token. --header adds an arbitrary HTTP header, but beware of the trap: --header="Authorization: Bearer ${DEPOT_TOKEN}" is expanded by the shell before the call, and the token ends up in clear text in the process arguments, readable through ps aux. The environment variable therefore protects nothing. The fix is to pass the header through a dedicated configuration file, in chmod 600, which --config loads instead of ~/.wgetrc:

umask 077   # the file created is readable by you only
printf 'header = Authorization: Bearer %s\n' "$DEPOT_TOKEN" > ~/.wgetrc-depot
wget --config="$HOME/.wgetrc-depot" \
     -O agent.deb https://depot.interne.lan/api/paquets/agent-2.4.1
rm -f ~/.wgetrc-depot

Since printf is a shell builtin, it creates no process and the token never shows up in ps. Measured with wget 1.21.4: the server does receive the Authorization header. For headers without a secret, such as Accept: or a custom header required by a corporate proxy, --header remains the simplest form and can be repeated as many times as needed.

Session cookie

For an application that protects its downloads behind a login form, you must first obtain a cookie, then replay it. --keep-session-cookies is essential: without it, wget only saves the persistent cookies and drops the session one, precisely the one you need.

umask 077
printf 'user=deploy&password=%s' "$MDP" > identifiants.txt
wget --save-cookies cookies.txt --keep-session-cookies \
     --post-file=identifiants.txt \
     -O /dev/null https://portail.interne.lan/login
rm -f identifiants.txt

wget --load-cookies cookies.txt -O rapport.csv https://portail.interne.lan/exports/rapport.csv

--post-file reads the request body from a file, for the same reason as above: a --post-data="...${MDP}" would expose the password in ps. The cookies.txt file then contains a valid session token: treat it like a password, keep it in chmod 600 (the umask takes care of it) and delete it at the end of the script.

FTP and FTPS

wget still handles FTP, a protocol you meet on old internal mirrors or industrial equipment. The credentials go through --ftp-user and --ftp-password, and a ftps:// URL switches to the encrypted variant.

wget --ftp-user=archives --ask-password ftp://miroir.interne.lan/isos/debian-12.iso

Plain FTP carries the credentials in clear text over the network. If it is still used where you work, it is a point to raise: prefer ftps://, or better, HTTPS access.

Proxy and network stack

On a server that has no direct Internet access, everything goes through an outbound proxy. wget honours the standard environment variables, which makes it the simplest and most portable method.

export http_proxy="http://proxy.interne.lan:3128/"
export https_proxy="http://proxy.interne.lan:3128/"
export no_proxy="localhost,127.0.0.1,depot.interne.lan"
wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/SHA256SUMS

no_proxy lists the destinations that must stay on a direct connection, typically your internal repositories. For a one-off call without touching the environment, -e injects the matching configuration directive, and --proxy-user provides the credentials if the proxy authenticates. The directive follows the scheme of the target URL: an https:// URL reads https_proxy, and an -e "http_proxy=..." would be ignored without the slightest message:

wget -e "https_proxy=http://proxy.interne.lan:3128" \
     --proxy-user=deploy --ask-password \
     https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb

Conversely, --no-proxy ignores the configured proxy for this request: it is the first test to run when an internal download fails for no apparent reason. Finally, on a dual-stack network, -4 and -6 force the address family and fix many slowdowns caused by an IPv6 that is announced but not routed.

wget -4 https://depot.interne.lan/paquets/agent_2.4.1_amd64.deb

The .wgetrc configuration file

Repeating the same proxy and timeout options in each script is a source of error. The ~/.wgetrc file (personal configuration) holds these default values once and for all; /etc/wgetrc does the same for the whole machine. Each line is a directive, not a shell command.

~/.wgetrc

http_proxy = http://proxy.interne.lan:3128/
https_proxy = http://proxy.interne.lan:3128/
no_proxy = localhost,127.0.0.1,depot.interne.lan
dns_timeout = 5
connect_timeout = 10
read_timeout = 20
tries = 3
limit_rate = 2m

The -e option of the command line accepts exactly these same directives, and nothing else: -e "tries=3" works, whereas -e "chmod 644" is rejected with the message Invalid --execute command. It is not a shell command, and the confusion is frequent.

wget has no post-download hook

There is no .wgetrc directive that runs a command once the file has been fetched: a line such as execute = chmod 644 makes wget fail at startup with Unknown command ‘execute’ and return code 2. To chain an action, do it in the shell, after checking the return code.

wget -nv -O jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64 \
  && echo "5942c9b0934e510ee61eb3e30273f1b3fe2590df93933a93d7c58b81d19c8ff5  jq" | sha256sum -c - \
  && chmod +x jq

This form is also the security good practice: you download, you check the checksum published in the release's sha256sum.txt file, and only then do you make the binary executable. A plain sha256sum jq would not be enough: it prints the checksum without comparing it to anything, and returns code 0 whatever the content. Never pipe a download straight into an interpreter: a compromised server, or a simple cut in the middle of the transfer, would run arbitrary code on your machine before you could even read it.

Common pitfalls

These errors come back with beginners as well as experienced teams. Knowing them saves hours of diagnosis: in most cases, the visible symptom is very far from the real cause.

Symptom	Cause	Fix
.1, .2 files pile up	wget does not replace, it numbers	Add -nc (cache) or -N (refresh on the date)
The script carries on despite a 404	-q hides the errors	Test $?, or use -nv
rapport-0[1-5].csv returns 404	No HTTP globbing	Shell brace expansion: rapport-0{1..5}.csv
The downloaded file is HTML	The server returned an error or login page	--spider -S to read the code and the headers
The script hangs for several minutes	20 retries by default	--tries=2 --timeout=10
The mirror fetches the whole site	Infinite depth with -m	Bound with -l, -np, -Q
The file does not have the right name after a redirect	Name taken from the original URL	--content-disposition or --trust-server-names
The proxy given with -e is ignored	http_proxy set for an https:// URL	Use -e "https_proxy=..."

Swipe to see more

Key points

• wget downloads, curl talks: for a REST API or fine HTTP diagnosis, switch to curl.
• wget does not overwrite an existing file, it creates fichier.1: use -nc to download nothing again, -N to refresh based on the date.
• -c resumes an interrupted transfer if the server accepts partial requests, which changes everything on a multi-gigabyte ISO.
• -q hides errors: in a script, always test the return code (0 success, 4 network failure, 8 server error).
• Bound your scripts with --tries and --timeout, otherwise wget retries 20 times by default.
• URL globbing does not exist in HTTP: go through shell brace expansion (fichier-{1..5}.csv), the bracket pattern being specific to curl.
• -m mirrors without any depth limit: frame it with -np, -l, -Q and --limit-rate so as not to overwhelm the remote server.
• wget runs nothing after the download: chain in the shell, checking the checksum before making a binary executable, and never pipe a download straight into an interpreter.

Knowledge check

Check that the essentials of this guide are acquired. The questions only cover what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →