Ollama: run LLMs locally on your own PC

Ollama runs AI models, Llama, Mistral, Gemma, Qwen, directly on your computer, with no subscription, without sending your data to a third-party server and even without an internet connection once the model is downloaded. This guide takes you from installation to your first conversation. Allow about 15 minutes to get a working local AI. It covers Ollama 0.40+ on Windows, macOS and Linux.

It is written for beginners and developers alike. No prior knowledge of artificial intelligence is required: if you can open a terminal and type a command, you have the level. Ollama is the right choice when an online assistant does not fit: code under NDA, regulated data, a tight budget or offline work.

What you will learn

By the end of this guide, you will be able to:

• Install Ollama on your computer (Windows, Mac or Linux)
• Download and use AI models such as Llama, Mistral or Qwen Coder
• Chat with the AI from the command line
• Pick the right model for your needs (code, text, translation...)
• Know where to troubleshoot common errors and secure access to Ollama

Installation and the command line are covered here from start to finish. The Python integration, in-depth troubleshooting and the fine choice of a model are covered in separate guides, linked throughout the text.

What is Ollama? A detailed explanation

Ollama is free, open source software (MIT license) that downloads and runs large language models (LLMs) directly on your machine, through a single command and a local API on port 11434. It makes generative AI usable without a subscription, without the cloud and offline once the model has been fetched.

Before diving into the installation, let us take a few minutes to understand what Ollama really is and why it matters. That understanding will help you use it better afterwards.

The problem Ollama solves

To use AI in your work as a developer, three routes exist, and none of them is free of effort. They differ in what they cost you in privacy, money and setup time.

• An online service (ChatGPT, Claude, Gemini): very convenient, but your code (possibly confidential) goes through third-party servers, usage is paid (monthly subscription or billing by volume of text processed) and the service is unusable during an outage.
• A model installed by hand: technically possible, but you have to download weights of several gigabytes, install the right versions of Python, PyTorch and CUDA, tune dozens of parameters and write the loading code. Count on hours, even for an experienced developer.
• Ollama: a single command installs a model, the technical parameters are set automatically, a simple interface lets you chat with it and your data stays local.

Ollama is not the only local inference engine, and it is not always the best fit. llama.cpp gives fine control over quantization and runs on modest hardware, vLLM aims at throughput on GPU to serve several users at once, TGI (Text Generation Inference, from Hugging Face) targets the same production ground. Ollama wins on ease of setup and loses on fine tuning. The comparison of inference backends details these trade-offs according to your hardware and your volume of use.

What exactly is an LLM?

You have surely seen the term LLM (Large Language Model). Concretely, it is a computer program trained on huge amounts of text (books, articles, source code, web pages...). Thanks to this training, it has learned:

• The rules of language: grammar, spelling, syntax
• Knowledge of the world: history, science, current events (up to its cutoff date)
• Reasoning patterns: how to solve problems, how to structure an answer
• Code conventions: language syntax, good practices, common patterns

When you ask an LLM a question, it does not "look up" the answer in a database. It generates the answer word by word, predicting which word is most likely after the previous one, given your question and everything it has learned.

Why "local" changes everything

With an online service, your question travels to the provider's servers (OpenAI, for example, in the United States), which computes the answer on its GPUs and sends it back. With Ollama, the model computes the answer on your own machine and nothing leaves your computer. The table spells out the concrete consequences.

Aspect	ChatGPT (cloud)	Ollama (local)
Privacy	Your data goes through third-party servers	Your data stays on your machine
Cost	Subscription or pay-per-use billing	Free (once you own the computer)
Internet	Required	Optional (after the model is downloaded)
Speed	Depends on network latency	Depends on your hardware
Availability	Can be saturated or under maintenance	Always available
Customization	Limited	Total (you can create your own models)

Swipe to see more

Concrete use cases

Four real scenarios show where Ollama shines. In each of them, it is control over data or cost that makes local the better choice.

• Privacy: a developer works on a client project under NDA (non-disclosure agreement). She cannot paste her code into an online service without breaching the contract; with Ollama, it never leaves her workstation.
• Budget: a student cannot afford the 20 EUR a month of a subscription. Ollama gives him a free assistant with no message limit to explain complex concepts.
• Unreliable network: a consultant who spends a lot of time on trains keeps working with AI, even in a tunnel.
• Regulated data: a bank's data team cannot send its internal reports to an American provider. Ollama lets it run AI in-house.

Is your computer compatible?

Before installing Ollama, let us check that your machine can run it. Rest assured: most recent computers (less than 5-6 years old) are compatible.

RAM: the key factor

Memory (RAM) is the most important factor. Here is why.

When you launch an AI model with Ollama, it is loaded entirely into memory. If your computer does not have enough RAM, the model cannot be loaded, or will be very slow because it has to use the hard drive (this is called swap).

Available RAM	Usable models	Experience
4 GB	Very small models (1 to 2 GB) only	Possible but poor: slow, limited quality
8 GB	Light models (3B)	Functional but limited
16 GB	Medium models (7B)	Good experience for most uses
32 GB	Large models (13B)	Excellent experience
64 GB+	Very large models (70B)	Professional use

Swipe to see more

How do you check your RAM?

• Windows: right-click the taskbar, "Task Manager", "Performance" tab, then "Memory".
• macOS: Apple menu, "About This Mac"; the RAM is shown (for example "16 GB").
• Linux: the command below prints the total RAM.

# Show total RAM
free -h

You will see something like Mem: 15Gi (for 16 GB).

Disk space

Every AI model takes up space on your hard drive. Here are the typical sizes:

Model	Size on disk
Llama 3.2 (3B)	~2 GB
Mistral (7B)	~4 GB
Qwen2.5-Coder (7B)	~4.7 GB
Llama 3.1 (70B)	~40 GB

Swipe to see more

Recommendation: keep at least 20 GB free so you can download several models and try them out.

The GPU: optional but useful

A graphics card (GPU) is not mandatory. Ollama works very well on a CPU (a regular processor). The difference:

• Without a GPU: a response takes 2-10 seconds
• With an NVIDIA GPU: a response takes 0.5-2 seconds

If you have an NVIDIA GPU with compute capability 5.0 or higher and a driver 550 or newer, Ollama will use it automatically. Cards with compute capability 5.0 to 6.2 require a driver 570 or newer (official Ollama documentation, docs/gpu.mdx). VRAM does not decide whether the GPU is used but the size of model that fits on it (about 8 GB for a 7B model). Otherwise, no worries, the CPU will do the job.

Apple Silicon Macs (M1 and later) are a special and particularly favorable case. Their unified memory is shared between the processor and the integrated GPU: the model does not have to be copied into a separate VRAM, and a 16 GB MacBook can load models that a laptop without a dedicated graphics card struggles with. Ollama uses this GPU automatically, with no driver or configuration.

Prerequisites summary

Before installing Ollama, check that your machine meets the requirements below. RAM is the most limiting factor: it determines the size of the models you will be able to load.

Component	Minimum	Recommended	Ideal
RAM	8 GB	16 GB	32 GB
Disk	20 GB free	50 GB free	100 GB+ (SSD)
Processor	Modern 64-bit	Recent multicore	Apple M1+ or Intel i7+
GPU	Not required	NVIDIA 8 GB VRAM	NVIDIA 16 GB+ VRAM
System	Windows 10 22H2, macOS 14 (Sonoma), modern Linux

Swipe to see more

Installing Ollama step by step

Now that you know your machine is compatible, let us move on to the installation. I will guide you step by step according to your operating system.

Installing on Linux (Ubuntu, Debian, Fedora...)

Linux is the easiest system on which to install Ollama. Three commands are enough: download the script, read it, then run it.

1. Open a terminal

   Use the Ctrl + Alt + T shortcut or search for "Terminal" in your applications.

2. Run the installation script

   Copy and paste this command and press Enter:

   # Download the official script, inspect it, then run it
   curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
   less ollama-install.sh
   sh ollama-install.sh

   What do these commands do?

   • curl ... -o ollama-install.sh downloads the installation script from the official Ollama site into a file
   • less ollama-install.sh lets you read the script before running it (never blindly run a remote script)
   • sh ollama-install.sh runs the script once you have read it; it detects your Linux distribution and installs Ollama

   Download and installation usually take 1-2 minutes.

3. Check the installation

   Once finished, check that Ollama is installed:

   ollama --version

   You should see something like ollama version is 0.40.4 (the number may vary).

4. Check that the service is running

   Ollama runs as a background service. Check its status:

   sudo systemctl status ollama

   You should see Active: active (running) in green.

If something goes wrong:

# Read the service logs to diagnose
sudo journalctl -u ollama -f

# Restart the service
sudo systemctl restart ollama

Installing on Windows

On Windows, installation goes through a classic graphical installer, with no command line. Once in place, Ollama runs as a background service shown by the llama icon in the notification area: this service is what answers your commands, and closing it shuts Ollama down.

1. Download the installer

   Go to ollama.com/download and click the "Download for Windows" button.

   The OllamaSetup.exe file will be downloaded (about 200 MB).

2. Run the installation

   Double-click the downloaded file. If Windows asks for confirmation ("Do you want to allow this app..."), click "Yes".

   The setup wizard appears. Click "Install" and wait a few moments.

3. Check the installation

   Open PowerShell (search for "PowerShell" in the Start menu) and type:

   ollama --version

   If you see a version number, the installation succeeded!

4. Important note: the taskbar icon

   After the installation, you will see a small Ollama icon (a llama) in the notification area (bottom right). Ollama runs in the background, which is normal and necessary for it to work.

One last Windows-specific point: on the first download of a model, Windows Defender scans the files received, and a 2 GB model takes a while to be scanned. The download then looks stuck when it is not. The scan happens only once per model, and later launches are immediate.

Installing on macOS

On macOS, Ollama is distributed as an application that you drag into the Applications folder. On Apple Silicon Macs (M1 and later), the integrated GPU is used automatically, which often gives better performance than a regular laptop without a dedicated graphics card.

1. Download the application

   Go to ollama.com/download and click "Download for macOS".

   The Ollama.dmg file will be downloaded. It requires macOS 14 (Sonoma) or newer; on an Intel Mac, Ollama runs on the processor only, with no GPU acceleration.

2. Install the application

   Double-click Ollama.dmg to mount the disk image, then drag and drop Ollama.app into your Applications folder. On first launch, the application checks that the ollama command is available in the terminal and offers to create the link in /usr/local/bin.

3. Launch Ollama

   Double-click Ollama in your Applications. macOS may ask you to confirm ("Ollama is an app downloaded from the Internet..."), click "Open".

   A llama icon will appear in the menu bar at the top of the screen.

4. Check the installation

   Open the Terminal (Applications → Utilities → Terminal) and type:

   ollama --version

   You should see the version number.

Downloading your first model

Ollama is installed, but it is missing the most important thing: an AI model. The model is what generates the answers; with no model loaded, Ollama cannot produce anything.

Understanding the available models

Ollama gives access to dozens of models. To start, I recommend Llama 3.2, developed by Meta (the company behind Facebook). Why this choice?

• Proven: released in September 2024, it is widely documented and used as the example in most guides
• Capable: excellent abilities in English and French
• Light: the 3B version weighs only 2 GB
• Versatile: good for text, code, translation...

A model's name carries two pieces of information, its family and its size: in "Llama 3.2 3B", the "3B" means 3 billion parameters. Parameters are the adjustable weights of the neural network, each one a number learned during training. The more there are, the more regularities of language the model captures, and the more memory and compute it consumes. A smaller model is therefore not "worse" in absolute terms, it is faster and less accurate on reasoning tasks. For personal use, 3B is an excellent compromise.

Model size	Minimum RAM	Typical use
1-3B	4 GB (1-2B) to 8 GB (3B)	Simple questions, summaries, translation
7B	8-16 GB	Code, writing, reasoning
13B	16-32 GB	Complex analysis, creativity
70B	64 GB+	Research, professional use

Swipe to see more

If you have a GPU, the sizing rule changes: it is the VRAM (the graphics card's memory) that decides, not the system RAM. With Q4_K_M quantization, the default format of models distributed by Ollama, plan for roughly as much VRAM as the size of the downloaded file, plus 1 to 2 GB for the context. Concretely, 8 GB of VRAM runs a 7B or 8B model (Llama 3.1 8B, Qwen3 8B), 16 GB a 13B-14B model, 24 GB and more a 32B model. Without a GPU, stick to 1B-3B, otherwise generation becomes painful.

The point to remember is the threshold: as soon as the model overflows from VRAM into system RAM, speed drops by 2 to 10 times. A smaller model that fits entirely in VRAM beats an ambitious model that overflows. The detail by use case and quantization level is covered in the guide which model to pick in 2026.

Downloading Llama 3.2

Open your terminal (or PowerShell on Windows) and type:

ollama pull llama3.2

What exactly happens?

1. Ollama contacts its registry (a sort of model "store")
2. It downloads the model piece by piece (you will see a progress bar)
3. It verifies the integrity of the downloaded files (to make sure they are not corrupted)
4. It saves the model in Ollama's local storage

The download usually takes 2 to 10 minutes depending on your internet connection. You will see something like:

pulling manifest
pulling dde5aa3fc5ff...  50% ▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░  1.1 GB/2.0 GB  45 MB/s

The command is safe to replay: if the model is already present, ollama pull llama3.2 just checks the layers and finishes within seconds. It is also the command that updates a model when a new version is published on the registry.

Checking that the download worked

Once the download is finished, check that the model is installed:

ollama list

You should see:

NAME              ID              SIZE      MODIFIED
llama3.2:latest   a80c4f17acd5    2.0 GB    About a minute ago

Well done! You now have a local AI ready to use.

Your first conversation with the AI

This is the moment you have been waiting for: talking to your local AI.

Starting an interactive session

Type this command:

ollama run llama3.2

After a few seconds of loading, a command prompt appears:

>>>

This is where you will type your questions. The model is waiting for your message!

A complete conversation example

Here is an example of what you can do. Type exactly what follows the >>>:

You:

>>> Hello! Can you introduce yourself?

The AI answers:

Hello! I am an AI assistant based on the Llama 3.2 model, developed by
Meta. I am designed to help you with various tasks such as answering your
questions, generating text, translating, explaining concepts, and much more.

I run entirely on your computer thanks to Ollama, which means that your
conversations stay private and are not sent anywhere.

How can I help you today?

You:

>>> Explain Docker to me as if I were 10 years old

The AI answers:

Imagine you want to play a video game at your friend's house. But the game
needs lots of things on the computer: special files, particular settings...
If your friend does not have exactly the same things, the game will not
work!

Docker is like a magic box where you put your game with EVERYTHING it needs.
When you give this box to your friend, they can open the box and the game
works right away, even if their computer is different from yours.

Developers use Docker to put their programs in "boxes" (they are called
"containers"). That way, the program works the same everywhere, on any
computer!

You:

>>> Great! Now, write me a Python function that counts the words in a sentence

The AI answers:

def count_words(sentence):
    """
    Count the number of words in a sentence.

    Args:
        sentence (str): The sentence to analyze

    Returns:
        int: The number of words
    """
    # Remove spaces at the start and end
    sentence = sentence.strip()

    # If the sentence is empty, return 0
    if not sentence:
        return 0

    # Split the sentence into words and count them
    words = sentence.split()
    return len(words)

# Usage example
text = "Hello, how are you doing today?"
number = count_words(text)
print(f"The sentence contains {number} words.")

Leaving the session

To end the conversation, type:

>>> /bye

You will return to your normal terminal.

Tips for better answers

The AI works better when you give it clear and detailed instructions. Here are a few techniques:

Vague question, to avoid:

>>> Tell me about Python

Precise question:

>>> Explain the differences between lists and tuples in Python, with examples of cases where to use one rather than the other

Generic request, to avoid:

>>> Write some code

Contextualized request:

>>> Write a Python function that takes a list of numbers and returns the average, handling the case where the list is empty. Add explanatory comments.

Essential Ollama commands

Now that you know how to chat with the AI, here are the essential commands to know.

Managing your models

Four commands cover the whole life cycle of a model: list it, download it, delete it and inspect it. The most useful day to day is ollama rm: models pile up fast and a 70B model takes about forty gigabytes, so it pays to clean up the models you no longer test, to free up disk space.

# List all models installed on your machine
ollama list

# Download a new model (example: Mistral)
ollama pull mistral

# Delete a model to free up disk space
ollama rm mistral

# Show a model's details (size, parameters, license...)
ollama show llama3.2

Running one-off commands

You do not have to open an interactive session. You can ask a single question:

# Ask a question and get the answer immediately
ollama run llama3.2 "What is the capital of Australia?"

The answer is printed, then you return to the terminal. This is handy for quick questions or to integrate Ollama into scripts.

Using several models

You can install several models and switch between them:

# Install several models
ollama pull llama3.2
ollama pull mistral
ollama pull qwen2.5-coder:7b

# Use Mistral for this question
ollama run mistral "Write a haiku about autumn"

# Use Qwen2.5-Coder for code
ollama run qwen2.5-coder:7b "Write a regex to validate an email address"

Each model takes up disk space, so there is no point installing them all up front. The table below gives each one's strength and the command to get it: choose according to what you do most often, and delete another one later with ollama rm if needed.

Model	Size	Strength	Command
llama3.2	2 GB	Versatile, good at several languages	ollama pull llama3.2
mistral	4 GB	Reasoning, logic	ollama pull mistral
qwen2.5-coder:7b	4.7 GB	Code generation	ollama pull qwen2.5-coder:7b
gemma:2b	1.7 GB	Light and fast, for modest machines	ollama pull gemma:2b

Swipe to see more

To get started, llama3.2 is plenty. Add qwen2.5-coder:7b if you do a lot of development (for the fine choice, see the 2026 model comparison), and gemma:2b if your machine is modest.

Using Ollama with Python

Ollama can also be called from your Python scripts, through the REST API on port 11434, the native ollama SDK, litellm or the OpenAI client pointed at your local instance. The guide Advanced Ollama: Python SDK, Structured Output and embeddings checks the REST API, then walks through the native SDK and the OpenAI client before building a complete assistant.

Three needs then come back systematically as soon as you plug a local LLM into code, and each has its own guide.

• Getting an answer a program can use, that is JSON that follows a schema rather than a paragraph: structured outputs with Ollama.
• Letting the model call your own functions (query a database, run a command) instead of describing what it would do: tool calling with Ollama.
• Having a multimodal model analyze images, a screenshot or a diagram: vision models with Ollama.

Troubleshooting and configuration

Common errors (Ollama does not respond, model not found, slowness) and configuration (storage location, memory limits, network exposure and security of port 11434, the context pitfall) are gathered in a dedicated guide: Ollama: troubleshooting and configuration.

Do not expose port 11434 on the network

By default, Ollama only listens on 127.0.0.1:11434, so only from your own machine. Setting OLLAMA_HOST=0.0.0.0 to reach it from another workstation makes the API reachable with no authentication at all: anyone on the network can list your models, download some, delete some and use up your GPU. If remote access is needed, put Ollama behind an authenticated reverse proxy or restrict the port to the local network with the firewall.

Checklist

Go through these points in order to confirm that your installation is complete and working. The first two blocks are essential; the third is a matter of comfort and can wait.

Installation

These four points confirm that Ollama is properly in place and that its service is really running on your machine.

• RAM checked (8 GB minimum, 16 GB recommended)
• Disk space available (20 GB minimum)
• Ollama installed (ollama --version works)
• Ollama service active (icon visible or systemctl status ollama)

First model

A downloaded model and a first answer prove that the whole chain, from registry to terminal, works.

• Model downloaded (ollama pull llama3.2)
• Interactive session tested (ollama run llama3.2)
• Test question asked and answer received

Advanced use (optional)

These points are not essential to chat with the AI, they prepare the integration of Ollama into your scripts and Python projects.

• Several models installed (llama3.2, qwen2.5-coder:7b, mistral...)
• I know where to find the Python integration if needed

Key takeaways

• Ollama = local LLM: run AI models such as Llama, Mistral or Qwen Coder directly on your machine, without sending your data to third-party servers.
• Two commands are enough: ollama pull llama3.2 downloads the model, ollama run llama3.2 starts the conversation. No complex configuration.
• RAM is the key factor: 8 GB minimum for small models (3B), 16 GB recommended for medium models (7B), 32 GB for large ones (13B+).
• Free and unlimited: no subscription, no message limit, no waiting queue.
• Works offline: once the model is downloaded, no more internet needed. Ideal for privacy or when traveling.
• Local API: the REST API on port 11434 lets you call the model from your Python scripts, without exposing it on the network.

Knowledge check

Check that you have the essentials of this guide. The questions cover only what was just explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

FAQ: frequently asked questions

What is Ollama?

What is the difference between Ollama and ChatGPT?

What is an LLM (Large Language Model)?

What hardware do you need for Ollama?

Which Ollama model should you pick to get started?

How do you install Ollama on Linux?

How do you install Ollama on Windows?

How do you install Ollama on macOS?

How do you fix the 'Out of memory' error?

How do you fix the 'Connection refused' error?

How do you fix the 'Model not found' error?

How do you fix model download problems?

How do you fix the port 11434 conflict?

How do you use Ollama with an NVIDIA GPU?

How do you use Ollama's REST API?

Where are Ollama models stored?

Is Ollama free?

Ollama or LM Studio: what is the difference?

Official resources

These four sources are the reference when a command or an option in this guide changes: the API documentation describes the endpoints of port 11434, the catalog gives the exact tags of each model, and the GitHub repository carries the release notes.

• Ollama website: ollama.com
• Source code on GitHub: github.com/ollama/ollama
• Model catalog: ollama.com/library
• API documentation: docs.ollama.com/api

×

See in the glossary →