With Gemma 4 (Google) or Llama 3.2 Vision (Meta), a model running on your machine can describe an image, classify it and read its text. No image leaves your computer: that is useful when the images are sensitive (health, defense, privacy) or when you do not want to depend on a paid API. This guide puts it into practice with Ollama and Python, on images that you create yourself.
What you will learn
- Choose a vision model based on its size, its declared capabilities and your memory
- Describe an image locally, in French
- Extract structured JSON from an image, by combining vision and structured outputs (Structured Outputs)
- Do local OCR: read the text of a screenshot
- Know the limits of local vision models: fine details, reasoning, image size
Which vision models in 2026?
A vision model accepts an image in addition to text. Each model's page on ollama.com/library shows the vision capability; the sizes below are those of that library, that is, the size of the file to download.
| Model | Ollama tag | File size | Notes |
|---|---|---|---|
| Gemma 4 | gemma4 | 6.6 GB for the latest tag (e4b) | vision, tools and thinking capabilities declared, 131072-token context |
| Llama 3.2 Vision 11B | llama3.2-vision | 7.8 GB | Meta multimodal, English only declared for requests with an image |
| Llama 3.2 Vision 90B | llama3.2-vision:90b | 55 GB | Reserved for machines with very large memory |
| Qwen 2.5 VL | qwen2.5vl (7b) | 6.0 GB (3b: 3.2 GB) | Alibaba multimodal |
| LLaVA | llava:7b | 4.7 GB | The classic, a lightweight alternative |
Gemma 4 comes in several variants, and its size ranges from 4.6 to 19 GB depending on the tag you download. The 6.6 GB figure in the table only holds for latest. The other tags weigh e2b (4.6 GB), 12b (7.7 GB), 26b (16 GB) and 31b (19 GB). Watch the name of Qwen: the tag is qwen2.5vl, with no hyphen between 2.5 and vl.
Three terms in the last column deserve a definition. A tools capability means the model can ask for a function to be called; thinking means it can produce reasoning before its answer. The context is the amount of text, counted in tokens (pieces of words), that the model processes at once; an image uses up its share of it too.
This guide uses gemma4, because its page declares both vision and tools: a single model is therefore enough to describe an image and to propose a function call. The Tool calling guide, for its part, uses qwen3. The file size is not the memory used. After a call, ollama ps shows the memory occupied in the SIZE column, and the PROCESSOR column says whether the model runs on the CPU, on the GPU or on both.
Prerequisites and installation
-
Check Ollama's version (the outputs in this guide come from version 0.40.16):
Terminal window ollama --version# To update (download the script, inspect it, run it):curl -fsSL https://ollama.com/install.sh -o ollama-install.shless ollama-install.shsh ollama-install.sh -
Download the model (
ollama pull):Terminal window ollama pull gemma4 # 6.6 GB, the model for this guide# orollama pull llama3.2-vision # 7.8 GB, alternative -
Install the Python dependencies:
pyproject.toml [project]name = "lab-ollama-vision"version = "0.1.0"requires-python = ">=3.10"dependencies = ["ollama==0.6.3","pydantic==2.14.3","pillow==11.3.0",]Terminal window uv sync
First call: describe an image
To reproduce this guide's outputs, first create two images with Pillow: shapes/sample.png (512×256 pixels, white background, a red square on the left and a blue circle on the right) and ocr/sample.png (640×120 pixels, white background, a line of black text). Run this script from your project folder: it creates the shapes and ocr folders.
from pathlib import Pathfrom PIL import Image, ImageDraw, ImageFont
Path("shapes").mkdir(exist_ok=True)Path("ocr").mkdir(exist_ok=True)
im = Image.new("RGB", (512, 256), "white")d = ImageDraw.Draw(im)d.rectangle([40, 48, 200, 208], fill=(220, 20, 20))d.ellipse([300, 48, 460, 208], fill=(20, 40, 200))im.save("shapes/sample.png")
im2 = Image.new("RGB", (640, 120), "white")d2 = ImageDraw.Draw(im2)d2.text((20, 40), "Lab Ollama Vision 2026 - dev H100", fill="black", font=ImageFont.load_default(size=32))im2.save("ocr/sample.png")Run this script with uv run 00-creer-images.py from the project folder. The following scripts are run the same way with uv run <script>.py, including from the shapes and ocr subfolders: uv climbs up to the project's pyproject.toml to find its environment.
The following script, run from the shapes folder, sends the first image to the model:
from pathlib import Pathfrom ollama import Client
MODEL = "gemma4"HOST = "http://localhost:11434"IMAGE = Path("sample.png") # Une image PNG ou JPEG, n'importe laquelle
client = Client(host=HOST)response = client.chat( model=MODEL, messages=[ { "role": "user", "content": "Décris cette image en 3-4 phrases.", "images": [str(IMAGE)], # chemin vers l'image } ], options={"temperature": 0.2},)print(response["message"]["content"])The prompt, "Décris cette image en 3-4 phrases." ("Describe this image in 3-4 sentences."), is in French, so the model answers in French.
Real output with gemma4 (image: a red square and a blue circle on a white background, run on CPU only):
Cette image est une composition graphique très simple et contrastée. Elle est dominée par deux formes géométriques distinctes, placées côte à côte sur un fond blanc. Sur la gauche, on trouve un carré de couleur rouge vif. À sa droite, un cercle de couleur bleu intense est positionné. Ces deux formes offrent un contraste visuel marqué entre le rouge et le bleu, créant une juxtaposition nette et épurée.The model spotted the two shapes, their colors and their relative position: it describes a very simple, high-contrast graphic composition, with a bright red square on the left and an intense blue circle on its right, on a white background. Its wording varies from one run to the next, even at a low temperature.
Vision and structured outputs: typed JSON from an image
A structured output (Structured Outputs) means passing Ollama a JSON schema in the format parameter: the model then only produces a response that conforms to that schema. Combined with an image, it spares you free text that you would have to parse with regular expressions: you get a typed Pydantic object directly, filled in from the image.
This schema is useful, for example, to catalog product photos for an online store, or to sort images uploaded by users.
Run this script from the shapes folder, like the previous one: it reads sample.png in the current folder.
from pathlib import Pathfrom typing import Literalfrom ollama import Clientfrom pydantic import BaseModel, Field
MODEL = "gemma4"HOST = "http://localhost:11434"IMAGE = Path("sample.png")
class AnalyseImage(BaseModel): """Analyse structurée d'une image.""" type_image: Literal["photo", "schema", "capture_ecran", "illustration", "autre"] objets_principaux: list[str] = Field(description="3-5 objets visibles, en français.") couleurs_dominantes: list[str] = Field(description="2-4 couleurs majoritaires.") contient_texte: bool texte_extrait: str | None = Field(default=None, description="Texte visible si présent.") description_courte: str = Field(description="1 phrase de résumé.")
client = Client(host=HOST)response = client.chat( model=MODEL, messages=[ { "role": "user", "content": "Analyse cette image et remplis le JSON demandé.", "images": [str(IMAGE)], } ], format=AnalyseImage.model_json_schema(), # sortie structurée options={"temperature": 0},)analyse = AnalyseImage.model_validate_json(response["message"]["content"])print(analyse.model_dump_json(indent=2))The schema's field names are in French (type_image is the image type, objets_principaux the main objects, couleurs_dominantes the dominant colors, contient_texte whether it contains text, texte_extrait the extracted text, description_courte the short description), and so are the values the model returns.
Real output obtained:
{ "type_image": "illustration", "objets_principaux": [ "carré", "cercle" ], "couleurs_dominantes": [ "rouge", "bleu", "blanc" ], "contient_texte": false, "texte_extrait": null, "description_courte": "Une image simple contenant un carré rouge à gauche et un cercle bleu à droite, sur fond blanc."}The model lists a square (carré) and a circle (cercle), the colors red, blue and white (rouge, bleu, blanc), and sums up the image as "a simple image containing a red square on the left and a blue circle on the right, on a white background". The JSON respects the schema: type_image is one of the Literal values, contient_texte is a boolean and texte_extrait is null, since the image contains no text. The model classified the image as illustration, a choice the schema allows but that nothing forces: the value's content remains to be checked, only the shape is guaranteed.
Local OCR: extract text from a screenshot
OCR (Optical Character Recognition) means extracting the text present in an image. A vision model reads a line of clean text with no difficulty, as in the example below. On dense or handwritten text, a dedicated tool (Tesseract, docTR) is a safer choice; the box that follows says why.
This script also reads sample.png in the current folder: run it from the ocr folder.
from pathlib import Pathfrom ollama import Client
client = Client(host="http://localhost:11434")response = client.chat( model="gemma4", messages=[ { "role": "user", "content": ( "Extrait UNIQUEMENT le texte visible dans cette image. " "Ne décris rien d'autre. Conserve la mise en forme (sauts de ligne, listes)." ), "images": [str(Path("sample.png"))], } ], options={"temperature": 0},)print(response["message"]["content"])Run from the ocr folder, where sample.png lives, the script gives the following output with gemma4. The image contains the text "Lab Ollama Vision 2026 - dev H100":
Lab Ollama Vision 2026 - dev H100The model returns the image's text with no added description. The prompt's "UNIQUEMENT le texte" ("ONLY the text") instruction matters: without it, a chat model tends to describe the image as well.
Pitfalls and limits in 2026
Questions that require several reasoning steps about an image pose a second problem. For example: "how many steps are there in this diagram, and what is the dependency between the 3rd and the 5th?". A local model of a few billion parameters has less room than a very large hosted model on this kind of question. Test it on your own cases; if the answers are wrong too often, a cloud service remains the fallback option, at the price of sending your images.
Use case: analyzing product photos
Here is the skeleton of a cataloging loop that combines vision and structured outputs. It cannot run as is: dossier_photos (a Path), client, db and the contents of messages are yours to provide, following the model of the 02-structured.py script.
class FicheProduit(BaseModel): categorie: Literal["vetement", "electronique", "alimentation", "mobilier", "autre"] couleurs: list[str] materiaux_apparents: list[str] style: Literal["moderne", "vintage", "industriel", "minimaliste", "classique"] qualite_photo: Literal["pro", "amateur", "floue"] description_seo: str = Field(description="Description optimisée SEO en 150 caractères.")
# Boucle sur 1000 photos produitsfor photo in dossier_photos.glob("*.jpg"): response = client.chat(model="gemma4", messages=[...], format=FicheProduit.model_json_schema(), ...) fiche = FicheProduit.model_validate_json(response["message"]["content"]) db.insert(fiche.model_dump())Everything stays local and no paid API is involved. The time per photo depends on your machine: time the loop on a few photos (time.perf_counter()) before running it on the whole catalog.
Key takeaways
- This guide uses Gemma 4, whose Ollama page declares the vision and tools capabilities; the
latesttag weighs 6.6 GB, the other variants 4.6 to 19 GB. - Llama 3.2 Vision 11B is a 7.8 GB alternative whose page declares English only for requests with an image.
- The file size is not the memory used: read the
SIZEcolumn ofollama psafter a call. - An image plus a Pydantic schema gives typed JSON, with no cloud. The schema guarantees the shape of the response, not the correctness of its content.
- A vision model reads a line of clean text well, but for critical OCR, prefer Tesseract or docTR.
- Resizing images before the call can reduce time and memory; time it on your machine.
Knowledge check
Check that you have the essentials of this guide. The questions cover only what was just explained here.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
A new full quiz with random questions
Work again only on the questions you missed
FAQ
gemma4 by default (4.6 to 19 GB depending on the tag), 7.8 GB for llama3.2-vision. The memory used is larger, because of the context. With 8 GB, the gemma4:e2b tag (4.6 GB) leaves some headroom. For gemma4 or llama3.2-vision, run a call then read the SIZE and PROCESSOR columns of ollama ps: a CPU/GPU split signals a model that overflows the GPU's memory, and is therefore slower.images is a list. The useful number of images depends on the model and the memory: try with two or three images and compare the quality of the answers on your own cases.import base64
img_b64 = base64.b64encode(file_bytes).decode()
client.chat(model="gemma4", messages=[{"role": "user", "content": "...", "images": [img_b64]}])
temperature reduces variation from one run to the next, without removing it.