You already know how to use Ollama from the command line (installation and first steps). This guide teaches you to call it from Python. You first check the local REST API, then compare the native SDK, LiteLLM and the OpenAI client pointed at Ollama. Next you get typed JSON with structured outputs, you compute embeddings for semantic search, and you assemble everything into a DevOps assistant that runs locally. The native SDK has one advantage: it gives direct access to Ollama's own options.
What you will learn
- REST API, native SDK, LiteLLM and OpenAI client: when to use each approach
- Streaming: display responses in real time
- Structured output: get typed JSON with Pydantic
- Embeddings: generate vectors for semantic search
- Multi-turn conversation: keep context between exchanges
- Running project: build DevOps Buddy, a local DevOps assistant
Prerequisites
Before you start, make sure you have:
- Ollama installed (installation guide)
- Python 3.10 or newer: script 07 uses the
str | Nonenotation, which does not exist before that version - Two models downloaded:
llama3.2:3bfor chat, andnomic-embed-text, a model dedicated to embeddings, required by script 06:
ollama pull llama3.2:3bollama pull nomic-embed-text- A dedicated Python environment, with the SDK in pinned versions so the
examples behave the same on your machine. The
~/lab-ollamadirectory will hold all the scripts of this guide:
mkdir -p ~/lab-ollama && cd ~/lab-ollamapython3 -m venv .venv && source .venv/bin/activatepip install ollama==0.6.3 pydantic==2.14.3Check the versions:
ollama --version # 0.40.x (replayed version: 0.40.16)pip show ollama # 0.6.3, the version pinned aboveCheck that the REST API responds
When Ollama is running, it exposes a REST API on port 11434 of your machine. Any program can then talk to the model: the Python SDK, LiteLLM and the OpenAI client seen below are only layers on top of this API. Before testing a script, check that the server responds:
curl http://localhost:11434/api/tagsThe output is a JSON object whose models key lists the downloaded models,
with their name and size. If the command fails with
Connection refused, the server is not started: see
Ollama: troubleshooting and configuration.
Native SDK vs LiteLLM: what is the difference?
Both libraries talk to the same Ollama server: the question is not
which one "works", but which one locks you in. The native SDK exposes Ollama's
API as it is, including the parameters that have no equivalent at
other providers. LiteLLM normalizes calls to the OpenAI format,
which makes the code portable at the price of the lowest common denominator. In
the table below, structured outputs exist on both sides: the native
SDK requests them with format=, LiteLLM with response_format (a JSON
schema or a Pydantic class, according to the
LiteLLM documentation for Ollama).
The deciding criterion is therefore portability versus direct access to
Ollama's own options, such as think=True or options.
| Aspect | ollama SDK | LiteLLM |
|---|---|---|
| Installation | pip install ollama | pip install litellm |
| Scope | Ollama only | 100+ providers (OpenAI, Anthropic...) |
| Structured output | Yes, native with format= | Yes, via response_format (JSON schema) |
| Embeddings | Yes, built-in embed() | Yes, embedding() |
| Thinking | Yes, think=True | Not documented for Ollama |
| Use case | Scripts dedicated to Ollama | Portable multi-provider code |
In practice, take the native SDK if you only target Ollama and
want to use its specific features. Take LiteLLM if you
want to be able to switch to other providers. A third case exists: code
that already uses the openai library, covered just below.
The OpenAI client: reuse existing code
Ollama also exposes an OpenAI-compatible endpoint under the /v1/ path
(chat completions, embeddings, model list). If your code already calls
the openai library, there is no need to rewrite it: you only have to change the
base URL to point at your local server. The client requires an API key,
but Ollama ignores it: any string will do, here ollama.
pip install openai==3.49.0from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")response = client.chat.completions.create( model="llama3.2:3b", messages=[{"role": "user", "content": "Explique une API REST en 3 phrases simples"}],)print(response.choices[0].message.content)The prompt is in French: it asks the model to explain a REST API in 3 simple sentences, so the model answers in French.
The syntax also differs from the native SDK. The response follows the OpenAI
format (response.choices[0].message.content), whereas the native SDK returns
response.message.content directly, as in the scripts of this lesson.
The other Ollama lessons on the site write response["message"]["content"]: the
two forms are equivalent, because the SDK returns an object that can be accessed
by attribute as well as by key.
Ollama's own options, such as think=True or options, remain the
territory of the native SDK. So choose the OpenAI client to plug Ollama
into code already written for an OpenAI service, and the native SDK for every
new script dedicated to Ollama.
Running project: DevOps Buddy
Throughout this guide, you build DevOps Buddy, a local DevOps assistant that:
- Answers DevOps questions in streaming
- Uses a knowledge base with embeddings (mini-RAG)
- Returns structured commands with Pydantic
The numbered scripts (01_... to 07_...) all go in the
~/lab-ollama directory created in the prerequisites, with the virtual environment activated. If
you closed your terminal in the meantime, reactivate it with
cd ~/lab-ollama && source .venv/bin/activate.
Basic chat with the native SDK
The SDK's chat() function is the equivalent of litellm.completion(), but
with a slightly different API.
Script 01: First call
This first script is mainly meant to show you the shape of the response object.
Look at the metadata part (headed "Métadonnées" in the output) rather than the generated text: prompt_eval_count
and eval_count count the tokens (the pieces of words the model reads
and produces) in input and output, and total_duration is expressed in
nanoseconds, hence the division by 1e9. These three values let you
measure the real cost of a local call, since no billing comes to remind
you of it.
The scripts of this lesson are kept as they were tested, with their prompts, comments and messages in French: the outputs shown below match them exactly, and the model answers in French. Here, "Réponse du modèle" simply means "the model's answer".
#!/usr/bin/env python3"""TP 01 : Appel basique avec le SDK Ollama natifObjectif : Comprendre la différence avec litellm"""
from ollama import chat
# Appel simple avec le SDK natifresponse = chat( model='llama3.2:3b', messages=[ {'role': 'user', 'content': 'Explique Docker en une phrase pour un débutant.'} ])
print("=== Réponse du modèle ===")print(response.message.content)
print("\n=== Métadonnées ===")print(f"Modèle utilisé : {response.model}")print(f"Tokens prompt : {response.prompt_eval_count}")print(f"Tokens réponse : {response.eval_count}")print(f"Durée totale : {response.total_duration / 1e9:.2f}s")Run the script:
python 01_chat_basique.pyExpected output:
=== Réponse du modèle ===Docker est une technologie qui permet de containeriser les applications, ce quisignifie que chaque application est placée dans son propre environnement isolé etsécurisé, avec ses propres dépendances et ressources, ce qui facilite ladéploiement, la gestion et la sécurité des applications.
=== Métadonnées ===Modèle utilisé : llama3.2:3bTokens prompt : 36Tokens réponse : 65Durée totale : 4.51sThe text and the duration vary from one run to the next, and this output comes from a processor only (Ollama 0.40.16).
Streaming: real-time responses
Streaming displays the response word by word, as it is generated. On a local model, generating a response takes several seconds: without streaming, the screen stays empty until the last line.
Script 02: Streaming
With stream=True, chat() no longer returns an object but a generator:
the for loop consumes the fragments at the pace of generation. Two
practical consequences. The closing metadata (eval_count, total_duration)
only arrives in the last fragment, not at the start. And if you need
the full text afterwards, you have to concatenate it yourself during the loop,
as DevOps Buddy does further down.
#!/usr/bin/env python3"""TP 02 : Streaming avec le SDK OllamaObjectif : Afficher la réponse en temps réel, mot par mot"""
from ollama import chat
# Streaming : la réponse arrive progressivementprint("=== Réponse en streaming ===")stream = chat( model='llama3.2:3b', messages=[ {'role': 'user', 'content': 'Donne 5 bonnes pratiques Dockerfile, très bref.'} ], stream=True # Active le streaming)
# Afficher chaque chunk au fur et à mesurefor chunk in stream: print(chunk.message.content, end='', flush=True)
print("\n\n=== Fin du streaming ===")The flush=True parameter forces each fragment to be displayed immediately, without
waiting for the end of the line.
Structured output with Pydantic
This feature saves you from writing a parser for free text. Instead of a paragraph, you get typed JSON that matches a Pydantic schema. Pydantic is a Python library that describes a data structure with a class, and checks that a piece of data conforms to it.
Why does it matter?
When you ask "answer in JSON" without a schema, the model often returns
clean JSON, but not always: it sometimes wraps it in a Markdown code block
(three backticks), and json.loads then rejects it. A single response of this
kind is enough to break a script in production. With format=,
Ollama constrains decoding token by token: the output is forced to
follow the schema, as long as generation runs to the end (script 04 explains
the opposite case). The four benefits below follow from this constraint
on the server side, not from an instruction in the prompt.
- Reliable parsing: no more need for regexes or manual parsing
- Automatic validation: Pydantic checks that the response conforms
- Typing: your IDE knows the field types
- Integration: the data is directly usable in your code
Script 03: Basic structured output
Two calls do all the work, and they must not be confused.
DockerCommand.model_json_schema() converts the Pydantic class into a JSON schema
sent to the server, which uses it to constrain generation.
DockerCommand.model_validate_json() reads the response back and instantiates the Python object.
The schema constrains the shape, not the content: the level field will indeed be
a string, but nothing prevents the model from writing intermédiaire instead of the
three expected values. To lock the values too, declare a
Literal["debutant", "intermediaire", "avance"] rather than a str.
#!/usr/bin/env python3"""TP 03 : Structured Output avec PydanticObjectif : Obtenir des réponses JSON typées et validées automatiquementFeature : Disponible depuis Ollama 0.5.0+"""
from ollama import chatfrom pydantic import BaseModel
# Définir le schéma de sortie avec Pydanticclass DockerCommand(BaseModel): """Une commande Docker expliquée""" command: str # La commande complète description: str # Explication en une phrase example: str # Un exemple concret level: str # "debutant", "intermediaire", "avance"
# Appeler le modèle avec le schémaresponse = chat( model='llama3.2:3b', messages=[ { 'role': 'user', 'content': 'Explique la commande docker build' } ], format=DockerCommand.model_json_schema() # Forcer le format JSON)
# Valider et parser la réponse avec Pydanticresult = DockerCommand.model_validate_json(response.message.content)
# Afficher le résultat structuréprint("=== Sortie structurée (Pydantic) ===")print(f"Commande : {result.command}")print(f"Description : {result.description}")print(f"Exemple : {result.example}")print(f"Niveau : {result.level}")
print("\n=== JSON brut ===")print(response.message.content)Expected output:
=== Sortie structurée (Pydantic) ===Commande : docker buildDescription : Construction d'un conteneur Docker à partir d'un fichier DockerfileExemple : docker build -t mon-image:latest .Niveau : utilise pour construire un conteneur Docker à partir d'un fichier Dockerfile dans le répertoire courant.
=== JSON brut ==={"command": "docker build", "description": "Construction d'un conteneur Docker à partir d'un fichier Dockerfile", "example": "docker build -t mon-image:latest .", "level": "utilise pour construire un conteneur Docker à partir d'un fichier Dockerfile dans le répertoire courant."}The generated text changes with every run. Look at the "Niveau" line (the label
of the level field): level is a plain str, so nothing forces the model to answer
"debutant". Here, it wrote a whole sentence in place of the level (Ollama
0.40.16), and depending on the run Administrateur, avancé or Intermediate.
To constrain the values, use an enumeration (the lesson on structured
outputs, cited after script 04, covers it in detail).
Script 04: Lists and nested objects
A Pydantic schema can contain other schemas: Tutorial references a
list of Step, and Pydantic generates a nested JSON schema that Ollama can
follow. Two options deserve attention in the call. temperature: 0 makes the
output almost deterministic, which matters when you extract data rather than
write prose. num_predict: 500 caps the number of generated tokens: a nested structure
cut off mid-way produces invalid JSON and makes
validation fail. The lower the cap, the more this risk grows with a nested
schema.
#!/usr/bin/env python3"""TP 04 : Structured Output avancé (listes et objets imbriqués)Objectif : Extraire plusieurs éléments structurés en une seule requête"""
from ollama import chatfrom pydantic import BaseModel
class Step(BaseModel): """Une étape d'un processus""" number: int action: str command: str
class Tutorial(BaseModel): """Un tutoriel complet""" title: str steps: list[Step] total_time: str
print("Définition du schéma Pydantic...")
print("\nEnvoi de la requête à Ollama (llama3.2:3b)...")
# Demander un tutoriel structuréresponse = chat( model='llama3.2:3b', messages=[ { 'role': 'user', 'content': 'Donne un tutoriel Docker en 2 étapes avec titre, steps (number, action, command) et total_time' } ], format=Tutorial.model_json_schema(), options={'temperature': 0, 'num_predict': 500})
print("Réponse reçue!")print(f" Durée: {response.total_duration / 1e9:.2f}s")
# Parser la réponsetutorial = Tutorial.model_validate_json(response.message.content)
# Afficher le tutoriel formatéprint(f"\nTutoriel : {tutorial.title}")print(f"Durée : {tutorial.total_time}")for step in tutorial.steps: print(f" {step.number}. {step.action} → {step.command}")Expected output:
Définition du schéma Pydantic...
Envoi de la requête à Ollama (llama3.2:3b)...Réponse reçue! Durée: 12.68s
Tutoriel : Tutoriel Docker en 2 étapesDurée : } 1. Créer un nouveau projet Docker → docker run -it ubuntu /bin/bash 2. Créer un fichier Dockerfile et l'exécuter → docker build -t mon-image .As with script 03, the content varies from one run to the next. With a
3-billion-parameter model, the duration field comes back here with an
unexpected value (}, observed on Ollama 0.40.16), while the structure
remains valid.
To go further on schemas (hand-written JSON Schema,
enumerations, limits of the guarantee), the lesson Structured outputs with
Ollama covers
the format= mechanism in detail, with recent models tested.
Multi-turn conversation with history
To keep context between several exchanges, you have to manage the message history.
Script 05: Conversation
The Ollama server remembers nothing between two calls: the conversation's
memory is your list messages. At each turn, you send back the whole
history, questions and answers included. The cost therefore grows with each
exchange, which you can see in prompt_eval_count, until it saturates the
model's context window (the maximum amount of text it reads at once).
For an assistant meant to last, plan a truncation of the history or
a summary of old turns; the script below, limited to four exchanges,
does not need one.
#!/usr/bin/env python3"""TP 05 : Conversation multi-turn avec historiqueObjectif : Maintenir le contexte entre plusieurs échanges"""
from ollama import chat
# Historique de la conversationmessages = []
# Contexte systèmesystem_message = { 'role': 'system', 'content': '''Tu es DevOps Buddy, un assistant spécialisé DevOps.Tu réponds de manière concise et technique.Tu donnes des exemples de commandes quand c'est pertinent.'''}messages.append(system_message)
print("DevOps Buddy initialisé")print("=" * 50)
def ask(question: str) -> str: """Pose une question et met à jour l'historique""" print(f"\nVous : {question}")
# Ajouter la question à l'historique messages.append({'role': 'user', 'content': question})
# Appeler le modèle avec tout l'historique response = chat( model='llama3.2:3b', messages=messages, options={'temperature': 0.3} )
# Ajouter la réponse à l'historique assistant_message = response.message.content messages.append({'role': 'assistant', 'content': assistant_message})
print(f"Buddy : {assistant_message}") return assistant_message
# Simulation d'une conversationask("C'est quoi un Pod Kubernetes en une phrase ?")ask("Et un Deployment, c'est quoi par rapport à un Pod ?")ask("Donne-moi la commande pour créer un Deployment avec 3 réplicas")ask("Comment je scale à 5 réplicas maintenant ?")
print("\n" + "=" * 50)print(f"Total messages dans l'historique : {len(messages)}")Real output (Ollama 0.40.16, ollama SDK 0.6.3, llama3.2:3b on processor only; the response varies from one run to the next):
DevOps Buddy initialisé==================================================
Vous : C'est quoi un Pod Kubernetes en une phrase ?Buddy : Un Pod Kubernetes est une unité de déployement et de gestion de conteneurs, qui est une combinaison logique de l'ensemble d'un conteneur et de ses dépendances, permettant une gestion plus fine-graine des ressources et des communications entre les conteneurs.
Vous : Et un Deployment, c'est quoi par rapport à un Pod ?Buddy : Un Deployment Kubernetes est une ressource qui définit la stratégie de déploiement d'un ou plusieurs Pods, en spécifiant les conditions de déploiement, les versions des images, les stratégies d'actualisation et les politiques de gestion des ressources.
Vous : Donne-moi la commande pour créer un Deployment avec 3 réplicasBuddy : La commande pour créer un Deployment avec 3 réplicas est la suivante :
kubectl apply -f deployment.yaml
Et dans votre fichier deployment.yaml (extrait) :
metadata: name: mon-deploymentspec: replicas: 3
Notez que vous devez remplacer mon-deployment, mon-app, mon-image par vos propres valeurs.
Vous : Comment je scale à 5 réplicas maintenant ?Buddy : Pour scaler un Deployment à 5 réplicas, vous pouvez utiliser la commande suivante :
kubectl scale deployment mon-deployment --replicas=5
Cette commande utilise la commande scale pour ajuster le nombre de réplicas du Deployment mon-deployment à 5.
==================================================Total messages dans l'historique : 9The output is abridged: the full YAML manifest of the third answer and the model's code fences are not reproduced.
Look at the last question, "Comment je scale à 5 réplicas maintenant ?" (how do I scale to 5 replicas now?). It names no Deployment, and yet the answer targets mon-deployment. The model made up that name itself in its previous answer: it picks it up because the messages list is sent back to it on every call. This is the proof that the context holds.
A 3-billion-parameter model sometimes writes approximate French, such as "déployement" and "fine-graine" in the first answer. This flaw comes from the size of the model, not from your script.
Embeddings for semantic search
Embeddings turn text into numerical vectors. Two similar texts will have close vectors. This is the basis of RAG (Retrieval-Augmented Generation): retrieve the relevant documents, then give them to the model before it answers.
Script 06: Embeddings and similarity
The embed() function always returns a list of vectors, even for
a single input string, hence the repeated response.embeddings[0] in the script.
Cosine similarity measures the angle between two vectors, not their
distance: it equals 1 for two vectors pointing the same way. The scores of the
script nevertheless all stay between 0.55 and 0.66, even for documents that
answer the question poorly. So never compare a score in absolute terms:
only the relative ranking between documents is meaningful, and it depends
entirely on the model used.
#!/usr/bin/env python3"""TP 06 : Embeddings avec OllamaObjectif : Générer des vecteurs pour la recherche sémantiqueFeature : endpoint /api/embed disponible depuis Ollama 0.3.4"""
from ollama import embedimport math
def cosine_similarity(vec1: list[float], vec2: list[float]) -> float: """Calcule la similarité cosinus entre deux vecteurs""" dot_product = sum(a * b for a, b in zip(vec1, vec2)) norm1 = math.sqrt(sum(a ** 2 for a in vec1)) norm2 = math.sqrt(sum(b ** 2 for b in vec2)) return dot_product / (norm1 * norm2)
# Base de connaissances DevOpsdocuments = [ "Docker est une plateforme de conteneurisation qui permet de packager des applications", "Kubernetes orchestre les conteneurs à grande échelle dans un cluster", "Ansible automatise la configuration des serveurs et le déploiement", "Terraform gère l'infrastructure as code pour le cloud", "GitLab CI/CD permet l'intégration et le déploiement continus"]
print("Génération des embeddings pour 5 documents...")
# Générer les embeddings pour chaque documentdoc_embeddings = []for i, doc in enumerate(documents): response = embed(model='nomic-embed-text', input=doc) doc_embeddings.append(response.embeddings[0]) print(f" Doc {i+1} : {len(response.embeddings[0])} dimensions")
print(f"\nDimension des vecteurs : {len(doc_embeddings[0])}")
# Question de testquery = "Comment déployer des conteneurs en production ?"print(f"\nRequête : {query}")
# Générer l'embedding de la requêtequery_response = embed(model='nomic-embed-text', input=query)query_embedding = query_response.embeddings[0]
# Calculer la similarité avec chaque documentprint("\nScores de similarité :")scores = []for i, doc_emb in enumerate(doc_embeddings): score = cosine_similarity(query_embedding, doc_emb) scores.append((score, documents[i])) print(f" {score:.4f} → {documents[i][:50]}...")
# Trier par score décroissantscores.sort(key=lambda s: s[0], reverse=True)print(f"\nDocument le plus pertinent :")print(f" Score: {scores[0][0]:.4f}")print(f" {scores[0][1]}")Expected output:
Génération des embeddings pour 5 documents... Doc 1 : 768 dimensions Doc 2 : 768 dimensions Doc 3 : 768 dimensions Doc 4 : 768 dimensions Doc 5 : 768 dimensions
Dimension des vecteurs : 768
Requête : Comment déployer des conteneurs en production ?
Scores de similarité : 0.6602 → Docker est une plateforme de conteneurisation qui ... 0.6462 → Kubernetes orchestre les conteneurs à grande échel... 0.5821 → Ansible automatise la configuration des serveurs e... 0.5903 → Terraform gère l'infrastructure as code pour le cl... 0.5528 → GitLab CI/CD permet l'intégration et le déploiemen...
Document le plus pertinent : Score: 0.6602 Docker est une plateforme de conteneurisation qui permet de packager des applicationsThe ranking is consistent: for the question "Comment déployer des conteneurs en production" (how to deploy containers in production), Docker and Kubernetes come first, ahead of Terraform, Ansible and GitLab CI/CD. Your scores may differ by a few hundredths depending on the model version and the processor, but Docker and Kubernetes must stay on top. This is what a dedicated embedding model brings: its vectors are trained so that texts with a close meaning end up as neighbors.
A chat model such as llama3.2:3b is not made for this. On
Ollama 0.40.16, embed(model='llama3.2:3b', ...) fails with HTTP error 501
"This server does not support embeddings. Start it with --embeddings". If you
run into it, replace the model with nomic-embed-text.
The lesson Embeddings and similarity (in French) covers these notions (dimension, cosine similarity, model choice) outside any assistant.
DevOps Buddy: the complete assistant
Let us combine all the features into an interactive DevOps assistant.
Script 07: DevOps Buddy
The script is long, read it by blocks rather than line by line. Three functions
carry the logic. find_relevant_context() performs the retrieval: it
compares the embedding of the question with those of the knowledge base and returns the two
closest documents. chat_with_rag() performs the augmentation: it
pastes that context in front of the question before sending it to the model.
get_structured_command() serves the /cmd mode with a structured output. The
rest is terminal dressing.
Three limits set this prototype apart from a real RAG system. The embeddings are
recomputed at every startup and live in memory: no persistence, no
vector database (storage designed to search for close vectors). The
search is linear: it compares the question with every document. On
six entries that is instant, but on thousands of documents you need a dedicated
index. Finally, the embedding model (the EMBED_MODEL constant) is distinct from the
chat model (MODEL), as in script 06: if you have not pulled nomic-embed-text,
startup fails.
#!/usr/bin/env python3"""TP 07 : DevOps Buddy - Assistant local completObjectif : Combiner toutes les fonctionnalités Ollama dans un assistant interactifFeatures utilisées : chat, streaming, structured output, embeddings"""
from ollama import chat, embedfrom pydantic import BaseModelimport math
# === Configuration ===MODEL = 'llama3.2:3b'EMBED_MODEL = 'nomic-embed-text' # modèle dédié aux embeddings (ollama pull nomic-embed-text)SYSTEM_PROMPT = '''Tu es DevOps Buddy, un assistant expert DevOps/DevSecOps.Tu réponds en français de manière concise et technique.Tu donnes des exemples de commandes quand c'est pertinent.Tu structures tes réponses avec des listes à puces.'''
# === Base de connaissances embarquée ===KNOWLEDGE_BASE = [ {"topic": "docker", "content": "Docker utilise des conteneurs pour isoler les applications. Commandes: docker build, docker run, docker compose"}, {"topic": "kubernetes", "content": "Kubernetes orchestre les conteneurs. Concepts: Pod, Deployment, Service, Ingress. Commandes: kubectl get, kubectl apply"}, {"topic": "ansible", "content": "Ansible automatise la configuration. Concepts: playbook, inventory, role, task. Commande: ansible-playbook"}, {"topic": "terraform", "content": "Terraform gère l'infrastructure as code. Concepts: provider, resource, module, state. Commandes: terraform init, plan, apply"}, {"topic": "gitlab", "content": "GitLab CI/CD pipeline avec .gitlab-ci.yml. Concepts: stages, jobs, artifacts, cache, runners"}, {"topic": "prometheus", "content": "Prometheus collecte les métriques. Concepts: scraping, PromQL, alertmanager, exporters"},]
# Pré-calculer les embeddings de la base de connaissancesprint("Chargement de la base de connaissances...")knowledge_embeddings = []for item in KNOWLEDGE_BASE: response = embed(model=EMBED_MODEL, input=item["content"]) knowledge_embeddings.append(response.embeddings[0])print(f"{len(KNOWLEDGE_BASE)} documents indexés\n")
# === Fonctions utilitaires ===def cosine_similarity(vec1: list[float], vec2: list[float]) -> float: dot = sum(a * b for a, b in zip(vec1, vec2)) norm1 = math.sqrt(sum(a ** 2 for a in vec1)) norm2 = math.sqrt(sum(b ** 2 for b in vec2)) return dot / (norm1 * norm2) if norm1 and norm2 else 0
def find_relevant_context(query: str, top_k: int = 2) -> str: """Recherche les documents pertinents pour enrichir le contexte""" query_emb = embed(model=EMBED_MODEL, input=query).embeddings[0]
scores = [] for i, doc_emb in enumerate(knowledge_embeddings): score = cosine_similarity(query_emb, doc_emb) scores.append((score, KNOWLEDGE_BASE[i]))
scores.sort(key=lambda s: s[0], reverse=True) top_docs = scores[:top_k]
context = "\n".join([f"- {doc['content']}" for _, doc in top_docs]) return context
# === Schéma pour les commandes structurées ===class CommandSuggestion(BaseModel): command: str description: str warning: str | None = None
def get_structured_command(query: str) -> CommandSuggestion | None: """Obtenir une suggestion de commande structurée""" try: response = chat( model=MODEL, messages=[ {'role': 'system', 'content': 'Tu suggères des commandes DevOps. Réponds uniquement en JSON.'}, {'role': 'user', 'content': f'Suggère une commande pour: {query}'} ], format=CommandSuggestion.model_json_schema(), options={'temperature': 0, 'num_predict': 200} ) return CommandSuggestion.model_validate_json(response.message.content) except Exception as e: print(f"Erreur structured output: {e}") return None
# === Chat avec RAG ===def chat_with_rag(question: str, history: list) -> str: """Répond à une question avec contexte enrichi""" # Trouver le contexte pertinent context = find_relevant_context(question)
# Construire le prompt enrichi augmented_question = f"""Contexte (base de connaissances):{context}
Question de l'utilisateur: {question}"""
# Ajouter au historique history.append({'role': 'user', 'content': augmented_question})
# Appeler le modèle avec streaming response_text = "" stream = chat( model=MODEL, messages=history, stream=True, options={'temperature': 0.3} )
for chunk in stream: content = chunk.message.content print(content, end='', flush=True) response_text += content
print() # Nouvelle ligne après le streaming
# Ajouter la réponse à l'historique history.append({'role': 'assistant', 'content': response_text})
return response_text
# === Main ===def main(): print("=" * 60) print("DevOps Buddy - Assistant DevOps Local (Ollama)") print("=" * 60) print(f"Modèle: {MODEL}") print("Commandes spéciales:") print(" /cmd <action> : Obtenir une commande structurée") print(" /quit : Quitter") print("-" * 60)
history = [{'role': 'system', 'content': SYSTEM_PROMPT}]
while True: try: user_input = input("\nVous: ").strip()
if not user_input: continue
if user_input.lower() == '/quit': print("À bientôt!") break
if user_input.startswith('/cmd '): # Mode commande structurée query = user_input[5:] print("\nRecherche de commande...") result = get_structured_command(query) if result: print(f"\nCommande: {result.command}") print(f"Description: {result.description}") if result.warning: print(f"Attention: {result.warning}") else: print("Impossible de générer la commande") else: # Mode conversation avec RAG print("\nDevOps Buddy: ", end='') chat_with_rag(user_input, history)
except EOFError: print("\nÀ bientôt!") break except KeyboardInterrupt: print("\nInterruption - À bientôt!") break
if __name__ == "__main__": main()Using DevOps Buddy
Startup takes a few seconds, the time to compute the six embeddings of
the base. The assistant then offers two modes that are worth comparing:
a free question goes through RAG with streaming, while the /cmd prefix
switches to the structured output and displays a command, its
description and an optional warning. The difference can be felt in use: the
conversation mode displays the first words almost immediately, whereas the
/cmd mode stays silent until the full JSON has been generated and validated,
since a partial JSON would be unusable.
python 07_devops_buddy.pyExample session:
Chargement de la base de connaissances...6 documents indexés
============================================================DevOps Buddy - Assistant DevOps Local (Ollama)============================================================Modèle: llama3.2:3bCommandes spéciales: /cmd <action> : Obtenir une commande structurée /quit : Quitter------------------------------------------------------------
Vous: C'est quoi un Pod ?
DevOps Buddy: **Définition d'un Pod**
Un Pod est la principale unité de déployement et de gestion des conteneurs dans Kubernetes. [...]
Vous: /cmd lister les pods kubernetes
Recherche de commande...
Commande: kubectlDescription: Lister les pods KubernetesAttention: Assurez-vous d'être connecté à votre cluster Kubernetes pour exécuter cette commandeThe session above is real output from Ollama 0.40.16, abridged at the spot
marked [...]. It shows the limit of a 3-billion-parameter model:
the returned command is plain kubectl, without get pods, even though the JSON
schema is perfectly respected. The schema constrains the shape, never the
correctness of the content. It does not protect against a cut-off either: if
num_predict (200 tokens here) interrupts generation, the JSON is incomplete and
model_validate_json raises an error, which is why get_structured_command()
catches it and returns None.
Checklist
Go through this list before moving on: each script isolates one capability of the
SDK, and DevOps Buddy brings them together. If 03_structured_output.py fails, there is no point
launching DevOps Buddy, which uses the same mechanism for its /cmd mode.
Installation
These four checks rule out installation failures before you touch the code. The first two read the version of the server and that of the SDK, which evolve separately. The third checks Pydantic, the fourth that both models are downloaded.
- Server reachable (
curl http://localhost:11434/api/tags) and version read (ollama --version) - ollama SDK installed (
pip show ollama) - Pydantic installed (
pip show pydantic) - Models downloaded:
llama3.2:3bandnomic-embed-textboth appear inollama list
Tested scripts
Run them in order. If 06_embeddings.py or 07_devops_buddy.py fails
with model not found, run ollama pull nomic-embed-text. A 501 error
means a chat model was passed to embed(). Close scores (around 0.55 to 0.66) are normal output: the good sign is Docker and
Kubernetes at the top of the ranking.
-
01_chat_basique.py: response + metadata -
02_streaming.py: real-time display -
03_structured_output.py: JSON validated by Pydantic -
04_structured_avance.py: lists of objects -
05_conversation.py: multi-turn history -
06_embeddings.py: cosine similarity -
07_devops_buddy.py: complete assistant
Key takeaways
-
Native SDK or LiteLLM. Both produce structured JSON (
format=on the SDK side,response_formaton the LiteLLM side). The ollama SDK adds direct access to Ollama's own options (think=True,options), but it only talks to Ollama. -
Structured output. Pass
Model.model_json_schema()toformat=to get JSON that conforms to the schema. The shape is constrained, not the content, and a response cut off bynum_predictremains invalid. -
Streaming. With
stream=True, the response arrives in fragments, and the metadata in the last one. -
Embeddings.
embed()returns vectors, to be computed with a dedicated model such asnomic-embed-text, never with a chat model. -
History. The server retains nothing: you send back the complete list of messages on every call.
-
Options.
optionssets the temperature, the maximum number of tokens, etc. -
Local RAG. Embeddings find the close documents, which you paste in front of the question before sending it to the model.
Knowledge check
Check that you have the essentials of this guide. The questions cover only what was just explained here.
Knowledge check
Check what you have learned with this interactive quiz
How it works
- The timer starts when you click Start
- Multiple choice, true/false and short answer questions
- You can move between questions
- Detailed results are shown at the end
Starts the quiz and the timer
Review
(0/0)Score by domain
A domain groups several guides and gets several questions, so its score is more reliable than a per-topic score, which often rests on a single question.
Summary
Strengths
No topic passed at 80% on at least two questions yet.
Weak areas
No topic below the pass mark.
Recommended next
Hints before you try again?
A new full quiz with random questions
Work again only on the questions you missed