Skip to content
Français
Account: not signed in, sync available My learning
Select theme
Français

CO-RE and BTF: making an eBPF program portable

By updated

Improve this page

CO-RE lets you compile an eBPF program once and load it on different kernels, without recompiling it on each machine. It is what moved eBPF from fragile tinkering to a production tool. This page explains the problem that CO-RE solves, the role of BTF (the kernel's type information that makes it all possible), and shows it on a real machine: BTF present, kernel types exposed, a generated vmlinux.h. This lesson is optional for an administrator: it mostly serves those who develop and distribute eBPF tools, but it explains why your kernel exposes /sys/kernel/btf/vmlinux. Outputs captured on an Ubuntu 24.04 VM (kernel 6.8).

What you will learn

  • Understand why an eBPF program used to break when changing kernel
  • Define BTF and its role in portability
  • Explain the principle of CO-RE: compile once, adapt at load time
  • Check that your kernel exposes the BTF it needs
  • Place CO-RE against the old recompilation approach

Why did an eBPF program break between two kernels?

An eBPF program often reads kernel data structures: a task, an open file, a network packet. Yet the layout of these structures in memory, the offset (the position, in bytes, of each field), changes from one kernel version to another. A field located at byte 40 on one kernel may be at byte 48 on the next.

A program compiled for a specific kernel therefore reads the fields in the wrong places on another kernel: at best absurd values, at worst a loading rejection. The program is glued to its compilation kernel.

The old answer, that of BCC, was to recompile on each machine, against the local kernel headers. It works, but it requires a compiler and the kernel headers on all targets, makes images heavier, and costs time and memory at each start. On a fleet of servers, it is a heavy constraint.

What is BTF?

The solution rests on BTF (BPF Type Format), a compact description of all the kernel types: every structure, every field, every size. Recent kernels embed this map and expose it at a single point. On the test VM, it is there:

Terminal window
ls -l /sys/kernel/btf/vmlinux
-r--r--r-- 1 root root 6096599 Jul 14 11:47 /sys/kernel/btf/vmlinux

This 6 MB file describes all the types of the running kernel. It can be read with bpftool, which renders it as C:

Terminal window
sudo bpftool btf dump file /sys/kernel/btf/vmlinux format c | head -20
#ifndef __VMLINUX_H__
#define __VMLINUX_H__
...
typedef unsigned char __u8;
typedef short unsigned int __u16;
typedef unsigned int __u32;
typedef long long unsigned int __u64;

What is vmlinux.h for?

From this BTF, a single header file is generated, vmlinux.h, which contains all the type definitions of the current kernel. The eBPF program includes it, and no longer needs the kernel headers installed on the compilation machine.

Terminal window
sudo bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
wc -l vmlinux.h
150631 vmlinux.h

More than 150,000 lines describing the kernel, produced from the machine itself. It is the first building block of portability: compiling against an exact description of the kernel, without depending on header packages.

How does CO-RE compile once and adapt at load time?

BTF solves the description; CO-RE (Compile Once Run Everywhere) solves the adaptation. The idea is elegant and unfolds in three steps.

  1. At compilation, the program does not write "read the field at byte 40". Thanks to the preserve_access_index attribute (present in the generated header), it notes "read field such-field of such structure", leaving the offset to be resolved at load time. These notes are relocations.

  2. The binary is distributed as is, compiled once, carrying its relocations.

  3. At load time, on the target machine, the libbpf library reads the local kernel's BTF, finds the real offset of each requested field, and rewrites the accesses accordingly before handing the program to the verifier.

Result: the same binary reads the right fields on a 5.15 kernel as on a 6.8 kernel, because the offsets are resolved at the last moment, machine by machine. No more compiler on the targets, no more recompilation at startup.

CO-RE or BCC: which should you choose?

The table compares the two approaches on what matters in operations: where you compile, what must be installed on the target and the startup cost.

BCC (recompilation)libbpf + CO-RE
Compilationon each target machineonly once
Dependencies on the targetclang + kernel headersnone (just the kernel's BTF)
Startupslow (compiles every time)fast (binary ready)
Tool sizeheavylight
Portabilityby recompilationby relocation at load time

This does not condemn BCC: it remains excellent for learning and prototyping, because writing a program in a Python string and running it is immediate. But to distribute an eBPF tool to a fleet of machines, CO-RE is the way. The BCC and libbpf lesson compares the two in practice.

Points of attention

Three limits of CO-RE are met in practice, and each one is checked before a deployment.

  • No BTF, no CO-RE. On an old kernel without /sys/kernel/btf/vmlinux, you must fall back on recompilation, or provide an external BTF (some projects distribute BTF files for kernels that do not embed one).
  • CO-RE relocates fields, not logic. If a field disappears or changes meaning between two kernels, CO-RE does not invent it: the program must handle this variability explicitly (libbpf offers field-presence tests).
  • vmlinux.h reflects the kernel that generated it. Regenerating it on a very different kernel gives a different file; this is normal, it describes a given kernel. Portability comes from the relocation at load time, not from the vmlinux.h used at compilation.

Key takeaways

  • The offsets of kernel fields change between versions: a hard-compiled program is glued to its kernel.
  • BTF is the description of all the kernel types, exposed in /sys/kernel/btf/vmlinux on recent kernels.
  • vmlinux.h, generated from the BTF, replaces the kernel headers at compilation.
  • CO-RE compiles the program once with relocations, which libbpf resolves at load time by reading the local BTF.
  • The same binary then runs on different kernels, without a compiler or headers on the targets.
  • BCC remains ideal for learning; libbpf + CO-RE is the way to distribute a tool.

Knowledge check

Check that the essentials of this lesson are in place. The questions only cover what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions
6 min
70% to pass

How it works

  • The timer starts when you click Start
  • Multiple choice, true/false and short answer questions
  • You can move between questions
  • Detailed results are shown at the end

Starts the quiz and the timer