I'm getting started with eBPF programming with Aya. The idea behind this series of articles is to get you started too.

In the previous parts, we created a single program. In this part, we'll take a look at the full power of eBPF. We're going to create eBPF programs and make them communicate with each other. We'll use tail calls, a mechanism for chaining eBPF programs together.

FYI, this is the English version of an article originally published in French.

Tail calls: Structuring the program

What is it?

As we saw in the previous section, eBPF programs have a stack memory limit. This is limited to 512 bytes. To get around this limit, we use eBPF maps. But this may still prove insufficient as the program grows. And we could certainly reach other limits...
So, instead of creating a monolith with a single program, we're going to divide this program into several smaller programs which will communicate via jumps. This is known as tail calling or program chaining.
Here's an outline of the possible change:

As has been the case from the outset, we have a single hook (the syscall:sys_enter_execve tracepoint, for example). In the input program, we'll define different programs that will continue the work, and so on. Thanks to eBPF maps, the various small eBPF programs will be able to retrieve output values created by another program.
Apart from the interest in going beyond imposed limits and staying within the Linux kernel, there is also a software engineering interest: to structure code better, to make it clearer and more extensible. In a large project, eBPF programs could also be managed by team.

A few limitations though...

Programs in tail calls have a memory stack limit of 256 bytes, except for the hook, which always has 512 bytes:

Edit: It's a bit more complicated than that. Thanks to Dylan Reimerink for his comment. This limitation only applies when using functions in parallel.

Another more anecdotal limitation: to avoid infinite tail call loops, we're limited to 32 jumps. So you can't write more than 33 programs. That's a lot of fun 😝

What's all this magic?

You're probably wondering how it works. To activate tail calls, you need a dedicated map. It's a Program Array type.

As its name suggests, it must be filled with a list of programs. Or, to be more precise, a list of program file descriptors (fd). When an eBPF program is loaded, a file descriptor is created. This is an integer that identifies the program.

Thanks to this map, you can start another eBPF program using the helper function bpf_tail_call :

To perform a jump, all you need to know is the index of the list corresponding to the program in question.

Let's see how to do this with our guiding program:

how to create the map
how to fill this map with program file descriptors
how to perform jumps using this map

Restructuring kernel-side code

Project cloning

If you follow the articles carefully, you can use the previous code, but I advise you to clone the repo and go to the dedicated directory to be able to follow what's coming next:

git clone https://github.com/littlejo/aya-examples
cd aya-examples/tracepoint-binary-maps/
cargo run # Checking if it works

You can also do the Killercoda lab, which follows step-by-step the creation and use of tail calls :

killercoda.com

Reminder

If you jump on the bandwagon, the initial program logs all binaries running on your computer. In the previous part, we also created a filter to prevent logging when certain binaries are executed, I externalized them in a tracepoint-binary/src/constant.rs file:

pub const EXCLUDE_LIST: [&str; 2] = ["/usr/bin/ls", "/usr/bin/top"];

You can test this with the command :

RUST_LOG=info cargo run

Divide and conquer

As you can imagine, we're going to start creating additional eBPF programs in kernel space. The code will start to grow. It becomes urgent to separate the tracepoint-binary-ebpf/src/main.rs file to prevent it from growing too quickly. The structure is fairly simple:

One file per kernel program.
A main.rs file containing the inclusions of the various files.
A common.rs file containing a catch-all (maps, constants, etc.).

At present, we have a single program, and we're going to put it in a file called hook.rs because it's the program's entry point.

Common People

So we'll create a common.rs file in the same directory as main.rs. We're already going to put the maps and constants in it:

const ZEROED_ARRAY: [u8; MAX_PATH_LEN] = [0u8; MAX_PATH_LEN];

#[map]
static BUF: PerCpuArray<[u8; MAX_PATH_LEN]> = PerCpuArray::with_max_entries(1, 0);

#[map]
static EXCLUDED_CMDS: HashMap<[u8; 512], u8> = HashMap::with_max_entries(10, 0);

We need to add the library for accessing MAX_PATH_LEN :

use tracepoint_binary_common::MAX_PATH_LEN;

We also need to add the #[map] macro and the PerCpuArray and HashMap maps:

use aya_ebpf::{
    macros::map,
    maps::{HashMap, PerCpuArray},
};

On the main.rs side, I've removed the constant and the maps that I put in common.rs and added :

mod common;
use crate::common::*;

This allows the common.rs file to be included in the main code.

I put a * so as not to bother with all the inclusions.

I test a cargo run. I get quite a few errors. I look at the first one:

The maps and the constant are not accessible. This means that they need to be made public. Here's what happens to the common.rs file:

use aya_ebpf::{
    macros::map,
    maps::{HashMap, PerCpuArray},
};

use tracepoint_binary_common::MAX_PATH_LEN;

pub const ZEROED_ARRAY: [u8; MAX_PATH_LEN] = [0u8; MAX_PATH_LEN];

#[map]
pub static BUF: PerCpuArray<[u8; MAX_PATH_LEN]> = PerCpuArray::with_max_entries(1, 0);

#[map]
pub static EXCLUDED_CMDS: HashMap<[u8; 512], u8> = HashMap::with_max_entries(10, 0);

We compile :

there are a few warnings, but it's no big deal. We'll clean it up at the end.

Hook

Now we'll create a hook.rs file in the same directory as main.rs, containing only the Tracepoint program:

#[tracepoint]
pub fn tracepoint_binary(ctx: TracePointContext) -> u32 {
    match try_tracepoint_binary(ctx) {
        Ok(ret) => ret,
        Err(ret) => ret as u32,
    }
}
fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;
    let filename = unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        let filename_bytes = bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
        if EXCLUDED_CMDS.get(&*buf).is_some() {
            info!(&ctx, "No log for this Binary");
            return Ok(0);
        }
        from_utf8_unchecked(filename_bytes)
    };
    info!(
        &ctx,
        "tracepoint sys_enter_execve called. Binary: {}", filename
    );
    Ok(0)
}

In terms of libraries and constants, we now need everything in main.rs:

use aya_ebpf::{
    helpers::bpf_probe_read_user_str_bytes,
    macros::{map, tracepoint},
    maps::{HashMap, PerCpuArray},
    programs::TracePointContext,
};

use aya_log_ebpf::info;
use core::str::from_utf8_unchecked;
use tracepoint_binary_common::MAX_PATH_LEN;
use crate::common::*;

const FILENAME_OFFSET: usize = 16;

main.rs will be fairly empty, mainly to declare the hook.rs and common.rs files:

#![no_std]
#![no_main]

mod common;
mod hook;

#[cfg(not(test))]
#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop {}
}

This separates out everything that isn't useful for developers (panic, no_main and no_std) but is intended to make the program work at kernel level.
Attempt compilation:

With a little clean-up for hook.rs , we end up with :

use aya_ebpf::{
    helpers::bpf_probe_read_user_str_bytes,
    macros::tracepoint,
    programs::TracePointContext,
};

use aya_log_ebpf::info;
use core::str::from_utf8_unchecked;
use crate::common::*;
const FILENAME_OFFSET: usize = 16;

#[tracepoint]
pub fn tracepoint_binary(ctx: TracePointContext) -> u32 {
    match try_tracepoint_binary(ctx) {
        Ok(ret) => ret,
        Err(ret) => ret as u32,
    }
}

fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;
    let filename = unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        let filename_bytes = bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
        if EXCLUDED_CMDS.get(&*buf).is_some() {
            info!(&ctx, "No log for this Binary");
            return Ok(0);
        }
        from_utf8_unchecked(filename_bytes)
    };
    info!(
        &ctx,
        "tracepoint sys_enter_execve called. Binary: {}", filename
    );
    Ok(0)
}

Now that we've structured the code better. Let's start the transition to programs with tail calls.

Let's create the foundations for tail calls

What are we going to do?

In part 2, we created a program that observes the various binaries that are executed in the computer. In part 3, we added a filter so as not to see certain binaries. Everything was in the same eBPF program.
We'll separate the different parts of the code into different eBPF programs:

data retrieval (hook)
data filtering (filter)
data display (display)

red: data retrieval
purple: data filtering
green: data display

What we're going to do as a transformation :

We'll have three eBPF programs.

In the “best” case, the programs will be linked as follows: hook ➡️ filter ➡️ display
If the data is filtered, the programs will be linked as follows: hook ➡️ filter

The global eBPF program will ultimately do the same thing, but in a segmented way. If you think you're using a sledgehammer to crack a nut, you're not wrong. But this code will allow us to be more flexible for the next part of the article series.

One last detail: if you've been following along, we need to create an eBPF map for tail calls. This has not been shown in the previous diagram, for simplicity's sake. By adding this map, we have the following diagram:

In user space, the program will fill in the JUMP_TABLE map, explaining that index 0 corresponds to the filter program and index 1 corresponds to the display program.
In kernel space, we'll use a tail call function to say: “I want to go to the program with such and such an index”.

Process

In this section, we're not going to do everything at once. We're already going to create the foundations.
In kernel space, we'll create :

the map dedicated to tail calls
the eBPF programs, which will all be identical (we'll just change the names)

In user space, we'll :

retrieve the map dedicated to tail calls
load the eBPF programs we've just created
fill the tail call map with the loaded programs.

The next section will be devoted to modifying the various eBPF programs and the famous jumps between programs.
This will ensure that the code is compilable and functional at every stage.

Creating an eBPF map dedicated to tail calls

We're going to create an eBPF map of type ProgramArray.
Here's the documentation:

This must be done in kernel space, in the file containing the eBPF maps, i.e. tracepoint-binary-ebpf/src/common.rs.
We'll create it as follows:

#[map]
pub static JUMP_TABLE: ProgramArray = ProgramArray::with_max_entries(2, 0);

We have 2 inputs:

Program filter: data filtering
Program display: data display

We'll set the flags to zero so as not to get bored.

Don't forget to add the library for this type of map:

use aya_ebpf::{
    macros::map,
    maps::{HashMap, PerCpuArray, ProgramArray},
};

Check that the code compiles with cargo run.

Creating eBPF programs

To create the eBPF programs, we'll carefully copy the tracepoint-binary-ebpf/src/hook.rs file.
For the filter program, we'll name it filter.rs:

cp tracepoint-binary-ebpf/src/hook.rs tracepoint-binary-ebpf/src/filter.rs

add _filter to the functions:

sed -i 's/tracepoint_binary/tracepoint_binary_filter/' tracepoint-binary-ebpf/src/filter.rs

Similarly, the display program is named display.rs :

cp tracepoint-binary-ebpf/src/hook.rs tracepoint-binary-ebpf/src/display.rs
sed -i 's/tracepoint_binary/tracepoint_binary_display/' tracepoint-binary-ebpf/src/display.rs

As you can see, the previous restructuring is very useful.
Don't forget to add these new files to the main.rs file:

#![no_std]
#![no_main]

mod common;
mod hook;
mod filter;
mod display;

#[cfg(not(test))]
#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop {}
}

Check that all 3 eBPF programs compile with cargo run.
So we've created the map dedicated to tail calls and we have 3 identical eBPF programs. We'll modify them later, just to have functional programs with a different nomenclature.

Retrieving the eBPF map from user space

We're now going to work in user space to load the programs and fill the eBPF map we've just created. We're going to modify the tracepoint-binary/src/main.rs file.
We need to add some code before the following line:

let ctrl_c = signal::ctrl_c();

We'll take a look at the user-side documentation for the ProgramArray map:

So we need to add :

let map = ebpf.take_map("JUMP_TABLE").unwrap();
let mut tail_call_map = ProgramArray::try_from(map)?;

I prefer to write this as two instructions, as I think it's easier to understand:

Retrieve the map
Convert it to ProgramArray

But you can also write :

let mut tail_call_map = ProgramArray::try_from(ebpf.take_map("JUMP_TABLE").unwrap())?;

Don't forget the library at the start of the program:

use aya::maps::ProgramArray;

Loading programs and eBPF map

We'll continue working in the tracepoint-binary/src/main.rs file. Now that we've retrieved the eBPF map, we'll load the eBPF programs we've just created and tell the eBPF map what index 0 and index 1 correspond to.

We'll continue to follow the example in the documentation:

Let's test this piece of code:

let prog_0: &TracePoint = ebpf.program("tracepoint_binary_filter").unwrap().try_into()?;
let prog_0_fd = prog_0.fd().unwrap();
tail_call_map.set(0, &prog_0_fd, 0);

With cargo run we get the following error:

Well, it doesn't work properly. The documentation is no good. I see the error:

thread 'main' panicked at tracepoint-binary/src/main.rs:58:33:
called `Result::unwrap()` on an `Err` value: NotLoaded

So it wants the programs to be loaded. So I test:

let prog_0: &TracePoint = ebpf.program("tracepoint_binary_filter").unwrap().try_into()?;
prog_0.load()?;
let prog_0_fd = prog_0.fd().unwrap();
tail_call_map.set(0, &prog_0_fd, 0);

prog_0 must be mutable:

let prog_0: &mut TracePoint = ebpf.program_mut("tracepoint_binary_filter").unwrap().try_into()?;
prog_0.load()?;
let prog_0_fd = prog_0.fd().unwrap();
tail_call_map.set(0, &prog_0_fd, 0);

It works!

Of course, you can copy and paste the previous code for the second program. But you can also make a small loop to make it a little more concise and readable:

let prg_list = ["tracepoint_binary_filter", "tracepoint_binary_display"];

for (i, prg) in prg_list.iter().enumerate() {
    {
        let program: &mut TracePoint = ebpf.program_mut(prg).unwrap().try_into()?;
        program.load()?;
        let fd = program.fd().unwrap();
        tail_call_map.set(i as u32, fd, 0)?;
    }
}

Things to remember about jumps:

index 0 of the map corresponds to the tracepoint_binary_filter program
index 1 of the map corresponds to the tracepoint_binary_display program.

For more complex tail calls, we'll have to find a simple way of finding our way around.
Now we can compile the code. It should work. If you test the program, only the hook program is launched. The filter and display programs won't do anything for the moment, but they are loaded and the tail calls map has been filled by these programs.

Here's what we've done schematically:

We've put all the building blocks in place, and all that's left is to modify the various programs so that :

do only what we ask them to do (retrieve data, filter or display)
create communication between each program

Let's set up the tails calls

We must therefore transform the previous diagram into:

Program modification data retrieval (hook)

The user-space code is now complete. We'll now concentrate on kernel space. In particular, the program that retrieves the binary name:

We therefore modify the tracepoint-binary-ebpf/src/hook.rs file.

The part of the hook program we're going to modify is :

fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;

    let filename = unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        let filename_bytes = bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
        if EXCLUDED_CMDS.get(&*buf).is_some() {
            info!(&ctx, "No log for this Binary");
            return Ok(0);
        }
        from_utf8_unchecked(filename_bytes)
    };

    info!(
        &ctx,
        "tracepoint sys_enter_execve called. Binary: {}", filename
    );
    Ok(0)
}

We're going to remove the display (the display program will do this):

info!(
        &ctx,
        "tracepoint sys_enter_execve called. Binary: {}", filename
    );

We're also going to remove the filter (the filter program will do this):

if EXCLUDED_CMDS.get(&*buf).is_some() {
            info!(&ctx, "No log for this Binary");
            return Ok(0);
}

This gives us:

fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;

    let filename = unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        let filename_bytes = bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
        from_utf8_unchecked(filename_bytes)
    };

    Ok(0)
}

the filename and filename_bytes variables are no longer used in this program, so we have:

fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "hook");
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;

    unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
    }

    Ok(0)
}

We've added debugging to check that we're in the hook program. We now need to add the debugging library:

use aya_log_ebpf::debug;

Nothing extraordinary so far.
Now we'll see how to tell the hook program to go to the filter program. Following the example given in the documentation:

The example looks quite complicated... We'll add bpf_probe_read_user_str_bytes just after it:

JUMP_TABLE.tail_call(ctx, 0);

0 being the index of the JUMP_TABLE map which corresponds to the filter program defined in user space.

I see the following error:

So we need to set :

JUMP_TABLE.tail_call(&ctx, 0);

It works:

The hook program is passed to the filter program.
However, there's a warning:

I don't really like let _ =.
Let's look at the signature of the tail_call() method:

So it returns Result<!, c_long>. Not very clear. What we need to know is whether there's an error in Result<>. There's a method for that:

We'll add these lines here:

let res = unsafe { JUMP_TABLE.tail_call(&ctx, 0)};
if res.is_err() {
     error!(&ctx, "hook: tail_call failed");
}

0 is the first tail call program defined in user space, corresponding to the filter program.

the error macro is not imported, so we need to add this library:

use aya_log_ebpf::{debug,error};

The result is:

use aya_ebpf::{
    helpers::bpf_probe_read_user_str_bytes,
    macros::tracepoint,
    programs::TracePointContext,
};
use aya_log_ebpf::{debug,error};

use crate::common::*;

const FILENAME_OFFSET: usize = 16;

#[tracepoint]
pub fn tracepoint_binary(ctx: TracePointContext) -> u32 {
    match try_tracepoint_binary(ctx) {
        Ok(ret) => ret,
        Err(ret) => ret as u32,
    }
}

fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "hook");
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;

    unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
    }
    let res = unsafe { JUMP_TABLE.tail_call(&ctx, 0)};
    if res.is_err() {
        error!(&ctx, "hook: tail_call failed");
    }

    Ok(0)
}

That's all for the hook program. You've done the hard part, the rest is the same.

Modifying the filter program

For filter, we need to retrieve the contents of the BUF map:

let buf = BUF.get(0).ok_or(0)?; //buf is a reference of a list of 512 entries

In the same way as for hook, we're going to remove the unnecessary stuff.

This results in the main filter.rs code:

fn try_tracepoint_binary_filter(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "filter");
    let buf = BUF.get(0).ok_or(0)?;

    let is_excluded = unsafe {
        EXCLUDED_CMDS.get(buf).is_some()
    };

    if is_excluded {
        debug!(&ctx, "No log for this Binary");
        return Ok(0);
    }

    Ok(0)
}

To tell it to go to the display program, we must also add:

let res = unsafe { JUMP_TABLE.tail_call(&ctx, 1) };
if res.is_err() {
    error!(&ctx, "filter: tail_call failed");
}

1 being the index of the JUMP_TABLE map array, which corresponds to the display program defined in the user space.

To complicate things a little, we could imagine that if it's a filtered command, it would go to another program for further processing. Let your imagination run wild...
In the end, we end up with:

use aya_ebpf::{
    macros::tracepoint,
    programs::TracePointContext,
};

use aya_log_ebpf::{debug,error};

use crate::common::*;

#[tracepoint]
pub fn tracepoint_binary_filter(ctx: TracePointContext) -> u32 {
    match try_tracepoint_binary_filter(ctx) {
        Ok(ret) => ret,
        Err(ret) => ret as u32,
    }
}

fn try_tracepoint_binary_filter(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "filter");
    let buf = BUF.get(0).ok_or(0)?;

    let is_excluded = unsafe {
        EXCLUDED_CMDS.get(buf).is_some()
    };

    if is_excluded {
        debug!(&ctx, "No log for this Binary");
        return Ok(0);
    }

    let res = unsafe { JUMP_TABLE.tail_call(&ctx, 1) };
    if res.is_err() {
        error!(&ctx, "filter: tail_call failed");
    }

    Ok(0)
}

Let's look at what we've done by testing in debug mode:

I ran the ls command: hook ➡️ filter
I ran the man command: hook ➡️ filter ➡️ display

This is exactly what we wanted!

Modifying the display program

For the last program, we'll just retrieve what's in the map buffer and transform it into &str as we saw in the previous section. We'll then modify the display.rs file:

fn try_tracepoint_binary_display(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "display");
    let buf = BUF.get(0).ok_or(0)?;
    let cmd = &buf[..];
    let filename = unsafe { from_utf8_unchecked(cmd) };
    info!(&ctx, "tracepoint sys_enter_execve called. Binary: {}", filename);
    Ok(0)
}

Now we'll test in debug mode:

RUST_LOG=debug cargo run

Without debug mode, we wouldn't be able to see the difference with the program created in the previous section.

Inline function

Finally, we're going to factorize the code for the jumps:

let res = unsafe { JUMP_TABLE.tail_call(&ctx, 1) };
if res.is_err() {
     error!(&ctx, "filter: tail_call failed");
}

It's used twice (in hook.rs and in filter.rs ) and for another project it might be a good idea to already have this bit of code. Let's put it in the common.rs file.

What's an inline function?

For performance reasons, it's not advisable to create too many functions. It's a bit complicated, but when you call up a function, there's a JMP (Jump) at assembler level, so it's expensive if you're looking for performances (CPU cycle cost) and minimizing instructions, as is the case with eBPF. To solve this problem, you can inline your function. This allows the compiler to automatically replace the function in the main code, making it transparent to the developer.
To inline a function in Rust, simply add the following code above the function:

#[inline(always)]

Creating an inline function

What is the input to this function?

ctx: the Tracepoint context
index: the program index

This function returns nothing: res doesn't interest us.
So the signature is something like :

fn try_tail_call(ctx: TracePointContext, index: u32) {
   [...]
}

So we'd have :

fn try_tail_call(ctx: TracePointContext, index: u32) {
    let res = unsafe { JUMP_TABLE.tail_call(&ctx, index) };
    if res.is_err() {
        error!(&ctx, "tail_call failed");
    }
}

This function is totally valid. However, for borrowing problems, it's more interesting to use the TracePointContext reference:

As we want to inline the function for performance reasons, we'll have in the common.rs file :

#[inline(always)]
pub fn try_tail_call(ctx: &TracePointContext, index: u32) {
    let res = unsafe { JUMP_TABLE.tail_call(ctx, index) };
    if res.is_err() {
        error!(ctx, "exit: tail_call failed");
    }
}

The complete common.rs file with the libraries:

use aya_ebpf::{
    macros::map,
    maps::{HashMap, PerCpuArray, ProgramArray},
    programs::TracePointContext,
};
use aya_log_ebpf::error;

use tracepoint_binary_common::MAX_PATH_LEN;

pub const ZEROED_ARRAY: [u8; MAX_PATH_LEN] = [0u8; MAX_PATH_LEN];

#[map]
pub static BUF: PerCpuArray<[u8; MAX_PATH_LEN]> = PerCpuArray::with_max_entries(1, 0);

#[map]
pub static EXCLUDED_CMDS: HashMap<[u8; 512], u8> = HashMap::with_max_entries(10, 0);

#[map]
pub static JUMP_TABLE: ProgramArray = ProgramArray::with_max_entries(2, 0);

#[inline(always)]
pub fn try_tail_call(ctx: &TracePointContext, index: u32) {
    let res = unsafe { JUMP_TABLE.tail_call(ctx, index) };
    if res.is_err() {
        error!(ctx, "tail_call failed");
    }
}

In the hook program, we'll replace the code :

let res = unsafe { JUMP_TABLE.tail_call(&ctx, 0) };
if res.is_err() {
    error!(&ctx, "filter: tail_call failed");
}

with: try_tail_call(&ctx, 0);
This gives us :

fn try_tracepoint_binary(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "hook");
    let buf = BUF.get_ptr_mut(0).ok_or(0)?;

    unsafe {
        *buf = ZEROED_ARRAY;
        let filename_src_addr = ctx.read_at::<*const u8>(FILENAME_OFFSET)?;
        bpf_probe_read_user_str_bytes(filename_src_addr, &mut *buf)?;
    }
    try_tail_call(&ctx, 0);

    Ok(0)
}

and in the filter program :

let res = unsafe { JUMP_TABLE.tail_call(&ctx, 1) };
if res.is_err() {
    error!(&ctx, "filter: tail_call failed");
}

by try_tail_call(&ctx, 1);
Which gives us :

fn try_tracepoint_binary_filter(ctx: TracePointContext) -> Result<u32, i64> {
    debug!(&ctx, "filter");
    let buf = BUF.get(0).ok_or(0)?;

    let is_excluded = unsafe {
        EXCLUDED_CMDS.get(buf).is_some()
    };

    if is_excluded {
        debug!(&ctx, "No log for this Binary");
        return Ok(0);
    }
    try_tail_call(&ctx, 1);

    Ok(0)
}

Don't forget to check that it continues to compile.

The hook and filter program code is still easier to read, and we've been able to see the creation of an inlined function 😆

The code

The code is in the usual git repo :

https://github.com/littlejo/aya-examples/tree/main/tracepoint-binary-tail-calls

That's all for this section. We've seen quite a lot in this tail calls eBPF section: creation and filling of the dedicated map, jumps and specialization of eBPF programs. We've also seen inline functions and a bit of Rust code restructuring.
In the last section, we'll introduce another eBPF program and see how to make it interact with the one we've created.

Joseph Ligier @littlejo

Modularizing your Aya program with tail calls