hp77 | Add Option for build without libbpf

[hp77-creator]

Description

Adding an option to allow bpftime to built without libbpf

Fixes #175

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How Has This Been Tested?

• Running bpftime on MacOS

Test Configuration:

• Firmware version:
• Hardware:
• Toolchain:
• SDK:

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I have checked my code and corrected any misspellings

[hp77-creator]

TODO: had to also change path for BOOST_ROOT to opt/homebrew/include to fix boost import errors. Will have to mention same in the documentation.

[hp77-creator]

@yunwei37 @Officeyutong right now, I am facing issue in bpftime_shm.hpp
It was giving error on sys/epoll.h not being present on system, so I added a macro-processor check but after that there are many functions which are giving error. What to do for that? Should I also make the dependent functions enclosed in a preprocessor check?

[yunwei37]

Could you please provide some detailed error information?

[Officeyutong]

@yunwei37 @Officeyutong right now, I am facing issue in bpftime_shm.hpp
It was giving error on sys/epoll.h not being present on system, so I added a macro-processor check but after that there are many functions which are giving error. What to do for that? Should I also make the dependent functions enclosed in a preprocessor check?

epoll is not supported on platforms except Linux. But in bpftime, we should have only used some types and structs defined in this header. You may find out these types and structs, define them in the code base of bpftime

[hp77-creator]

getting these now:
@Officeyutong @yunwei37

bpftime-hp/runtime/src/handler/map_handler.hpp:37:11: error: unknown type name 'pthread_spinlock_t'; did you mean 'pthread_rwlock_t'?
        volatile pthread_spinlock_t &spinlock;
                 ^~~~~~~~~~~~~~~~~~

should i replace spinlock with rwlock? for apple systems?

[Officeyutong]

getting these now: @Officeyutong @yunwei37

bpftime-hp/runtime/src/handler/map_handler.hpp:37:11: error: unknown type name 'pthread_spinlock_t'; did you mean 'pthread_rwlock_t'?
        volatile pthread_spinlock_t &spinlock;
                 ^~~~~~~~~~~~~~~~~~

should i replace spinlock with rwlock? for apple systems?

You may write a spinlock. It's very simple. Don't use mutex, they are very slow

[hp77-creator]

@Officeyutong @yunwei37
I am getting following errors

/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:366:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:382:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:400:18: error: use of undeclared identifier 'sigtimedwait'
                        if (int sig = sigtimedwait(&to_block, &sig_info, &ts);

It is coming from sys/epoll , I looked for alternatives, kevent is one, should we go for that for BSD specific kernels?

[hp77-creator]

@Officeyutong @yunwei37
Also getting error on

error: use of undeclared identifier 'CLOCK_MONOTONIC_COARSE'; did you mean '_CLOCK_MONOTONIC_RAW'?
        clock_gettime(CLOCK_MONOTONIC_COARSE, &spec);
                      ^~~~~~~~~~~~~~~~~~~~~~

will try to refer this: https://stackoverflow.com/questions/41509505/clock-gettime-on-macos

[hp77-creator]

Also getting this:

error: use of undeclared identifier 'gettid'
                tid = gettid();

[hp77-creator]

also getting this:

runtime/src/bpf_helper.cpp:330:14: error: use of undeclared identifier 'bpf_test_run_opts'
        LIBBPF_OPTS(bpf_test_run_opts, run_opts, .ctx_in = context,

do we need to include LIBBPF_OPTS, it is coming from bpftime_tail_call
what should I do with it?

[Officeyutong]

@Officeyutong @yunwei37 I am getting following errors

/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:366:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:382:10: error: use of undeclared identifier 'EPOLLIN'
                                                                        EPOLLIN,
                                                                        ^
/Users/hp77/repos/bpftime-hp/runtime/src/bpftime_shm.cpp:400:18: error: use of undeclared identifier 'sigtimedwait'
                        if (int sig = sigtimedwait(&to_block, &sig_info, &ts);

It is coming from sys/epoll , I looked for alternatives, kevent is one, should we go for that for BSD specific kernels?

• EPOLLIN is a constant defined in epoll headers. We didn't use epoll syscalls in bpftime, but just used constants, types, structs from epoll
• sigtimedwait is a POSIX signal API. Does darwin support it?

[Officeyutong]

Also getting this:

error: use of undeclared identifier 'gettid'
                tid = gettid();

gettid returns the thread id of the currently executing thread. It's a linux specified syscall but I think there should be alternatives on darwin

[Officeyutong]

also getting this:

runtime/src/bpf_helper.cpp:330:14: error: use of undeclared identifier 'bpf_test_run_opts'
        LIBBPF_OPTS(bpf_test_run_opts, run_opts, .ctx_in = context,

do we need to include LIBBPF_OPTS, it is coming from bpftime_tail_call what should I do with it?

You may disable this helper function on darwin. It relies kernel eBPF to work.

[hp77-creator]

• EPOLLIN is a constant defined in epoll headers. We didn't use epoll syscalls in bpftime, but just used constants, types, structs from epoll

I have defined the value in a separate header file for this

• sigtimedwait is a POSIX signal API. Does darwin support it?

It doesn't, have created a custom implementation and added in a header file.

gettid returns the thread id of the currently executing thread. It's a linux specified syscall but I think there should be alternatives on darwin

Yea, I found some alternative in pthread_threadid_np(NULL, &tid); and using that.

You may disable this helper function on darwin. It relies kernel eBPF to work.

I have disabled this function

I am getting a lot of error in prog_array.cpp now, a lot of linux/libc related functions and constants are used.

[hp77-creator]

also what to do of bpf_object.cpp ?

[Officeyutong]

• EPOLLIN is a constant defined in epoll headers. We didn't use epoll syscalls in bpftime, but just used constants, types, structs from epoll

I have defined the value in a separate header file for this

• sigtimedwait is a POSIX signal API. Does darwin support it?

It doesn't, have created a custom implementation and added in a header file.

gettid returns the thread id of the currently executing thread. It's a linux specified syscall but I think there should be alternatives on darwin

Yea, I found some alternative in pthread_threadid_np(NULL, &tid); and using that.

You may disable this helper function on darwin. It relies kernel eBPF to work.

I have disabled this function

I am getting a lot of error in prog_array.cpp now, a lot of linux/libc related functions and constants are used.

prog_array.cpp is only supported on Linux (it relies on kernel eBPF). you may disable this map implementation on other platforms

[hp77-creator]

I did that, Now for some reason boost library is not recognized
getting this error now:

fatal error: 'boost/interprocess/interprocess_fwd.hpp' file not found
#include <boost/interprocess/interprocess_fwd.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[3]: *** [tools/bpftimetool/CMakeFiles/bpftimetool.dir/main.cpp.o] Error 1
make[2]: *** [tools/bpftimetool/CMakeFiles/bpftimetool.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [build-wo-libbpf] Error 2

[hp77-creator]

I have done so many changes to rectify errors now that I have lost track of my changes, will need to look at them again.
I had faced this boost error before as well, setting BOOST_ROOT fixed this before. will try again.

[Officeyutong]

I have done so many changes to rectify errors now that I have lost track of my changes, will need to look at them again. I had faced this boost error before as well, setting BOOST_ROOT fixed this before. will try again.

boost may not be recognized automically. Sometimes you may need to set BOOST_ROOT manualy

[Officeyutong]

also what to do of bpf_object.cpp ?

This file is mainly used for parsing ELF files. It calls API provided by libbpf to pick up eBPF programs defined in an ELF file. It may be replaced with other libraries that can parse ELF, such as libelf.

eBPF programs are stored in program sections of an ELF file. section names would be like uprobe/XXXXX or tracepoint/syscall/XXXX or other things alike. (You may use readelf to get some details). Some eBPF programs might require linking with other eBPF programs (such as an eBPF program calling another non-inline function), but in my opinion we may temporarily not provide support for linking

You may refer to https://github.com/libbpf/libbpf/blob/42065ea6627ff6e1ab4c65e51042a70fbf30ff7c/src/libbpf.c#L7892 for how libbpf loads ELF files

[hp77-creator]

I will read more on the above. An update that I have been able to compile bpftime successfully. But when I tried running it on my system, I am getting following error:

2024-06-09 02:47:53.495] [info] Injecting to 99920
[2024-06-09 02:47:53.664] [error] Failed to inject: Module not found at "/usr/lib/libz.1.dylib"

I checked, macOS comes with a zlib installation which is symlinked to libz.1.dylib as libz.1.2.1.dylib, I thought of adding one more symlink but there are various write permission issues. I will try it again.

[Officeyutong]

Also, options added should be documented in https://github.com/eunomia-bpf/eunomia.dev/blob/main/docs/bpftime/documents/build-and-test.md, remember to update it

@hp77-creator

[hp77-creator]

Will do