目录
基本语法套路
综述
常用接口
io_uring实践
跟踪io_uring_enter
iouring所有的跟踪点
旨在记录在跟踪内核时的问题及具体案例。
问题:
1) 如何跟踪某个函数内部的实现 ?
2) 如何跟踪setup过程中单独启用的sq_poll线程的工作?
基本语法套路
综述
bpftrace -e 'tracepoint:syscalls:sys_enter_io_uring_enter { printf ("%u %u\n",args->fd,args->to_submit); }'
1. -e 之后 tracepoint为类型,一般有kprobe ,krtprobe等,可以简写为t,k,kr等,具体有哪些可以参考最后一节 bpftrace -l的查询命令结果
2. 在具体的函数或者event之后,为在检测到此函数或者event后要进行的操作。
其中针对事件的操作,其语法看起来和c比较相近。
常用接口
io_uring实践
跟踪io_uring_enter
1)查询 io_uring_enter的args的具体的参数信息,实际上这个也可以通过直接阅读源代码获取。
bpftrace -vl tracepoint:syscalls:sys_enter_io_uring_enter
tracepoint:syscalls:sys_enter_io_uring_enter
int __syscall_nr;
unsigned int fd;
u32 to_submit;
u32 min_complete;
u32 flags;
const void * argp;
size_t argsz;
2)打印某些参数
bpftrace -e 'tracepoint:syscalls:sys_enter_io_uring_enter { printf ("%u %u\n",args->fd,args->to_submit); }'
Attaching 1 probe...
3 0
运行 https://github.com/axboe/liburing/issues/413 中reprodution的代码,当child: sqe submit failed: -130时,则输出fd 3 to_submit 0;而 child: sqe submit failed: 0时,则bpftrace没有任何输出。
跟踪io_fixed_file
1)明确要跟踪的符号
虽然在代码中函数为: io_fixed_file_set,但是经过编译器优化后,已经变为
bpftrace -l |grep fixed_file
kprobe:io_fixed_file_set.isra.95
2) 实施跟踪
在进行追踪时,采用*号代替,采用kprobe:io_fixed_file_set.isra.95会一直提示语法错误,也就是不能有.(点)或者通过什么符号进行转译。
bpftrace -e 'kprobe:io_fixed_file_set* {printf("%s",kstack);}'
结果如下:
io_fixed_file_set.isra.95+1
__io_uring_register+1499
__x64_sys_io_uring_register+179
do_syscall_64+58
entry_SYSCALL_64_after_hwframe+68
bpftrace -e 'kprobe:__io_file_supports_async {printf("%s",kstack);}'
__io_file_supports_async+1
io_sqe_files_register+388
__io_uring_register+1133
__x64_sys_io_uring_register+179
do_syscall_64+58
entry_SYSCALL_64_after_hwframe+68
__io_file_supports_async+1
io_issue_sqe+2403
__io_queue_sqe+127
io_req_task_submit+87
tctx_task_work+156
task_work_run+123
io_run_task_work_sig+98
io_cqring_wait+355
__x64_sys_io_uring_enter+682
do_syscall_64+58
entry_SYSCALL_64_after_hwframe+68
bpftrace -e 'kprobe:io_fixed_file_set* {printf("%llx_%llx_%llx\n",arg1,arg2,arg3);}'
static void io_fixed_file_set(struct io_fixed_file *file_slot, struct file *file)
这个函数只有两个参数,但优化后,可以打印出三个参数的信息:
ffff92a9925f6f00_ffff92a8cf4f1000_1
ffff92a9925f6e00_ffff92a8cf4f1000_1
ffff92a9925f6700_ffff92a8ccc76000_ffffb98942d67e90
ffff92a8c09d8700_ffff92a8ccc76000_ffffb98942d67e90
最后一个参数的形式是怎么样的?为什么差别这么大?
iouring所有的跟踪点
bpftrace -l |grep uring
tracepoint:io_uring:io_uring_create
tracepoint:io_uring:io_uring_register
tracepoint:io_uring:io_uring_file_get
tracepoint:io_uring:io_uring_queue_async_work
tracepoint:io_uring:io_uring_defer
tracepoint:io_uring:io_uring_link
tracepoint:io_uring:io_uring_cqring_wait
tracepoint:io_uring:io_uring_fail_link
tracepoint:io_uring:io_uring_complete
tracepoint:io_uring:io_uring_submit_sqe
tracepoint:io_uring:io_uring_poll_arm
tracepoint:io_uring:io_uring_poll_wake
tracepoint:io_uring:io_uring_task_add
tracepoint:io_uring:io_uring_task_run
tracepoint:syscalls:sys_enter_io_uring_enter
tracepoint:syscalls:sys_exit_io_uring_enter
tracepoint:syscalls:sys_enter_io_uring_setup
tracepoint:syscalls:sys_exit_io_uring_setup
tracepoint:syscalls:sys_enter_io_uring_register
tracepoint:syscalls:sys_exit_io_uring_register
kprobe:__traceiter_io_uring_create
kprobe:__traceiter_io_uring_register
kprobe:__traceiter_io_uring_file_get
kprobe:__traceiter_io_uring_queue_async_work
kprobe:__traceiter_io_uring_defer
kprobe:__traceiter_io_uring_link
kprobe:__traceiter_io_uring_cqring_wait
kprobe:__traceiter_io_uring_fail_link
kprobe:__traceiter_io_uring_complete
kprobe:__traceiter_io_uring_submit_sqe
kprobe:__traceiter_io_uring_poll_arm
kprobe:__traceiter_io_uring_poll_wake
kprobe:__traceiter_io_uring_task_add
kprobe:__traceiter_io_uring_task_run
kprobe:io_uring_get_socket
kprobe:io_uring_poll
kprobe:io_uring_fasync
kprobe:io_uring_del_tctx_node
kprobe:io_uring_alloc_task_context
kprobe:__io_uring_add_tctx_node
kprobe:io_uring_mmap
kprobe:io_uring_show_fdinfo
kprobe:__io_uring_register
kprobe:__ia32_sys_io_uring_register
kprobe:__x64_sys_io_uring_register
kprobe:io_uring_try_cancel_requests
kprobe:io_uring_release
kprobe:io_uring_setup
kprobe:__x64_sys_io_uring_setup
kprobe:__ia32_sys_io_uring_setup
kprobe:__ia32_sys_io_uring_enter
kprobe:__x64_sys_io_uring_enter
kprobe:__io_uring_free
kprobe:io_uring_cancel_generic
kprobe:__io_uring_cancel