Difference between revisions of "Kernel Trace Systems"
(→Special Purpose tracing systems: include link in definition line) |
(→Sampling Systems) |
||
(6 intermediate revisions by 3 users not shown) | |||
Line 6: | Line 6: | ||
*ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process) | *ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process) | ||
** see "man ptrace" and "man strace" | ** see "man ptrace" and "man strace" | ||
+ | * [[Ftrace]] | ||
+ | ** [[Ftrace Function Graph ARM]] - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems | ||
+ | *** The presentation has good introductory material on ftrace, as well as links to additional resources | ||
+ | ** tracer for kernel functions | ||
+ | ** can also be used for debugging or analyzing latencies and performance issues | ||
+ | ** in mainline since 2.6.27 | ||
+ | ** See [[Measuring Function Duration with FTrace]] - outline of presentation by Tim Bird for Linux Symposium 2009 | ||
* [[System Tap]] - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system | * [[System Tap]] - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system | ||
*LTTng - [[Linux Trace Toolkit]], next generation | *LTTng - [[Linux Trace Toolkit]], next generation | ||
Line 42: | Line 49: | ||
** see [http://oprofile.sourceforge.net/about/ oprofile] | ** see [http://oprofile.sourceforge.net/about/ oprofile] | ||
** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM] | ** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM] | ||
− | * | + | * BootChart - samples bootup and provides visualization of process startup and system utilization |
− | ** see [[ | + | ** see [[Bootchart]] |
== Related facilities == | == Related facilities == | ||
Line 61: | Line 68: | ||
Some trace system project leaders are trying to collaborate: see [[Tracing Collaboration Project]] | Some trace system project leaders are trying to collaborate: see [[Tracing Collaboration Project]] | ||
+ | |||
+ | [[Category:Linux tracing]] |
Latest revision as of 14:01, 28 July 2010
Here are some links to information about different kernel tracing systems:
Contents
General Purpose tracing systems
Some major Linux general-purpose tracing systems are:
- ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process)
- see "man ptrace" and "man strace"
- Ftrace
- Ftrace Function Graph ARM - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
- The presentation has good introductory material on ftrace, as well as links to additional resources
- tracer for kernel functions
- can also be used for debugging or analyzing latencies and performance issues
- in mainline since 2.6.27
- See Measuring Function Duration with FTrace - outline of presentation by Tim Bird for Linux Symposium 2009
- Ftrace Function Graph ARM - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
- System Tap - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system
- LTTng - Linux Trace Toolkit, next generation
- LKST - Linux Kernel State Tracer
Special Purpose tracing systems
There are some other notable special-purpose kernel tracing systems:
- KFT - Kernel Function Trace - traces functions to show function durations and call graphs
- latency trace - RT-preempt tool for measuring interrupt and mutex latency
- The latency tracer is embedded in the RT-preempt patch - see Realtime Preemption and RT-preempt Article
- block tracer (blktrace) - allows you to see exactly what is going on in the block layer for a given queue
- Introduction by Jens Axboe: Introduction
- Execellent presentation: blktrace.pdf
- Guide to using is at: blktrace guide
- This appears to have been mainlined as of 2.6.17
- Timeline utility (blktrace post-processing tool): blktrace timeline utility
- delay accounting patches - collect statistics about the delays that are experienced by each task on the system
Trace Infrastructure
- KProbes - grew out of dprobes, with information at: dprobes
- see an excellent tutorial at: kprobes
- The mainline version of the KProbes supports x86,Alpha and PPC64 architectures. A MIPS implementation has been completed on the 2.6.16 kernel and tested on the Toshiba TX49 platform. Patch is available in the Patch Archive.
- [would be nice to have some djprobe stuff here]
Sampling Systems
Note that profile systems (or "sampling systems") are slightly different, in that they involve sampling instead of event tracing. Some major ones for Linux are:
- top - provides a dynamic real-time view of a running system, including processes
- see "man top"
- see also ksysguard, Gnome system Monitor
- OProfile - system-wide profiler for Linux systems
- see oprofile
- also: oprofile at IBM
- BootChart - samples bootup and provides visualization of process startup and system utilization
- see Bootchart
Related facilities
- in-kernel statistics infrastructure - proposal for a generic implementation of statistics facilities inside the kernel
- see inkernel stats
- perfmon2 - interfaces to hardware performance monitoring features of the CPU
- see perfmon
- inotify - inotify
Other Systems
Here are some systems I haven't classified yet:
- Datastreams - a system for creating and monitoring tracepoints - see datastreams
Collaboration Efforts
Some trace system project leaders are trying to collaborate: see Tracing Collaboration Project