Difference between revisions of "Kernel Trace Systems"
From eLinux.org
(Legacy Content Conversion) |
(→Sampling Systems) |
||
| (9 intermediate revisions by 5 users not shown) | |||
| Line 6: | Line 6: | ||
*ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process) | *ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process) | ||
** see "man ptrace" and "man strace" | ** see "man ptrace" and "man strace" | ||
| − | * | + | * [[Ftrace]] |
| − | + | ** [[Ftrace Function Graph ARM]] - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems | |
| − | *LTTng - | + | *** The presentation has good introductory material on ftrace, as well as links to additional resources |
| − | + | ** tracer for kernel functions | |
| − | *LKST - | + | ** can also be used for debugging or analyzing latencies and performance issues |
| − | + | ** in mainline since 2.6.27 | |
| + | ** See [[Measuring Function Duration with FTrace]] - outline of presentation by Tim Bird for Linux Symposium 2009 | ||
| + | * [[System Tap]] - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system | ||
| + | *LTTng - [[Linux Trace Toolkit]], next generation | ||
| + | *LKST - [[Linux Kernel State Tracer]] | ||
== Special Purpose tracing systems == | == Special Purpose tracing systems == | ||
There are some other notable special-purpose kernel tracing systems: | There are some other notable special-purpose kernel tracing systems: | ||
| − | * KFT - Kernel Function Trace - traces functions to show function durations and call graphs | + | * KFT - [[Kernel Function Trace]] - traces functions to show function durations and call graphs |
| − | + | ||
*latency trace - RT-preempt tool for measuring interrupt and mutex latency | *latency trace - RT-preempt tool for measuring interrupt and mutex latency | ||
**The latency tracer is embedded in the RT-preempt patch - see [[Realtime Preemption]] and [http://lwn.net/Articles/97811/ RT-preempt Article] | **The latency tracer is embedded in the RT-preempt patch - see [[Realtime Preemption]] and [http://lwn.net/Articles/97811/ RT-preempt Article] | ||
| Line 31: | Line 34: | ||
== Trace Infrastructure == | == Trace Infrastructure == | ||
| − | *KProbes - grew out of dprobes, with information at: [http://dprobes.sourceforge.net/ | + | *KProbes - grew out of dprobes, with information at: [http://dprobes.sourceforge.net/ dprobes] |
**see an excellent tutorial at: [http://www-users.cs.umn.edu/~boutcher/kprobes/ kprobes] | **see an excellent tutorial at: [http://www-users.cs.umn.edu/~boutcher/kprobes/ kprobes] | ||
**The mainline version of the KProbes supports x86,Alpha and PPC64 architectures. A MIPS implementation has been completed on the 2.6.16 kernel and tested on the Toshiba TX49 platform. Patch is available in the [[Patch Archive]]. | **The mainline version of the KProbes supports x86,Alpha and PPC64 architectures. A MIPS implementation has been completed on the 2.6.16 kernel and tested on the Toshiba TX49 platform. Patch is available in the [[Patch Archive]]. | ||
| Line 46: | Line 49: | ||
** see [http://oprofile.sourceforge.net/about/ oprofile] | ** see [http://oprofile.sourceforge.net/about/ oprofile] | ||
** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM] | ** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM] | ||
| − | * | + | * BootChart - samples bootup and provides visualization of process startup and system utilization |
| − | ** see [[ | + | ** see [[Bootchart]] |
== Related facilities == | == Related facilities == | ||
| Line 65: | Line 68: | ||
Some trace system project leaders are trying to collaborate: see [[Tracing Collaboration Project]] | Some trace system project leaders are trying to collaborate: see [[Tracing Collaboration Project]] | ||
| + | |||
| + | [[Category:Linux tracing]] | ||
Latest revision as of 21:01, 28 July 2010
Here are some links to information about different kernel tracing systems:
Contents |
General Purpose tracing systems
Some major Linux general-purpose tracing systems are:
- ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process)
- see "man ptrace" and "man strace"
- Ftrace
- Ftrace Function Graph ARM - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
- The presentation has good introductory material on ftrace, as well as links to additional resources
- tracer for kernel functions
- can also be used for debugging or analyzing latencies and performance issues
- in mainline since 2.6.27
- See Measuring Function Duration with FTrace - outline of presentation by Tim Bird for Linux Symposium 2009
- Ftrace Function Graph ARM - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
- System Tap - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system
- LTTng - Linux Trace Toolkit, next generation
- LKST - Linux Kernel State Tracer
Special Purpose tracing systems
There are some other notable special-purpose kernel tracing systems:
- KFT - Kernel Function Trace - traces functions to show function durations and call graphs
- latency trace - RT-preempt tool for measuring interrupt and mutex latency
- The latency tracer is embedded in the RT-preempt patch - see Realtime Preemption and RT-preempt Article
- block tracer (blktrace) - allows you to see exactly what is going on in the block layer for a given queue
- Introduction by Jens Axboe: Introduction
- Execellent presentation: blktrace.pdf
- Guide to using is at: blktrace guide
- This appears to have been mainlined as of 2.6.17
- Timeline utility (blktrace post-processing tool): blktrace timeline utility
- delay accounting patches - collect statistics about the delays that are experienced by each task on the system
Trace Infrastructure
- KProbes - grew out of dprobes, with information at: dprobes
- see an excellent tutorial at: kprobes
- The mainline version of the KProbes supports x86,Alpha and PPC64 architectures. A MIPS implementation has been completed on the 2.6.16 kernel and tested on the Toshiba TX49 platform. Patch is available in the Patch Archive.
- [would be nice to have some djprobe stuff here]
Sampling Systems
Note that profile systems (or "sampling systems") are slightly different, in that they involve sampling instead of event tracing. Some major ones for Linux are:
- top - provides a dynamic real-time view of a running system, including processes
- see "man top"
- see also ksysguard, Gnome system Monitor
- OProfile - system-wide profiler for Linux systems
- see oprofile
- also: oprofile at IBM
- BootChart - samples bootup and provides visualization of process startup and system utilization
- see Bootchart
Related facilities
- in-kernel statistics infrastructure - proposal for a generic implementation of statistics facilities inside the kernel
- see inkernel stats
- perfmon2 - interfaces to hardware performance monitoring features of the CPU
- see perfmon
- inotify - inotify
Other Systems
Here are some systems I haven't classified yet:
- Datastreams - a system for creating and monitoring tracepoints - see datastreams
Collaboration Efforts
Some trace system project leaders are trying to collaborate: see Tracing Collaboration Project
