Difference between revisions of "Kernel Trace Systems"

From eLinux.org
Jump to: navigation, search
(General Purpose tracing systems)
(Sampling Systems)
 
(2 intermediate revisions by the same user not shown)
Line 6: Line 6:
 
*ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process)
 
*ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process)
 
** see "man ptrace" and "man strace"
 
** see "man ptrace" and "man strace"
* [http://free-electrons.com/kerneldoc/latest/ftrace.txt ftrace]
+
* [[Ftrace]]
 
** [[Ftrace Function Graph ARM]] - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
 
** [[Ftrace Function Graph ARM]] - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
 
*** The presentation has good introductory material on ftrace, as well as links to additional resources
 
*** The presentation has good introductory material on ftrace, as well as links to additional resources
Line 49: Line 49:
 
** see [http://oprofile.sourceforge.net/about/ oprofile]
 
** see [http://oprofile.sourceforge.net/about/ oprofile]
 
** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM]
 
** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM]
* !BootChart - samples bootup and provides visualization of process startup and system utilization
+
* BootChart - samples bootup and provides visualization of process startup and system utilization
** see [[Boot Chart]]
+
** see [[Bootchart]]
  
 
== Related facilities ==
 
== Related facilities ==

Latest revision as of 21:01, 28 July 2010

Here are some links to information about different kernel tracing systems:

General Purpose tracing systems

Some major Linux general-purpose tracing systems are:

  • ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process)
    • see "man ptrace" and "man strace"
  • Ftrace
    • Ftrace Function Graph ARM - presentations and patches by Tim Bird to add function graph and duration tracing to ARM systems
      • The presentation has good introductory material on ftrace, as well as links to additional resources
    • tracer for kernel functions
    • can also be used for debugging or analyzing latencies and performance issues
    • in mainline since 2.6.27
    • See Measuring Function Duration with FTrace - outline of presentation by Tim Bird for Linux Symposium 2009
  • System Tap - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system
  • LTTng - Linux Trace Toolkit, next generation
  • LKST - Linux Kernel State Tracer

Special Purpose tracing systems

There are some other notable special-purpose kernel tracing systems:

  • KFT - Kernel Function Trace - traces functions to show function durations and call graphs
  • latency trace - RT-preempt tool for measuring interrupt and mutex latency
  • block tracer (blktrace) - allows you to see exactly what is going on in the block layer for a given queue
  • delay accounting patches - collect statistics about the delays that are experienced by each task on the system

Trace Infrastructure

  • KProbes - grew out of dprobes, with information at: dprobes
    • see an excellent tutorial at: kprobes
    • The mainline version of the KProbes supports x86,Alpha and PPC64 architectures. A MIPS implementation has been completed on the 2.6.16 kernel and tested on the Toshiba TX49 platform. Patch is available in the Patch Archive.
  • [would be nice to have some djprobe stuff here]

Sampling Systems

Note that profile systems (or "sampling systems") are slightly different, in that they involve sampling instead of event tracing. Some major ones for Linux are:

  • top - provides a dynamic real-time view of a running system, including processes
  • OProfile - system-wide profiler for Linux systems
  • BootChart - samples bootup and provides visualization of process startup and system utilization

Related facilities

  • in-kernel statistics infrastructure - proposal for a generic implementation of statistics facilities inside the kernel
  • perfmon2 - interfaces to hardware performance monitoring features of the CPU
  • inotify - inotify

Other Systems

Here are some systems I haven't classified yet:

  • Datastreams - a system for creating and monitoring tracepoints - see datastreams

Collaboration Efforts

Some trace system project leaders are trying to collaborate: see Tracing Collaboration Project