Profilers

Linux systems count with a wide variety of profilers each of with their pros and cons, there is no magic bullet, it's recommended to use more than one tool when analyzing your application, I recommend you to use at least OProfile and Valgrind.

OProfile
OProfile is a non-obtrusive system-wide profiler for Linux, it can use system's performance counters to give you insights on where to optimize your code.

Helper script for simple runs (remember to run  once before this script):

sudo opcontrol --init sudo opcontrol --start sudo opcontrol --dump sudo opcontrol --reset $@ sudo opcontrol --stop opreport --symbols > oprof.txt
 * 1) !/bin/sh

see also oprof_start

oprof_start
Qt3 GUI for OProfile, ships in official distribution, see http://oprofile.sourceforge.net/doc/oprofile-gui.html

KCachegrind
One can use reports that are usable by kcachegrind with (for more instruction see http://docs.kde.org/kde3/en/kdesdk/kcachegrind/using-kcachegrind.html):

opreport -gdf | op2callgrind

OProfileUI
Another useful application is OProfileUI, it provides a GTK+2 user interface to run and show statistics, including basic call graph.

Documentation
In depth explanation can be found at:
 * IBM DeveloperWorks
 * OProfile Manual
 * OProfile FAQ

Caveats
mkdir /var/lib/oprofile mount -t ramfs none /var/lib/oprofile mount -t ext3 /dev/sda1 /var/lib/oprofile
 * JFFS2 does not provide OProfile requirements, running OProfile to output data into JFFS2 partition will fail, instead use a ramfs or some external card with ext2/3:
 * 1) ram
 * 1) ext3
 * settings event counter too low or too high may hang your test machine. Check the recommended value for each event on specific platforms.

GProf
Unlike OProfile, GProf requires applications to be compiled with special flag  so compiler will introduce marks in generated binary to measure runs. After run the  file will be created with measured data, this file can be viewed with tools like gprof or kprof.

KProf
Outdated, but exists: http://kprof.sourceforge.net/

Documentation

 * GProf Manual

Caveats

 * Imprecise
 * Huge overhead

Valgrind Cachegrind/Callgrind
Valgrind is tool suite for debugging and profiling. It's most famous for it's memcheck tool to check for memory leaks, invalid access, double frees and more, but also ships with interesting callgrind and cachegrind.

cachegrind will accurately simulate L1, L2 and D1 CPU caches so you can figure out what in your code is trashing your memory access.

callgrind is an extension of cachegrind that will collect call graphs.

KCachegrind
The most famous user interface is KCachegrind

Documentation

 * Valgrind information in this wiki
 * Valgrind Manual
 * FAQ
 * http://docs.kde.org/kde3/en/kdesdk/kcachegrind/using-kcachegrind.html

Caveats

 * Since these are simulators, your application will run very slow, from 20 to 100 times or more.