Difference between revisions of "KDB"

From eLinux.org
Jump to: navigation, search
(invocation by breakpoint: add more explanatory text)
(invocation by panic)
Line 96: Line 96:
=== invocation by panic ===
=== invocation by panic ===
** panic
When a kernel panic occurs, then something has gone seriously wrong and the kernel automatically enters
kdb.  From here you can look at memory, do a traceback, examine registers, and do other operations to
find out more about the state of the system and debug the problem.
=== invocation by breakpoint ===
=== invocation by breakpoint ===

Revision as of 10:57, 23 June 2011

Introduction and basic resources

Here is some information about KDB - the in-kernel debugger for the Linux kernel.

The KDB and KGDB official wiki: https://kgdb.wiki.kernel.org/ (this only has 2 pages?)

Jason Wessel is the current KDB maintainer. Here is a presentation from him at LinuxCon 2010 (August 2010): http://kernel.org/pub/linux/kernel/people/jwessel/dbg_webinar/State_Of_kernel_debugging_LinuxCon2010.pdf

Here are some videos showing use of KDB and KGDB:

Documentation, up-to-date as of 2010, for KDB and KGDB is at: http://kernel.org/pub/linux/kernel/people/jwessel/kdb/

Older Information

See http://www.ibm.com/developerworks/linux/library/l-kdbug/ for a tutorial for the 2.4.20 kernel (from June 2003)

Here's an article from 2002 on KDB vs. KGDB: http://kerneltrap.org/node/112 It has a good discussion excerpt between Andrew Morton and Keith Owens about the relative merits of KDB versus KGDB.

General Information

Kernel Versions supported

kgdb was added to the mainline Linux kernel in version 2.6.26.

kdb support was added to the mainline Linux kernel in version 2.6.35.

Before those versions, kgdb and kdb were available as patches which could be applied to the Linux kernel source.

Kernel configuration

The following descriptions are for a 2.6.35 kernel, using KDB over a serial line between host and target:

All these options on are the "Kernel Hacking" menu.

In order to support KDB, "KGDB" support must be turned on first (even if you aren't using kgdb/gdb)

  • CONFIG_DEBUG_KERNEL=y - includes debug information in the kernel compilation - required for basic kernel debugging support
  • CONFIG_KGDB=Y - turn on basic kernel debug agent support
  • CONFIG_KGDB_SERIAL_CONSOLE=Y - to share a serial console with kgdb.
    • Sysrq-g must be used to break in initially.
    • Selecting this will automatically set:
      • CONFIG_MAGIC_SYSRQ=Y - turn on MAGIC-SYSRQ key support
  • CONFIG_KGDB_KDB=Y - actually turn on the KDB debugger feature

Optional other configuration settings:

  • CONFIG_FRAME_POINTER=Y - this allows for better backtrace support in KDB
  • CONFIG_DEBUG_RODATA=N - disable this in order to support hardware breakpoints on data accesses
  • CONFIG_KALLSYMS=Y - this adds symbolic information to the kernel, useful to see symbols instead of addresses
  • CONFIG_KDB_KEYBOARD - use KDB with an attached keyboard (not for use with serial console)
  • CONFIG_KGDB_TESTS - used to turn on kgdb internal self-tests - see the config help for this for more information

Enabling kdb at kernel runtime

Once the kernel is compiled with kdb support and is running on your target board, you need to enable it. This can be done on a running system, binding the kdb/kgdb feature to a serial port, by writing a value into the sys filesystem.

If your machine starts a serial console on ttyS0, you can bind kdb/kgdb to this serial console by writing the string "ttyS0" to /sys/module/kgdboc/parameters/kgdboc. The kernel will respond with a message indicating that that driver is registered.

$ echo ttyS0 >/sys/module/kgdboc/parameters/kgdboc
kgdb: Registered I/O driver kgdboc.

[ANSWER THIS - can you also do this binding at boot time - you can for kgdb, but what about kdb?]

Invoking kdb

Once the kernel is running, and the kgdb/kdb is bound to the serial console, you can invoke the debugger in numerous ways.

First, you can enter the debugger using a [Magic SysRq] command.

kdb will also be entered automatically if the kernel panics.

Finally, you can set a breakpoint (either hardware or software), such that the debugger is invoked when the breakpoint condition is met. For a code breakpoint, this means when the instruction is executed at the breakpoint location, and for a data breakpoint, when the particular access is made at the breakpoint address.

Invoking with Magic SysRq 'g'

To invoke the debugger using the Magic SysRq command, you use the 'g' command, which can be issued any of the ways supported by the Magic SysRq feature. This can be done by 1) typing the key sequence on a connected keyboard, 2) echoing a value to /proc/sysrq-trigger, or 3) sending a break key sequence through the serial console


  • sysrq trigger from local shell, via procfs: 'echo g >/proc/sysrq-trigger'
  • sysrq trigger via serial console break sequence:
    • in minicom, type 'ctrl-a f g' (quickly)
    • in telnet, through a server that supports sending a break: type

invocation by panic

When a kernel panic occurs, then something has gone seriously wrong and the kernel automatically enters kdb. From here you can look at memory, do a traceback, examine registers, and do other operations to find out more about the state of the system and debug the problem.

invocation by breakpoint

To enter kdb using a breakpoint, first invoke kdb using the Magic SysRq key (see above), then set a breakpoint. Then type 'go' to continue execution. When the breakpoint is hit, the debugger shell will appear.

In the example that follows, items in italics are commands typed by a user. Items following a '$' are commands entered at a shell command (normal Linux user-space runtime), and items following 'kgdb>' are commands entered at the kdb interactive shell.

$ echo g >/proc/sysrq-trigger

Entering kdb (current=0xdfdff040, pid 71) due to Keyboard Entry
kdb> bp sys_sync+4
Instruction(i) BP #0 at 0xc00c9f00 (sys_sync+0x4)
   is enabled  addr at 00000000c00c9f00, hardtype=0 installed=0

kdb> go
$ sync

Entering kdb (current=0xdfdaa360, pid 72) due to Breakpoint @ 0xc00c9f00
kdb> bt
Stack traceback for pid 72
0xdfdaa360       72       71  1    0   R  0xdfdaa560 *sync
[<c0028cb4>] (unwind_backtrace+0x0/0xe4) from [<c0026d50>] (show_stack+0x10/0x14)
[<c0026d50>] (show_stack+0x10/0x14) from [<c0079e78>] (kdb_show_stack+0x58/0x80)
[<c0079e78>] (kdb_show_stack+0x58/0x80) from [<c0079f1c>] (kdb_bt1.clone.0+0x7c/0xcc)
[<c0079f1c>] (kdb_bt1.clone.0+0x7c/0xcc) from [<c007a240>] (kdb_bt+0x2d4/0x338)
[<c007a240>] (kdb_bt+0x2d4/0x338) from [<c0078328>] (kdb_parse+0x4d4/0x5f8)
[<c0078328>] (kdb_parse+0x4d4/0x5f8) from [<c0078a8c>] (kdb_main_loop+0x448/0x6b0)
[<c0078a8c>] (kdb_main_loop+0x448/0x6b0) from [<c007acb4>] (kdb_stub+0x210/0x398)
[<c007acb4>] (kdb_stub+0x210/0x398) from [<c0073280>] (kgdb_handle_exception+0x384/0x574)
[<c0073280>] (kgdb_handle_exception+0x384/0x574) from [<c0028518>] (kgdb_brk_fn+0x18/0x20)
[<c0028518>] (kgdb_brk_fn+0x18/0x20) from [<c0022198>] (do_undefinstr+0x10c/0x1a8)
[<c0022198>] (do_undefinstr+0x10c/0x1a8) from [<c0022b84>] (__und_svc+0x44/0x60)
Exception stack(0xdfe09f58 to 0xdfe09fa0)
9f40:                                                       00000000 bec93e74
9f60: 000437ac 00034738 00000001 bec93e74 00000049 00000024 c00230e8 dfe08000
9f80: 00000000 bec93e54 00000000 dfe09fa0 c0022f40 c00c9f00 80000013 ffffffff
[<c0022b84>] (__und_svc+0x44/0x60) from [<c00c9f00>] (sys_sync+0x4/0x28)
[<c00c9f00>] (sys_sync+0x4/0x28) from [<c0022f40>] (ret_fast_syscall+0x0/0x30)

This example shows an invocation of kdb, followed by setting a breakpoint, then resuming execution with 'go'. Then, at the Linux user-space shell, the 'sync' command is run to cause the breakpoint to occur. When kdb is entered due to the breakpoint, then 'bt' is run to get a backtrace from the stack of the current process.

Using gdb to see the kernel source listing

You can use the addresses printed out in kdb, with a host-side gdb session, to see the source code or assembly instructions around a particular address.

The target address can come from a backtrace or register dump (e.g. instruction pointer).

To load the source for a kernel, start gdb (or the appropriate arch-specific gdb) with the vmlinux that matches the image running on target. The kernel should have been compiled with debug symbols (CONFIG_DEBUG_KERNEL=y). gdb will start, and load the symbol information for the kernel.

Use the following commands to see various bits of information:

  • source file and line number for an instruction address
    • info line *0x<target_addr>
  • source lines around an instruction address
    • list *0x<target_addr>
  • assembly instructions at an address
    • disas 0x<target_addr>, or
    • x/20i 0x<target_addr>

KDB environment variables

  • LINES - set the number of lines for paging output from KDB