Difference between revisions of "BeagleBoard/GSoC/2010 Projects/USBSniffer"

From eLinux.org
Jump to: navigation, search
(MUSB testing code)
 
(32 intermediate revisions by 3 users not shown)
Line 2: Line 2:
  
 
* Student: Nicolas Boichat
 
* Student: Nicolas Boichat
* Mentors: Hunyue Yau, Laine Walker-Avina, Frans Meulenbroeks
+
* Mentors: Hunyue Yau, Frans Meulenbroeks
  
 
BeagleBoard project: [http://beagleboard.org/project/usbsniffer/ usbsniffer]
 
BeagleBoard project: [http://beagleboard.org/project/usbsniffer/ usbsniffer]
Line 14: Line 14:
 
The goal of this project is to use the BeagleBoard as an USB sniffer. The host computer would be connected to the slave USB port of the BeagleBoard, and the device to be sniffed on the host USB port.
 
The goal of this project is to use the BeagleBoard as an USB sniffer. The host computer would be connected to the slave USB port of the BeagleBoard, and the device to be sniffed on the host USB port.
  
The BeagleBoard would then forward USB data, while logging it.
+
The BeagleBoard would then forward USB data, while logging it. In basic terms, there is a proxy driver running on the BeagleBoard, that acts both as a USB slave, and claims a USB device. According to the device descriptor, endpoints are activated on the USB slave controller, and packets are then forwarded back and forth between the host and the device.
  
This presents the following advantages over a software-based solution: No software modification is required; support of proprietary OSes; allows debugging of new USB stacks; and possibly lower-level debugging of USB frames...
+
This presents the following advantages over a software-based solution: No software modification is required; support of proprietary OSes; allows debugging of new USB stacks... However, this doesn't allow low-level debugging of USB transactions, that a hardware solution would allow. Also, the solution can only be as reliable as the USB host controller driver (which is well tested), and as the USB device controller driver (the MUSB driver in the Linux still has some bugs).
  
== Build and run instructions ==
+
== Build instructions ==
 +
 
 +
Latest recommended branches:
 +
* Kernel tree:
 +
**<tt>stable-20100726</tt> [http://gitorious.org/beagleboard-usbsniffer/beagleboard-usbsniffer-kernel/commits/stable-20100726], OR
 +
**<tt>stable-20100809</tt> [http://gitorious.org/beagleboard-usbsniffer/beagleboard-usbsniffer-kernel/commits/stable-20100809]: this version may fix some problems with some devices, but will affect performance, for example when used with USB mass storage devices.
 +
* Helper scripts: <tt>stable-20100730</tt> [http://gitorious.org/beagleboard-usbsniffer/helper-scripts/commits/stable-20100730].
  
 
To get the proxy driver to work, you need to follow these steps:
 
To get the proxy driver to work, you need to follow these steps:
 +
* Clone my kernel git tree. This can be done using the following commands:
 +
git clone git://gitorious.org/beagleboard-usbsniffer/beagleboard-usbsniffer-kernel.git
 +
cd beagleboard-usbsniffer-kernel
 +
git checkout origin/<i>branch</i> -b <i>branch</i>
 +
* Do not reconfigure the kernel (unless you need some extra modules): the git tree comes with a ready-made <tt>.config</tt>.
 +
* Compile and install the kernel.
 +
** Make sure your environment is set properly (at least <tt>CROSS_COMPILE=arm-angstrom-linux-gnueabi-</tt> and <tt>ARCH=arm</tt> should be set)
 +
** Run <tt>make uImage</tt>.
 +
** Copy the resulting <tt>uImage</tt> on the SD card.
 +
* Compile and install the kernel modules
 +
** <tt>make modules</tt>
 +
** To install the modules, the easiest is probably to set <tt>INSTALL_MOD_PATH</tt> to some directory on your host computer, run <tt>make modules_install</tt>, and copy the modules to the SD card, or via the network.
 +
** Note: in some cases, I had problem with the kernel not finding modules. In that case, run <tt>depmod -a</tt> on the BeagleBoard, and reboot.
 +
* Install libpcap-1.1.1 and tcpdump-4.1.1.
 +
** If you don't have a recent enough OpenEmbedded install, the recipes can be found in these 2 commits: [http://git.openembedded.org/cgit.cgi/openembedded/commit/?id=7b9e14891f7d69b5376041fc15df3d5f13f41855] and [http://git.openembedded.org/cgit.cgi/openembedded/commit/?id=d4f0fb310f7d40f7a50f50fb12083fa258aa1eed]: apply these 2 commits, or update your OpenEmbedded distribution to the latest git.
 +
** Build libpcap and tcpdump, this can be done with a command like <tt>bitbake libpcap tcpdump</tt> provided you have the environment set properly (i.e., source <tt>~/.oe/environment</tt> or use <tt>oebb.sh</tt>).
 +
** The 2 packages can be found in <tt>$OE_BASE/build/tmp-angstrom_2008_1/deploy/glibc/ipk/armv7a</tt>: <tt>libpcap_1.1.1-r1.5_armv7a.ipk</tt> and <tt>tcpdump_4.1.1-r1.5_armv7a.ipk</tt>.
 +
** Copy these on the BeagleBoard, and run <tt>opkg install <i>name</i>.ipk</tt> for both packages.
 +
* Clone the helper scripts git tree:
 +
git clone git://gitorious.org/beagleboard-usbsniffer/helper-scripts.git
 +
cd helper-scripts/
 +
git checkout origin/<i>branch</i> -b <i>branch</i>
 +
* Copy the content of the <tt>arm</tt> directory to the BeagleBoard
 +
 +
== Usage instructions ==
 +
 +
Then, you have 2 options, the automatic way:
 +
* Plug your device (through a USB hub if it is a low/full-speed device).
 +
* Run <tt>./sniff</tt>, and follow the instructions. Data transfers will be logged to <tt>/media/ram/dump</tt>. This resulting file can be displayed using wireshark.
 +
* Use the device, it should work, and packets are captured.
  
* Clone my kernel git tree. Use the <tt>stable-20100702</tt> branch.
+
or the manual way (mostly for testing purpose, as it does not log packets):
* Compile the kernel with the default beagleboard configuration (see [[BeagleBoard#Linux_kernel]]). You just need to add <tt>CONFIG_USB_G_PROXY=m</tt>. I also disabled MUSB in host and OTG mode, as well as USB suspend, but this may not be necessary.
+
* Install the new kernel on the board.
+
* Clone the helper scripts git tree, and copy the content of the arm directory to the BeagleBoard (you need to modify load/setup scripts if you do not have have a copy of <tt>musb_hdrc.ko</tt> and <tt>g_proxy.ko</tt> in the same directory).
+
 
* Run <tt>./setup</tt> on the BeagleBoard, this will unload the <tt>g_ether</tt> gadget driver.
 
* Run <tt>./setup</tt> on the BeagleBoard, this will unload the <tt>g_ether</tt> gadget driver.
 
* Plug your device (through a USB hub if it is a low/full-speed device).
 
* Plug your device (through a USB hub if it is a low/full-speed device).
 
* Plug your PC to the BeagleBoard USB slave port (this can be done earlier as well).
 
* Plug your PC to the BeagleBoard USB slave port (this can be done earlier as well).
 
* Run <tt>./unbind</tt>: This will unbind the device from the normal Linux driver.
 
* Run <tt>./unbind</tt>: This will unbind the device from the normal Linux driver.
* Run <tt>./load</tt>: this will (re)load the g_proxy driver.
+
* Run <tt>./load</tt>: this will (re)load the <tt>g_proxy</tt> driver.
 
* Use the device, it should work.
 
* Use the device, it should work.
  
Line 74: Line 107:
 
|style="background-color:#00ff00; font-style:bold" align="center"|OK
 
|style="background-color:#00ff00; font-style:bold" align="center"|OK
 
|<tt>>=stable-20100618</tt>
 
|<tt>>=stable-20100618</tt>
|Scanning devices (<tt>hcitool -i hci1 scan</tt>), and file transfers (<tt>odp</tt> and <tt>ussp-push</tt>) work.
+
|Scanning devices (<tt>hcitool -i hci1 scan</tt>), and file transfers (<tt>odp</tt> and <tt>ussp-push</tt>) work. Didn't check anything that would use isochronous endpoints.
 
|-
 
|-
 
|Imation 4GB Flash Drive||<tt>0718:0348</tt>
 
|Imation 4GB Flash Drive||<tt>0718:0348</tt>
Line 105: Line 138:
 
* EP3 IN, isochronous (audio)
 
* EP3 IN, isochronous (audio)
 
|style="background-color:#c0ff00; font-style:bold" align="center"|OK<br>(audio+video at the same time doesn't work)
 
|style="background-color:#c0ff00; font-style:bold" align="center"|OK<br>(audio+video at the same time doesn't work)
|<tt>stable-20100717</tt>
+
|<tt>>stable-20100717</tt>
 
| No visible problem (video 640x480 OR audio). Needs <tt>musb_hdrc</tt> parameter <tt>fifo_mode=6</tt>. The FIFO mode is required, because EP1 needs 768 bytes per packet (and the default is only 512). The DMA has been fixed to handle ISO packets properly.<br>
 
| No visible problem (video 640x480 OR audio). Needs <tt>musb_hdrc</tt> parameter <tt>fifo_mode=6</tt>. The FIFO mode is required, because EP1 needs 768 bytes per packet (and the default is only 512). The DMA has been fixed to handle ISO packets properly.<br>
 
Enabling both video + audio at the same time doesn't work (bandwidth allocation problem, but this is probably an USB hub problem).
 
Enabling both video + audio at the same time doesn't work (bandwidth allocation problem, but this is probably an USB hub problem).
Line 119: Line 152:
 
|<tt>>stable-20100717</tt>
 
|<tt>>stable-20100717</tt>
 
| No visible problem (audio loopback with something like: <tt>arecord -D default:CARD=Headset -f dat |  aplay -D default:CARD=Headset -</tt> works). Needs <tt>musb_hdrc</tt> parameter <tt>fifo_mode=7</tt>. The FIFO mode is required, because EP4 needs 96 bytes per packet (and the default is only 32).
 
| No visible problem (audio loopback with something like: <tt>arecord -D default:CARD=Headset -f dat |  aplay -D default:CARD=Headset -</tt> works). Needs <tt>musb_hdrc</tt> parameter <tt>fifo_mode=7</tt>. The FIFO mode is required, because EP4 needs 96 bytes per packet (and the default is only 32).
 +
|-
 +
|[http://fpga4u.epfl.ch FPGA4U], without firmware (Cypress Semiconductor Corp. CY7C68013 EZ-USB FX2 USB 2.0 Development Kit)||<tt>04b4:8613</tt>
 +
|align="center"|HS
 +
|
 +
* Many, for testing (only control is used for firmware loading)
 +
|style="background-color:#00ff00; font-style:bold" align="center"|OK
 +
|<tt>>=stable-20100809</tt>
 +
| Firmware loading using fxload works. The device then resets, and appears as a new device on the BeagleBoard. Restarting <tt>sniff</tt> claims the new device.
 +
|-
 +
|[http://fpga4u.epfl.ch FPGA4U], with firmware (Altera compatible)||<tt>09fb:6001</tt>
 +
|align="center"|HS
 +
|
 +
* EP1 IN, bulk (JTAG emulation)
 +
* EP2 OUT, bulk (JTAG emulation)
 +
|style="background-color:#00ff00; font-style:bold" align="center"|OK
 +
|<tt>>=stable-20100809</tt>
 +
| <tt>nios2-terminal</tt> works.
 
|-
 
|-
 
|}
 
|}
 +
 +
== Known issues ==
 +
 +
=== Bandwidth allocation ===
 +
 +
One major problem with the device emulation model, using a proxy driver, is about bandwidth allocation.
 +
 +
The USB is polling-based, that is, the controller regularly interrogates all the endpoints of all the devices attached to it.
 +
 +
When a new USB device is added, some bandwidth is allocated for each of its endpoints, depending on some polling interval and packet size defined in the device descriptor. This bandwidth allocation defines the polling schedule, across all the endpoints, for all the devices present on the bus.
 +
 +
If insufficient bandwidth is available, the device may not work. For isochronous endpoints, the device usually defines several interfaces, with different bandwidth requirements (i.e., different packet sizes), and the driver can choose the appropriate interface, according to the bandwidth available on the bus.
 +
 +
One of the problems that could occur with our proxy driver, is that the host may decide to allocate more bandwidth than the bandwidth available on the USB host controller of the BeagleBoard. In that case, bandwidth allocation may fail on the BeagleBoard (with an <tt>ENOSPC</tt>, or <tt>-28</tt> error), and transfers on the affected endpoint would not work. From my experience, this seems to happen only with full-speed devices, that cannot be connected directly to the BeagleBoard, and need an intermediate USB hub.
  
 
== MUSB testing code ==
 
== MUSB testing code ==
Line 151: Line 215:
 
  ... S Bi:2:101:1 -115 1024 <
 
  ... S Bi:2:101:1 -115 1024 <
 
  ... C Bi:2:101:1 -121 514 = 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 
  ... C Bi:2:101:1 -121 514 = 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 +
 +
 +
[[Category:BeagleBoard]]
 +
[[Category:GSoC]]

Latest revision as of 00:31, 4 February 2014

Google Summer of Code 2010 Project

  • Student: Nicolas Boichat
  • Mentors: Hunyue Yau, Frans Meulenbroeks

BeagleBoard project: usbsniffer

Project at gitorious.org: http://gitorious.org/beagleboard-usbsniffer

Blog: http://beagleboard-usbsniffer.blogspot.com/ (RSS)

Abstract

The goal of this project is to use the BeagleBoard as an USB sniffer. The host computer would be connected to the slave USB port of the BeagleBoard, and the device to be sniffed on the host USB port.

The BeagleBoard would then forward USB data, while logging it. In basic terms, there is a proxy driver running on the BeagleBoard, that acts both as a USB slave, and claims a USB device. According to the device descriptor, endpoints are activated on the USB slave controller, and packets are then forwarded back and forth between the host and the device.

This presents the following advantages over a software-based solution: No software modification is required; support of proprietary OSes; allows debugging of new USB stacks... However, this doesn't allow low-level debugging of USB transactions, that a hardware solution would allow. Also, the solution can only be as reliable as the USB host controller driver (which is well tested), and as the USB device controller driver (the MUSB driver in the Linux still has some bugs).

Build instructions

Latest recommended branches:

  • Kernel tree:
    • stable-20100726 [1], OR
    • stable-20100809 [2]: this version may fix some problems with some devices, but will affect performance, for example when used with USB mass storage devices.
  • Helper scripts: stable-20100730 [3].

To get the proxy driver to work, you need to follow these steps:

  • Clone my kernel git tree. This can be done using the following commands:
git clone git://gitorious.org/beagleboard-usbsniffer/beagleboard-usbsniffer-kernel.git
cd beagleboard-usbsniffer-kernel
git checkout origin/branch -b branch
  • Do not reconfigure the kernel (unless you need some extra modules): the git tree comes with a ready-made .config.
  • Compile and install the kernel.
    • Make sure your environment is set properly (at least CROSS_COMPILE=arm-angstrom-linux-gnueabi- and ARCH=arm should be set)
    • Run make uImage.
    • Copy the resulting uImage on the SD card.
  • Compile and install the kernel modules
    • make modules
    • To install the modules, the easiest is probably to set INSTALL_MOD_PATH to some directory on your host computer, run make modules_install, and copy the modules to the SD card, or via the network.
    • Note: in some cases, I had problem with the kernel not finding modules. In that case, run depmod -a on the BeagleBoard, and reboot.
  • Install libpcap-1.1.1 and tcpdump-4.1.1.
    • If you don't have a recent enough OpenEmbedded install, the recipes can be found in these 2 commits: [4] and [5]: apply these 2 commits, or update your OpenEmbedded distribution to the latest git.
    • Build libpcap and tcpdump, this can be done with a command like bitbake libpcap tcpdump provided you have the environment set properly (i.e., source ~/.oe/environment or use oebb.sh).
    • The 2 packages can be found in $OE_BASE/build/tmp-angstrom_2008_1/deploy/glibc/ipk/armv7a: libpcap_1.1.1-r1.5_armv7a.ipk and tcpdump_4.1.1-r1.5_armv7a.ipk.
    • Copy these on the BeagleBoard, and run opkg install name.ipk for both packages.
  • Clone the helper scripts git tree:
git clone git://gitorious.org/beagleboard-usbsniffer/helper-scripts.git
cd helper-scripts/
git checkout origin/branch -b branch
  • Copy the content of the arm directory to the BeagleBoard

Usage instructions

Then, you have 2 options, the automatic way:

  • Plug your device (through a USB hub if it is a low/full-speed device).
  • Run ./sniff, and follow the instructions. Data transfers will be logged to /media/ram/dump. This resulting file can be displayed using wireshark.
  • Use the device, it should work, and packets are captured.

or the manual way (mostly for testing purpose, as it does not log packets):

  • Run ./setup on the BeagleBoard, this will unload the g_ether gadget driver.
  • Plug your device (through a USB hub if it is a low/full-speed device).
  • Plug your PC to the BeagleBoard USB slave port (this can be done earlier as well).
  • Run ./unbind: This will unbind the device from the normal Linux driver.
  • Run ./load: this will (re)load the g_proxy driver.
  • Use the device, it should work.

Tested devices

Device USB ID Speed Endpoints & types Status Branch Notes
Logitech (M-BJ58/M-BJ69) Optical Wheel Mouse 046d:c00e/046d:c018 LS
  • EP1 IN, interrupt
OK >=stable-20100618
Logitech Internet Keyboard 046d:c309 LS
  • EP1 IN, interrupt (keyboard itself)
  • EP2 IN, interrupt (wheel on the side, like a mouse)
OK stable-20100618
FTDI FT232 USB-Serial (on Arduino) 0403:6001 FS
  • EP1 IN, bulk (Arduino->host)
  • EP2 OUT, bulk (host->Arduino)
OK >=stable-20100618 Programming + bidirectional serial communication
CSR Bluetooth Dongle 0a12:0001 FS
  • EP1 IN, interrupt
  • EP2 IN, bulk
  • EP2 OUT, bulk
  • EP3 IN, iso
  • EP3 OUT, iso
OK >=stable-20100618 Scanning devices (hcitool -i hci1 scan), and file transfers (odp and ussp-push) work. Didn't check anything that would use isochronous endpoints.
Imation 4GB Flash Drive 0718:0348 HS
  • EP1 OUT, bulk
  • EP2 IN, bulk
  • EP3 IN, interrupt
OK >=stable-20100702 Works hdparm -t --direct /dev/sdX. On PC: 19.76 MB/sec; on BeagleBoard: 17.64 MB/sec; on PC through the BeagleBoard proxy: 19.18 MB/sec.
AVerMedia Volar DVB-T dongle 07ca:b808 HS
  • EP1 OUT, bulk
  • EP1 IN, bulk
  • EP2 IN, bulk
  • EP3 IN, bulk
OK >=stable-20100618 No visible problem.
Logitech, Inc. QuickCam Messanger (sic) 046d:08da FS
  • EP1 IN, isochronous (video)
  • EP2 IN, interrupt (button status)
  • EP3 IN, isochronous (audio)
OK
(audio+video at the same time doesn't work)
>stable-20100717 No visible problem (video 640x480 OR audio). Needs musb_hdrc parameter fifo_mode=6. The FIFO mode is required, because EP1 needs 768 bytes per packet (and the default is only 512). The DMA has been fixed to handle ISO packets properly.

Enabling both video + audio at the same time doesn't work (bandwidth allocation problem, but this is probably an USB hub problem).

Logitech, Inc. Premium Stereo USB Headset 350 046d:0a02 FS
  • EP1 OUT, isochronous (audio stereo output, max 192 bytes per packet)
  • EP3 IN, interrupt (button status)
  • EP4 IN, isochronous (audio input, 96 bytes per packet)
OK >stable-20100717 aplay -D default:CARD=Headset - works). Needs musb_hdrc parameter fifo_mode=7. The FIFO mode is required, because EP4 needs 96 bytes per packet (and the default is only 32).
FPGA4U, without firmware (Cypress Semiconductor Corp. CY7C68013 EZ-USB FX2 USB 2.0 Development Kit) 04b4:8613 HS
  • Many, for testing (only control is used for firmware loading)
OK >=stable-20100809 Firmware loading using fxload works. The device then resets, and appears as a new device on the BeagleBoard. Restarting sniff claims the new device.
FPGA4U, with firmware (Altera compatible) 09fb:6001 HS
  • EP1 IN, bulk (JTAG emulation)
  • EP2 OUT, bulk (JTAG emulation)
OK >=stable-20100809 nios2-terminal works.

Known issues

Bandwidth allocation

One major problem with the device emulation model, using a proxy driver, is about bandwidth allocation.

The USB is polling-based, that is, the controller regularly interrogates all the endpoints of all the devices attached to it.

When a new USB device is added, some bandwidth is allocated for each of its endpoints, depending on some polling interval and packet size defined in the device descriptor. This bandwidth allocation defines the polling schedule, across all the endpoints, for all the devices present on the bus.

If insufficient bandwidth is available, the device may not work. For isochronous endpoints, the device usually defines several interfaces, with different bandwidth requirements (i.e., different packet sizes), and the driver can choose the appropriate interface, according to the bandwidth available on the bus.

One of the problems that could occur with our proxy driver, is that the host may decide to allocate more bandwidth than the bandwidth available on the USB host controller of the BeagleBoard. In that case, bandwidth allocation may fail on the BeagleBoard (with an ENOSPC, or -28 error), and transfers on the affected endpoint would not work. From my experience, this seems to happen only with full-speed devices, that cannot be connected directly to the BeagleBoard, and need an intermediate USB hub.

MUSB testing code

Some instructions on how to use the code to trigger the MUSB bug with short isochronous packets:

Checkout [6]. There are 2 directories: host, and device.

For the device side (on the beagleboard), you need to cross-compile usbtest. You need libaio, which can be built using Angstrom, and gadgetfs in the Linux kernel. Then, the gadgetfs driver is loaded as follows:

modprobe gadgetfs
mkdir /dev/gadget/ -p
mount -t gadgetfs none /dev/gadget
./usbtest -v -s 512 -p 2 -a 1 -I0 -x 18

The host code must be run on your host PC. It requires the usbtest module (CONFIG_USB_TEST=m), then isochronous IN transfers are tested with the following command:

./testusb -a -t 16 -g 1 -c 10

While running the test, you can monitor the USB traffic using usbmon, you should see isochronous packets of length 18, i.e., something like this:

... S Zi:2:100:1 -115:8:5232 1 -18:0:512 512 <
... C Zi:2:100:1 0:8:5240:0 1 0:0:18 18 = e6010203 e6050607 e6090a0b e60d0e0f e611

Every 4 bytes contains some kind of packet id (incrementing), the rest of the bytes are given by a mod 63 counter.

Bulk transfers can also be tested, with the following commands (device and host):

./usbtest -a 5 -s 514
./testusb -a -t 4 -c 10 -s 1024

testusb will complain, since the packet is short (514 instead of 1024), but usbmon still shows that the transfer has been done correctly:

... S Bi:2:101:1 -115 1024 <
... C Bi:2:101:1 -121 514 = 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000