Difference between revisions of "Jetson/Cameras"

From eLinux.org
Jump to: navigation, search
m (minor edit)
(Fixed redundancy in USB title.)
Line 108: Line 108:
 
[[File:DUO3D.jpg|200px|thumb|right|DUO Stereo Camera]]
 
[[File:DUO3D.jpg|200px|thumb|right|DUO Stereo Camera]]
  
* DUO3D USB M/MLX: USB ([https://duo3d.com/product/duo-minilx-lv1 DUO MLX]) (Working/Manufatuer Supported - [https://www.youtube.com/watch?v=ZoBVCSGQXN8 DUO3D on Jetson]).  
+
* DUO3D M/MLX: USB ([https://duo3d.com/product/duo-minilx-lv1 DUO MLX]) (Working/Manufatuer Supported - [https://www.youtube.com/watch?v=ZoBVCSGQXN8 DUO3D on Jetson]).  
 
* IDS N10 Stereo: USB ([http://en.ids-imaging.com/store/n10-stereo-3d-kamera-400.html IDS N10]) (untested).
 
* IDS N10 Stereo: USB ([http://en.ids-imaging.com/store/n10-stereo-3d-kamera-400.html IDS N10]) (untested).
 
* Leopard Imaging: USB/UVC ([https://www.leopardimaging.com/LI-USB30-V024STEREO.html LI-USB30]) (untested).
 
* Leopard Imaging: USB/UVC ([https://www.leopardimaging.com/LI-USB30-V024STEREO.html LI-USB30]) (untested).
 
* Point Grey Bumblebee 2/XB3: Firewire ([http://www.ptgrey.com/products/stereo.asp Bumblebee 2]) (untested).
 
* Point Grey Bumblebee 2/XB3: Firewire ([http://www.ptgrey.com/products/stereo.asp Bumblebee 2]) (untested).
 
  
 
= Specialized camera sensors =
 
= Specialized camera sensors =

Revision as of 15:09, 6 January 2015

Camera interfaces on Jetson TK1

The Jetson TK1 board provides several different interfaces that allow camera input:

  • USB 3.0 port (supports many USB 2.0 or USB 3.0 webcams). Additional USB 3.0 ports can be added through mini-PCIe, eg: mini-PCIe (untested).
  • USB 2.0 port (through a micro-B to female USB-A adapter, that is sometimes included with Jetson TK1).
  • Gigabit Ethernet port. Additional Gigabit or Dual-Gigabit Ethernet ports can be added through mini-PCIE, eg: mini-PCIe (untested).
  • CSI-2 MIPI dual camera ports (one 4-lane camera port and one 1-lane camera port) through the GPIO expansion port.
  • Firewire (available through mini-PCIe), eg: mini-PCIe (untested).
  • Analog video capture cards, available through mini-PCIe (eg: mini-PCIe (untested)), USB 3.0 (eg: USB 3.0 (untested)), or USB 2.0 (eg: USB 2.0 (untested)).


Interface details

USB 2.0

USB 2.0 (480 Mbps) is the most common & cheapest method for camera input, since USB 2.0 webcams range from $5 - $200. However, USB 2.0 is the slowest of the possible camera interfaces, so it usually only supports upto 720p 30fps (eg: Logitech C310 $45 (untested)), (except for the few USB 2.0 cameras supporting 1080p video compression, such as Logitech C910 (untested)). Logitech C910 is known to work on Jetson TK1 in uncompressed modes but has not been tested in 1080p compressed mode.

Typically, webcams are successfully giving the same frame-rates & resolutions on Jetson TK1 as on an x86 desktop running Linux (eg: 640x480 @ 30 FPS, 960x720 @ 15 FPS, and 1920x1080 @ 15 FPS). Note that many webcams will run slower if there is low lighting, so for example, it might be 30 FPS when pointed at a bright light but only 10 FPS when pointed at a shadow.

Disabling USB auto-suspend

Some USB devices & cameras have problems on Jetson TK1 due to automatic suspending of inactive USB ports in L4T 19.2 OS to save power. So you might need to disable USB auto-suspend mode. This will disable it temporarily until you reboot:

sudo bash -c 'echo -1 > /sys/module/usbcore/parameters/autosuspend'

To automatically disable USB auto-suspend on bootup, follow the instructions to Run a command automatically on bootup.

USB 2.0 webcams known to be working

USB 2.0 webcams with known problems

Note: If a webcam or video capture card is supported by UVC or Video4Linux (see examples) then it will probably work the same on Jetson TK1 as on a desktop, but there is a possibility that it requires the firmware to be copied to your "/lib/firmware" folder or potentially even compiling & flashing your own custom Linux kernel with appropriate drivers enabled.

USB 3.0

USB 3.0 (5 Gbps) on the full-sized USB port (J1C2 connector) has enough bandwidth to allow sending uncompressed 1080p video streams. But USB 3.0 hasn't reached the mass consumer market like USB 2.0 has, so USB 3.0 cameras are mostly still limited to expensive professional-grade cameras. eg: PointGrey Flea3 1080p @ 120 FPS (untested) or e-Con See3CAM_80 1080p @ 30 FPS (working!).

Enabling support for USB 3.0 on the full-sized USB port

Note that early versions of Linux4Tegra didn't come with USB 3.0 support enabled, and thus you might need to enable USB 3.0 support and re-flash your whole L4T system. According to the L4T R19.2 Release Notes, you should modify the odmdata settings (in "pm375_release_armhf/Linux_for_Tegra/jetson-tk1.conf") to enable USB 3.0:

# Use ODMDATA=0x6009C000 for USB 2.0 operation or for use as a root device.
# Use ODMDATA=0x6209C000 for USB 3.0 operation. Requires firmware loaded from userspace or initial ramdisk.
ODMDATA=0x6209C000;

Then flash your L4T system and reboot.

USB 3.0 webcams known to be working

See3CAM_80 USB 3.0 1080p Camera
  • e-Con See3CAM_80 (~$90) has been tested as fully working on Jetson TK1 for Full-HD 1920x1080 @ 30 FPS (see video) by Korneliusz Jarzębski. It requires flashing a custom L4T Linux kernel that has the CONFIG_HIDRAW setting enabled. This camera board has good Linux support and contains an 8 Mega-Pixel MIPI camera with auto-focus ability and an onboard ISP including 10 GPIOs that can potentially be used for other purposes besides flash & camera activation.






Gigabit Ethernet

TRENDnet IP camera for just $120

"IP Cameras" are fairly easy to find and are a good option for single or multiple cameras. Gigabit Ethernet (1 Gbps) has enough bandwidth for uncompressed 1080p video streams. eg: GigE cameras (untested) or regular IP network cameras. Nico notes that older GigE cameras typically do not apply any compression, since compression is a feature of the newer GigE Vision 2 standard, and thus most industrial GigE cameras for computer vision don't have compression and send Bayer images instead. IP cameras typically have M-JPEG compression as well as a video streaming server such as by providing a URL where M-JPEG or JPEG frames are available.





Firewire

DFK31BF03 Firewire camera

Firewire has been used for professional video capture for a long time, since Firewire cameras do a lot of both the compression and the data transmission in hardware and thus don't put nearly as much stress on the host CPU compared to USB.





CSI MIPI

CSI camera such as in a smartphone

CSI-2 MIPI is the most efficient method for camera input, since the cameras are tiny and the images can be processed directly by the ISP rather than needing to be stored into the computer's DRAM first before processing. However using a CSI MIPI camera directly requires much more work than using a USB, Ethernet or Firewire camera.

Licensing costs of the ISP has been reported as the constraint as it is only generally viable for millions of units.

An untested example is 720p 60fps. However there isn't a single standard for CSI cameras, thus each customer typically designs their own connector that is specific to a camera module and a computer or ISP. No CSI MIPI camera has currently been tested on Jetson TK1.



Multiple Cameras

There are several ways to handle multiple cameras on Jetson TK1 at the same time:

  • Ethernet: multiple IP cameras on a network (eg: several TRENDnet Internet Cameras (untested) at $118 each). Theoretically you could have 6 x FullHD 1080p 30fps cameras using 3 x Gigabit Ethernet ports on Jetson TK1 (discussed on the forum)!
  • CSI MIPI: 2 cameras (eg: a front camera and a rear camera (untested)).


Stereo Cameras

DUO Stereo Camera

Specialized camera sensors

  • Time-Of-Flight 3D depth "RGBD" cameras (eg: Microsoft Kinect is known to work, see details below).
  • Night-vision or Thermal-imaging Infrared cameras, typically with interfaces similar to standard cameras.

Microsoft Kinect Structured-Light 3D Depth camera

MS Kinect 3D camera

Several users (eg: HellMood) have been able to use a Kinect camera on Jetson TK1, after disabling the USB auto-suspend (mentioned above).

Then you need to follow the Kinect Manual Installation Guide rather than the many x86 based guides, since the Tegra CPU uses ARM architecture.

User HellMood also posted a video showing the new Kinect camera working on Jetson TK1: MoodShades Preview.

Asus XTion Pro Live RGB-D Sensor using OpenNI2 library

MS Kinect 3D camera

Asus XTion Pro Live is a RGB-D sensor based on the Primesense's patent (Primesense has been acquired and deleted by Apple in the end of 2013) that drives the first Microsoft Kinect. Asus Xtion sensor, differently from Microsoft Kinect, can be powered directly by USB cable not requiring an external power source.

You can find a guide to compile OpenNI2 library for Jetson TK1 and use it to interface Asus XTion Pro Live on the blog of the robotic project MyzharBot.

Camera access

To see which video capture devices you have available:

ls /dev/video*

To see the supported video formats of your webcam:

sudo apt-get install luvcview
luvcview -d /dev/video0 -L

You can also use luvcview to record video from your webcam to a video file. It will open a GUI window showing the camera preview. Then along the middle of the bottom row there are 4 rectangle buttons, click the bottom-left of the 4 buttons to start/stop video recording, and the bottom-right button to quit. To record video using the default settings of your webcam:

luvcview -o out.avi

To record video using a custom resolution (of those shown when running luvcview -L above):

luvcview -o out.avi -s 640x480

Normally, guvcview can be used to adjust default camera settings, but it seems that the default guvcview is not working correctly on ARM due to a kernel change.