Difference between revisions of "Jetson/Cameras"

From eLinux.org
Jump to: navigation, search
(Added a Camera access section)
(Added some known-working webcams)
Line 9: Line 9:
  
  
== Interface details ==
+
= Interface details =
  
=== USB 2.0 ===
+
== USB 2.0 ==
USB 2.0 (480 Mbps) is the most common & cheapest method for camera input, since USB 2.0 webcams range from $5 - $200. However, USB 2.0 is the slowest of the possible camera interfaces, so it usually only supports upto 720p 30fps (eg: [http://www.newegg.com/Product/Product.aspx?Item=N82E16826104368 Logitech C310 $45] (untested)),(except for the few USB 2.0 cameras supporting 1080p video compression, such as [http://www.amazon.com/Logitech-HD-Pro-Webcam-C910/dp/B003M2YT96 Logitech C910] (untested)). [http://www.amazon.com/Logitech-HD-Pro-Webcam-C910/dp/B003M2YT96 Logitech C910] is known to work on Jetson TK1 in uncompressed modes but has not been tested in 1080p compressed mode.  
+
USB 2.0 (480 Mbps) is the most common & cheapest method for camera input, since USB 2.0 webcams range from $5 - $200. However, USB 2.0 is the slowest of the possible camera interfaces, so it usually only supports upto 720p 30fps (eg: [http://www.newegg.com/Product/Product.aspx?Item=N82E16826104368 Logitech C310 $45] (untested)), (except for the few USB 2.0 cameras supporting 1080p video compression, such as [http://www.amazon.com/Logitech-HD-Pro-Webcam-C910/dp/B003M2YT96 Logitech C910] (untested)). [http://www.amazon.com/Logitech-HD-Pro-Webcam-C910/dp/B003M2YT96 Logitech C910] is known to work on Jetson TK1 in uncompressed modes but has not been tested in 1080p compressed mode.  
  
==== Disabling USB auto-suspend ====
+
Typically, webcams are successfully giving the same frame-rates & resolutions on [[Jetson TK1]] as on an x86 desktop running Linux (eg: 640x480 @ 30 FPS, 960x720 @ 15 FPS, and 1920x1080 @ 15 FPS). Note that many webcams will run slower if there is low lighting, so for example, it might be 30 FPS when pointed at a bright light but only 10 FPS when pointed at a shadow.
The L4T 19.2 OS includes automatic suspending of inactive USB ports to save power. This can cause problems with some USB devices & cameras, so you might need to disable USB auto-suspend mode:
+
 
 +
=== Disabling USB auto-suspend ===
 +
Many USB devices & cameras have problems on Jetson TK1 due to automatic suspending of inactive USB ports in L4T 19.2 OS to save power. So you might need to disable USB auto-suspend mode:
 
  sudo bash -c 'echo -1 > /sys/module/usbcore/parameters/autosuspend'
 
  sudo bash -c 'echo -1 > /sys/module/usbcore/parameters/autosuspend'
 
To disable USB auto-suspend on bootup, you could put "echo -1 > /sys/module/usbcore/parameters/autosuspend" into one of your bootup scripts, such as "/etc/rc.local" with root permissions (untested).
 
To disable USB auto-suspend on bootup, you could put "echo -1 > /sys/module/usbcore/parameters/autosuspend" into one of your bootup scripts, such as "/etc/rc.local" with root permissions (untested).
  
=== USB 3.0 ===
+
=== USB 2.0 webcams known to be working ===
 +
* [http://www.myrocksoul.com/webcam-5-million-pixel.html Rocksoul WC-102 5MP]
 +
* [http://www.newegg.com/Product/Product.aspx?Item=N82E16826623075 Gearhead WC4500AFW]
 +
* [http://www.logitech.com/en-us/support/webcam-c210?bit=64&osid=14&crid=405 Logitech C210]
 +
* [http://www.logitech.com/en-us/product/hd-pro-webcam-c920 Logitech C920]
 +
* [http://www.rocketfishproducts.com/products/computer-accessories/RF-HDWEB10.html Rocketfish HD Webcam Pro]
 +
* [http://us.creative.com/p/web-cameras/live-cam-chat-hd Creative Live! Cam Chat VF0700]
 +
* [http://www.jaycar.com.au/productView.asp?ID=QC3205 Generic "Full HD 1080p" webcams using the Realtek chipset]
 +
 
 +
=== USB 2.0 webcams with known problems ===
 +
* [http://www.microsoft.com/hardware/en-us/d/lifecam-hd-5000 Microsoft Lifecam HD-5000] does not work. NVIDIA is investigating some Microsoft Lifecam models soon.
 +
 
 +
 
 +
== USB 3.0 ==
 
USB 3.0 (5 Gbps) has enough bandwidth to allow sending uncompressed 1080p video streams. But USB 3.0 hasn't reached the mass consumer market like USB 2.0 has, so USB 3.0 cameras are still limited to expensive professional-grade cameras. eg: [http://ww2.ptgrey.com/USB3/Flea3 1080p 120fps] (untested) or [http://www.e-consystems.com/8MP-USB3-Autofocus-Camera.asp 1080p 30fps] (untested).
 
USB 3.0 (5 Gbps) has enough bandwidth to allow sending uncompressed 1080p video streams. But USB 3.0 hasn't reached the mass consumer market like USB 2.0 has, so USB 3.0 cameras are still limited to expensive professional-grade cameras. eg: [http://ww2.ptgrey.com/USB3/Flea3 1080p 120fps] (untested) or [http://www.e-consystems.com/8MP-USB3-Autofocus-Camera.asp 1080p 30fps] (untested).
  
==== Enabling support for USB 3.0 ====
+
=== Enabling support for USB 3.0 ===
 
Note that early versions of [[Jetson TK1]] don't come with USB 3.0 support enabled, and thus you might need to enable USB 3.0 support and re-flash your whole L4T system. According to the [https://developer.nvidia.com/sites/default/files/akamai/mobile/files/L4T/Tegra_Linux_Driver_Package_Release_Notes_R19.2.pdf L4T R19.2 Release Notes], you should modify the odmdata settings in "jetson-tk1.conf" to enable USB 3.0:
 
Note that early versions of [[Jetson TK1]] don't come with USB 3.0 support enabled, and thus you might need to enable USB 3.0 support and re-flash your whole L4T system. According to the [https://developer.nvidia.com/sites/default/files/akamai/mobile/files/L4T/Tegra_Linux_Driver_Package_Release_Notes_R19.2.pdf L4T R19.2 Release Notes], you should modify the odmdata settings in "jetson-tk1.conf" to enable USB 3.0:
 
  # USB 2.0 operation on USB2 port(J1C2 connector)/for use as root device use ODMDATA=0x6009C000;
 
  # USB 2.0 operation on USB2 port(J1C2 connector)/for use as root device use ODMDATA=0x6009C000;
Line 29: Line 44:
 
Then flash your L4T system and reboot.
 
Then flash your L4T system and reboot.
  
=== Gigabit Ethernet ===
+
 
 +
== Gigabit Ethernet ==
 
"IP Cameras" are fairly easy to find and tend to be a good option. Gigabit Ethernet cameras (1 Gbps) has enough bandwidth for uncompressed 1080p video streams. eg: [http://ww2.ptgrey.com/GIGE GigE cameras] (untested) or regular [http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100023202%20600046262%20600046249%20600030158%204115%204114&IsNodeId=1&bop=And&Order=RATING&PageSize=20 IP network cameras]. Some IP cameras provide a URL where M-JPEG frames are available.
 
"IP Cameras" are fairly easy to find and tend to be a good option. Gigabit Ethernet cameras (1 Gbps) has enough bandwidth for uncompressed 1080p video streams. eg: [http://ww2.ptgrey.com/GIGE GigE cameras] (untested) or regular [http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100023202%20600046262%20600046249%20600030158%204115%204114&IsNodeId=1&bop=And&Order=RATING&PageSize=20 IP network cameras]. Some IP cameras provide a URL where M-JPEG frames are available.
  
=== Firewire ===
+
 
 +
== Firewire ==
 
Firewire has been used for professional video capture for a long time, since Firewire cameras do a lot of both the compression and the data transmission in hardware and thus don't put nearly as much stress on the host CPU compared to USB.
 
Firewire has been used for professional video capture for a long time, since Firewire cameras do a lot of both the compression and the data transmission in hardware and thus don't put nearly as much stress on the host CPU compared to USB.
  
=== CSI MIPI ===
+
 
 +
== CSI MIPI ==
 
CSI is the ideal method for camera input, since the cameras are tiny and the images can be processed directly by the ISP rather than needing to be stored into the computer's DRAM first before processing. eg: [http://www.e-consystems.com/70mm-MIPI-Camera-Module.asp 720p 60fps] (untested). However there isn't a single standard for CSI cameras, thus each customer typically designs their own connector that is specific to a camera module and a computer or ISP.
 
CSI is the ideal method for camera input, since the cameras are tiny and the images can be processed directly by the ISP rather than needing to be stored into the computer's DRAM first before processing. eg: [http://www.e-consystems.com/70mm-MIPI-Camera-Module.asp 720p 60fps] (untested). However there isn't a single standard for CSI cameras, thus each customer typically designs their own connector that is specific to a camera module and a computer or ISP.
  
  
== Specialized camera sensors ==
+
= Specialized camera sensors =
 
* Stereo cameras (eg: [http://www.ptgrey.com/products/stereo.asp Bumblebee 2]) (untested).
 
* Stereo cameras (eg: [http://www.ptgrey.com/products/stereo.asp Bumblebee 2]) (untested).
 
* Time-Of-Flight 3D depth "RGBD" cameras (eg: Microsoft Kinect is known to work, see details below).
 
* Time-Of-Flight 3D depth "RGBD" cameras (eg: Microsoft Kinect is known to work, see details below).
 
* Night-vision or Thermal-imaging Infrared cameras, typically with interfaces similar to standard cameras.
 
* Night-vision or Thermal-imaging Infrared cameras, typically with interfaces similar to standard cameras.
  
=== Microsoft Kinect TOF 3D Depth camera ===
+
== Microsoft Kinect TOF 3D Depth camera ==
 
Several users (eg: [https://devtalk.nvidia.com/default/topic/751916/embedded-systems/microsoft-kinect-with-jetson-tk1/post/4239497/#4239497 HellMood] have been able to use a Kinect camera on [[Jetson TK1]], after disabling the USB auto-suspend (mentioned above).
 
Several users (eg: [https://devtalk.nvidia.com/default/topic/751916/embedded-systems/microsoft-kinect-with-jetson-tk1/post/4239497/#4239497 HellMood] have been able to use a Kinect camera on [[Jetson TK1]], after disabling the USB auto-suspend (mentioned above).
  
Line 50: Line 68:
  
  
== Camera access ==
+
= Camera access =
 
To see which video capture devices you have available:
 
To see which video capture devices you have available:
 
  ls /dev/video*
 
  ls /dev/video*
Line 62: Line 80:
 
  luvcview -o out.avi -s 640x480
 
  luvcview -o out.avi -s 640x480
 
Normally, guvcview can be used to adjust default camera settings, but it seems that the default guvcview is not working correctly on ARM due to a [http://lists.infradead.org/pipermail/linux-arm-kernel/2013-June/178645.html kernel change].
 
Normally, guvcview can be used to adjust default camera settings, but it seems that the default guvcview is not working correctly on ARM due to a [http://lists.infradead.org/pipermail/linux-arm-kernel/2013-June/178645.html kernel change].
 
When testing a "Full HD 1080p" USB 2.0 webcam containing a Realtek chipset, the same speeds & resolutions worked on Jetson TK1 as on the x86 desktop running Linux (ie: 640x480 @ 30 FPS, 960x720 @ 15 FPS, and 1920x1080 @ 15 FPS). Note that many webcams will run slower if there is low lighting, so for example, it might be 30 FPS when pointed at a bright light but only 10 FPS when pointed at a shadow.
 

Revision as of 16:33, 17 June 2014

Camera interfaces on Jetson TK1

The Jetson TK1 board provides several different interfaces that allow camera input:

  • USB 3.0 port (supports many USB 2.0 or USB 3.0 webcams). Additional USB 3.0 ports can be added through mini-PCIe, eg: mini-PCIe (untested).
  • USB 2.0 port (through a micro-B to female USB-A adapter, that is sometimes included with Jetson TK1).
  • Gigabit Ethernet port. Additional Gigabit or Dual-Gigabit Ethernet ports can be added through mini-PCIE, eg: mini-PCIe (untested).
  • CSI-2 MIPI dual camera ports (one 4-lane camera port and one 1-lane camera port) through the GPIO expansion port.
  • Firewire (available through mini-PCIe), eg: mini-PCIe (untested).
  • Analog video capture cards, available through mini-PCIe (eg: mini-PCIe (untested)), USB 3.0 (eg: USB 3.0 (untested)), or USB 2.0 (eg: USB 2.0 (untested)).


Interface details

USB 2.0

USB 2.0 (480 Mbps) is the most common & cheapest method for camera input, since USB 2.0 webcams range from $5 - $200. However, USB 2.0 is the slowest of the possible camera interfaces, so it usually only supports upto 720p 30fps (eg: Logitech C310 $45 (untested)), (except for the few USB 2.0 cameras supporting 1080p video compression, such as Logitech C910 (untested)). Logitech C910 is known to work on Jetson TK1 in uncompressed modes but has not been tested in 1080p compressed mode.

Typically, webcams are successfully giving the same frame-rates & resolutions on Jetson TK1 as on an x86 desktop running Linux (eg: 640x480 @ 30 FPS, 960x720 @ 15 FPS, and 1920x1080 @ 15 FPS). Note that many webcams will run slower if there is low lighting, so for example, it might be 30 FPS when pointed at a bright light but only 10 FPS when pointed at a shadow.

Disabling USB auto-suspend

Many USB devices & cameras have problems on Jetson TK1 due to automatic suspending of inactive USB ports in L4T 19.2 OS to save power. So you might need to disable USB auto-suspend mode:

sudo bash -c 'echo -1 > /sys/module/usbcore/parameters/autosuspend'

To disable USB auto-suspend on bootup, you could put "echo -1 > /sys/module/usbcore/parameters/autosuspend" into one of your bootup scripts, such as "/etc/rc.local" with root permissions (untested).

USB 2.0 webcams known to be working

USB 2.0 webcams with known problems


USB 3.0

USB 3.0 (5 Gbps) has enough bandwidth to allow sending uncompressed 1080p video streams. But USB 3.0 hasn't reached the mass consumer market like USB 2.0 has, so USB 3.0 cameras are still limited to expensive professional-grade cameras. eg: 1080p 120fps (untested) or 1080p 30fps (untested).

Enabling support for USB 3.0

Note that early versions of Jetson TK1 don't come with USB 3.0 support enabled, and thus you might need to enable USB 3.0 support and re-flash your whole L4T system. According to the L4T R19.2 Release Notes, you should modify the odmdata settings in "jetson-tk1.conf" to enable USB 3.0:

# USB 2.0 operation on USB2 port(J1C2 connector)/for use as root device use ODMDATA=0x6009C000;
# USB 3.0 operation on USB2 port(J1C2 connector) use ODMDATA=0x6209C000, requires firmware load from userspace or initial ramdisk
ODMDATA=0x6209C000;

Then flash your L4T system and reboot.


Gigabit Ethernet

"IP Cameras" are fairly easy to find and tend to be a good option. Gigabit Ethernet cameras (1 Gbps) has enough bandwidth for uncompressed 1080p video streams. eg: GigE cameras (untested) or regular IP network cameras. Some IP cameras provide a URL where M-JPEG frames are available.


Firewire

Firewire has been used for professional video capture for a long time, since Firewire cameras do a lot of both the compression and the data transmission in hardware and thus don't put nearly as much stress on the host CPU compared to USB.


CSI MIPI

CSI is the ideal method for camera input, since the cameras are tiny and the images can be processed directly by the ISP rather than needing to be stored into the computer's DRAM first before processing. eg: 720p 60fps (untested). However there isn't a single standard for CSI cameras, thus each customer typically designs their own connector that is specific to a camera module and a computer or ISP.


Specialized camera sensors

  • Stereo cameras (eg: Bumblebee 2) (untested).
  • Time-Of-Flight 3D depth "RGBD" cameras (eg: Microsoft Kinect is known to work, see details below).
  • Night-vision or Thermal-imaging Infrared cameras, typically with interfaces similar to standard cameras.

Microsoft Kinect TOF 3D Depth camera

Several users (eg: HellMood have been able to use a Kinect camera on Jetson TK1, after disabling the USB auto-suspend (mentioned above).

Then you need to follow the Kinect Manual Installation Guide rather than the many x86 based guides, since the Tegra CPU uses ARM architecture.


Camera access

To see which video capture devices you have available:

ls /dev/video*

To see the supported video formats of your webcam:

sudo apt-get install luvcview
luvcview -d /dev/video0 -L

You can also use luvcview to record video from your webcam to a video file. It will open a GUI window showing the camera preview. Then along the middle of the bottom row there are 4 rectangle buttons, click the bottom-left of the 4 buttons to start/stop video recording, and the bottom-right button to quit. To record video using the default settings of your webcam:

luvcview -o out.avi

To record video using a custom resolution (of those shown when running luvcview -L above):

luvcview -o out.avi -s 640x480

Normally, guvcview can be used to adjust default camera settings, but it seems that the default guvcview is not working correctly on ARM due to a kernel change.