Revision as of 13:28, 26 May 2022

This page contains instructions for installing various open source add-on packages and frameworks on NVIDIA Jetson, in addition to a collection of DNN models for inferencing.

Below are links to container images and precompiled binaries built for aarch64 (arm64) architecture. These are intended to be installed on top of JetPack.

Note that JetPack comes with various pre-installed components such as the L4T kernel, CUDA Toolkit, cuDNN, TensorRT, VisionWorks, OpenCV, GStreamer, Docker, and more.

Machine Learning

Jetson is able to natively run the full versions of popular machine learning frameworks, including TensorFlow, PyTorch, Caffe2, Keras, and MXNet.

There are also helpful deep learning examples and tutorials available, created specifically for Jetson - like Hello AI World and JetBot.

Docker Containers

There are ready-to-use ML and data science containers for Jetson hosted on NVIDIA GPU Cloud (NGC), including the following:

l4t-tensorflow - TensorFlow for JetPack 4.4 (and newer)
l4t-pytorch - PyTorch for JetPack 4.4 (and newer)
l4t-ml - TensorFlow, PyTorch, scikit-learn, scipy, pandas, JupyterLab, ect.

If you wish to modify them, the Dockerfiles and build scripts for these containers can be found on GitHub.

There are also the following ready-to-use container images for Jetson hosted on DockerHub and third-party registries:

ROS/ROS2 containers: https://github.com/dusty-nv/jetson-containers
ONNX Runtime for Jetson: mcr.microsoft.com/azureml/onnxruntime:v.1.4.0-jetpack4.4-l4t-base-r32.4.3

These containers are highly recommended to reduce the installation time of the frameworks below, and for beginners getting started.

TensorFlow

Website: https://tensorflow.org
Source: https://github.com/tensorflow/tensorflow
Container: l4t-tensorflow
Version: 1.15, 2.x (Python 3.6/3.8)
Packages:

	TensorFlow 1.x	TensorFlow 2.x
JetPack 5.0	1.15.5 pip wheel	2.8 pip wheel
JetPack 4.6	1.15.5 pip wheel	2.5 pip wheel
JetPack 4.5	1.15.5 pip wheel	2.5 pip wheel
JetPack 4.4.1	1.15.3 pip wheel	2.3 pip wheel
JetPack 4.4	1.15.2 pip wheel	2.2 pip wheel
JetPack 4.4 DP	1.15.2 pip wheel	2.1 pip wheel
JetPack 4.3	1.15.2 pip wheel	2.1 pip wheel

Supports: JetPack >= 4.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier / AGX Orin)
Install Guide: Installing TensorFlow on Jetson
Forum Topic: devtalk.nvidia.com/default/topic/1048776/jetson-nano/official-tensorflow-for-jetson-nano-/
Build from Source: https://devtalk.nvidia.com/default/topic/1055131/jetson-agx-xavier/building-tensorflow-1-13-on-jetson-xavier/

# install prerequisites
$ sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran

# install and upgrade pip3
$ sudo apt-get install python3-pip
$ sudo pip3 install -U pip testresources setuptools=49.6.0 

# install the following python packages
$ sudo pip3 install -U numpy==1.19.4 future==0.18.2 mock==3.0.5 h5py==2.10.0 keras_preprocessing==1.1.1 keras_applications==1.0.8 gast==0.2.2 futures protobuf pybind11

# to install TensorFlow 1.15 for JetPack 4.6:
$ sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 'tensorflow<2'

# or install the latest version of TensorFlow (2.x) for JetPack 4.6:
$ sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow

# or install the latest version of TensorFlow (2.x) for JetPack 5.0:
$ sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v50 tensorflow

PyTorch (Caffe2)

Website: https://pytorch.org/
Source: https://github.com/pytorch/pytorch
Container: l4t-pytorch
Version: PyTorch v1.0.0 - v1.12.0
Packages:

JetPack 5.0 Developer Preview

	Python 3.8
v1.11.0	pip wheel
v1.12.0	pip wheel

JetPack 4.4 / 4.4.1 / 4.5 / 4.5.1 / 4.6

	Python 3.6
v1.6.0	pip wheel
v1.7.0	pip wheel
v1.8.0	pip wheel
v1.9.0	pip wheel
v1.10.0	pip wheel

JetPack 4.4 Developer Preview

	Python 2.7	Python 3.6
v1.2.0	pip wheel	pip wheel
v1.3.0	pip wheel	pip wheel
v1.4.0	pip wheel	pip wheel
v1.5.0		pip wheel

JetPack 4.2 / 4.3

	Python 2.7	Python 3.6
v1.0.0	pip wheel	pip wheel
v1.1.0	pip wheel	pip wheel
v1.2.0	pip wheel	pip wheel
v1.3.0	pip wheel	pip wheel
v1.4.0	pip wheel	pip wheel

As per the PyTorch Release Notes, Python 2 is not longer supported as of PyTorch v1.5 and newer.
Supports: JetPack >= 4.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier / AGX Orin)
Forum Topic: devtalk.nvidia.com/default/topic/1049071/jetson-nano/pytorch-for-jetson-nano/
Build from Source: https://devtalk.nvidia.com/default/topic/1049071/#5324123

note — the PyTorch and Caffe2 projects have merged, so installing PyTorch will also install Caffe2

# Python 2.7 (download pip wheel from above)
$ sudo apt-get install libopenblas-base libopenmpi-dev
$ pip install future torch-1.4.0-cp27-cp27mu-linux_aarch64.whl

# Python 3.6 (download pip wheel from above)
$ sudo apt-get install libopenblas-base libopenmpi-dev libomp-dev python3-pip
$ pip3 install Cython
$ pip3 install numpy torch-1.6.0-cp36-cp36m-linux_aarch64.whl

As per the the PyTorch 1.4 Release Notes, Python 2 support is now deprecated and PyTorch 1.4 is the last version to support Python 2.

ONNX Runtime

Website: https://microsoft.github.io/onnxruntime/
Source: https://github.com/microsoft/onnxruntime
Container: mcr.microsoft.com/azureml/onnxruntime:v.1.4.0-jetpack4.4-l4t-base-r32.4.3
Version: 1.4.0, 1.5.2, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0
Packages:

JetPack 4.4 / 4.4.1 / 4.5 / 4.5.1 / 4.6 / 4.6.1

	Python 3.6	Python 3.7	Python 3.8	Python 3.9
onnxruntime 1.4.0	pip wheel
onnxruntime 1.5.2	pip wheel
onnxruntime 1.6.0	pip wheel	pip wheel	pip wheel
onnxruntime 1.7.0	pip wheel	pip wheel	pip wheel	pip wheel
onnxruntime 1.8.0	pip wheel	pip wheel	pip wheel	pip wheel
onnxruntime 1.9.0	pip wheel	pip wheel	pip wheel	pip wheel
onnxruntime 1.10.0	pip wheel	pip wheel	pip wheel	pip wheel
onnxruntime 1.11.0	pip wheel	pip wheel	pip wheel	pip wheel

Supports: JetPack >= 4.4 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
Forum Support: https://github.com/microsoft/onnxruntime
Build from Source: Refer to these instructions
ONNX Runtime 1.10.0 Install instructions

# Download pip wheel from location mentioned above
$ wget https://nvidia.box.com/shared/static/jy7nqva7l88mq9i8bw3g3sklzf4kccn2.whl -O onnxruntime_gpu-1.10.0-cp36-cp36m-linux_aarch64.whl

# Install pip wheel
$ pip3 install onnxruntime_gpu-1.10.0-cp36-cp36m-linux_aarch64.whl

MXNet

Website: https://mxnet.apache.org/
Source: https://github.com/apache/incubator-mxnet
Version: 1.4, 1.6, 1.7
Packages:

	Python 2.7	Python 3.6
v1.4 (JetPack 4.2.x)	pip wheel	pip wheel
v1.6 (JetPack 4.3)	pip wheel	pip wheel
v1.7 (JetPack 4.4)	-	forum post

Supports: JetPack >= 4.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
Forum Topics: v1.4 | v1.6 | v1.7
Build from Source: v1.6 | v1.7

MXNet 1.7 Install Instructions:

$ wget https://raw.githubusercontent.com/AastaNV/JEP/master/MXNET/autoinstall_mxnet.sh
$ sudo chmod +x autoinstall_mxnet.sh
$ ./autoinstall_mxnet.sh <Nano/TX1/TX2/Xavier>

MXNet 1.4 / 1.6 Install Instructions:

# Python 2.7
sudo apt-get install -y git build-essential libatlas-base-dev libopencv-dev graphviz python-pip
sudo pip install mxnet-1.4.0-cp27-cp27mu-linux_aarch64.whl

# Python 3.6
sudo apt-get install -y git build-essential libatlas-base-dev libopencv-dev graphviz python3-pip
sudo pip install mxnet-1.4.0-cp36-cp36m-linux_aarch64.whl

Keras

Website: https://keras.io/
Source: https://github.com/keras-team/keras
Version: 2.2.4
Forum Topic: https://devtalk.nvidia.com/default/topic/1049362/#5325752

First, install TensorFlow from above.

# beforehand, install TensorFlow (https://eLinux.org/Jetson_Zoo#TensorFlow)
$ sudo apt-get install -y build-essential libatlas-base-dev gfortran
$ sudo pip install keras

Hello AI World

Website: https://developer.nvidia.com/embedded/twodaystoademo
Source: https://github.com/dusty-nv/jetson-inference
Supports: Jetson Nano, TX1, TX2, Xavier NX, AGX Xavier
Run the Container

$ git clone --recursive https://github.com/dusty-nv/jetson-inference
$ cd jetson-inference
$ docker/run.sh

Build from Source

# download the repo
$ git clone --recursive https://github.com/dusty-nv/jetson-inference
$ cd jetson-inference

# configure build tree
$ mkdir build
$ cd build
$ cmake ../

# build and install
$ make -j$(nproc)
$ sudo make install
$ sudo ldconfig

Model Zoo

Below are various DNN models for inferencing on Jetson with support for TensorRT. Included are links to code samples with the model and the original source.

Note that many other models are able to run natively on Jetson by using the Machine Learning frameworks like those listed above.

For performance benchmarks, see these resources:

Classification

Network	Dataset	Resolution	Classes	Framework	Format	TensorRT	Samples	Original
AlexNet	ILSVRC12	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	BVLC
GoogleNet	ILSVRC12	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	BVLC
ResNet-18	ILSVRC15	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
ResNet-50	ILSVRC15	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
ResNet-101	ILSVRC15	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
ResNet-152	ILSVRC15	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
VGG-16	ILSVRC14	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
VGG-19	ILSVRC14	224x224	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
Inception-v4	ILSVRC12	299x299	1000	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
Inception-v4	ILSVRC12	299x299	1000	TensorFlow	`TF-TRT (UFF)`	Yes	tf_trt_models	TF-slim
Mobilenet-v1	ILSVRC12	224x224	1000	TensorFlow	`TF-TRT (UFF)`	Yes	tf_trt_models	TF-slim

Object Detection

Network	Dataset	Resolution	Classes	Framework	Format	TensorRT	Samples	Original
SSD-Mobilenet-v1	COCO	300x300	91	TensorFlow	`UFF`	Yes	Hello AI World TRT_object_detection	TF Zoo
SSD-Mobilenet-v2	COCO	300x300	91	TensorFlow	`UFF`	Yes		TF Zoo
SSD-Inception-v2	COCO	300x300	91	TensorFlow	`UFF`	Yes		TF Zoo
YOLO-v2	COCO	608x608	80	Darknet	`Custom`	Yes	trt-yolo-app	YOLO
YOLO-v3	COCO	608x608	80	Darknet	`Custom`	Yes	trt-yolo-app	YOLO
YOLO-v4	COCO	608x608	80	Darknet	`Custom`	Yes	tensorrt-yolov4	YOLO
Tiny YOLO-v3	COCO	416x416	80	Darknet	`Custom`	Yes	trt-yolo-app	YOLO
Tiny YOLO-v3	COCO	416x416	80	Darknet	`Custom`	Yes	trt-yolo-app	YOLO
Tiny YOLO-v4	COCO	416x416	80	Darknet	`Custom`	Yes	tensorrt-yolov4	YOLO
Faster-RCNN	Pascal VOC	500x375	21	Caffe	`caffemodel`	Yes	TensorRT sample	GitHub

Segmentation

Network	Dataset	Resolution	Classes	Framework	Format	TensorRT	Samples	Original
FCN-ResNet18	Cityscapes	2048x1024	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	Cityscapes	1024x512	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	Cityscapes	512x256	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	DeepScene	864x480	5	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	DeepScene	576x320	5	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	DeepScene	864x480	5	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	Multi-Human	640x360	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	Multi-Human	512x320	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	Pascal VOC	512x320	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	Pascal VOC	320x320	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	SUN RGB-D	640x512	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-ResNet18	SUN RGB-D	512x400	21	PyTorch	`ONNX`	Yes	Hello AI World	GitHub
FCN-Alexnet	Cityscapes	2048x1024	21	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
FCN-Alexnet	Cityscapes	1024x512	21	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
FCN-Alexnet	Pascal VOC	500x356	21	Caffe	`caffemodel`	Yes	Hello AI World	GitHub
U-Net	Carvana	512x512	1	TensorFlow	`UFF`	Yes	Nano Benchmarks	GitHub

Pose Estimation

Network	Dataset	Resolution	Classes	Framework	Format	TensorRT	Samples	Original
ResNet18_att	COCO	224x224	16	PyTorch	torch2trt	Yes	trt_pose	GitHub
DenseNet121_att	COCO	224x224	16	PyTorch	torch2trt	Yes	trt_pose	GitHub

Computer Vision

OpenCV

Website: https://opencv.org/
Source: https://github.com/opencv/opencv
Version: 3.3.1 (JetPack <= 4.2.x), 4.1.1 (JetPack 4.3, JetPack 4.4, JetPack 4.5)
Supports: Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier

OpenCV is included with JetPack, compiled with support for GStreamer. To build a newer version or to enable CUDA support, see these guides:
- nano_build_opencv (GitHub)
- Installing OpenCV 3.4.6

Robotics

ROS

Website: http://ros.org/
Source: https://github.com/ros
Version: ROS Melodic, ROS Noetic, ROS2 Eloquent, ROS2 Foxy, ROS2 Galactic, ROS2 Humble
Supports: JetPack >= 4.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
Installation: http://wiki.ros.org/melodic/Installation/Ubuntu
Containers: https://github.com/dusty-nv/jetson-containers

# install ROS Melodic
$ sudo apt-add-repository universe
$ sudo apt-add-repository multiverse
$ sudo apt-add-repository restricted

# add ROS repository to apt sources
$ sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
$ sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

# install ROS Base
$ sudo apt-get update
$ sudo apt-get install ros-melodic-ros-base

# add ROS paths to environment
sudo sh -c 'echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc'

You can also find various resources, libraries, and extra NVIDIA packages for ROS2 at https://nvidia-ai-iot.github.io/ros2_jetson/

NVIDIA Isaac SDK

Website: https://developer.nvidia.com/isaac-sdk
Version: 2019.2, 2019.3, 2020.1, 2020.2
Supports: JetPack 4.2.x, JetPack 4.3, JetPack 4.4 (Jetson Nano / TX2 / Xavier)
Downloads: https://developer.nvidia.com/isaac/downloads
Documentation: https://docs.nvidia.com/isaac

Isaac SIM

Website: https://developer.nvidia.com/isaac-sdk
Documentation: http://docs.nvidia.com/isaac/isaac_sim/index.html

IoT / Edge

AWS Greengrass

Website: https://aws.amazon.com/greengrass/
Source: https://github.com/aws/aws-greengrass-core-sdk-c
Version: v1.9.1
Supports: JetPack 4.2.x, JetPack 4.3, JetPack 4.4 (Jetson Nano / TX1 / TX2 / Xavier)
Forum Thread: https://devtalk.nvidia.com/default/topic/1052324/#5341970

1. Create Greengrass user group:

$ sudo adduser --system ggc_user
$ sudo addgroup --system ggc_group

2. Setup your AWS account and Greengrass group during this page: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html
After downloading your unique security resource keys to your Jetson that were created in this step, proceed to #3 below.

3. Download the AWS IoT Greengrass Core Software (v1.9.1) for ARMv8 (aarch64):

$ wget https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.9.1/greengrass-linux-aarch64-1.9.1.tar.gz

4. Following step #4 from this page, extract Greengrass core and your unique security keys on your Jetson:

$ sudo tar -xzvf greengrass-linux-aarch64-1.9.1.tar.gz -C /
$ sudo tar -xzvf <hash>-setup.tar.gz -C /greengrass   # these are the security keys downloaded above

5. Download AWS ATS endpoint root certificate (CA):

$ cd /greengrass/certs/
$ sudo wget -O root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem

6. Start Greengrass core on your Jetson:

$ cd /greengrass/ggc/core/
$ sudo ./greengrassd start

You should get a message in your terminal Greengrass sucessfully started with PID: xxx

NVIDIA DeepStream

Website: https://developer.nvidia.com/deepstream-sdk
Version: 5.0
Supports: JetPack >= 4.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
FAQ: https://developer.nvidia.com/deepstream-faq
GitHub Samples:

Containers

Docker

Website: https://docker.com/
Source: https://github.com/docker
Version: 18.06
Support: ≥ JetPack 3.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
Installed by default in JetPack-L4T

Launch your container with --runtime nvidia to enable GPU-passthrough. Launch your container with --volume /tmp/argus_socket:/tmp/argus_socket to enable access to MIPI CSI cameras.

See https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-Container-Runtime-on-Jetson for more documentation on using Docker on Jetson.

To enable IPVLAN for Docker Swarm mode: https://blog.hypriot.com/post/nvidia-jetson-nano-build-kernel-docker-optimized/

Kubernetes

Website: https://kubernetes.io/
Source: https://github.com/kubernetes/
Support: ≥ JetPack 3.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
Distributions:
- MicroK8s (v1.14) $ sudo snap install microk8s --classic
- k3s (v0.5.0) $ wget https://github.com/rancher/k3s/releases/download/v0.5.0/k3s-arm64

To configure L4T kernel for K8S: https://medium.com/@jerry_liang/deploy-gpu-enabled-kubernetes-pod-on-nvidia-jetson-nano-ce738e3bcda9
See also: https://medium.com/jit-team/building-a-gpu-enabled-kubernets-cluster-for-machine-learning-with-nvidia-jetson-nano-7b67de74172a

@@ Line 462: / Line 462: @@
 * Website:  [http://ros.org/ http://ros.org/]
 * Source:  [https://github.com/ros https://github.com/ros]
-* Version:  ROS Melodic, ROS Noetic, ROS2 Eloquent, ROS2 Foxy
+* Version:  ROS Melodic, ROS Noetic, ROS2 Eloquent, ROS2 Foxy, ROS2 Galactic, ROS2 Humble
 * Supports:  JetPack >= 4.2 (Jetson Nano / TX1 / TX2 / Xavier NX / AGX Xavier)
 * Installation:  [http://wiki.ros.org/melodic/Installation/Ubuntu http://wiki.ros.org/melodic/Installation/Ubuntu]

Difference between revisions of "Jetson Zoo"

Revision as of 13:28, 26 May 2022

Contents