Jetson Zoo

From eLinux.org
Revision as of 12:39, 22 April 2020 by Dusty-nv (talk | contribs) (→‎OpenCV)
Jump to: navigation, search

This page contains instructions for installing various open source add-on packages and frameworks on NVIDIA Jetson, in addition to a collection of DNN models for inferencing.

Below are links to precompiled binaries built for aarch64 (arm64) architecture, including support for CUDA where applicable. These are intended to be installed on top of JetPack.

Note that JetPack comes with various pre-installed components such as the L4T kernel, CUDA Toolkit, cuDNN, TensorRT, VisionWorks, OpenCV, GStreamer, Docker, and more.

For the latest updates and support, refer to the listed forum topics. Feel free to contribute to the list below if you know of software packages that are working & tested on Jetson.

Machine Learning

Jetson is able to natively run the full versions of popular machine learning frameworks, including TensorFlow, PyTorch, Caffe2, Keras, and MXNet.

There are also helpful deep learning examples and tutorials available, created specifically for Jetson - like Hello AI World and JetBot.

TensorFlow

TensorFlow Logo.png
# install prerequisites
$ sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev

# install and upgrade pip3
$ sudo apt-get install python3-pip
$ sudo pip3 install -U pip testresources setuptools

# install the following python packages
$ sudo pip3 install -U numpy==1.16.1 future==0.17.1 mock==3.0.5 h5py==2.9.0 keras_preprocessing==1.0.5 keras_applications==1.0.8 gast==0.2.2 enum34 futures protobuf

# to install TensorFlow 1.15:
$ sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v43 'tensorflow-gpu<2'

# or install the latest version of TensorFlow (2.0)
$ sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v43 tensorflow-gpu

PyTorch (Caffe2)

PyTorch Logo.png

       JetPack 4.4 DP

Python 2.7 Python 3.6
v1.2.0 pip wheel pip wheel
v1.3.0 pip wheel pip wheel
v1.4.0 pip wheel pip wheel

       JetPack 4.2 / 4.3

Python 2.7 Python 3.6
v1.0.0 pip wheel pip wheel
v1.1.0 pip wheel pip wheel
v1.2.0 pip wheel pip wheel
v1.3.0 pip wheel pip wheel
v1.4.0 pip wheel pip wheel

note — the PyTorch and Caffe2 projects have merged, so installing PyTorch will also install Caffe2 

# for PyTorch v1.4.0, install OpenBLAS
$ sudo apt-get install libopenblas-base

# Python 2.7 (download pip wheel from above)
$ pip install torch-1.4.0-cp27-cp27mu-linux_aarch64.whl

# Python 3.6 (download pip wheel from above)
$ pip3 install Cython
$ pip3 install numpy torch-1.4.0-cp36-cp36m-linux_aarch64.whl

As per the the PyTorch 1.4 Release Notes, Python 2 support is now deprecated and PyTorch 1.4 is the last version to support Python 2.

MXNet

MXNet Logo.png
Python 2.7 Python 3.6
v1.4 (JetPack 4.2.x) pip wheel pip wheel
v1.6 (JetPack 4.3) pip wheel pip wheel 
# Python 2.7
sudo apt-get install -y git build-essential libatlas-base-dev libopencv-dev graphviz python-pip
sudo pip install mxnet-1.4.0-cp27-cp27mu-linux_aarch64.whl

# Python 3.6
sudo apt-get install -y git build-essential libatlas-base-dev libopencv-dev graphviz python3-pip
sudo pip install mxnet-1.4.0-cp36-cp36m-linux_aarch64.whl

Keras

Keras Logo.png

First, install TensorFlow from above. 

# beforehand, install TensorFlow (https://eLinux.org/Jetson_Zoo#TensorFlow)
$ sudo apt-get install -y build-essential libatlas-base-dev gfortran
$ sudo pip install keras

Hello AI World

Hello-AI-World-CV.png
# download the repo
$ git clone --recursive https://github.com/dusty-nv/jetson-inference
$ cd jetson-inference

# configure build tree
$ mkdir build
$ cd build
$ cmake ../

# build and install
$ make -j$(nproc)
$ sudo make install
$ sudo ldconfig

Model Zoo

Below are various DNN models for inferencing on Jetson with support for TensorRT. Included are links to code samples with the model and the original source.

Note that many other models are able to run natively on Jetson by using the Machine Learning frameworks like those listed above.

For performance benchmarks, see these resources:

Classification

Network Dataset Resolution Classes Framework Format TensorRT Samples Original
AlexNet ILSVRC12 224x224 1000 Caffe caffemodel Yes Hello AI World BVLC
GoogleNet ILSVRC12 224x224 1000 Caffe caffemodel Yes Hello AI World BVLC
ResNet-18 ILSVRC15 224x224 1000 Caffe caffemodel Yes Hello AI World GitHub
ResNet-50 ILSVRC15 224x224 1000 Caffe caffemodel Yes Hello AI World GitHub
ResNet-101 ILSVRC15 224x224 1000 Caffe caffemodel Yes Hello AI World GitHub
ResNet-152 ILSVRC15 224x224 1000 Caffe caffemodel Yes Hello AI World GitHub
VGG-16 ILSVRC14 224x224 1000 Caffe caffemodel Yes Hello AI World GitHub
VGG-19 ILSVRC14 224x224 1000 Caffe caffemodel Yes Hello AI World GitHub
Inception-v4 ILSVRC12 299x299 1000 Caffe caffemodel Yes Hello AI World GitHub
Inception-v4 ILSVRC12 299x299 1000 TensorFlow TF-TRT (UFF) Yes tf_trt_models TF-slim
Mobilenet-v1 ILSVRC12 224x224 1000 TensorFlow TF-TRT (UFF) Yes tf_trt_models TF-slim

Object Detection

Network Dataset Resolution Classes Framework Format TensorRT Samples Original
SSD-Mobilenet-v1 COCO 300x300 91 TensorFlow UFF Yes Hello AI World
TRT_object_detection		 TF Zoo
SSD-Mobilenet-v2 COCO 300x300 91 TensorFlow UFF Yes TF Zoo
SSD-Inception-v2 COCO 300x300 91 TensorFlow UFF Yes TF Zoo
YOLO-v2 COCO 608x608 80 Darknet Custom Yes trt-yolo-app YOLO
YOLO-v3 COCO 608x608 80 Darknet Custom Yes trt-yolo-app YOLO
Tiny YOLO-v3 COCO 416x416 80 Darknet Custom Yes trt-yolo-app YOLO
Tiny YOLO-v3 COCO 416x416 80 Darknet Custom Yes trt-yolo-app YOLO
Faster-RCNN Pascal VOC 500x375 21 Caffe caffemodel Yes TensorRT sample GitHub

Segmentation

Network Dataset Resolution Classes Framework Format TensorRT Samples Original
FCN-ResNet18 Cityscapes 2048x1024 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 Cityscapes 1024x512 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 Cityscapes 512x256 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 DeepScene 864x480 5 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 DeepScene 576x320 5 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 DeepScene 864x480 5 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 Multi-Human 640x360 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 Multi-Human 512x320 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 Pascal VOC 512x320 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 Pascal VOC 320x320 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 SUN RGB-D 640x512 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-ResNet18 SUN RGB-D 512x400 21 PyTorch ONNX Yes Hello AI World GitHub
FCN-Alexnet Cityscapes 2048x1024 21 Caffe caffemodel Yes Hello AI World GitHub
FCN-Alexnet Cityscapes 1024x512 21 Caffe caffemodel Yes Hello AI World GitHub
FCN-Alexnet Pascal VOC 500x356 21 Caffe caffemodel Yes Hello AI World GitHub
U-Net Carvana 512x512 1 TensorFlow UFF Yes Nano Benchmarks GitHub

Pose Estimation

Network Dataset Resolution Classes Framework Format TensorRT Samples Original
ResNet18_att COCO 224x224 16 PyTorch torch2trt Yes trt_pose GitHub
DenseNet121_att COCO 224x224 16 PyTorch torch2trt Yes trt_pose GitHub

Computer Vision

OpenCV

OpenCV Logo.png

Robotics

ROS

Ros logo.png
# enable all Ubuntu packages:
$ sudo apt-add-repository universe
$ sudo apt-add-repository multiverse
$ sudo apt-add-repository restricted

# add ROS repository to apt sources
$ sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
$ sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

# install ROS Base
$ sudo apt-get update
$ sudo apt-get install ros-melodic-ros-base

# add ROS paths to environment
sudo sh -c 'echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc'

NVIDIA Isaac SDK

Isaac Gems.jpg

Isaac SIM

IoT / Edge

AWS Greengrass

Greengrass Logo.png

1. Create Greengrass user group: 

$ sudo adduser --system ggc_user
$ sudo addgroup --system ggc_group

2. Setup your AWS account and Greengrass group during this page: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html
    After downloading your unique security resource keys to your Jetson that were created in this step, proceed to #3 below.

3. Download the AWS IoT Greengrass Core Software (v1.9.1) for ARMv8 (aarch64): 

$ wget https://d1onfpft10uf5o.cloudfront.net/greengrass-core/downloads/1.9.1/greengrass-linux-aarch64-1.9.1.tar.gz

4. Following step #4 from this page, extract Greengrass core and your unique security keys on your Jetson: 

$ sudo tar -xzvf greengrass-linux-aarch64-1.9.1.tar.gz -C /
$ sudo tar -xzvf <hash>-setup.tar.gz -C /greengrass   # these are the security keys downloaded above

5. Download AWS ATS endpoint root certificate (CA): 

$ cd /greengrass/certs/
$ sudo wget -O root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem

6. Start Greengrass core on your Jetson: 

$ cd /greengrass/ggc/core/
$ sudo ./greengrassd start

You should get a message in your terminal Greengrass sucessfully started with PID: xxx

NVIDIA DeepStream

DeepStream 30 Stream.png

Containers

Docker

Docker Logo.png

To enable GPU passthrough, enable access to these device nodes with the --device flag when launching Docker containers: 

/dev/nvhost-ctrl
/dev/nvhost-ctrl-gpu
/dev/nvhost-prof-gpu
/dev/nvmap
/dev/nvhost-gpu
/dev/nvhost-as-gpu

The /usr/lib/aarch64-linux-gnu/tegra directory also needs mounted.

Below is an example command line for launching Docker with access to the GPU: 

docker run --device=/dev/nvhost-ctrl --device=/dev/nvhost-ctrl-gpu --device=/dev/nvhost-prof-gpu --device=/dev/nvmap --device=/dev/nvhost-gpu --device=/dev/nvhost-as-gpu -v /usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra <container-name>

To enable IPVLAN for Docker Swarm mode: https://blog.hypriot.com/post/nvidia-jetson-nano-build-kernel-docker-optimized/

Kubernetes

Kubernetes-Logo.png

To configure L4T kernel for K8S: https://medium.com/@jerry_liang/deploy-gpu-enabled-kubernetes-pod-on-nvidia-jetson-nano-ce738e3bcda9
See also: https://medium.com/jit-team/building-a-gpu-enabled-kubernets-cluster-for-machine-learning-with-nvidia-jetson-nano-7b67de74172a

Retrieved from "https://elinux.org/index.php?title=Jetson_Zoo&oldid=513646"