TensorRT/Build TF1 14 onXavier

System Setup

- Product: Jetson AGX Xavier
- JetPack: 4.2
- TensorFlow: 1.13
- Cuda: 10.0
- Compute Capability: 7.2
- Cudnn: 7.4
- TensorRT: 5.0.6
- Python: 3.6
- bazel: 0.19.2
- gcc : 7.4.0
- pip: 1.19.1

Building bazel

1. Install java and some dependencies if you haven't already done so

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install openjdk-11-jdk
sudo apt-get install pkg-config zip g++ zlib1g-dev unzip 

2. Install Bazel

wget https://github.com/bazelbuild/bazel/releases/download/0.25.3/bazel-0.25.3-dist.zip 
unzip bazel-0.25.3-dist.zip -d bazel-0.25.3 
cd bazel-0.25.3/ 

3. cd to the unzipped directory and build bazel

env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh.

4. The output should be produced in output/bazel. Feel free to add this binary to your environment, i.e. ~/.bashrc:

vim ~/.bashrc
export PATH=/pathToYourBazelDirectory/output${PATH:+:${PATH}} # add this at the end of your file

Building Tensorflow

1. Download sources

git clone https://github.com/tensorflow/tensorflow.git 
cd tensorflow 
git checkout r1.14 

2. apply this patch for our arm architecture

diff --git a/tensorflow/lite/kernels/internal/BUILD b/tensorflow/lite/kernels/internal/BUILD
index 4be3226938..7226f96fdf 100644
--- a/tensorflow/lite/kernels/internal/BUILD
+++ b/tensorflow/lite/kernels/internal/BUILD
@@ -22,15 +22,12 @@ HARD_FP_FLAGS_IF_APPLICABLE = select({
 NEON_FLAGS_IF_APPLICABLE = select({
     ":arm": [
         "-O3",
-        "-mfpu=neon",
     ],
     ":armeabi-v7a": [
         "-O3",
-        "-mfpu=neon",
     ],
     ":armv7a": [
         "-O3",
-        "-mfpu=neon",
     ],
     "//conditions:default": [
         "-O3",

diff --git a/third_party/aws/BUILD.bazel b/third_party/aws/BUILD.bazel
index 5426f79e46..e08f8fc108 100644
--- a/third_party/aws/BUILD.bazel
+++ b/third_party/aws/BUILD.bazel
@@ -24,7 +24,7 @@ cc_library(
         "@org_tensorflow//tensorflow:raspberry_pi_armeabi": glob([
             "aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",
         ]),
-        "//conditions:default": [],
+        "//conditions:default": glob(["aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",]),
     }) + glob([
         "aws-cpp-sdk-core/include/**/*.h",
         "aws-cpp-sdk-core/source/*.cpp", 

3. Configure system build

nvidia@nvidia-desktop:~/Work/tensorflow$ ./configure 
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3
Found possible Python library paths:
  /usr/lib/python3.6/dist-packages
  /usr/lib/python3/dist-packages
  /usr/local/lib/python3.6/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3.6/dist-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n
No XLA JIT support will be enabled for TensorFlow. 

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow. 

Do you wish to build TensorFlow with TensorRT support? [y/N]: y
TensorRT support will be enabled for TensorFlow.

Found CUDA 10.0 in:
    /usr/local/cuda/lib64
    /usr/local/cuda/include
Found cuDNN 7 in:
    /usr/lib/aarch64-linux-gnu
    /usr/include
Found TensorRT 5 in:
    /usr/lib/aarch64-linux-gnu
    /usr/include/aarch64-linux-gnu 

Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]: 7.2

Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: 

Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:  

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=gdr         	# Build with GDR support.
	--config=verbs       	# Build with libverbs support.
	--config=ngraph      	# Build with Intel nGraph support.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
 	--config=noaws       	# Disable AWS S3 filesystem support.
	--config=nogcp       	# Disable GCP support.
	--config=nohdfs      	# Disable HDFS support.
	--config=noignite    	# Disable Apache Ignite support.
	--config=nokafka     	# Disable Apache Kafka support.
	--config=nonccl      	# Disable NVIDIA NCCL support. 
Configuration finished

Build your own application with Bazel
1. make a directory under tensorflow/cc/

mkdir tensorflow/cc/example

2. Implement your own project in test.cc
3. Write BUILD file for yout application like this:

load("//tensorflow:tensorflow.bzl", "tf_cc_binary")

tf_cc_binary(
    name = "example",
    srcs = ["test.cc"],
    deps = [ 
	 "//tensorflow/cc:cc_ops",
        "//tensorflow/cc:client_session",
        "//tensorflow/core:tensorflow", 
        "//tensorflow/compiler/tf2tensorrt:trt_engine_op_op_lib", 
	 "//tensorflow/compiler/tf2tensorrt:trt_op_kernels",
        "//tensorflow/core:protos_all_cc"
    ],
)

Note: trt_engine_op_op_lib and trt_op_kernels is for running inference with TF-TRT optimized model

4. Build your application

bazel build --config=opt --config=nonccl //tensorflow/cc/example:example --verbose_failures --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"

5. Run

./bazel-bin/tensorflow/cc/example/example