TensorRT/Build TF1 14 onXavier

System Setup

- Product: Jetson AGX Xavier - JetPack: 4.2 - TensorFlow: 1.13 - Cuda: 10.0 - Compute Capability: 7.2 - Cudnn: 7.4 - TensorRT: 5.0.6 - Python: 3.6 - bazel: 0.19.2 - gcc : 7.4.0 - pip: 1.19.1

Building bazel

1. Install java and some dependencies if you haven't already done so sudo apt-get update sudo apt-get upgrade sudo apt-get install openjdk-11-jdk sudo apt-get install pkg-config zip g++ zlib1g-dev unzip 2. Install Bazel wget https://github.com/bazelbuild/bazel/releases/download/0.25.3/bazel-0.25.3-dist.zip unzip bazel-0.25.3-dist.zip -d bazel-0.25.3 cd bazel-0.25.3/ 3. cd to the unzipped directory and build bazel env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh. 4. The output should be produced in output/bazel. Feel free to add this binary to your environment, i.e. ~/.bashrc: vim ~/.bashrc export PATH=/pathToYourBazelDirectory/output${PATH:+:${PATH}} # add this at the end of your file

Building Tensorflow

1. Download sources git clone https://github.com/tensorflow/tensorflow.git cd tensorflow git checkout r1.14 2. apply this patch for our arm architecture diff --git a/tensorflow/lite/kernels/internal/BUILD b/tensorflow/lite/kernels/internal/BUILD index 4be3226938..7226f96fdf 100644 --- a/tensorflow/lite/kernels/internal/BUILD +++ b/tensorflow/lite/kernels/internal/BUILD @@ -22,15 +22,12 @@ HARD_FP_FLAGS_IF_APPLICABLE = select({ NEON_FLAGS_IF_APPLICABLE = select({ ":arm": [ "-O3", -       "-mfpu=neon", ],     ":armeabi-v7a": [ "-O3", -       "-mfpu=neon", ],     ":armv7a": [ "-O3", -       "-mfpu=neon", ],     "//conditions:default": [ "-O3", diff --git a/third_party/aws/BUILD.bazel b/third_party/aws/BUILD.bazel index 5426f79e46..e08f8fc108 100644 --- a/third_party/aws/BUILD.bazel +++ b/third_party/aws/BUILD.bazel @@ -24,7 +24,7 @@ cc_library(         "@org_tensorflow//tensorflow:raspberry_pi_armeabi": glob([ "aws-cpp-sdk-core/source/platform/linux-shared/*.cpp", ]), -       "//conditions:default": [], +        "//conditions:default": glob(["aws-cpp-sdk-core/source/platform/linux-shared/*.cpp",]),      }) + glob([          "aws-cpp-sdk-core/include/**/*.h",          "aws-cpp-sdk-core/source/*.cpp", 3. Configure system build nvidia@nvidia-desktop:~/Work/tensorflow$ ./configure  Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3 Found possible Python library paths:   /usr/lib/python3.6/dist-packages   /usr/lib/python3/dist-packages   /usr/local/lib/python3.6/dist-packages Please input the desired Python library path to use.  Default is [/usr/lib/python3.6/dist-packages] Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n No XLA JIT support will be enabled for TensorFlow.  Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n No OpenCL SYCL support will be enabled for TensorFlow. Do you wish to build TensorFlow with ROCm support? [y/N]: n No ROCm support will be enabled for TensorFlow. Do you wish to build TensorFlow with CUDA support? [y/N]: y CUDA support will be enabled for TensorFlow. Do you wish to build TensorFlow with TensorRT support? [y/N]: y TensorRT support will be enabled for TensorFlow. Found CUDA 10.0 in: /usr/local/cuda/lib64 /usr/local/cuda/include Found cuDNN 7 in: /usr/lib/aarch64-linux-gnu /usr/include Found TensorRT 5 in: /usr/lib/aarch64-linux-gnu /usr/include/aarch64-linux-gnu Please specify a list of comma-separated CUDA compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]: 7.2 Do you want to use clang as CUDA compiler? [y/N]: n nvcc will be used as CUDA compiler. Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: Do you wish to build TensorFlow with MPI support? [y/N]: n No MPI support will be enabled for TensorFlow. Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n Not configuring the WORKSPACE for Android builds. Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details. --config=mkl        	# Build with MKL support. --config=monolithic 	# Config for mostly static monolithic build. --config=gdr        	# Build with GDR support. --config=verbs      	# Build with libverbs support. --config=ngraph     	# Build with Intel nGraph support. --config=numa       	# Build with NUMA support. --config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects. Preconfigured Bazel build configs to DISABLE default on features: --config=noaws      	# Disable AWS S3 filesystem support. --config=nogcp      	# Disable GCP support. --config=nohdfs     	# Disable HDFS support. --config=noignite   	# Disable Apache Ignite support. --config=nokafka    	# Disable Apache Kafka support. --config=nonccl     	# Disable NVIDIA NCCL support. Configuration finished

Build your own application with Bazel 1. make a directory under tensorflow/cc/ mkdir tensorflow/cc/example 2. Implement your own project in test.cc 3. Write BUILD file for yout application like this: load("//tensorflow:tensorflow.bzl", "tf_cc_binary") tf_cc_binary(    name = "example",     srcs = ["test.cc"],     deps = [  	 "//tensorflow/cc:cc_ops",         "//tensorflow/cc:client_session",         "//tensorflow/core:tensorflow",          "//tensorflow/compiler/tf2tensorrt:trt_engine_op_op_lib",  	 "//tensorflow/compiler/tf2tensorrt:trt_op_kernels",         "//tensorflow/core:protos_all_cc"     ], ) Note: trt_engine_op_op_lib and trt_op_kernels is for running inference with TF-TRT optimized model

4. Build your application bazel build --config=opt --config=nonccl //tensorflow/cc/example:example --verbose_failures --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" 5. Run ./bazel-bin/tensorflow/cc/example/example