Difference between revisions of "Jetson/FAQ/BSP/RootFS Reduction"
(→How to fix library missing issue after reduction) |
(→How to fix library missing issue after reduction) |
||
| Line 205: | Line 205: | ||
lrwxrwxrwx 1 root root 18 Dec 10 15:08 /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999 -> tegra/libnvv4l2.so | lrwxrwxrwx 1 root root 18 Dec 10 15:08 /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999 -> tegra/libnvv4l2.so | ||
| − | After restore libv4l2 and libnvjpeg, remember to run "sudo ldconfig" to update /etc/ld.so.cache | + | After restore libv4l2(/usr/lib/aarch64-linux-gnu/libv4l2.so.0, /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.0, /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999 and /usr/lib/aarch64-linux-gnu/tegra/libnvv4l2.so) and libnvjpeg(/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so), remember to run "sudo ldconfig" to update /etc/ld.so.cache |
Now, re-run video_decode, it still fails. That's because the new added libv4l2 and libnvjpeg depends on other missing libraries like below: | Now, re-run video_decode, it still fails. That's because the new added libv4l2 and libnvjpeg depends on other missing libraries like below: | ||
Revision as of 02:55, 3 February 2020
Contents
How to reduce disk usage on Jetson platform
Jetson platform files system is based on ubuntu sample file system. Some Nvidia platform related binaries (BSP related) are added to support platform basic function. Jetson SDK Components including CUDA, cuDNN, TensorRT, OpenCV, VisionWorks, DeepStream and multimedia API provide AI related support.
This page breaks down Nvidia platform related binaries and Jetson SDK Component, so user can remove unused files to save space. A command to list all installed ubuntu packages by size is also provided for user to remove unused package based on needs.
Below is a breakdown of Nvidia platform related binaries.
/
├── boot (43M) Image(kernel binary), dtb and extlinux.conf. All are not used by Xavier, Image and extlinux.conf are used by Nano, TX1 and TX2
├── etc (536K) Nvidia specified config file. DO NOT touch
├── lib
│ ├── firmware (4.3M) Firmwares used by platform. DO NOT touch
│ ├── modules (47M) Modules(.ko) used by platform. Can delete some based on needs
│ └── systemd (32K) Nvidia specified config file. DO NOT touch
├── opt
│ └── nvidia (17M) Jetson gpio python lib and usb device mode service. Can delete based on needs
├── usr
│ ├── bin (356K) Nvidia speficied tools. DO NOT touch
│ ├── lib (130M) Nvidia speficied libs. DO NOT touch
│ ├── sbin (2M) Nvidia speficied tools. DO NOT touch
│ ├── share (2.5M) Nvidia specified boot logo, license, icon, etc. DO NOT touch
│ └── src (234M) Linux header files and grahics demos. Can delete based on needs
└── var
└── nvidia (188K) Placeholder for nvcam. Can delete based on needs
Jetson SDK Components
Based on Jetson OS, Jetson SDK Components, including CUDA, cuDNN, TensorRT, OpenCV, VisionWorks, MultiMedia API and DeepStream, are installed by sdkmanager(or jetpack tool for older version). MultiMedia API is installed by tbz2 package while others are installed by deb package.
Both Jetson SDK libs(both static and dynamic libs) and sample code are installed and sample code can be deleted to save space. And if user's application is dynamicly linked to these libs, all static libs can be deleted.
After Jetson SDK Components are installed, run below commands can remove all deb files.
sudo apt clean # clean debs in /var/cache/apt/archives sudo rm /etc/apt/sources.list.d/* # remove /var/cudaxxxx, /var/visonworksxxxx from apt source list sudo rm /var/cuda-repo-10-0-local-10.0.326/ /var/visionworks-repo/ /var/visionworks-sfm-repo/ /var/visionworks-tracking-repo/ -rf # remove nvidia's debs
Below table list all Jetson SDK Components brief breakdown.
CUDA
| Elememt | Path | Size |
|---|---|---|
| sample | /usr/local/cuda/samples | 198M |
| dynamic lib | /usr/local/cuda/targets/aarch64-linux/lib/*.so* | 723M |
| static lib | /usr/local/cuda/targets/aarch64-linux/lib/*.a | 898M |
| doc | /usr/local/cuda/doc | 231M |
cuDNN
| Elememt | Path | Size |
|---|---|---|
| sample | /usr/src/cudnn_samples_v7 | 11M |
| dynamic lib | /usr/lib/aarch64-linux-gnu/libcudnn*.so* | 367M |
| static lib | /usr/lib/aarch64-linux-gnu/libcudnn*.a | 358M |
TensorRT
| Elememt | Path | Size |
|---|---|---|
| sample | /usr/src/tensorrt | 783M |
| dynamic lib | /usr/lib/aarch64-linux-gnu/libnvcaffe_parser*.so* /usr/lib/aarch64-linux-gnu/libnvinfer*.so* /usr/lib/aarch64-linux-gnu/libnvonnxparser*.so* /usr/lib/aarch64-linux-gnu/libnvparsers*.so* |
156M |
| static lib | /usr/lib/aarch64-linux-gnu/libnvcaffe_parser*.a /usr/lib/aarch64-linux-gnu/libnvinfer*.a /usr/lib/aarch64-linux-gnu/libnvonnxparser*.a /usr/lib/aarch64-linux-gnu/libnvparsers*.a |
172M |
| doc | /usr/share/doc/libnvinfer* | 24K |
OpenCV
| Elememt | Path | Size |
|---|---|---|
| sample | /usr/share/OpenCV | 11M |
| dynamic lib | /usr/lib/libopencv*.so* | 19M |
VisionWorks
| Elememt | Path | Size |
|---|---|---|
| sample | /usr/share/visionworks* ~/VisionWorks-SFM*Samples |
131M |
| dynamic lib | /usr/lib/libvisionworks*.so* | 28M |
| doc | /usr/share/doc/libvisionworks* | 352K |
DeepStream
| Elememt | Path | Size |
|---|---|---|
| sample | /opt/nvidia/deepstream/deepstream*/samples | 205M |
| dynamic lib | /opt/nvidia/deepstream/deepstream*/lib | 52M |
| doc | /opt/nvidia/deepstream/deepstream*/doc | 16K |
MultiMedia API
| Path | Size |
|---|---|
| /usr/src/tegra_multimedia_api | 107M |
Remove installed deb packages
Run below command can remove all cached .deb file.
sudo apt clean
Run below command can remove all non-required installed packages.
sudo apt autoremove
Below command can list all install deb packages by size. User can remove the undesired packages by hand to save space.
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -rn
Example
Remove all deb installation packages
sudo apt clean sudo rm /etc/apt/sources.list.d/* sudo rm -rf /var/cuda-repo-10-0-local-10.0.326/ /var/visionworks-repo/ /var/visionworks-sfm-repo/ /var/visionworks-tracking-repo
Remove ubuntu-desktop and the accessories
sudo apt-get purge gnome-shell sudo apt-get purge docker* sudo apt-get purge chromium-browser thunderbird fonts-noto-cjk libreoffice-common containerd snapd sudo apt-get remove --purge python* libpython* sudo apt-get purge libwebkit2gtk-4.0-37 libqt5webkit5 ubuntu-wallpapers-bionic freepats libreoffice-writer libicu60 libreoffice-calc vim-runtime libflite1 libperl5.26 libmozjs-52-0 humanity-icon-theme samba-libs iso-codes kwin-data perl-modules-5.26 light-themes libjavascriptcoregtk-4.0-18 chromium-browser-l10n sudo apt-get purge libvisionworks libvisionworks-sfm-dev libvisionworks-sfm-repo sudo apt-get autoremove
Remove some source code and sample code
cd /usr/src && sudo rm -rf cudnn_samples_v7 linux-headers-4.9.140-tegra-ubuntu18.04_aarch64 linux-headers-4.9.140-tegra-linux_x86_64 tensorrt nvidia rm -rf /home/nvidia/VisionWorks-SFM-0.90-Samples
How to fix library missing issue after reduction
Sometimes after removing ubuntu packages, application execution may fail due to depended libraries missing.
For example, after removing packages by above example, Jetson Multimedia API sample cannot run like below.
nvidia@nvidia-desktop:~/jetson_multimedia_api/samples/00_video_decode$ ./video_decode ./video_decode: error while loading shared libraries: libv4l2.so.0: cannot open shared object file: No such file or directory
ldd command shows both libv4l2.so.0 and libnvjpeg.so are missing like below. So we need to copy these libraries from clean platform.
nvidia@nvidia-desktop:~/jetson_multimedia_api/samples/00_video_decode$ ldd video_decode
linux-vdso.so.1 (0x0000007f869e8000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f86916000)
libv4l2.so.0 => not found
libEGL.so.1 => /usr/lib/aarch64-linux-gnu/libEGL.so.1 (0x0000007f868f5000)
libGLESv2.so.2 => /usr/lib/aarch64-linux-gnu/libGLESv2.so.2 (0x0000007f868bf000)
libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f86796000)
libnvbuf_utils.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so.1.0.0 (0x0000007f8677c000)
libnvjpeg.so => not found
libdrm.so.2 => /usr/lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007f8674a000)
libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f865b7000)
libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f86593000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f8643a000)
/lib/ld-linux-aarch64.so.1 (0x0000007f869bd000)
libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/libGLdispatch.so.0 (0x0000007f8630e000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f862f9000)
libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f862c9000)
libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f86287000)
libnvrm_graphics.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so (0x0000007f86268000)
libnvddk_vic.so => /usr/lib/aarch64-linux-gnu/tegra/libnvddk_vic.so (0x0000007f8624a000)
libnvbuf_fdmap.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_fdmap.so.1.0.0 (0x0000007f86237000)
libnvll.so => /usr/lib/aarch64-linux-gnu/tegra/libnvll.so (0x0000007f8621b000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f86161000)
libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f8614e000)
libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f86139000)
libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f8611b000)
libnvdc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so (0x0000007f860fc000)
libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007f860da000)
librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f860c3000)
libnvimp.so => /usr/lib/aarch64-linux-gnu/tegra/libnvimp.so (0x0000007f860ae000)
On clean platform, we can use "find" and "ls" command to get all the necessary libraries like below.
nvidia@nvidia-xavier:~$ find /usr/lib/ -name "libv4l2.so*" /usr/lib/aarch64-linux-gnu/libv4l2.so.0 /usr/lib/aarch64-linux-gnu/libv4l2.so /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.0 /usr/lib/aarch64-linux-gnu/tegra/libv4l2.so.0 /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999
nvidia@nvidia-xavier:/usr/lib/aarch64-linux-gnu$ ls -l /usr/lib/aarch64-linux-gnu/libv4l2.so* lrwxrwxrwx 1 root root 12 Dec 26 16:15 /usr/lib/aarch64-linux-gnu/libv4l2.so -> libv4l2.so.0 lrwxrwxrwx 1 root root 21 Dec 19 14:43 /usr/lib/aarch64-linux-gnu/libv4l2.so.0 -> libv4l2.so.0.0.999999 -rw-r--r-- 1 root root 55424 Feb 10 2018 /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.0 lrwxrwxrwx 1 root root 18 Dec 10 15:08 /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999 -> tegra/libnvv4l2.so
After restore libv4l2(/usr/lib/aarch64-linux-gnu/libv4l2.so.0, /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.0, /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999 and /usr/lib/aarch64-linux-gnu/tegra/libnvv4l2.so) and libnvjpeg(/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so), remember to run "sudo ldconfig" to update /etc/ld.so.cache
Now, re-run video_decode, it still fails. That's because the new added libv4l2 and libnvjpeg depends on other missing libraries like below:
nvidia@nvidia-desktop:~/jetson_multimedia_api/samples/00_video_decode$ ldd video_decode
linux-vdso.so.1 (0x0000007f9abff000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f9ab2d000)
libv4l2.so.0 => /usr/lib/aarch64-linux-gnu/libv4l2.so.0 (0x0000007f9aa18000)
libEGL.so.1 => /usr/lib/aarch64-linux-gnu/libEGL.so.1 (0x0000007f9a9f7000)
libGLESv2.so.2 => /usr/lib/aarch64-linux-gnu/libGLESv2.so.2 (0x0000007f9a9c1000)
libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f9a898000)
libnvbuf_utils.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so.1.0.0 (0x0000007f9a87e000)
libnvjpeg.so => /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so (0x0000007f9a826000)
libdrm.so.2 => /usr/lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007f9a7f4000)
libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f9a661000)
libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f9a63d000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f9a4e4000)
/lib/ld-linux-aarch64.so.1 (0x0000007f9abd4000)
libv4lconvert.so.0 => not found
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f9a4cf000)
libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/libGLdispatch.so.0 (0x0000007f9a3a3000)
libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f9a373000)
libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f9a331000)
libnvrm_graphics.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so (0x0000007f9a312000)
libnvddk_vic.so => /usr/lib/aarch64-linux-gnu/tegra/libnvddk_vic.so (0x0000007f9a2f4000)
libnvbuf_fdmap.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_fdmap.so.1.0.0 (0x0000007f9a2e1000)
libnvddk_2d_v2.so => /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so (0x0000007f9a2bc000)
libnvbufsurface.so.1.0.0 => not found
libnvll.so => /usr/lib/aarch64-linux-gnu/tegra/libnvll.so (0x0000007f9a2a0000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f9a1e6000)
libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f9a1d3000)
libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f9a1be000)
libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f9a1a0000)
libnvdc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so (0x0000007f9a181000)
libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007f9a15f000)
librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f9a148000)
libnvimp.so => /usr/lib/aarch64-linux-gnu/tegra/libnvimp.so (0x0000007f9a133000)
After restore libv4lconvert and libnvbufsurface just like libv4l2, all dependencies are available now.
nvidia@nvidia-desktop:~/jetson_multimedia_api/samples/00_video_decode$ ldd video_decode
linux-vdso.so.1 (0x0000007f8bb5c000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f8ba8a000)
libv4l2.so.0 => /usr/lib/aarch64-linux-gnu/libv4l2.so.0 (0x0000007f8b975000)
libEGL.so.1 => /usr/lib/aarch64-linux-gnu/libEGL.so.1 (0x0000007f8b954000)
libGLESv2.so.2 => /usr/lib/aarch64-linux-gnu/libGLESv2.so.2 (0x0000007f8b91e000)
libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f8b7f5000)
libnvbuf_utils.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so.1.0.0 (0x0000007f8b7db000)
libnvjpeg.so => /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so (0x0000007f8b783000)
libdrm.so.2 => /usr/lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007f8b751000)
libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f8b5be000)
libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f8b59a000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f8b441000)
/lib/ld-linux-aarch64.so.1 (0x0000007f8bb31000)
libv4lconvert.so.0 => /usr/lib/aarch64-linux-gnu/libv4lconvert.so.0 (0x0000007f8b3bc000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f8b3a7000)
libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/libGLdispatch.so.0 (0x0000007f8b27b000)
libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f8b24b000)
libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f8b209000)
libnvrm_graphics.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so (0x0000007f8b1ea000)
libnvddk_vic.so => /usr/lib/aarch64-linux-gnu/tegra/libnvddk_vic.so (0x0000007f8b1cc000)
libnvbuf_fdmap.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_fdmap.so.1.0.0 (0x0000007f8b1b9000)
libnvddk_2d_v2.so => /usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so (0x0000007f8b194000)
libnvbufsurface.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so.1.0.0 (0x0000007f8b116000)
libnvll.so => /usr/lib/aarch64-linux-gnu/tegra/libnvll.so (0x0000007f8b0fa000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f8b040000)
librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f8b029000)
libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f8b016000)
libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f8b001000)
libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f8afe3000)
libnvdc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so (0x0000007f8afc4000)
libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007f8afa2000)
libnvimp.so => /usr/lib/aarch64-linux-gnu/tegra/libnvimp.so (0x0000007f8af8d000)
