Difference between revisions of "Jetson/L4T BSP development tips"

From eLinux.org
Jump to: navigation, search
(How to update pinmux)
(Ramdisk customization)
(31 intermediate revisions by the same user not shown)
Line 9: Line 9:
  
 
'''Sample Root Filesystem'''
 
'''Sample Root Filesystem'''
 +
  
 
Source code for kernel and other components:
 
Source code for kernel and other components:
  
 
'''L4T Driver Package (BSP) Sources'''
 
'''L4T Driver Package (BSP) Sources'''
 +
  
 
Toolchain for kernel building:
 
Toolchain for kernel building:
  
 
'''GCC 7.3.1 for 64 bit BSP and Kernel'''
 
'''GCC 7.3.1 for 64 bit BSP and Kernel'''
 +
  
 
Secure package if secure-boot is necessary:
 
Secure package if secure-boot is necessary:
  
 
'''Jetson Platform Fuse Burning and Secure Boot Documentation and Tools'''
 
'''Jetson Platform Fuse Burning and Secure Boot Documentation and Tools'''
 
  
  
Line 27: Line 29:
  
 
Then the device can be flashed by command line.
 
Then the device can be flashed by command line.
 
 
With this method, user can have more controls for the BSP configuration, like pinmux, kernel/kernel DTB customization, etc.
 
With this method, user can have more controls for the BSP configuration, like pinmux, kernel/kernel DTB customization, etc.
  
Line 35: Line 36:
 
Refer to https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fmb1_platform_config_xavier.html%23wwpID0E0240HA
 
Refer to https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fmb1_platform_config_xavier.html%23wwpID0E0240HA
  
There are several types of Jetson reference boards are supported in SDK.
+
There are several types of Jetson reference boards which are supported in SDK.
===== One way is to check the configuration file =====
+
===== Check the configuration file =====
For example, when the flash command is:
+
For example, when the flash command is run:
  
 
   sudo ./flash.sh jetson-xavier mmcblk0p1
 
   sudo ./flash.sh jetson-xavier mmcblk0p1
Line 44: Line 45:
  
 
(Note: the value of PINMUX_CONFIG may be overwritten. So the later one should take effect.)
 
(Note: the value of PINMUX_CONFIG may be overwritten. So the later one should take effect.)
===== Another way is to check the flash log =====
+
 
 +
===== Or check the flash log =====
 
Run following command:
 
Run following command:
  
Line 72: Line 74:
  
 
Still, please make sure the PINMUX configuration file name is correct.
 
Still, please make sure the PINMUX configuration file name is correct.
 +
 +
===== Edit the pinmux configuration file =====
 +
That's the direct and simple way, assumed the developer is familiar with PINMUX setting.
 +
 +
Download technical reference manual from https://developer.nvidia.com/embedded/downloads#?search=TRM for different platforms. And those documents contain detailed information.
 +
 +
===== Edit the prod configuration =====
 +
L4T document shows another way to override the PINMUX:
 +
 +
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fmb1_platform_config_xavier.html%23wwpID0E0A40HA
 +
 +
Same command as [[Jetson/L4T_BSP_development_tips#Or_check_the_flash_log|'''2.1.1.2''']], and check the prod configuration in following log:
 +
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
...
 +
 +
copying prod_config(/home/Work/jetson_customer/32.4.3/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-prod-p2888-0000-p2822-0000.cfg)... done.
 +
 +
...</pre>
 +
 +
Follow the guide in above link.
 +
 +
===== How to update device PINMUX =====
 +
After the pinmux configuration files are changed, re-flash the device:
 +
 +
  sudo ./flash.sh jetson-xavier mmcblk0p1        #flash the whole device
 +
 +
or
 +
 +
  sudo ./flash.sh -k MB1_BCT jetson-xavier mmcblk0p1      #only flash the MB1_BCT
 +
 +
===== How to verify the new PINUX configuration works =====
 +
User can read the PINMUX registers to confirm the new configuration works.
 +
 +
For example,
 +
 +
In PINMUX configuration file, there's an entry:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
pinmux.0x0c302030 = 0x00000540; # gen2_i2c_scl_pcc7: i2c2, tristate-disable, input-enable, io_high_voltage-disable, lpdr-enable</pre>
 +
 +
And in prod configuration file, another entry:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
prod.0x0c302030.0x0000100 = 0x00000000; #gen2_i2c_scl_pcc7: LPDR disable</pre>
 +
 +
Run a physical memory access tool, like devmem2 in Jetson device:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
root@nvidia-desktop:/home/nvidia# devmem2 0x0c302030
 +
/dev/mem opened.
 +
Memory mapped at address 0x7f84359000.
 +
Value at address 0xC302030 (0x7f84359030): 0x440</pre>
 +
 +
 +
=== Linux kernel ===
 +
==== Kernel building ====
 +
There are several resources in internet introducing how to build Jetson Linux kernel from source.
 +
But it seems that the NV-provided nvbuild.sh never works in my side.
 +
Here's the script I'm using for kernel building:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
#!/bin/bash
 +
set -e
 +
export KERNEL_SRC_DIR=${HOME}/Work/jetson_sdk/32.4.3/source/Linux_for_Tegra/source/public/kernel-source/kernel/kernel-4.9
 +
export CROSS_COMPILE=${HOME}/Tools/kernel-toolchain/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
 +
export TEGRA_KERNEL_OUT=${HOME}/Work/jetson_sdk/32.4.3/source/Linux_for_Tegra/source/public/kernel-build
 +
export ARCH=arm64
 +
make -C $KERNEL_SRC_DIR ARCH=arm64 LOCALVERSION="-tegra" O=$TEGRA_KERNEL_OUT tegra_defconfig
 +
make -C $KERNEL_SRC_DIR ARCH=arm64 LOCALVERSION="-tegra" O=$TEGRA_KERNEL_OUT -j8
 +
make -C $KERNEL_SRC_DIR ARCH=arm64 INSTALL_MOD_PATH=$TEGRA_KERNEL_OUT/modules_install INSTALL_MOD_STRIP=1 O=$TEGRA_KERNEL_OUT modules_install -j8
 +
</pre>
 +
Note the parameter 'INSTALL_MOD_STRIP=1' should be added, otherwise the built module will be quite large. For example, nvgpu.ko size will increase from 2.6MB to 89MB.
 +
==== Ramdisk customization ====
 +
The original ram-disk image can be copied from device (/boot/initrd) or host SDK directory (Linux_for_Tegra/bootloader/l4t_initrd.img).
 +
 +
    1. Extract the <span style="color:red">initrd</span> by following command:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">zcat xxx/initrd | cpio -idmv</pre>
 +
    2. Change all files in this directory owner as root:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">sudo chown root.root * -R</pre>
 +
    3. Make some private changes. (For example, to add some special echo message in init script.)
 +
    4. Repack the <span style="color:red">initrd</span>:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">find . | cpio -o -H newc | gzip > ../initrd.debug</pre>
 +
    5. Replace the file <span style="color:red">/boot/initrd</span> with generated file <span style="color:red">initrd.debug</span> in above step.
 +
    6. Reboot the device, and check the kernel log to confirm the new <span style="color:red">initrd</span> works.
 +
 +
 +
In addition, the initrd can also be built-in to kernel image.
 +
 +
    1. Extract the initrd by following command:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">zcat xxx/initrd | cpio -idmv</pre>
 +
    2. Change all files in this directory owner as root:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">sudo chown root.root * -R</pre>
 +
    3. Make some private changes. (For example, to add some special echo message in init script.)
 +
    4. Repack the initrd:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">find . | cpio -o -H newc > ../initrd.debug.cpio</pre>
 +
    5. Edit the kernel config, and add the following line:
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
    CONFIG_BLK_DEV_INITRD=y
 +
    CONFIG_INITRAMFS_SOURCE="/home/temp/initr.debug.cpio"
 +
</pre>
 +
    6. Re-build the kernel, replace the kernel image in device.
 +
    7. Comment out the default INITRD in /boot/extlinux/extlinux.conf
 +
<pre style="margin-left:30px; color:#B0B0B0; background-color:#111111; white-space:pre-wrap;">
 +
    ...
 +
    LINUX /boot/Image
 +
    #INITRD /boot/initrd
 +
    ...
 +
</pre>
 +
    8. Reboot the device, and the kernel should use the built-in initrd.

Revision as of 00:46, 16 July 2021

Preparation

First, please download BSP package from internet. It's preferred to download BSP package, instead of Jetpack.

Go to https://developer.nvidia.com/embedded/linux-tegra.

The following packages are necessary:

L4T Driver Package (BSP)

Sample Root Filesystem


Source code for kernel and other components:

L4T Driver Package (BSP) Sources


Toolchain for kernel building:

GCC 7.3.1 for 64 bit BSP and Kernel


Secure package if secure-boot is necessary:

Jetson Platform Fuse Burning and Secure Boot Documentation and Tools


Refer to https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fquick_start.html

Then the device can be flashed by command line. With this method, user can have more controls for the BSP configuration, like pinmux, kernel/kernel DTB customization, etc.

BSP customization

PINMUX

How to find out the actual pinmux configuration file

Refer to https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fmb1_platform_config_xavier.html%23wwpID0E0240HA

There are several types of Jetson reference boards which are supported in SDK.

Check the configuration file

For example, when the flash command is run:

 sudo ./flash.sh jetson-xavier mmcblk0p1

Check the configuration file: jetson-xavier.conf → p2822-0000+p2888-0004.conf → PINMUX_CONFIG="tegra19x-mb1-pinmux-p2888-0000-a04-p2822-0000-b01.cfg";

(Note: the value of PINMUX_CONFIG may be overwritten. So the later one should take effect.)

Or check the flash log

Run following command:

sudo ./flash.sh -r --no-flash jetson-xavier mmcblk0p1

And check the log:

...

copying pinmux_config(/home/Work/jetson_customer/32.4.3/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-pinmux-p2888-0000-a04-p2822-0000-b01.cfg)... done.

...

How to update pinmux

There are several ways to customize the device PINMUX. The simple way is to generate PINMUX configuration through pre-defined excel. Another way is to edit the PINMUX configuration file directly, but that may need more knowledge about PINMUX setting for the chip.

Edit the excel and generate the Configuration

Refer to https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fadaptation_and_bringup_xavier_nx.html%23wwpID0E0WL0HA

Search 'Pinmux Changes'

Also, another good reference: https://elinux.org/Jetson/AGX_Xavier_Update_Pinmux

Still, please make sure the PINMUX configuration file name is correct.

Edit the pinmux configuration file

That's the direct and simple way, assumed the developer is familiar with PINMUX setting.

Download technical reference manual from https://developer.nvidia.com/embedded/downloads#?search=TRM for different platforms. And those documents contain detailed information.

Edit the prod configuration

L4T document shows another way to override the PINMUX:

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fmb1_platform_config_xavier.html%23wwpID0E0A40HA

Same command as 2.1.1.2, and check the prod configuration in following log:

...

copying prod_config(/home/Work/jetson_customer/32.4.3/Linux_for_Tegra/bootloader/t186ref/BCT/tegra19x-mb1-prod-p2888-0000-p2822-0000.cfg)... done.

...

Follow the guide in above link.

How to update device PINMUX

After the pinmux configuration files are changed, re-flash the device:

 sudo ./flash.sh jetson-xavier mmcblk0p1        #flash the whole device

or

 sudo ./flash.sh -k MB1_BCT jetson-xavier mmcblk0p1      #only flash the MB1_BCT
How to verify the new PINUX configuration works

User can read the PINMUX registers to confirm the new configuration works.

For example,

In PINMUX configuration file, there's an entry:

pinmux.0x0c302030 = 0x00000540; # gen2_i2c_scl_pcc7: i2c2, tristate-disable, input-enable, io_high_voltage-disable, lpdr-enable

And in prod configuration file, another entry:

prod.0x0c302030.0x0000100 = 0x00000000; #gen2_i2c_scl_pcc7: LPDR disable

Run a physical memory access tool, like devmem2 in Jetson device:

root@nvidia-desktop:/home/nvidia# devmem2 0x0c302030
/dev/mem opened.
Memory mapped at address 0x7f84359000.
Value at address 0xC302030 (0x7f84359030): 0x440


Linux kernel

Kernel building

There are several resources in internet introducing how to build Jetson Linux kernel from source. But it seems that the NV-provided nvbuild.sh never works in my side. Here's the script I'm using for kernel building:

#!/bin/bash
set -e
export KERNEL_SRC_DIR=${HOME}/Work/jetson_sdk/32.4.3/source/Linux_for_Tegra/source/public/kernel-source/kernel/kernel-4.9
export CROSS_COMPILE=${HOME}/Tools/kernel-toolchain/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
export TEGRA_KERNEL_OUT=${HOME}/Work/jetson_sdk/32.4.3/source/Linux_for_Tegra/source/public/kernel-build
export ARCH=arm64
make -C $KERNEL_SRC_DIR ARCH=arm64 LOCALVERSION="-tegra" O=$TEGRA_KERNEL_OUT tegra_defconfig
make -C $KERNEL_SRC_DIR ARCH=arm64 LOCALVERSION="-tegra" O=$TEGRA_KERNEL_OUT -j8
make -C $KERNEL_SRC_DIR ARCH=arm64 INSTALL_MOD_PATH=$TEGRA_KERNEL_OUT/modules_install INSTALL_MOD_STRIP=1 O=$TEGRA_KERNEL_OUT modules_install -j8

Note the parameter 'INSTALL_MOD_STRIP=1' should be added, otherwise the built module will be quite large. For example, nvgpu.ko size will increase from 2.6MB to 89MB.

Ramdisk customization

The original ram-disk image can be copied from device (/boot/initrd) or host SDK directory (Linux_for_Tegra/bootloader/l4t_initrd.img).

   1. Extract the initrd by following command: 
zcat xxx/initrd | cpio -idmv
   2. Change all files in this directory owner as root: 
sudo chown root.root * -R
   3. Make some private changes. (For example, to add some special echo message in init script.)
   4. Repack the initrd: 
find . | cpio -o -H newc | gzip > ../initrd.debug
   5. Replace the file /boot/initrd with generated file initrd.debug in above step.
   6. Reboot the device, and check the kernel log to confirm the new initrd works.


In addition, the initrd can also be built-in to kernel image.

   1. Extract the initrd by following command: 
zcat xxx/initrd | cpio -idmv
   2. Change all files in this directory owner as root: 
sudo chown root.root * -R
   3. Make some private changes. (For example, to add some special echo message in init script.)
   4. Repack the initrd: 
find . | cpio -o -H newc > ../initrd.debug.cpio
   5. Edit the kernel config, and add the following line:
    CONFIG_BLK_DEV_INITRD=y
    CONFIG_INITRAMFS_SOURCE="/home/temp/initr.debug.cpio"
   6. Re-build the kernel, replace the kernel image in device.
   7. Comment out the default INITRD in /boot/extlinux/extlinux.conf
    ...
    LINUX /boot/Image
    #INITRD /boot/initrd
    ...
   8. Reboot the device, and the kernel should use the built-in initrd.