Jetson/TX1 Upstream Kernel

From eLinux.org
Jump to: navigation, search

Upstream Linux kernel

It is possible to boot an upstream Linux kernel on Jetson TX1. The following are instructions that should get you set up. It assumes that you've followed the quickstart instructions from here.

Note that there seem to be slight incompatibilities between the Ubuntu root filesystem provided with L4T and the upstream kernel configuration which may cause weird behaviour during boot. The L4T root filesystem also ships some binaries which will not work on an upstream kernel because it lacks the required support. You might want to replace the Ubuntu root filesystem with something like Arch Linux which you can download from here. To do this, follow the quickstart instructions but instead of extracting the L4T sample root filesystem extract the Arch Linux root filesystem. If you do so, you might want to skip the step that installs the L4T-specific binaries as well.

Rudimentary support for Jetson TX1 is available in linux-next (as of next-20151118). You can clone the repository from here:

 $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git linux-next.git

To build a kernel image and the device tree blobs, run the following commands:

 $ cd linux-next.git
 $ make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnueabihf- O=build/jetson-tx1 defconfig
 $ make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnueabihf- O=build/jetson-tx1 -j32

If you haven't opened a terminal to the debug UART yet, now's the right time:

 $ screen /dev/ttyUSB0 115200

screen works really well as a terminal program, but any other should work just fine, too. You should see the U-Boot prompt in the terminal, if not reset the board and wait for U-Boot to boot up. Then interrupt the automatic boot by pressing any key.

Now put U-Boot on the Jetson TX1 into USB mass storage (UMS) mode:

 Tegra210 (P2371-2180) # ums 0 mmc 0

This will export the eMMC as block device over USB. Before running the above you might want to watch the kernel log to see what block device the eMMC is associated with:

 $ dmesg --follow

When you enter UMS mode (make sure you have the Jetson TX1 hooked up to your host via the micro-USB cable), the log should show something like this:

 Nov 17 12:16:21 ulmo kernel: usb 4-1.5.1.5: new high-speed USB device number 120 using ehci-pci
 Nov 17 12:16:21 ulmo kernel: usb-storage 4-1.5.1.5:1.0: USB Mass Storage device detected
 Nov 17 12:16:21 ulmo kernel: scsi host385: usb-storage 4-1.5.1.5:1.0
 Nov 17 12:16:22 ulmo kernel: scsi 385:0:0:0: Direct-Access     Linux    UMS disk         ffff PQ: 0 ANSI: 2
 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: Attached scsi generic sg3 type 0
 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] 61071360 512-byte logical blocks: (31.2 GB/29.1 GiB)
 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Write Protect is off
 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Mode Sense: 0f 00 00 00
 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 Nov 17 12:16:22 ulmo kernel:  sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc7 sdc8 sdc9 sdc10 sdc11 sdc12 sdc13 sdc14 sdc15 sdc16 sdc17 sdc18
 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Attached SCSI removable disk

The sdc (and more precisely sdc1) in the above is what you're going to need. Copy your kernel and DTB files to the root partition:

 # mount /dev/sdc1 /mnt
 # cp build/jetson-tx1/arch/arm64/boot/Image /mnt/boot/Image-upstream
 # cp build/jetson-tx1/arch/arm64/boot/dts/nvidia/*.dtb /mnt/boot/
 # umount /mnt

Note that this keeps the original kernel. If you're not interested in keeping the L4T installation (if you've opted for the Arch Linux root filesystem you probably aren't), you might just as well copy the upstream kernel image over the existing one.

Now the bootloader must be configured to be able to boot the upstream kernel. The L4T kernel requires a bunch of command-line parameters that the upstream kernel doesn't, so again, if you want to keep the L4T setup in place, you can simply add an extlinux.conf configuration snippet:

 # mount /dev/sdc1 /mnt
 # vim /mnt/boot/extlinux/extlinux.conf

Add the following to the end of the file, or overwrite the existing "primary kernel" entry with it:

 LABEL upstream
       MENU LABEL upstream kernel
       LINUX /boot/Image-upstream
       FDT /boot/tegra210-p2371-2180.dtb
       APPEND console=ttyS0,115200n8 root=/dev/mmcblk0p1 rw rootwait

Save the file and unmount the partition:

  # umount /mnt

You can now exit UMS mode by pressing Ctrl+C in the terminal window. Sometimes this doesn't react, so you can usually just reset the board and interrupt the automatic boot again.

The downstream kernel requires EMC timings to enable memory frequency scaling. For that purpose U-Boot by default copies some device tree nodes to the DTB that is passed to the kernel. Since upstream doesn't support memory frequency scaling yet, and hence the device tree doesn't contain such a node, this step will fail and hence must be disabled. You can do that by unsetting two environment variables in U-Boot:

 Tegra210 (P2371-2180) # env delete fdt_copy_node_names
 Tegra210 (P2371-2180) # env delete fdt_copy_src_addr

As it happens, this mechanism is useful, though, and it can be repurposed to enable SMP via PSCI. So instead of the above two commands you can execute this single command instead:

 Tegra210 (P2371-2180) # env set fdt_copy_node_names /cpus:/psci

Note that U-Boot will reset the environment to the default when you reset. To make these changes permanent you need to save the environment to eMMC:

 Tegra210 (P2371-2180) # env save

That's it. You're now ready to boot your upstream kernel:

 Tegra210 (P2371-2180) # boot

If you've opted to keep L4T in place, U-Boot will detect that you have two configurations and will ask you which one to boot. If you have a single configuration you can simply wait for the auto-boot timer to run out. If all goes well this should boot you to a login- or command- prompt.

Ethernet support

To add Ethernet support to the Linux kernel, the xusb.bin firmware needs to be embedded in the kernel image. Before building the kernel, download the xusb.bin through

  git clone http://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/

Then, copy the xusb.bin file inside the firmware/nvidia/tegra210/ directory in the Linux kernel root.

Ensure that the .config file contains:

  CONFIG_FW_LOADER=y
  CONFIG_FIRMWARE_IN_KERNEL=y
  CONFIG_EXTRA_FIRMWARE="nvidia/tegra210/xusb.bin"
  CONFIG_EXTRA_FIRMWARE_DIR="firmware"
  CONFIG_PHY_TEGRA_XUSB=y
  CONFIG_USB_RTL8150=y
  CONFIG_USB_RTL8152=y

Finally, build the kernel.

Dual-Boot

U-Boot will by default try to boot from the external SD card and fall back to eMMC if no external card is detected. This means that you can easily keep both L4T and an upstream setup easily by moving all your upstream related files onto an external SD card. The setup instructions are similar to the above, except that your U-Boot command to enter UMS mode becomes:

 Tegra210 (P2371-2180) # ums 0 mmc 1

You may have to partition and format the SD card first, which you can do with your favourite partitioning tool (such as cfdisk) and your regular mkfs tools:

 # cfdisk /dev/sdc
 [a single partition spanning the entire SD card is usually fine, but be as creative as you want]
 # mkfs -t ext4 /dev/sdc1

After creating and formatting a partition, mount the partition and extract the root filesystem into it:

 # mount /dev/sdc1 /mnt
 # tar xpf ArchLinuxARM-aarch64-generic-latest.tar.gz -C /mnt
 # umount /mnt

Now you can simply copy the kernel and DTB files as described above and write the following to /mnt/boot/extlinux/extlinux.conf:

 TIMEOUT 30
 DEFAULT primary
 
 MENU TITLE p2371-2180 SD boot options
 
 LABEL primary
       MENU LABEL primary kernel
       LINUX /boot/Image
       FDT /boot/tegra210-p2371-2180.dtb
       APPEND console=ttyS0,115200n8 root=/dev/mmcblk1p1 rw rootwait

Flashing the board

The open source flashing tools for Jetson do not work with the Jetson TX1, so the proprietary NVidia tools (including special binary versions of tegrarcm etc) are the only flashing tools that seem to work for this board.

These can be obtained as a tarball from https://developer.nvidia.com/embedded/downloads as the "Jetson TX1 Driver Package". Or, currently:

wget http://developer.download.nvidia.com/embedded/L4T/r23_Release_v1.0/Tegra210_Linux_R23.1.1_armhf.tbz2
tar -xf Tegra210_Linux_R23.1.1_armhf.tbz2
cd Linux_for_Tegra

The script flash.sh wraps distributed binaries, providing for flashing of the NVidia primary bootlader, chain-loaded U-Boot, kernel, as well as partitioning and flashing of the resulting partitions on the embedded eMMC device, via the board's USB recovery mode firmware.

The distributed tarball contains a pre-built U-Boot and kernel, but these may be replaced with any images you wish to flash, by replacing the distributed images in the directory tree. If desired, an unpacked rootfs tree may also be included. During flashing, from this the script assembles a system.img containing kernel, DTB and rootfs, which is then flashed to the eMMC.

The paths to place any files or images you wish to flash are as follows:

  • Extlinux configuration: bootloader/t210ref/p2371-2180/extlinux.conf.emmc
  • U-Boot: bootloader/t210ref/p2371-2180/u-boot*
  • Kernel: bootloader/t210ref/p2371-2180/Image
  • DTB: bootloader/t210ref/p2371-2180/tegra210-jetson-cv-base-p2597-2180-a00.dtb

The DTB name can be changed in p2371-2180.conf.

In version 23.2, the flash size defined in p2371-2180.conf is incorrect:

sed -i 's/EMMCSIZE=31276924928/EMMCSIZE=15758000128/' p2371-2180.conf

If you wish to create additional partitions, these can be added to bootloader/t210ref/cfg/gnu_linux_tegraboot_emmc_full.xml, which may also include a named image used for flashing to a given partition, specified in the <filename> tags.

To flash only the primary bootloader and U-Boot:

./flash.sh -k EBT p2371-2180 mmcblk0p1

To perform required partitioning, and flash the primary bootloader, U-Boot, kernel and rootfs:

./flash.sh p2371-2180 mmcblk0p1

The partition layout required by the primary bootloader, and created by the flash script is rather complicated, for example:

*** PARTITION LAYOUT(23 partitions) ***
[     BCT] BH            0         8191       4.0MiB 
[     NVC] BH         8192        16383       4.0MiB nvtboot.bin
[     PPT] UH            0         4095       2.0MiB 
[     GP1] UH         4096         8191       2.0MiB 
[     APP] UH         8192       270335     128.0MiB system.img
[     TBC] UV       270336       274431       2.0MiB nvtboot_cpu.bin
[     EBT] UV       274432       282623       4.0MiB u-boot-dtb.bin
[     BPF] UV       282624       286719       2.0MiB bpmp.bin
[     WB0] UV       286720       299007       6.0MiB warmboot.bin
[     RP1] UV       299008       307199       4.0MiB tegra210-jetson-cv-base-p2597-2180-a00.dtb
[     TOS] UV       307200       319487       6.0MiB tos.img
[     EKS] UV       319488       323583       2.0MiB 
[      FX] UV       323584       327679       2.0MiB 
[     SOS] UV       327680       368639      20.0MiB 
[     EXI] UV       368640       499711      64.0MiB 
[     LNX] UV       499712       630783      64.0MiB 
[     DTB] UV       630784       638975       4.0MiB tegra210-jetson-cv-base-p2597-2180-a00.dtb
[     NXT] UV       638976       643071       2.0MiB 
[     MXB] UV       643072       655359       6.0MiB 
[     MXP] UV       655360       667647       6.0MiB 
[     USP] UV       667648       671743       2.0MiB 
[     UDA] UV       671744     30756863   14690.0MiB
[     GPT] UH     30756864     30760959       2.0MiB

Bleeding Edge

The feature set supported in linux-next is currently very limited. It will get you to a login- or command-prompt and you can access SD/MMC storage, but that's pretty much it.

There is a development tree on github that serves as a staging tree for various new features. The staging/work branch is the top-of-tree and usually based on a recent linux-next. On top of that, there is support for DSI (in case you have the necessary display module) and HDMI, as well as xHCI for networking. There is also preliminary support for the GPU, though that will require pulling in a number of userspace changes as well.

Support

If you get stuck anywhere in the above, feel free to let me know. You can reach me, as well as other people working on upstream support via email or on IRC (#tegra on Freenode).

To help troubleshoot issues, you might want to add one or more of the following kernel command-line parameters:

  • earlycon=uart8250,mmio32,0x70006000 earlyprintk: Enable debug UART output early during the boot process. This is useful to diagnose problems that appear to be hangs with no output to the serial console at all.
  • ignore_loglevel: ignore kernel message log level (useful to see more verbose messages)