Jetson/TX1 Upstream Kernel
Upstream Linux kernel
It is possible to boot an upstream Linux kernel on Jetson TX1. The following are instructions that should get you set up. It assumes that you've followed the quickstart instructions from here.
Note that there seem to be slight incompatibilities between the Ubuntu root filesystem provided with L4T and the upstream kernel configuration which may cause weird behaviour during boot. The L4T root filesystem also ships some binaries which will not work on an upstream kernel because it lacks the required support. You might want to replace the Ubuntu root filesystem with something like Arch Linux which you can download from here. To do this, follow the quickstart instructions but instead of extracting the L4T sample root filesystem extract the Arch Linux root filesystem. If you do so, you might want to skip the step that installs the L4T-specific binaries as well.
Rudimentary support for Jetson TX1 is available in linux-next (as of next-20151118). You can clone the repository from here:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git linux-next.git
To build a kernel image and the device tree blobs, run the following commands:
$ cd linux-next.git $ make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnueabihf- O=build/jetson-tx1 defconfig $ make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnueabihf- O=build/jetson-tx1 -j32
If you haven't opened a terminal to the debug UART yet, now's the right time:
$ screen /dev/ttyUSB0 115200
screen works really well as a terminal program, but any other should work just fine, too. You should see the U-Boot prompt in the terminal, if not reset the board and wait for U-Boot to boot up. Then interrupt the automatic boot by pressing any key.
Now put U-Boot on the Jetson TX1 into USB mass storage (UMS) mode:
Tegra210 (P2371-2180) # ums 0 mmc 0
This will export the eMMC as block device over USB. Before running the above you might want to watch the kernel log to see what block device the eMMC is associated with:
$ dmesg --follow
When you enter UMS mode (make sure you have the Jetson TX1 hooked up to your host via the micro-USB cable), the log should show something like this:
Nov 17 12:16:21 ulmo kernel: usb 4-188.8.131.52: new high-speed USB device number 120 using ehci-pci Nov 17 12:16:21 ulmo kernel: usb-storage 4-184.108.40.206:1.0: USB Mass Storage device detected Nov 17 12:16:21 ulmo kernel: scsi host385: usb-storage 4-220.127.116.11:1.0 Nov 17 12:16:22 ulmo kernel: scsi 385:0:0:0: Direct-Access Linux UMS disk ffff PQ: 0 ANSI: 2 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: Attached scsi generic sg3 type 0 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] 61071360 512-byte logical blocks: (31.2 GB/29.1 GiB) Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Write Protect is off Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Mode Sense: 0f 00 00 00 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Nov 17 12:16:22 ulmo kernel: sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc7 sdc8 sdc9 sdc10 sdc11 sdc12 sdc13 sdc14 sdc15 sdc16 sdc17 sdc18 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Attached SCSI removable disk
The sdc (and more precisely sdc1) in the above is what you're going to need. Copy your kernel and DTB files to the root partition:
# mount /dev/sdc1 /mnt # cp build/jetson-tx1/arch/arm64/boot/Image /mnt/boot/Image-upstream # cp build/jetson-tx1/arch/arm64/boot/dts/nvidia/*.dtb /mnt/boot/ # umount /mnt
Note that this keeps the original kernel. If you're not interested in keeping the L4T installation (if you've opted for the Arch Linux root filesystem you probably aren't), you might just as well copy the upstream kernel image over the existing one.
Now the bootloader must be configured to be able to boot the upstream kernel. The L4T kernel requires a bunch of command-line parameters that the upstream kernel doesn't, so again, if you want to keep the L4T setup in place, you can simply add an extlinux.conf configuration snippet:
# mount /dev/sdc1 /mnt # vim /mnt/boot/extlinux/extlinux.conf
Add the following to the end of the file, or overwrite the existing "primary kernel" entry with it:
LABEL upstream MENU LABEL upstream kernel LINUX /boot/Image-upstream FDT /boot/tegra210-p2371-2180.dtb APPEND console=ttyS0,115200n8 root=/dev/mmcblk0p1 rw rootwait
Save the file and unmount the partition:
# umount /mnt
You can now exit UMS mode by pressing Ctrl+C in the terminal window. Sometimes this doesn't react, so you can usually just reset the board and interrupt the automatic boot again.
The downstream kernel requires EMC timings to enable memory frequency scaling. For that purpose U-Boot by default copies some device tree nodes to the DTB that is passed to the kernel. Since upstream doesn't support memory frequency scaling yet, and hence the device tree doesn't contain such a node, this step will fail and hence must be disabled. You can do that by unsetting two environment variables in U-Boot:
Tegra210 (P2371-2180) # env delete fdt_copy_node_names Tegra210 (P2371-2180) # env delete fdt_copy_src_addr
As it happens, this mechanism is useful, though, and it can be repurposed to enable SMP via PSCI. So instead of the above two commands you can execute this single command instead:
Tegra210 (P2371-2180) # env set fdt_copy_node_names /cpus:/psci
Note that U-Boot will reset the environment to the default when you reset. To make these changes permanent you need to save the environment to eMMC:
Tegra210 (P2371-2180) # env save
That's it. You're now ready to boot your upstream kernel:
Tegra210 (P2371-2180) # boot
If you've opted to keep L4T in place, U-Boot will detect that you have two configurations and will ask you which one to boot. If you have a single configuration you can simply wait for the auto-boot timer to run out. If all goes well this should boot you to a login- or command- prompt.
To add Ethernet support to the Linux kernel, the xusb.bin firmware needs to be embedded in the kernel image. Before building the kernel, download the xusb.bin through
Then, copy the xusb.bin file inside the firmware/nvidia/tegra210/ directory in the Linux kernel root.
Ensure that the .config file contains:
CONFIG_FW_LOADER=y CONFIG_FIRMWARE_IN_KERNEL=y CONFIG_EXTRA_FIRMWARE="nvidia/tegra210/xusb.bin" CONFIG_EXTRA_FIRMWARE_DIR="firmware" CONFIG_PHY_TEGRA_XUSB=y CONFIG_USB_RTL8150=y CONFIG_USB_RTL8152=y
Finally, build the kernel.
U-Boot will by default try to boot from the external SD card and fall back to eMMC if no external card is detected. This means that you can easily keep both L4T and an upstream setup easily by moving all your upstream related files onto an external SD card. The setup instructions are similar to the above, except that your U-Boot command to enter UMS mode becomes:
Tegra210 (P2371-2180) # ums 0 mmc 1
You may have to partition and format the SD card first, which you can do with your favourite partitioning tool (such as cfdisk) and your regular mkfs tools:
# cfdisk /dev/sdc [a single partition spanning the entire SD card is usually fine, but be as creative as you want] # mkfs -t ext4 /dev/sdc1
After creating and formatting a partition, mount the partition and extract the root filesystem into it:
# mount /dev/sdc1 /mnt # tar xpf ArchLinuxARM-aarch64-generic-latest.tar.gz -C /mnt # umount /mnt
Now you can simply copy the kernel and DTB files as described above and write the following to /mnt/boot/extlinux/extlinux.conf:
TIMEOUT 30 DEFAULT primary MENU TITLE p2371-2180 SD boot options LABEL primary MENU LABEL primary kernel LINUX /boot/Image FDT /boot/tegra210-p2371-2180.dtb APPEND console=ttyS0,115200n8 root=/dev/mmcblk1p1 rw rootwait
Flashing the board
The open source flashing tools for Jetson do not work with the Jetson TX1, so the proprietary NVidia tools (including special binary versions of tegrarcm etc) are the only flashing tools that seem to work for this board.
These can be obtained as a tarball from https://developer.nvidia.com/embedded/downloads as the "Jetson TX1 Driver Package". Or, currently:
wget http://developer.download.nvidia.com/embedded/L4T/r23_Release_v1.0/Tegra210_Linux_R23.1.1_armhf.tbz2 tar -xf Tegra210_Linux_R23.1.1_armhf.tbz2 cd Linux_for_Tegra
flash.sh wraps distributed binaries, providing for flashing of the NVidia primary bootlader, chain-loaded U-Boot, kernel, as well as partitioning and flashing of the resulting partitions on the embedded eMMC device, via the board's USB recovery mode firmware.
The distributed tarball contains a pre-built U-Boot and kernel, but these may be replaced with any images you wish to flash, by replacing the distributed images in the directory tree. If desired, an unpacked rootfs tree may also be included. During flashing, from this the script assembles a
system.img containing kernel, DTB and rootfs, which is then flashed to the eMMC.
The paths to place any files or images you wish to flash are as follows:
- Extlinux configuration:
The DTB name can be changed in
In version 23.2, the flash size defined in
p2371-2180.conf is incorrect:
sed -i 's/EMMCSIZE=31276924928/EMMCSIZE=15758000128/' p2371-2180.conf
If you wish to create additional partitions, these can be added to
bootloader/t210ref/cfg/gnu_linux_tegraboot_emmc_full.xml, which may also include a named image used for flashing to a given partition, specified in the
To flash only the primary bootloader and U-Boot:
./flash.sh -k EBT p2371-2180 mmcblk0p1
To perform required partitioning, and flash the primary bootloader, U-Boot, kernel and rootfs:
./flash.sh p2371-2180 mmcblk0p1
The partition layout required by the primary bootloader, and created by the flash script is rather complicated, for example:
*** PARTITION LAYOUT(23 partitions) *** [ BCT] BH 0 8191 4.0MiB [ NVC] BH 8192 16383 4.0MiB nvtboot.bin [ PPT] UH 0 4095 2.0MiB [ GP1] UH 4096 8191 2.0MiB [ APP] UH 8192 270335 128.0MiB system.img [ TBC] UV 270336 274431 2.0MiB nvtboot_cpu.bin [ EBT] UV 274432 282623 4.0MiB u-boot-dtb.bin [ BPF] UV 282624 286719 2.0MiB bpmp.bin [ WB0] UV 286720 299007 6.0MiB warmboot.bin [ RP1] UV 299008 307199 4.0MiB tegra210-jetson-cv-base-p2597-2180-a00.dtb [ TOS] UV 307200 319487 6.0MiB tos.img [ EKS] UV 319488 323583 2.0MiB [ FX] UV 323584 327679 2.0MiB [ SOS] UV 327680 368639 20.0MiB [ EXI] UV 368640 499711 64.0MiB [ LNX] UV 499712 630783 64.0MiB [ DTB] UV 630784 638975 4.0MiB tegra210-jetson-cv-base-p2597-2180-a00.dtb [ NXT] UV 638976 643071 2.0MiB [ MXB] UV 643072 655359 6.0MiB [ MXP] UV 655360 667647 6.0MiB [ USP] UV 667648 671743 2.0MiB [ UDA] UV 671744 30756863 14690.0MiB [ GPT] UH 30756864 30760959 2.0MiB
The feature set supported in linux-next is currently very limited. It will get you to a login- or command-prompt and you can access SD/MMC storage, but that's pretty much it.
There is a development tree on github that serves as a staging tree for various new features. The staging/work branch is the top-of-tree and usually based on a recent linux-next. On top of that, there is support for DSI (in case you have the necessary display module) and HDMI, as well as xHCI for networking. There is also preliminary support for the GPU, though that will require pulling in a number of userspace changes as well.
If you get stuck anywhere in the above, feel free to let me know. You can reach me, as well as other people working on upstream support via email or on IRC (#tegra on Freenode).
To help troubleshoot issues, you might want to add one or more of the following kernel command-line parameters:
- earlycon=uart8250,mmio32,0x70006000 earlyprintk: Enable debug UART output early during the boot process. This is useful to diagnose problems that appear to be hangs with no output to the serial console at all.
- ignore_loglevel: ignore kernel message log level (useful to see more verbose messages)