Jetson TX1

= Upstream Linux kernel =

It is possible to boot an upstream Linux kernel on Jetson TX1. The following are instructions that should get you set up. It assumes that you've followed the quickstart instructions from here.

Note that there seem to be slight incompatibilities between the Ubuntu root filesystem provided with L4T and the upstream kernel configuration which may cause weird behaviour during boot. The L4T root filesystem also ships some binaries which will not work on an upstream kernel because it lacks the required support. You might want to replace the Ubuntu root filesystem with something like Arch Linux which you can download from here. To do this, follow the quickstart instructions but instead of extracting the L4T sample root filesystem extract the Arch Linux root filesystem. If you do so, you might want to skip the step that installs the L4T-specific binaries as well.

Rudimentary support for Jetson TX1 is available in linux-next (as of next-20151118). You can clone the repository from here:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git linux-next.git

To build a kernel image and the device tree blobs, run the following commands:

$ cd linux-next.git $ make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnueabihf- O=build/jetson-tx1 defconfig $ make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnueabihf- O=build/jetson-tx1 -j32

If you haven't opened a terminal to the debug UART yet, now's the right time:

$ screen /dev/ttyUSB0 115200

screen works really well as a terminal program, but any other should work just fine, too. You should see the U-Boot prompt in the terminal, if not reset the board and wait for U-Boot to boot up. Then interrupt the automatic boot by pressing any key.

Now put U-Boot on the Jetson TX1 into USB mass storage (UMS) mode:

Tegra210 (P2371-2180) # ums 0 mmc 0

This will export the eMMC as block device over USB. Before running the above you might want to watch the kernel log to see what block device the eMMC is associated with:

$ dmesg --follow

When you enter UMS mode (make sure you have the Jetson TX1 hooked up to your host via the micro-USB cable), the log should show something like this:

Nov 17 12:16:21 ulmo kernel: usb 4-1.5.1.5: new high-speed USB device number 120 using ehci-pci Nov 17 12:16:21 ulmo kernel: usb-storage 4-1.5.1.5:1.0: USB Mass Storage device detected Nov 17 12:16:21 ulmo kernel: scsi host385: usb-storage 4-1.5.1.5:1.0 Nov 17 12:16:22 ulmo kernel: scsi 385:0:0:0: Direct-Access    Linux    UMS disk         ffff PQ: 0 ANSI: 2 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: Attached scsi generic sg3 type 0 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] 61071360 512-byte logical blocks: (31.2 GB/29.1 GiB) Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Write Protect is off Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Mode Sense: 0f 00 00 00 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Nov 17 12:16:22 ulmo kernel: sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc7 sdc8 sdc9 sdc10 sdc11 sdc12 sdc13 sdc14 sdc15 sdc16 sdc17 sdc18 Nov 17 12:16:22 ulmo kernel: sd 385:0:0:0: [sdc] Attached SCSI removable disk

The sdc (and more precisely sdc1) in the above is what you're going to need. Copy your kernel and DTB files to the root partition:

# mount /dev/sdc1 /mnt # cp build/jetson-tx1/arch/arm64/boot/Image /mnt/boot/Image-upstream # cp build/jetson-tx1/arch/arm64/boot/dts/nvidia/*.dtb /mnt/boot/ # umount /mnt

Note that this keeps the original kernel. If you're not interested in keeping the L4T installation (if you've opted for the Arch Linux root filesystem you probably aren't), you might just as well copy the upstream kernel image over the existing one.

Now the bootloader must be configured to be able to boot the upstream kernel. The L4T kernel requires a bunch of command-line parameters that the upstream kernel doesn't, so again, if you want to keep the L4T setup in place, you can simply add an extlinux.conf configuration snippet:

# mount /dev/sdc1 /mnt # vim /mnt/boot/extlinux/extlinux.conf

Add the following to the end of the file, or overwrite the existing "primary kernel" entry with it:

LABEL upstream MENU LABEL upstream kernel LINUX /boot/Image-upstream FDT /boot/tegra210-p2371-2180.dtb APPEND console=ttyS0,115200n8 root=/dev/mmcblk0p1 rw rootwait

Save the file and unmount the partition:

# umount /mnt

You can now exit UMS mode by pressing Ctrl+C in the terminal window. Sometimes this doesn't react, so you can usually just reset the board and interrupt the automatic boot again.

The downstream kernel requires EMC timings to enable memory frequency scaling. For that purpose U-Boot by default copies some device tree nodes to the DTB that is passed to the kernel. Since upstream doesn't support memory frequency scaling yet, and hence the device tree doesn't contain such a node, this step will fail and hence must be disabled. You can do that by unsetting two environment variables in U-Boot:

Tegra210 (P2371-2180) # env delete fdt_copy_node_names Tegra210 (P2371-2180) # env delete fdt_copy_src_addr

As it happens, this mechanism is useful, though, and it can be repurposed to enable SMP via PSCI. So instead of the above two commands you can execute this single command instead:

Tegra210 (P2371-2180) # env set fdt_copy_node_names /cpus:/psci

Note that U-Boot will reset the environment to the default when you reset. To make these changes permanent you need to save the environment to eMMC:

Tegra210 (P2371-2180) # env save

That's it. You're now ready to boot your upstream kernel:

Tegra210 (P2371-2180) # boot

If you've opted to keep L4T in place, U-Boot will detect that you have two configurations and will ask you which one to boot. If you have a single configuration you can simply wait for the auto-boot timer to run out. If all goes well this should boot you to a login- or command- prompt.

Dual-Boot
U-Boot will by default try to boot from the external SD card and fall back to eMMC if no external card is detected. This means that you can easily keep both L4T and an upstream setup easily by moving all your upstream related files onto an external SD card. The setup instructions are similar to the above, except that your U-Boot command to enter UMS mode becomes:

Tegra210 (P2371-2180) # ums 0 mmc 1

You may have to partition and format the SD card first, which you can do with your favourite partitioning tool (such as cfdisk) and your regular mkfs tools:

# cfdisk /dev/sdc [a single partition spanning the entire SD card is usually fine, but be as creative as you want] # mkfs -t ext4 /dev/sdc1

After creating and formatting a partition, mount the partition and extract the root filesystem into it:

# mount /dev/sdc1 /mnt # tar xpf ArchLinuxARM-aarch64-generic-latest.tar.gz -C /mnt # umount /mnt

Now you can simply copy the kernel and DTB files as described above and write the following to /mnt/boot/extlinux/extlinux.conf:

TIMEOUT 30 DEFAULT primary MENU TITLE p2371-2180 SD boot options LABEL primary MENU LABEL primary kernel LINUX /boot/Image FDT /boot/tegra210-p2371-2180.dtb APPEND console=ttyS0,115200n8 root=/dev/mmcblk1p1 rw rootwait

Flashing the board
The open source flashing tools for Jetson do not work with the Jetson TX1, so the proprietary NVidia tools (including special binary versions of tegrarcm etc) are the only flashing tools that seem to work for this board.

These can be obtained as a tarball from https://developer.nvidia.com/embedded/downloads as the "Jetson TX1 Driver Package". Or, currently:

wget http://developer.download.nvidia.com/embedded/L4T/r23_Release_v1.0/Tegra210_Linux_R23.1.1_armhf.tbz2 tar -xf Tegra210_Linux_R23.1.1_armhf.tbz2 cd Linux_for_Tegra

The script  wraps distributed binaries, providing for flashing of the NVidia primary bootlader, chain-loaded U-Boot, kernel, as well as partitioning and flashing of the resulting partitions on the embedded eMMC device, via the board's USB recovery mode firmware.

The distributed tarball contains a pre-built U-Boot and kernel, but these may be replaced with any images you wish to flash, by replacing the distributed images in the directory tree. If desired, an unpacked rootfs tree may also be included. During flashing, from this the script assembles a  containing kernel, DTB and rootfs, which is then flashed to the eMMC.

The paths to place any files or images you wish to flash are as follows:


 * Extlinux configuration:
 * U-Boot:
 * Kernel:
 * DTB:

The DTB name can be changed in.

In version 23.2, the flash size defined in  is incorrect:

sed -i 's/EMMCSIZE=31276924928/EMMCSIZE=15758000128/' p2371-2180.conf

If you wish to create additional partitions, these can be added to, which may also include a named image used for flashing to a given partition, specified in the   tags.

To flash only the primary bootloader and U-Boot:

./flash.sh -k EBT p2371-2180 mmcblk0p1

To perform required partitioning, and flash the primary bootloader, U-Boot, kernel and rootfs:

./flash.sh p2371-2180 mmcblk0p1

The partition layout required by the primary bootloader, and created by the flash script is rather complicated, for example:

*** PARTITION LAYOUT(23 partitions) *** [    BCT] BH            0         8191       4.0MiB [    NVC] BH         8192        16383       4.0MiB nvtboot.bin [    PPT] UH            0         4095       2.0MiB [    GP1] UH         4096         8191       2.0MiB [    APP] UH         8192       270335     128.0MiB system.img [    TBC] UV       270336       274431       2.0MiB nvtboot_cpu.bin [    EBT] UV       274432       282623       4.0MiB u-boot-dtb.bin [    BPF] UV       282624       286719       2.0MiB bpmp.bin [    WB0] UV       286720       299007       6.0MiB warmboot.bin [    RP1] UV       299008       307199       4.0MiB tegra210-jetson-cv-base-p2597-2180-a00.dtb [    TOS] UV       307200       319487       6.0MiB tos.img [    EKS] UV       319488       323583       2.0MiB [     FX] UV       323584       327679       2.0MiB [    SOS] UV       327680       368639      20.0MiB [    EXI] UV       368640       499711      64.0MiB [    LNX] UV       499712       630783      64.0MiB [    DTB] UV       630784       638975       4.0MiB tegra210-jetson-cv-base-p2597-2180-a00.dtb [    NXT] UV       638976       643071       2.0MiB [    MXB] UV       643072       655359       6.0MiB [    MXP] UV       655360       667647       6.0MiB [    USP] UV       667648       671743       2.0MiB [    UDA] UV       671744     30756863   14690.0MiB [    GPT] UH     30756864     30760959       2.0MiB

Bleeding Edge
The feature set supported in linux-next is currently very limited. It will get you to a login- or command-prompt and you can access SD/MMC storage, but that's pretty much it.

There is a development tree on github that serves as a staging tree for various new features. The staging/work branch is the top-of-tree and usually based on a recent linux-next. On top of that, there is support for DSI (in case you have the necessary display module) and HDMI, as well as xHCI for networking. There is also preliminary support for the GPU, though that will require pulling in a number of userspace changes as well.

Support
If you get stuck anywhere in the above, feel free to let me know. You can reach me, as well as other people working on upstream support via [mailto:linux-tegra@vger.kernel.org email] or on IRC (#tegra on Freenode).

To help troubleshoot issues, you might want to add one or more of the following kernel command-line parameters:
 * earlycon=uart8250,mmio32,0x70006000 earlyprintk: Enable debug UART output early during the boot process. This is useful to diagnose problems that appear to be hangs with no output to the serial console at all.
 * ignore_loglevel: ignore kernel message log level (useful to see more verbose messages)

= Serial Console Wiring =

The serial console is extremely valuable for many reasons, including the ability to choose menu entries for boot images during boot, as well as having access even when networking and direct monitor/keyboard/mouse are unavailable. This functionality is accessed via the rectangular connector J21, near the power-on button.

The Software Side
Any serial port program should work as a console. Examples are PuTTY, gtkTerm, and minicom. Connection speed is 115200, with 8 bits, no parity, and 1 stop bit (115200 8N1). Flow control will be RTS/CTS.

The USB Serial UART
Because of the custom nature of the TX1 board connector wiring, a USB serial UART is by far the easiest and most reliable way to connect the host side of the serial console. This serial UART is required to handle 3.3V I/O. Most of the serial UARTs available should work if the UART is designed for 3.3V I/O. Beware that other USB serial UART cables are available which support an incompatibile 5V or 12V I/O...do not use these.

Because the FTDI USB chips are so widely available and often have drivers by default for Linux host machines, this chipset is a good starting point when you have many choices. The example USB serial UART used here is the TTL-232R-3V3 by FTDI. One source of this serial UART is here: https://www.digikey.com/product-detail/en/TTL-232R-3V3/768-1015-ND/1836393.

About Serial Port Cabling
The example cable comes with a 6-wire cable and 0.1in spacing 6-pin female header. Five of the 6 pins are used. This cable could be used directly with the J17 UART header, but this information is concerned with showing how to wire the standard 6-wire serial UART cable to connector J21 (J21 has the serial console UART, not J17...don't try the cable with J17 unless you know what you are doing).

Color Codes and Pin Descriptions of the USB Serial UART Cable
Pin 1 of the serial USB UART 6-pin connector should be marked.


 * Pin 1 is black.
 * This is a ground wire in common with both x86 host and Jetson TX1 ground.
 * J21 pin 9 is convenient (there are several ground pins which would work).
 * Pin 2 is brown.
 * This is the host-side CTS (clear-to-send).
 * This will connect to Jetson TX1's RTS (request-to-send).
 * J21 pin 11 should be used.
 * Pin 3 is red.
 * This pin is supplied +5V from the host USB HUB, and is not used.
 * Pin 4 is orange.
 * This pin is TXD (transmit data).
 * This will connect to Jetson TX1's RXD (receive data).
 * J21 pin 10 should be used.
 * Pin 5 is yellow.
 * This pin is RXD (recieve data).
 * This will connect to Jetson TX1's TXD (transmit data).
 * J21 pin 8 should be used.
 * Pin 6 is green.
 * This pin is RTS (request-to-send).
 * This will connect to Jetson TX1's CTS (clear-to-send).
 * J21 pin 36 should be used.

The crossing over between send and receive (as well as request to send and clear to send) means this is a "NULL modem" or "crossover" cable.

Note that if you prefer to simplify the assembly, you can ignore the connection between CTS and RTS. Doing so implies using software flow control instead of CTS/RTS flow control

Required Parts
The USB serial UART has already been mentioned. The 6-pin connector is easily used in combination with soldering a ribbon cable to a standard 0.1in connector. The example here should work: https://www.digikey.com/product-detail/en/M20-9990646/952-2270-ND/3728234.

Any standard 0.1in spacing female header should work for the Jetson TX1's J21 connector. A connector capable of self-splicing of an inserted ribbon cable is recommended. Although many of these connectors were not intended to be re-opened and added to at a later date, there isn't usually an issue with opening the connector up and adding wires to a position which has never been used before. Here is a large list of example female 40-pin headers arranged in two rows: https://www.digikey.com/product-search/en?pv88=23&FV=fff40016%2Cfff802f5%2C1640001%2C1680002%2C800000f%2C8000014%2C8cc0003&k=ribbon&mnonly=0&newproducts=0&ColumnSort=0&page=1&quantity=0&ptm=0&fid=0&pageSize=500.

Soldering to the 6-pin header will not be easy without a proper soldering iron. Once soldered these pins need insulation. The example used small diameter heat shrink tubing around each conductor, and then a larger diameter section of heat shrink tubing was used to keep the entire assembly together. Any stray wires may also be insulated with heat shrink tubing.

The cable was generic ribbon cable which thankfully was color coded exactly the same as the 6-pin cable of the USB serial UART. This makes knowing which orientation to use for connecting male and female 6-pin connectors a non-issue. If there is no color coding, then some means of insuring pin 1 of one connector matches pin 1 of the other connector is required.

The Example USB Serial UART Assembly
Here's a picture of what a working assembly is like:

Connecting a Linux Host to Jetson TX1 Serial Console
The final step from your Linux host to connecting to the serial console is to name the serial device file which belongs to the USB serial UART. This can present a challenge and a lot of guessing if you move your USB devices around, as they might enumerate differently depending on order of detecting USB devices.

The general naming of serial USB UART devices is of the format /dev/ttyUSB0, /dev/ttyUSB1, /dev/ttyUSB2...these names work well, but may not be convenient in the long run.

The udev automated system of naming device special files also categorizes many devices by type and then by information to the specific device (something like a serial number). These alternate names remain constant, and are simply symbolic links to the more common ttyUSB# scheme. These files are found in /dev/serial/by-id/.

The example USB serial UART has this symbolic link in /dev/serial/by-id/: usb-FTDI_FT232R_USB_UART_A10171DT-if00-port0. Other USB related serial console labels may change, this particular file should stay constant. This makes it easy to have several USB serial devices without mixing them up when operated simultaneously.

Some Serial Port Administration Information
Typically, the serial port requires sudo or root authority to access, which can be irritating. The serial port devices are group dialout, and adding your user to this gains access to serial ports without sudo. Programs requiring lock files (some of which are serial console programs) may require sudo or root access unless your user is a member of group lock. Consider using a non-root developer account and granting these groups:

sudo usermod -a -G dialout  ''sudo usermod -a -G lock  ''

Note that if you have installed OpenOCD, then udev may attempt to set your ttyUSB group to "plugdev", rather than to "dialout". If "plugdev" does not exist, then your ttyUSB files will default to group "root". If you use Fedora (and who knows how many other distributions), then getting group "dialout" back requires edit of "/lib/udev/rules.d/99-openocd.rules", replacing "plugdev" with "dialout".