Difference between revisions of "R-Car/Virtualization/VFIO"

From eLinux.org
Jump to: navigation, search
m (Add Salvator-X)
(IOMMU groups work when using a device tied to an IOMMU (e.g. SATA))
 
(11 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
Device Pass-Through Using VFIO
 
Device Pass-Through Using VFIO
  
= Platform Device Pass-Through Prototype =
+
= GPIO Pass-Through Prototype =
  
 
This is a proof-of-concept showing how to provide guest access to an R-Car GPIO controller block on the Renesas [[R-Car/Boards/Salvator-X|Salvator-X]] and [[R-Car/Boards/Salvator-XS|Salvator-XS]] boards.
 
This is a proof-of-concept showing how to provide guest access to an R-Car GPIO controller block on the Renesas [[R-Car/Boards/Salvator-X|Salvator-X]] and [[R-Car/Boards/Salvator-XS|Salvator-XS]] boards.
Line 11: Line 11:
 
* Build and boot a host kernel using:
 
* Build and boot a host kernel using:
 
** Repository:  https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-drivers.git
 
** Repository:  https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-drivers.git
** Branch: "topic/rcar3-virt-gpio-passthrough"
+
** Branch: "topic/rcar3-virt-gpio-passthrough-v3"
 
** Config: "renesas_defconfig"
 
** Config: "renesas_defconfig"
  
 
* Configure workarounds for missing functionality:
 
* Configure workarounds for missing functionality:
 
<pre>
 
<pre>
$ echo 0 > /sys/module/vfio_platform/parameters/reset_required
 
 
$ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
 
$ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
 
</pre>
 
</pre>
Line 38: Line 37:
 
* Build a guest kernel using:
 
* Build a guest kernel using:
 
** Repository:  https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-drivers.git
 
** Repository:  https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-drivers.git
** Branch: "topic/rcar3-virt-gpio-passthrough"
+
** Branch: "topic/rcar3-virt-gpio-passthrough-v3"
 
** Config: "virt_defconfig"
 
** Config: "virt_defconfig"
  
 
* Build QEMU using:
 
* Build QEMU using:
 
** Repository: https://github.com/geertu/qemu.git
 
** Repository: https://github.com/geertu/qemu.git
** Branch: "TBD"
+
** Branch: "topic/rcar3-virt-gpio-passthrough-v3"
  
* When starting QEMU, specify pass-through of the GPIO6 platform device.  The device specifier consists of three parts:
+
* When starting QEMU, specify pass-through of the GPIO6 platform device.  The device specifier consists of either the device's node name, or the full path to the device's node in sysfs.
*; $1a
 
*: the device's node name, or
 
*; $1b
 
*: the full path to the device's node in sysfs,
 
*; $2
 
*: the manufacturer part of the device's compatible value,
 
*; $3
 
*: the model part of the device's compatible value.
 
 
I.e.:
 
I.e.:
 
<pre>
 
<pre>
-device vfio-platform,host=$1a,manufacturer=$2,model=$3
+
-device vfio-platform,host=e6055400.gpio
-device vfio-platform,host=e6055400.gpio,manufacturer=renesas,model=rcar-gen3-gpio
+
-device vfio-platform,sysfsdev=/sys/bus/platform/devices/e6055400.gpio
-device vfio-platform,sysfsdev=$1b,manufacturer=$2,model=$3
 
-device vfio-platform,sysfsdev=/sys/bus/platform/devices/e6055400.gpio,manufacturer=renesas,model=rcar-gen3-gpio
 
 
</pre>
 
</pre>
  
Line 66: Line 55:
 
$ qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
 
$ qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
 
         -kernel /path/to/guest/kernel/Image \
 
         -kernel /path/to/guest/kernel/Image \
         -device vfio-platform,host=e6055400.gpio,manufacturer=renesas,model=rcar-gen3-gpio
+
         -device vfio-platform,host=e6055400.gpio
</pre>
 
 
 
Host kernel output:
 
<pre>
 
vfio-platform e6055400.gpio: no reset function found!
 
vfio-platform e6055400.gpio: vfio-noiommu device opened by user (qemu-system-aar:3003)
 
 
</pre>
 
</pre>
  
Line 78: Line 61:
 
<pre>
 
<pre>
 
Booting Linux on physical CPU 0x0000000000 [0x411fd073]
 
Booting Linux on physical CPU 0x0000000000 [0x411fd073]
 +
Linux version 4.18.0-rc6-arm64-virt-00012-g33dccc007797d60a ...
 
...
 
...
gpio_rcar c000000.e6055400.gpio: missing clock, ignoring
 
gpio_rcar c000000.e6055400.gpio: missing IRQ, ignoring
 
 
gpio_rcar c000000.e6055400.gpio: driving 32 GPIOs
 
gpio_rcar c000000.e6055400.gpio: driving 32 GPIOs
 
...
 
...
 +
</pre>
 +
 +
Host kernel output:
 +
<pre>
 +
vfio-platform e6055400.gpio: reset
 +
vfio-platform e6055400.gpio: vfio-noiommu device opened by user (qemu-system-aar:3003)
 
</pre>
 
</pre>
  
Line 89: Line 77:
 
--- /sys/firmware/devicetree/base.orig
 
--- /sys/firmware/devicetree/base.orig
 
+++ /sys/firmware/devicetree/base
 
+++ /sys/firmware/devicetree/base
@@ -124,6 +124,13 @@
+
@@ -126,6 +126,14 @@
 
                 compatible = "qemu,platform", "simple-bus";
 
                 compatible = "qemu,platform", "simple-bus";
 
                 interrupt-parent = <0x8001>;
 
                 interrupt-parent = <0x8001>;
Line 96: Line 84:
 
+              e6055400.gpio@0 {
 
+              e6055400.gpio@0 {
 
+                      #gpio-cells = <0x2>;
 
+                      #gpio-cells = <0x2>;
+                      compatible = "renesas,rcar-gen3-gpio";
+
+                      #interrupt-cells = <0x2>;
 +
+                      compatible = "renesas,gpio-r8a7795", "renesas,rcar-gen3-gpio";
 
+                      gpio-controller;
 
+                      gpio-controller;
 +
+                      interrupt-controller;
 +
+                      interrupts = <0x0 0x70 0x4>;
 
+                      reg = <0x0 0x50>;
 
+                      reg = <0x0 0x50>;
 
+              };
 
+              };
Line 105: Line 96:
 
</pre>
 
</pre>
  
* Export GPIOs used for LEDs, and turn all LEDs off:
+
* Export GPIOs used for LEDs (LED4/5/6), and turn all LEDs off:
 
<pre>
 
<pre>
base=$(cat /sys/class/gpio/gpiochip*/base)
+
# base=$(cat /sys/class/gpio/gpiochip*/base)
for i in $(seq 11 13); do
+
# for i in 11 12 13; do
 
     echo $(($base + $i)) > /sys/class/gpio/export
 
     echo $(($base + $i)) > /sys/class/gpio/export
 
     echo low > /sys/class/gpio/gpio$(($base + $i))/direction
 
     echo low > /sys/class/gpio/gpio$(($base + $i))/direction
Line 116: Line 107:
 
* Cycle through all LEDs, letting them blink once:
 
* Cycle through all LEDs, letting them blink once:
 
<pre>
 
<pre>
for i in $(seq 11 13); do
+
# for i in 11 12 13; do
 
     echo high > /sys/class/gpio/gpio$(($base + $i))/direction
 
     echo high > /sys/class/gpio/gpio$(($base + $i))/direction
 
     sleep 1
 
     sleep 1
Line 123: Line 114:
 
</pre>
 
</pre>
  
== Future Work ==
+
* Change the first GPIO from output to input, and configure edge detection to enable its interrupt:
 +
<pre>
 +
# echo in > /sys/class/gpio/gpio$(($base + 11))/direction
 +
# echo both > /sys/class/gpio/gpio$(($base + 11))/edge
 +
# grep gpio /proc/interrupts
 +
  5:          0    GIC-0 144 Level    c000000.e6055400.gpio
 +
42:          0  c000000.e6055400.gpio  11 Edge      gpiolib
 +
</pre>
 +
 
 +
* Monitor the GPIO interrupt (press button SW20 to trigger):
 +
<pre>
 +
# while /bin/true; do grep gpiolib /proc/interrupts; done | uniq -c
 +
      71298  42:          0  c000000.e6055400.gpio  11 Edge      gpiolib
 +
        388  42:          1  c000000.e6055400.gpio  11 Edge      gpiolib
 +
        247  42:          2  c000000.e6055400.gpio  11 Edge      gpiolib
 +
        139  42:          3  c000000.e6055400.gpio  11 Edge      gpiolib
 +
        586  42:          4  c000000.e6055400.gpio  11 Edge      gpiolib
 +
^C
 +
</pre>
 +
 
 +
* Shut the guest down:
 +
<pre>
 +
# poweroff
 +
</pre>
 +
 
 +
Guest kernel output:
 +
<pre>
 +
reboot: Power down
 +
</pre>
 +
 
 +
Host kernel output:
 +
<pre>
 +
vfio-platform e6055400.gpio: reset
 +
</pre>
 +
 
 +
The GPIO module has been reset, turning on all three LEDs.
 +
 
 +
 
 +
= Serial Pass through =
 +
Creating a serial pass-through will allow you to test VFIO platform pass-through using a serial loopback (USB-A -> USB Micro-B) between a host and guest using the R-Car platform.
 +
 
 +
== Host Side ==
 +
* Utilise a host kernel configured as stated in the section [[R-Car/Virtualization/VFIO#Host_Side|Host Side]] above
 +
* To work around the lack of dynamic clock handling within QEmu, a [HACK] patch is provided to hardcode the SCIF clocks:
 +
  git://git.kernel.org/pub/scm/linux/kernel/git/kbingham/rcar.git virt/serial-passthrough
 +
 
 +
* Configure workarounds for no IOMMU support
 +
$ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
 +
 
 +
* Unbind the SCIF1 from the "sh-sci" (rcar-gen3-scif compatible) driver:
 +
$ echo e6e68000.serial > /sys/bus/platform/drivers/sh-sci/unbind
 +
 
 +
* Rebind the SCIF1 to the vfio-platform driver:
 +
$ echo vfio-platform > /sys/bus/platform/devices/e6e68000.serial/driver_override
 +
$ echo e6e68000.serial > /sys/bus/platform/drivers/vfio-platform/bind
 +
 
 +
== Guest Side ==
 +
Build QEMU using:
 +
    Repository: https://github.com/kbingham/qemu.git
 +
    Branch: "rcar3/serial/passthrough"
 +
 
 +
Launch with the Serial passthrough:
 +
    -device vfio-platform,host=e6e68000.serial,manufacturer=renesas,model=rcar-gen3-scif \
 +
 
 +
Full Q-Emu command:
 +
qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
 +
    -kernel /path/to/guest/kernel/Image \
 +
    -device vfio-platform,host=e6e68000.serial,manufacturer=renesas,model=rcar-gen3-scif \
 +
    -append "loglevel=7 root=/dev/nfs nfsroot=192.168.0.39:/path/to/nfsroot/,nolock,v3 rw ip=dhcp"
 +
 
 +
''Ensure that SLiRP is configured in your QEmu build to support networking, and NFS roots.''
 +
 
 +
== Serial loopback test ==
 +
 
 +
Dependencies:
 +
https://github.com/geertu/sertest
 +
* Compile and install the sertest utility in both the host, and guest root filesystem.
 +
 
 +
We can utilise a USB-A -> USB Micro-B cable as a serial-loopback to test the SCIF device passthrough.
 +
 
 +
* Connect a USB cable between the Debug Serial 1 port, and a free available USB2 Type A socket.
 +
* Launch a guest with SCIF pass through as above in [[R-Car/Virtualization/VFIO#Guest_Side_2|Guest Side]]
 +
 
 +
* 'In' the *guest*:
 +
  $ sertest --slave -s 9600 /dev/ttySC0
 +
 
 +
* 'On' the *host*:
 +
  $ sertest --master -s 9600 /dev/ttyUSB0
 +
 
 +
 
 +
= SATA Pass-Through =
 +
 
 +
This shows how to provide guest access to an R-Car SATA controller block on the Renesas [[R-Car/Boards/Salvator-X|Salvator-X]] and [[R-Car/Boards/Salvator-XS|Salvator-XS]] boards.
 +
 
 +
== Host Side ==
 +
 
 +
* Connect a SATA device to the SATA port on Salvator-X(S) (R-Car H3 only!).
 +
 
 +
* Build and boot a host kernel according to section [[R-Car/Virtualization/VFIO#Host_Side|Host Side]] above.
 +
 
 +
* Unbind SATA from the "sata_rcar" driver (actual SATA device output may vary):
 +
<pre>
 +
$ echo ee300000.sata > /sys/bus/platform/drivers/sata_rcar/unbind
 +
ata1.00: disabled
 +
sd 0:0:0:0: [sda] Synchronizing SCSI cache
 +
sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x04 driverbyte=0x00
 +
sd 0:0:0:0: [sda] Stopping disk
 +
sd 0:0:0:0: [sda] Start/Stop Unit failed: Result: hostbyte=0x04 driverbyte=0x00
 +
</pre>
 +
 
 +
* Bind SATA to the "vfio-platform" driver, for pass-through to guests:
 +
<pre>
 +
$ echo vfio-platform > /sys/bus/platform/devices/ee300000.sata/driver_override
 +
$ echo ee300000.sata > /sys/bus/platform/drivers/vfio-platform/bind
 +
</pre>
 +
 
 +
== Guest Side ==
  
* VFIO reset driver
+
* Build a guest kernel and QEMU according to section [[R-Car/Virtualization/VFIO#Guest_Side|Guest Side]] above.
* IOMMU groups
+
 
* Interrupts
+
* Launch QEMU:
* Clocks
+
<pre>
* PM Domains
+
$ qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
 +
        -kernel /path/to/guest/kernel/Image \
 +
        -device vfio-platform,host=ee300000.sata
 +
</pre>
 +
 
 +
Guest kernel output (actual SATA device output may vary):
 +
<pre>
 +
Booting Linux on physical CPU 0x0000000000 [0x411fd073]
 +
Linux version 4.18.0-rc6-arm64-virt-00012-g33dccc007797d60a ...
 +
...
 +
scsi host0: sata_rcar
 +
ata1: SATA max UDMA/133 irq 5
 +
...
 +
ata1: link resume succeeded after 1 retries
 +
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
 +
ata1.00: ATA-7: Maxtor 6L160M0, BANC1G10, max UDMA/133
 +
ata1.00: 320173056 sectors, multi 0: LBA48 NCQ (not used)
 +
ata1.00: configured for UDMA/133
 +
scsi 0:0:0:0: Direct-Access    ATA      Maxtor 6L160M0  1G10 PQ: 0 ANSI: 5
 +
sd 0:0:0:0: [sda] 320173056 512-byte logical blocks: (164 GB/153 GiB)
 +
sd 0:0:0:0: [sda] Write Protect is off
 +
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 +
sda: sda1
 +
sd 0:0:0:0: [sda] Attached SCSI disk
 +
...
 +
</pre>
 +
 
 +
Host kernel output:
 +
<pre>
 +
vfio-platform ee300000.sata: Using IPMMU context 1
 +
vfio-platform ee300000.sata: reset
 +
</pre>
 +
 
 +
* Analysis of "/sys/firmware/devicetree/base" using "dtx_diff" shows that "ee300000.sata@0" has been added as a subnode to the existing "platform@c000000" node:
 +
<pre>
 +
--- /sys/firmware/devicetree/base.orig
 +
+++ /sys/firmware/devicetree/base
 +
@@ -126,6 +126,12 @@
 +
                compatible = "qemu,platform", "simple-bus";
 +
                interrupt-parent = <0x8001>;
 +
                ranges = <0x0 0x0 0xc000000 0x2000000>;
 +
 +
+              ee300000.sata@0 {
 +
+                      compatible = "renesas,sata-r8a7795", "renesas,rcar-gen3-sata";
 +
+                      interrupts = <0x0 0x70 0x4>;
 +
+                      reg = <0x0 0x200000>;
 +
+              };
 +
        };
 +
 
 +
        pmu {
 +
</pre>
 +
 
 +
* Depending on the SATA device, you can mount a partition.
 +
 
 +
* Shut the guest down:
 +
<pre>
 +
# poweroff
 +
</pre>
 +
 
 +
Guest kernel output:
 +
<pre>
 +
reboot: Power down
 +
</pre>
 +
 
 +
Host kernel output:
 +
<pre>
 +
vfio-platform ee300000.sata: reset
 +
vfio-platform ee300000.sata: Reusing IPMMU context 0
 +
</pre>

Latest revision as of 09:46, 25 July 2018

Device Pass-Through Using VFIO

GPIO Pass-Through Prototype

This is a proof-of-concept showing how to provide guest access to an R-Car GPIO controller block on the Renesas Salvator-X and Salvator-XS boards.

Host Side

  • Configure workarounds for missing functionality:
$ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
  • Unbind GPIO6 from the "gpio-rcar" driver:
$ echo e6055400.gpio > /sys/bus/platform/drivers/gpio_rcar/unbind
gpio gpiochip6: REMOVING GPIOCHIP WITH GPIOS STILL REQUESTED
  • Bind GPIO6 to the "vfio-platform" driver, for pass-through to guests:
$ echo vfio-platform > /sys/bus/platform/devices/e6055400.gpio/driver_override
$ echo e6055400.gpio > /sys/bus/platform/drivers/vfio-platform/bind
iommu: Adding device e6055400.gpio to group 0
vfio-platform e6055400.gpio: Adding kernel taint for vfio-noiommu group on device

Guest Side

  • When starting QEMU, specify pass-through of the GPIO6 platform device. The device specifier consists of either the device's node name, or the full path to the device's node in sysfs.

I.e.:

-device vfio-platform,host=e6055400.gpio
-device vfio-platform,sysfsdev=/sys/bus/platform/devices/e6055400.gpio

Full command line:

$ qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
        -kernel /path/to/guest/kernel/Image \
        -device vfio-platform,host=e6055400.gpio

Guest kernel output:

Booting Linux on physical CPU 0x0000000000 [0x411fd073]
Linux version 4.18.0-rc6-arm64-virt-00012-g33dccc007797d60a ...
...
gpio_rcar c000000.e6055400.gpio: driving 32 GPIOs
...

Host kernel output:

vfio-platform e6055400.gpio: reset
vfio-platform e6055400.gpio: vfio-noiommu device opened by user (qemu-system-aar:3003)
  • Analysis of "/sys/firmware/devicetree/base" using "dtx_diff" shows that "e6055400.gpio@0" has been added as a subnode to the existing "platform@c000000" node:
--- /sys/firmware/devicetree/base.orig
+++ /sys/firmware/devicetree/base
@@ -126,6 +126,14 @@
                compatible = "qemu,platform", "simple-bus";
                interrupt-parent = <0x8001>;
                ranges = <0x0 0x0 0xc000000 0x2000000>;
+
+               e6055400.gpio@0 {
+                       #gpio-cells = <0x2>;
+                       #interrupt-cells = <0x2>;
+                       compatible = "renesas,gpio-r8a7795", "renesas,rcar-gen3-gpio";
+                       gpio-controller;
+                       interrupt-controller;
+                       interrupts = <0x0 0x70 0x4>;
+                       reg = <0x0 0x50>;
+               };
        };

        pmu {
  • Export GPIOs used for LEDs (LED4/5/6), and turn all LEDs off:
# base=$(cat /sys/class/gpio/gpiochip*/base)
# for i in 11 12 13; do
    echo $(($base + $i)) > /sys/class/gpio/export
    echo low > /sys/class/gpio/gpio$(($base + $i))/direction
done
  • Cycle through all LEDs, letting them blink once:
# for i in 11 12 13; do
    echo high > /sys/class/gpio/gpio$(($base + $i))/direction
    sleep 1
    echo low > /sys/class/gpio/gpio$(($base + $i))/direction
done
  • Change the first GPIO from output to input, and configure edge detection to enable its interrupt:
# echo in > /sys/class/gpio/gpio$(($base + 11))/direction
# echo both > /sys/class/gpio/gpio$(($base + 11))/edge
# grep gpio /proc/interrupts
  5:          0     GIC-0 144 Level     c000000.e6055400.gpio
 42:          0  c000000.e6055400.gpio  11 Edge      gpiolib
  • Monitor the GPIO interrupt (press button SW20 to trigger):
# while /bin/true; do grep gpiolib /proc/interrupts; done | uniq -c
      71298  42:          0  c000000.e6055400.gpio  11 Edge      gpiolib
        388  42:          1  c000000.e6055400.gpio  11 Edge      gpiolib
        247  42:          2  c000000.e6055400.gpio  11 Edge      gpiolib
        139  42:          3  c000000.e6055400.gpio  11 Edge      gpiolib
        586  42:          4  c000000.e6055400.gpio  11 Edge      gpiolib
^C
  • Shut the guest down:
# poweroff

Guest kernel output:

reboot: Power down

Host kernel output:

vfio-platform e6055400.gpio: reset

The GPIO module has been reset, turning on all three LEDs.


Serial Pass through

Creating a serial pass-through will allow you to test VFIO platform pass-through using a serial loopback (USB-A -> USB Micro-B) between a host and guest using the R-Car platform.

Host Side

  • Utilise a host kernel configured as stated in the section Host Side above
  • To work around the lack of dynamic clock handling within QEmu, a [HACK] patch is provided to hardcode the SCIF clocks:
 git://git.kernel.org/pub/scm/linux/kernel/git/kbingham/rcar.git virt/serial-passthrough
  • Configure workarounds for no IOMMU support
$ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
  • Unbind the SCIF1 from the "sh-sci" (rcar-gen3-scif compatible) driver:
$ echo e6e68000.serial > /sys/bus/platform/drivers/sh-sci/unbind
  • Rebind the SCIF1 to the vfio-platform driver:
$ echo vfio-platform > /sys/bus/platform/devices/e6e68000.serial/driver_override
$ echo e6e68000.serial > /sys/bus/platform/drivers/vfio-platform/bind

Guest Side

Build QEMU using:

   Repository: https://github.com/kbingham/qemu.git
   Branch: "rcar3/serial/passthrough"

Launch with the Serial passthrough:

   -device vfio-platform,host=e6e68000.serial,manufacturer=renesas,model=rcar-gen3-scif \

Full Q-Emu command:

qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
   -kernel /path/to/guest/kernel/Image \
   -device vfio-platform,host=e6e68000.serial,manufacturer=renesas,model=rcar-gen3-scif \
   -append "loglevel=7 root=/dev/nfs nfsroot=192.168.0.39:/path/to/nfsroot/,nolock,v3 rw ip=dhcp"

Ensure that SLiRP is configured in your QEmu build to support networking, and NFS roots.

Serial loopback test

Dependencies:

https://github.com/geertu/sertest
* Compile and install the sertest utility in both the host, and guest root filesystem.

We can utilise a USB-A -> USB Micro-B cable as a serial-loopback to test the SCIF device passthrough.

  • Connect a USB cable between the Debug Serial 1 port, and a free available USB2 Type A socket.
  • Launch a guest with SCIF pass through as above in Guest Side
  • 'In' the *guest*:
 $ sertest --slave -s 9600 /dev/ttySC0
  • 'On' the *host*:
 $ sertest --master -s 9600 /dev/ttyUSB0


SATA Pass-Through

This shows how to provide guest access to an R-Car SATA controller block on the Renesas Salvator-X and Salvator-XS boards.

Host Side

  • Connect a SATA device to the SATA port on Salvator-X(S) (R-Car H3 only!).
  • Build and boot a host kernel according to section Host Side above.
  • Unbind SATA from the "sata_rcar" driver (actual SATA device output may vary):
$ echo ee300000.sata > /sys/bus/platform/drivers/sata_rcar/unbind
ata1.00: disabled
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x04 driverbyte=0x00
sd 0:0:0:0: [sda] Stopping disk
sd 0:0:0:0: [sda] Start/Stop Unit failed: Result: hostbyte=0x04 driverbyte=0x00
  • Bind SATA to the "vfio-platform" driver, for pass-through to guests:
$ echo vfio-platform > /sys/bus/platform/devices/ee300000.sata/driver_override
$ echo ee300000.sata > /sys/bus/platform/drivers/vfio-platform/bind

Guest Side

  • Build a guest kernel and QEMU according to section Guest Side above.
  • Launch QEMU:
$ qemu-system-aarch64 -enable-kvm -M virt -cpu cortex-a57 -m 1024 -nographic \
        -kernel /path/to/guest/kernel/Image \
        -device vfio-platform,host=ee300000.sata

Guest kernel output (actual SATA device output may vary):

 
Booting Linux on physical CPU 0x0000000000 [0x411fd073]
Linux version 4.18.0-rc6-arm64-virt-00012-g33dccc007797d60a ... 
...
scsi host0: sata_rcar
ata1: SATA max UDMA/133 irq 5
...
ata1: link resume succeeded after 1 retries
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: ATA-7: Maxtor 6L160M0, BANC1G10, max UDMA/133
ata1.00: 320173056 sectors, multi 0: LBA48 NCQ (not used)
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      Maxtor 6L160M0   1G10 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 320173056 512-byte logical blocks: (164 GB/153 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1
sd 0:0:0:0: [sda] Attached SCSI disk
... 

Host kernel output:

vfio-platform ee300000.sata: Using IPMMU context 1
vfio-platform ee300000.sata: reset
  • Analysis of "/sys/firmware/devicetree/base" using "dtx_diff" shows that "ee300000.sata@0" has been added as a subnode to the existing "platform@c000000" node:
--- /sys/firmware/devicetree/base.orig
+++ /sys/firmware/devicetree/base
@@ -126,6 +126,12 @@
                compatible = "qemu,platform", "simple-bus";
                interrupt-parent = <0x8001>;
                ranges = <0x0 0x0 0xc000000 0x2000000>;
+   
+               ee300000.sata@0 {
+                       compatible = "renesas,sata-r8a7795", "renesas,rcar-gen3-sata";
+                       interrupts = <0x0 0x70 0x4>;
+                       reg = <0x0 0x200000>;
+               };
        };

        pmu {
  • Depending on the SATA device, you can mount a partition.
  • Shut the guest down:
# poweroff

Guest kernel output:

reboot: Power down

Host kernel output:

vfio-platform ee300000.sata: reset
vfio-platform ee300000.sata: Reusing IPMMU context 0