R-Car/Tests:rcar-vin

This document describes how to test the VIN and CSI-2 (only on Gen3) functionality on Renesas R-Car Gen2 and Gen3.

Get the code
Ideally all the code to test VIN should be available in the mainline kernel, unfortunately this is not the case yet. Renesas driver parts not yet picked-up are available in renesas-drivers tree while none Renesas drivers (e.g. the ADV7482 driver for Gen3 testing) are only available in the out-of-tree repository bellow.

There are also some patches needed to facilitate testing that are not part of renesas-drivers such as kernel configurations and DT changes. To ease testing a tag is available where all patches are included which can be build and used to test at:

https://git.ragnatech.se/linux rcar-vin-elinux-v9

Gen2
All that is needed is included in v4.10.

Gen3

 * VIN and CSI2 driver support is included in renesas-drivers.
 * Prototype driver for ADV7482 are not available in renesas-drivers but in the out-of-tree repository mentioned above, this driver is incomplete and requires more work before it's ready for mainline.
 * To test CVBS or HDMI input the RCAR_VIN, RCAR_CSI2 and ADV7482 needs to be enabled (not default in defconfig) and added to DT, see example in out-of-tree repository mentioned above.

Boot the kernel
You should see this on successful probe (this is from Gen3):

rcar-vin e6ef0000.video: Device registered as video0 rcar-vin e6ef1000.video: Device registered as video1 rcar-vin e6ef2000.video: Device registered as video2 rcar-vin e6ef3000.video: Device registered as video3 rcar-vin e6ef4000.video: Device registered as video4 rcar-vin e6ef5000.video: Device registered as video5 rcar-vin e6ef6000.video: Device registered as video6 rcar-vin e6ef7000.video: Device registered as video7

Media Controller (Gen3)
The hardware available on Gen3 is much more complex then on Gen2. For this reason it's no longer enough to interact with only the video device to control the hardware, the entire media pipeline needs to configured. To facilitate this the Media Controller (MC) framework is used exclusivity on Gen3. By using the MC framework it is up to user-space to first configure the media pipeline and then setup the correct format for each entity involved in the pipeline before streaming can be started on the video device.

The pipeline will consist of a video sensor (CSI-2 Transmitter, ADV7482 on Salvator-X) a CSI-2 receiver (R-Car CSI-2) and a dma engine (R-Car VIN). The video sensor is immutably connected to a single R-Car CSI-2 while the routing between the R-Car CSI-2 and R-Car VIN can be configured at runtime. There are limitations on which channel from the R-Car CSI-2 node can be connected to which R-Car VIN node, this is a hardware limitation that is different for each SoC in the Gen3 series. All links between these different nodes and all possible routing configurations are exposed as a media device to user-space. After the links in the pipeline have been setup by the user the proper format needs to be set on all nodes involved in the pipeline before the user can start a stream at the video device.

To control the routing and format from user-space the user needs to interact with the media device, the subdevices for video sensor and R-Car CSI-2, the video node for R-Car VIN. For configuring the media device and subdevices 'media-ctl' can be used and to configure the video device itself a grabber can be used, such as 'yavta' or 'qv4l2'. The following sections will briefly describe how to use these tools to configure the pipeline for streaming.

Show current routing
Examine the current routing setup with 'media-ctl -d /dev/mediaX -p'.

Note: In the example bellow the ADV7482 node is shown twice, this do not represent reality but is one of the hacks in the ADV7482 driver which needs to be solved before it's ready for upstream.

$ media-ctl -d /dev/media0 -p Media controller API version 0.1.0

Media device information

driver         Renesas VIN model          video serial bus info       /soc/video@e6ef0000 hw revision    0x0 driver version 4.11.0

Device topology - entity 1: rcar_vin e6ef0000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "rcar_csi2 feaa0000.csi2":1 [ENABLED] <- "rcar_csi2 fea80000.csi2":1 []

- entity 5: rcar_csi2 feaa0000.csi2 (5 pads, 8 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:unknown/0x0] <- "adv7482 4-0070":10 [ENABLED,IMMUTABLE] pad1: Source [fmt:unknown/0x0] -> "rcar_vin e6ef0000.video":0 [ENABLED] -> "rcar_vin e6ef1000.video":0 [] -> "rcar_vin e6ef2000.video":0 [] pad2: Source [fmt:unknown/0x0] -> "rcar_vin e6ef1000.video":0 [] -> "rcar_vin e6ef3000.video":0 [ENABLED] pad3: Source [fmt:unknown/0x0] -> "rcar_vin e6ef2000.video":0 [] pad4: Source [fmt:unknown/0x0] -> "rcar_vin e6ef3000.video":0 []

- entity 11: rcar_csi2 fea80000.csi2 (5 pads, 15 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:unknown/0x0] <- "adv7482 4-0034":11 [ENABLED,IMMUTABLE] pad1: Source [fmt:unknown/0x0] -> "rcar_vin e6ef0000.video":0 [] -> "rcar_vin e6ef1000.video":0 [ENABLED] -> "rcar_vin e6ef2000.video":0 [] -> "rcar_vin e6ef4000.video":0 [] -> "rcar_vin e6ef5000.video":0 [ENABLED] -> "rcar_vin e6ef6000.video":0 [] pad2: Source [fmt:unknown/0x0] -> "rcar_vin e6ef1000.video":0 [] -> "rcar_vin e6ef3000.video":0 [] -> "rcar_vin e6ef5000.video":0 [] -> "rcar_vin e6ef7000.video":0 [] pad3: Source [fmt:unknown/0x0] -> "rcar_vin e6ef2000.video":0 [] -> "rcar_vin e6ef6000.video":0 [] pad4: Source [fmt:unknown/0x0] -> "rcar_vin e6ef3000.video":0 [] -> "rcar_vin e6ef7000.video":0 []

- entity 17: adv7482 4-0034 (12 pads, 1 link) type V4L2 subdev subtype Decoder flags 0 device node name /dev/v4l-subdev2 pad0: Sink pad1: Sink pad2: Sink pad3: Sink pad4: Sink pad5: Sink pad6: Sink pad7: Sink pad8: Sink pad9: Sink pad10: Source [fmt:RGB888_1X24/720x480 field:interlaced] pad11: Source [fmt:UYVY8_2X8/720x576 field:interlaced] -> "rcar_csi2 fea80000.csi2":0 [ENABLED,IMMUTABLE]

- entity 30: adv7482 4-0070 (12 pads, 1 link) type V4L2 subdev subtype Decoder flags 0 device node name /dev/v4l-subdev3 pad0: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad1: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad2: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad3: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad4: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad5: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad6: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad7: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad8: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad9: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad10: Source [fmt:RGB888_1X24/640x480 field:none] [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] [dv.detect:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic] [dv.current:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic] -> "rcar_csi2 feaa0000.csi2":0 [ENABLED,IMMUTABLE] pad11: Source [fmt:UYVY8_2X8/720x576 field:interlaced] [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] [dv.detect:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic] [dv.current:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic]

- entity 59: rcar_vin e6ef1000.video (1 pad, 4 links) type Node subtype V4L flags 0 device node name /dev/video1 pad0: Sink <- "rcar_csi2 feaa0000.csi2":1 [] <- "rcar_csi2 feaa0000.csi2":2 [] <- "rcar_csi2 fea80000.csi2":1 [ENABLED] <- "rcar_csi2 fea80000.csi2":2 []

- entity 71: rcar_vin e6ef2000.video (1 pad, 4 links) type Node subtype V4L flags 0 device node name /dev/video2 pad0: Sink <- "rcar_csi2 feaa0000.csi2":1 [] <- "rcar_csi2 feaa0000.csi2":3 [] <- "rcar_csi2 fea80000.csi2":1 [] <- "rcar_csi2 fea80000.csi2":3 []

- entity 83: rcar_vin e6ef3000.video (1 pad, 4 links) type Node subtype V4L flags 0 device node name /dev/video3 pad0: Sink <- "rcar_csi2 feaa0000.csi2":2 [ENABLED] <- "rcar_csi2 feaa0000.csi2":4 [] <- "rcar_csi2 fea80000.csi2":2 [] <- "rcar_csi2 fea80000.csi2":4 []

- entity 95: rcar_vin e6ef4000.video (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video4 pad0: Sink <- "rcar_csi2 fea80000.csi2":1 []

- entity 101: rcar_vin e6ef5000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video5 pad0: Sink <- "rcar_csi2 fea80000.csi2":1 [ENABLED] <- "rcar_csi2 fea80000.csi2":2 []

- entity 109: rcar_vin e6ef6000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video6 pad0: Sink <- "rcar_csi2 fea80000.csi2":1 [] <- "rcar_csi2 fea80000.csi2":3 []

- entity 117: rcar_vin e6ef7000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video7 pad0: Sink <- "rcar_csi2 fea80000.csi2":2 [] <- "rcar_csi2 fea80000.csi2":4 []

Activate/Deactivate a link
The Media Controller framework allows user-space to enable/disable a link and that way control the routing of video data from a R-Car CSI-2 to a R-Car VIN instance. Not all CSI-2 channels are possible to route to any and all VIN instances, this is limited by the SoC hardware and is specific for each SoC version. Look at the previous section on how to show which links are possible on your particular SoC. Further more not all links can be activated independent of other links, this is also a limitation set by the hardware. If you try to enable a link which would not be possible with regard to other already active links the operation will fail with a EMLINK error. Try deactivating some other link to crate more routing possibilities and try again, all possible routing setups for a specific SoC are documented in the datasheet.

A link is always configured from a R-Car CSI-2 entity to a R-Car VIN entity, the same way the video data is flowing.

To enable a link use the media-ctl utility from v4l-utils package:


 * 1) media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin e6ef0000.video':0 [1]"

To disable the same link use:


 * 1) media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin.e6ef0000.video':0 [0]"

Deactivate all active links
This is a useful command to reset all links before you start enabling new links to make sure you got the biggest possible routing space to start out with.


 * 1) media-ctl -d /dev/media0 -r

Example configuring the pipeline and propagate format
Once the user have configured a pipeline using 'media-ctl' as described above the format needs to be propagated in the pipeline before streaming can start. In this example we use a R-Car H3 board where the HDMI input will be configured to output on VIN2 and CVBS configured to VIN4. If we look at the media graph as show above we can identify which nodes are involved in each of these two pipelines:

HDMI: 'adv7482 4-0070' -> 'rcar_csi2 feaa0000.csi2' -> 'rcar_vin e6ef2000.video' CVBS: 'adv7482 4-0034' -> 'rcar_csi2 fea80000.csi2' -> 'rcar_vin e6ef4000.video'

Start by clearing any previous links and configure the ones we want, remember that the link between the adv7482 and the R-Car CSI-2 entities are immutable and won't be clear and therefore do not need to be set again.

media-ctl -d /dev/media0 -r media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'rcar_vin e6ef2000.video':0 [1]" media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin e6ef4000.video':0 [1]"

When the pipelines are configured it's time to propagate the format in the pipeline, start by examine which format is present at the video sources sinks.

$ media-ctl -d /dev/media0 --get-v4l2 "'adv7482 4-0070':10" [fmt:RGB888_1X24/640x480 field:none]
 * 1) HDMI

$ media-ctl -d /dev/media0 --get-v4l2 "'adv7482 4-0034':11" [fmt:UYVY8_2X8/720x576 field:interlaced]
 * 1) CVBS

Now we know the formats so we can propagate them:

media-ctl -d /dev/media0 -V "'adv7482 4-0070':10 [fmt:RGB888_1X24/640x480 field:none]" media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/640x480 field:none]"
 * 1) HDMI

media-ctl -d /dev/media0 -V "'adv7482 4-0070':11 [fmt:UYVY8_2X8/720x576 field:interlaced]" media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x576 field:interlaced]"
 * 1) CVBS

Once the pipeline format is setup we can then grab frames using for example 'yavta' or look use a GUI like 'qv4l2' yavta -f RGB565 -s 640x480 --field none -n 4 --capture=10 -F /dev/video2
 * 1) HDMI

yavta -f RGB565 -s 720x576 --field interlaced -n 4 --capture=10 -F /dev/video4
 * 1) CVBS

Multi-channel CSI-2 tests
Multi-channel CSI-2 testing uses an expansion board for the R-Car H3 Salvator-X based on a Maxim MAX9286 quad-channel GMSL (Gigabit Multimedia Serial Link) deserializer. Up to four cameras can be connected to the deserializer, which outputs the four video streams on separate virtual channels over one CSI-2 connections.

The cameras used in this setup are IMI RDACM20 camera modules based on an Omnivision 10635 camera sensor and a Maxim MAX9271 GMSL serializer. The GMSL link carries power, control (I2C) and video data over a single coax cable.

At this time only capture a single virtual channel is supported, connected to the first GMSL camera.

On the hardware side, make sure to set SW29 to OFF-OFF on the Salvator-X board to select the external CSI-2 devices, to remove the J19 jumper from the MAX9286 expansion board to disable the second MAX9286, and to connect RDACM20 cameras to the first four GMSL ports (J1 to J4).

On the software side, the latest version of the work-in-progress kernel code is available from

git://linuxtv.org/pinchartl/media.git vin-gmsl-20170515

The kernel must be compiled with the following options.

CONFIG_VIDEO_MAX9286=y CONFIG_VIDEO_RDACM20=y CONFIG_VIDEO_RCAR_CSI2=y CONFIG_VIDEO_RCAR_VIN=y
 * 1) CONFIG_VIDEO_RENESAS_FDP1 is not set
 * 2) CONFIG_VIDEO_RENESAS_JPU is not set
 * 3) CONFIG_VIDEO_RENESAS_VSP1 is not set

Disabling FDP1, JPU and VSP1 ensure that the media controller and video device nodes will have known names. You can enable those drivers if you update the device names accordingly in the commands below.

You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply run

Device /dev/video0 opened. Device `R_Car_VIN' on `platform:e6ef0000.video' is a video capture device. Video format set: YUYV (56595559) 1280x800 (stride 2560) buffer size 2048000 Video format: YUYV (56595559) 1280x800 (stride 2560) buffer size 2048000 8 buffers requested. length: 2048000 offset: 0 Buffer 0 mapped at address 0x7f80c94000. length: 2048000 offset: 2048000 Buffer 1 mapped at address 0x7f80aa0000. length: 2048000 offset: 4096000 Buffer 2 mapped at address 0x7f808ac000. length: 2048000 offset: 6144000 Buffer 3 mapped at address 0x7f806b8000. length: 2048000 offset: 8192000 Buffer 4 mapped at address 0x7f804c4000. length: 2048000 offset: 10240000 Buffer 5 mapped at address 0x7f802d0000. length: 2048000 offset: 12288000 Buffer 6 mapped at address 0x7f800dc000. length: 2048000 offset: 14336000 Buffer 7 mapped at address 0x7f7fee8000. 0 (0) [-] 0 2048000 bytes 182.333337 182.333348 8.889 fps 1 (1) [-] 1 2048000 bytes 182.366871 182.366878 29.820 fps 2 (2) [-] 2 2048000 bytes 182.400405 182.400412 29.820 fps 3 (3) [-] 3 2048000 bytes 182.433939 182.433946 29.820 fps 4 (4) [-] 4 2048000 bytes 182.467473 182.467480 29.820 fps 5 (5) [-] 5 2048000 bytes 182.501007 182.501014 29.820 fps 6 (6) [-] 6 2048000 bytes 182.534541 182.534548 29.820 fps 7 (7) [-] 7 2048000 bytes 182.568076 182.568082 29.820 fps 8 (0) [-] 8 2048000 bytes 182.601611 182.601622 29.820 fps 9 (1) [-] 9 2048000 bytes 182.635145 182.635157 29.820 fps Captured 10 frames in 0.414323 seconds (24.135744 fps, 49430003.394668 B/s). 8 buffers released.
 * 1) media-ctl -V "'rcar_csi2 feaa0000.csi2':0 [fmt:UYVY8_2X8/1280x800 field:none]"
 * 2) yavta -f YUYV -s 1280x800 -c10 --skip 7 -F /dev/video0

This will capture three images from the first camera to files frame-000007.bin, frame-000008.bin and frame-000009.bin. You can convert them to pnm with the raw2rgbpnm tool (which doesn't have to be run on the target system, you can use it in your development environment).

$ raw2rgbpnm -f YUYV -s 1280x800 frame-000007.bin frame-000007.pnm 0 padding bytes detected at end of line Image size: 1280x800, bytes per pixel: 16, format: YVYU (16 YUV 4:2:2) Writing to file `frame.pnm'...

Use v4l-utils to test
The v4l2-utils (git://linuxtv.org/v4l-utils.git) provide two tools which are helpful in testing rcar-vin.

qv4l2
A GUI tool which lets you view the video from a rcar-vin device and test a wide range of different input parameters. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.


 * 1) qv4l2 -d 23

v4l2-compliance
A compliance test tool that verifies the drivers user-space interface. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.

There are a few other options worth testing
 * On Gen2 --stream-all-formats (-f) and --streaming= (-s) will test that the driver actually can grab frames in different formats.
 * On Gen2 --stream-all-formats (-f) will test that the driver actually can grab frames. The --stream-all-formats option can't be used on Gen3 since the format needs to be changed in the entire pipeline and not just at the video device.

$ v4l2-compliance -d 2 -s v4l2-compliance SHA  : 098aa386ea6cf08cacb44e0982431c5e837e63d0

Driver Info: Driver name  : rcar_vin Card type    : R_Car_VIN Bus info     : platform:e6ef2000.video Driver version: 4.11.0 Capabilities : 0x85200001 Video Capture Read/Write Streaming Extended Pix Format Device Capabilities Device Caps  : 0x05200001 Video Capture Read/Write Streaming Extended Pix Format

Compliance test for device /dev/video2 (not using libv4l2):

Required ioctls: test VIDIOC_QUERYCAP: OK

Allow for multiple opens: test second video open: OK	test VIDIOC_QUERYCAP: OK	test VIDIOC_G/S_PRIORITY: OK	test for unlimited opens: OK

Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK

Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK	test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Test input 0:

Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0

Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK		test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK		test VIDIOC_TRY_FMT: OK		test VIDIOC_S_FMT: OK		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK

Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK		test VIDIOC_EXPBUF: OK

Test input 0:

Streaming ioctls: test read/write: OK	test MMAP: OK	test USERPTR: OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device

Total: 46, Succeeded: 46, Failed: 0, Warnings: 0