R-Car/Tests:rcar-vin

This document describes how to test the VIN and CSI-2 (only on Gen3) functionality on Renesas R-Car Gen2 and Gen3.

Get the code
Ideally all the code to test VIN should be available in the mainline kernel, unfortunately this is not the case yet. Renesas driver parts not yet picked-up are available in renesas-drivers tree while none Renesas drivers (e.g. the ADV7482 driver for Gen3 testing) are only available in the out-of-tree repository bellow.

There are also some patches needed to facilitate testing that are not part of renesas-drivers such as kernel configurations and DT changes. To ease testing a tag is available where all patches are included which can be build and used to test at:

https://git.ragnatech.se/linux rcar-vin-elinux-v8

Gen2
All that is needed is included in v4.10.

Gen3

 * VIN and CSI2 driver support is included in renesas-drivers.
 * Prototype driver for ADV7482 are not available in renesas-drivers but in the out-of-tree repository mentioned above, this driver is incomplete and requires more work before it's ready for mainline.
 * To test CVBS or HDMI input the RCAR_VIN, RCAR_CSI2 and ADV7482 needs to be enabled (not default in defconfig) and added to DT, see example in out-of-tree repository mentioned above.

Boot the kernel
You should see this on successful probe (this is from Gen3):

rcar-vin e6ef0000.video: Device registered as video0 rcar-vin e6ef1000.video: Device registered as video1 rcar-vin e6ef2000.video: Device registered as video2 rcar-vin e6ef3000.video: Device registered as video3 rcar-vin e6ef4000.video: Device registered as video4 rcar-vin e6ef5000.video: Device registered as video5 rcar-vin e6ef6000.video: Device registered as video6 rcar-vin e6ef7000.video: Device registered as video7

Media Controller (Gen3)
The hardware available on Gen3 is much more complex then on Gen2. For this reason it's no longer enough to interact with only the video device to control the hardware, the entire media pipeline needs to configured. To facilitate this the Media Controller (MC) framework is used exclusivity on Gen3. By using the MC framework it is up to user-space to first configure the media pipeline and then setup the correct format for each entity involved in the pipeline before streaming can be started on the video device.

The pipeline will consist of a video sensor (CSI-2 Transmitter, ADV7482 on Salvator-X) a CSI-2 receiver (R-Car CSI-2) and a dma engine (R-Car VIN). The video sensor is immutably connected to a single R-Car CSI-2 while the routing between the R-Car CSI-2 and R-Car VIN can be configured at runtime. There are limitations on which channel from the R-Car CSI-2 node can be connected to which R-Car VIN node, this is a hardware limitation that is different for each SoC in the Gen3 series. All links between these different nodes and all possible routing configurations are exposed as a media device to user-space. After the links in the pipeline have been setup by the user the proper format needs to be set on all nodes involved in the pipeline before the user can start a stream at the video device.

To control the routing and format from user-space the user needs to interact with the media device, the subdevices for video sensor and R-Car CSI-2, the video node for R-Car VIN. For configuring the media device and subdevices 'media-ctl' can be used and to configure the video device itself a grabber can be used, such as 'yavta' or 'qv4l2'. The following sections will briefly describe how to use these tools to configure the pipeline for streaming.

Show current routing
Examine the current routing setup with 'media-ctl -d /dev/mediaX -p'.

Note: In the example bellow the ADV7482 node is shown twice, this do not represent reality but is one of the hacks in the ADV7482 driver which needs to be solved before it's ready for upstream.

$ media-ctl -d /dev/media0 -p Media controller API version 0.1.0

Media device information

driver         Renesas VIN model          video serial bus info       /soc/video@e6ef0000 hw revision    0x0 driver version 4.11.0

Device topology - entity 1: rcar_vin e6ef0000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "rcar_csi2 feaa0000.csi2":1 [ENABLED] <- "rcar_csi2 fea80000.csi2":1 []

- entity 5: rcar_csi2 feaa0000.csi2 (5 pads, 8 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:unknown/0x0] <- "adv7482 4-0070":10 [ENABLED,IMMUTABLE] pad1: Source [fmt:unknown/0x0] -> "rcar_vin e6ef0000.video":0 [ENABLED] -> "rcar_vin e6ef1000.video":0 [] -> "rcar_vin e6ef2000.video":0 [] pad2: Source [fmt:unknown/0x0] -> "rcar_vin e6ef1000.video":0 [] -> "rcar_vin e6ef3000.video":0 [ENABLED] pad3: Source [fmt:unknown/0x0] -> "rcar_vin e6ef2000.video":0 [] pad4: Source [fmt:unknown/0x0] -> "rcar_vin e6ef3000.video":0 []

- entity 11: rcar_csi2 fea80000.csi2 (5 pads, 15 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:unknown/0x0] <- "adv7482 4-0034":11 [ENABLED,IMMUTABLE] pad1: Source [fmt:unknown/0x0] -> "rcar_vin e6ef0000.video":0 [] -> "rcar_vin e6ef1000.video":0 [ENABLED] -> "rcar_vin e6ef2000.video":0 [] -> "rcar_vin e6ef4000.video":0 [] -> "rcar_vin e6ef5000.video":0 [ENABLED] -> "rcar_vin e6ef6000.video":0 [] pad2: Source [fmt:unknown/0x0] -> "rcar_vin e6ef1000.video":0 [] -> "rcar_vin e6ef3000.video":0 [] -> "rcar_vin e6ef5000.video":0 [] -> "rcar_vin e6ef7000.video":0 [] pad3: Source [fmt:unknown/0x0] -> "rcar_vin e6ef2000.video":0 [] -> "rcar_vin e6ef6000.video":0 [] pad4: Source [fmt:unknown/0x0] -> "rcar_vin e6ef3000.video":0 [] -> "rcar_vin e6ef7000.video":0 []

- entity 17: adv7482 4-0034 (12 pads, 1 link) type V4L2 subdev subtype Decoder flags 0 device node name /dev/v4l-subdev2 pad0: Sink pad1: Sink pad2: Sink pad3: Sink pad4: Sink pad5: Sink pad6: Sink pad7: Sink pad8: Sink pad9: Sink pad10: Source [fmt:RGB888_1X24/720x480 field:interlaced] pad11: Source [fmt:UYVY8_2X8/720x576 field:interlaced] -> "rcar_csi2 fea80000.csi2":0 [ENABLED,IMMUTABLE]

- entity 30: adv7482 4-0070 (12 pads, 1 link) type V4L2 subdev subtype Decoder flags 0 device node name /dev/v4l-subdev3 pad0: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad1: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad2: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad3: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad4: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad5: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad6: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad7: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad8: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad9: Sink [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] pad10: Source [fmt:RGB888_1X24/640x480 field:none] [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] [dv.detect:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic] [dv.current:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic] -> "rcar_csi2 feaa0000.csi2":0 [ENABLED,IMMUTABLE] pad11: Source [fmt:UYVY8_2X8/720x576 field:interlaced] [dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive] [dv.detect:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic] [dv.current:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic]

- entity 59: rcar_vin e6ef1000.video (1 pad, 4 links) type Node subtype V4L flags 0 device node name /dev/video1 pad0: Sink <- "rcar_csi2 feaa0000.csi2":1 [] <- "rcar_csi2 feaa0000.csi2":2 [] <- "rcar_csi2 fea80000.csi2":1 [ENABLED] <- "rcar_csi2 fea80000.csi2":2 []

- entity 71: rcar_vin e6ef2000.video (1 pad, 4 links) type Node subtype V4L flags 0 device node name /dev/video2 pad0: Sink <- "rcar_csi2 feaa0000.csi2":1 [] <- "rcar_csi2 feaa0000.csi2":3 [] <- "rcar_csi2 fea80000.csi2":1 [] <- "rcar_csi2 fea80000.csi2":3 []

- entity 83: rcar_vin e6ef3000.video (1 pad, 4 links) type Node subtype V4L flags 0 device node name /dev/video3 pad0: Sink <- "rcar_csi2 feaa0000.csi2":2 [ENABLED] <- "rcar_csi2 feaa0000.csi2":4 [] <- "rcar_csi2 fea80000.csi2":2 [] <- "rcar_csi2 fea80000.csi2":4 []

- entity 95: rcar_vin e6ef4000.video (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video4 pad0: Sink <- "rcar_csi2 fea80000.csi2":1 []

- entity 101: rcar_vin e6ef5000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video5 pad0: Sink <- "rcar_csi2 fea80000.csi2":1 [ENABLED] <- "rcar_csi2 fea80000.csi2":2 []

- entity 109: rcar_vin e6ef6000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video6 pad0: Sink <- "rcar_csi2 fea80000.csi2":1 [] <- "rcar_csi2 fea80000.csi2":3 []

- entity 117: rcar_vin e6ef7000.video (1 pad, 2 links) type Node subtype V4L flags 0 device node name /dev/video7 pad0: Sink <- "rcar_csi2 fea80000.csi2":2 [] <- "rcar_csi2 fea80000.csi2":4 []

Activate/Deactivate a link
The Media Controller framework allows user-space to enable/disable a link and that way control the routing of video data from a R-Car CSI-2 to a R-Car VIN instance. Not all CSI-2 channels are possible to route to any and all VIN instances, this is limited by the SoC hardware and is specific for each SoC version. Look at the previous section on how to show which links are possible on your particular SoC. Further more not all links can be activated independent of other links, this is also a limitation set by the hardware. If you try to enable a link which would not be possible with regard to other already active links the operation will fail with a EMLINK error. Try deactivating some other link to crate more routing possibilities and try again, all possible routing setups for a specific SoC are documented in the datasheet.

A link is always configured from a R-Car CSI-2 entity to a R-Car VIN entity, the same way the video data is flowing.

To enable a link use the media-ctl utility from v4l-utils package:


 * 1) media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin e6ef0000.video':0 [1]"

To disable the same link use:


 * 1) media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin.e6ef0000.video':0 [0]"

Deactivate all active links
This is a useful command to reset all links before you start enabling new links to make sure you got the biggest possible routing space to start out with.


 * 1) media-ctl -d /dev/media0 -r

Example configuring the pipeline and propagate format
Once the user have configured a pipeline using 'media-ctl' as described above the format needs to be propagated in the pipeline before streaming can start. In this example we use a R-Car H3 board where the HDMI input will be configured to output on VIN2 and CVBS configured to VIN4. If we look at the media graph as show above we can identify which nodes are involved in each of these two pipelines:

HDMI: 'adv7482 4-0070' -> 'rcar_csi2 feaa0000.csi2' -> 'rcar_vin e6ef2000.video' CVBS: 'adv7482 4-0034' -> 'rcar_csi2 fea80000.csi2' -> 'rcar_vin e6ef4000.video'

Start by clearing any previous links and configure the ones we want, remember that the link between the adv7482 and the R-Car CSI-2 entities are immutable and won't be clear and therefore do not need to be set again.

media-ctl -d /dev/media0 -r media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'rcar_vin e6ef2000.video':0 [1]" media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin e6ef4000.video':0 [1]"

When the pipelines are configured it's time to propagate the format in the pipeline, start by examine which format is present at the video sources sinks.

$ media-ctl -d /dev/media0 --get-v4l2 "'adv7482 4-0070':10" [fmt:RGB888_1X24/640x480 field:none]
 * 1) HDMI

$ media-ctl -d /dev/media0 --get-v4l2 "'adv7482 4-0034':11" [fmt:UYVY8_2X8/720x576 field:interlaced]
 * 1) CVBS

Now we know the formats so we can propagate them:

media-ctl -d /dev/media0 -V "'adv7482 4-0070':10 [fmt:RGB888_1X24/640x480 field:none]" media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/640x480 field:none]"
 * 1) HDMI

media-ctl -d /dev/media0 -V "'adv7482 4-0070':11 [fmt:UYVY8_2X8/720x576 field:interlaced]" media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x576 field:interlaced]"
 * 1) CVBS

Once the pipeline format is setup we can then grab frames using for example 'yavta' or look use a GUI like 'qv4l2' yavta -f RGB565 -s 640x480 --field none -n 4 --capture=10 -F /dev/video2
 * 1) HDMI

yavta -f RGB565 -s 720x576 --field interlaced -n 4 --capture=10 -F /dev/video4
 * 1) CVBS

Use v4l-utils to test
The v4l2-utils (git://linuxtv.org/v4l-utils.git) provide two tools which are helpful in testing rcar-vin.

qv4l2
A GUI tool which lets you view the video from a rcar-vin device and test a wide range of different input parameters. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.


 * 1) qv4l2 -d 23

v4l2-compliance
A compliance test tool that verifies the drivers user-space interface. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.

There are a few other options worth testing
 * On Gen2 --stream-all-formats (-f) and --streaming= (-s) will test that the driver actually can grab frames in different formats.
 * On Gen2 --stream-all-formats (-f) will test that the driver actually can grab frames. The --stream-all-formats option can't be used on Gen3 since the format needs to be changed in the entire pipeline and not just at the video device.

$ v4l2-compliance -d 2 -s v4l2-compliance SHA  : 098aa386ea6cf08cacb44e0982431c5e837e63d0

Driver Info: Driver name  : rcar_vin Card type    : R_Car_VIN Bus info     : platform:e6ef2000.video Driver version: 4.11.0 Capabilities : 0x85200001 Video Capture Read/Write Streaming Extended Pix Format Device Capabilities Device Caps  : 0x05200001 Video Capture Read/Write Streaming Extended Pix Format

Compliance test for device /dev/video2 (not using libv4l2):

Required ioctls: test VIDIOC_QUERYCAP: OK

Allow for multiple opens: test second video open: OK	test VIDIOC_QUERYCAP: OK	test VIDIOC_G/S_PRIORITY: OK	test for unlimited opens: OK

Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK

Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK	test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Test input 0:

Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0

Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK		test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK		test VIDIOC_TRY_FMT: OK		test VIDIOC_S_FMT: OK		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK

Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK		test VIDIOC_EXPBUF: OK

Test input 0:

Streaming ioctls: test read/write: OK	test MMAP: OK	test USERPTR: OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device

Total: 46, Succeeded: 46, Failed: 0, Warnings: 0