Raspberry Pi VideoCore APIs

Warning
The information below is not invalid but is significantly out of date (and doesn't cover the Pi 4 at all, which uses a newer GPU). See this forum post for a summary of the current status as of August 2021. In short, the Broadcom-specific APIs below (OpenMAX, DispmanX and the like) are deprecated in favor of industry/Linux standard APIs including DRM, KMS, and OpenGL.

Architecture


The Raspberry Pi contains a Broadcom VideoCore IV GPU providing OpenGL ES 1.1, OpenGL ES 2.0, hardware-accelerated OpenVG 1.1, Open EGL, OpenMAX and 1080p30 H.264 high-profile decode. There are 24 GFLOPS of general purpose compute and a bunch of texture filtering and DMA infrastructure. Eben worked on the architecture team for this and the Raspberry Pi team are looking at how they can make some of the proprietary features available to application programmers.

Currently C header files and libraries for many of the Broadcom APIs are located in  and   respectively, or available from GitHub within the same directory structure. Some documentation is contained within comments in the header files, as well as documentation for the OpenMAX IL components in the documentation directory on GitHub, however it is severely lacking and difficult to understand in a general sense for people wanting to experiment with the device due to it being proprietary. As such, we encourage you to edit this page to help us build much better open documentation.

Quirks
(See this sample code which does initialisation for vc_tvservice_* APIs)
 * The Raspberry Pi requires that the  function is called first before any GPU calls can be made.
 * Before calling any of the vc_* functions, you need to initialise vcos and vchi, and make a vchi connection, before then calling the corresponding vc_*_init function for the part of the library you want to use:,  ,   or.
 * Linking against the EGL library (-lEGL) also requires you to link against the GLESv2 library (-lGLESv2).
 * is a Broadcom-specific function.

Built-in Sample Programs
The Raspberry Pi comes with seven sample programs and two helper libraries in the /opt/vc/src/hello_pi directory showing some of the system's capabilities and sample code.

hello_encode
This sample is a GPU-assisted H.264 encoder using the OpenMAX APIs, and currently works with raw frames. It outputs a raw H.264 file.

hello_video
This sample decodes H.264 video using the OpenMAX APIs and the ilclient library and is distributed with a short scene of Big Buck Bunny

hello_audio
This sample plays a sine wave for ten seconds using the OpenMAX APIs.

hello_triangle
This sample shows how to get a OpenGL ES context on the Raspberry Pi.

ilclient helper library
This library is described as:

libbcm_host
bcm_host is the Broadcom hardware interface library.

vc_dispmanx_*
Dispmanx is a windowing system in the process of being deprecated in favour of OpenWF (or similar), however dispmanx is still used in all API demos and it's replacement may not yet be available. (confirmation required?)

vc_tvservice_*
These APIs allow controlling the HDMI and SDTV video outputs of the Raspberry Pi, as well as allowing the user to query the supported HDMI resolutions and audio formats and turn on/off copy protection. The Raspberry Pi can only have one output active at a time - turning on the HDMI automatically turns off the SDTV and vice-versa. However either can be turned off using the  function.

The header file vc_tvservice.h documents the functions quite well via comments above each prototype function, however before any of these functions will succeed you must initialise the connection to the host API like so:

void tvservice_init { VCHI_INSTANCE_T vchi_instance; VCHI_CONNECTION_T *vchi_connections; // initialise bcm_host bcm_host_init; // initialise vcos/vchi vcos_init; if (vchi_initialise(&vchi_instance) != VCHIQ_SUCCESS) { fprintf(stderr, "failed to open vchiq instance\n"); exit(-2); }   // create a vchi connection if ( vchi_connect( NULL, 0, vchi_instance ) != 0) { fprintf(stderr, "failed to connect to VCHI\n"); exit(-3); }   // connect to tvservice if ( vc_vchi_tv_init( vchi_instance, &vchi_connections, 1) != 0) { fprintf(stderr, "failed to connect to tvservice\n"); exit(-4); } }

TODO: Deinitialization

Samples

 * rpi-output-swapper Uses tvservice apis to power on HDMI or SDTV display in specific or preferred mode.

vc_cec_*
The vc_cec APIs allow programs to interface with the Consumer Electronics Control (CEC) host middleware on the system to send CEC packets, set strings for the middleware to auto-respond to certain requests or override the middleware handling by adding custom callbacks.

TODO: Samples/Howto

vc_gencmd_*
TODO

EGL
EGL (Native Platform Graphics Interface) is an interface between Khronos rendering APIs such as OpenGL ES or OpenVG and the underlying native platform. The Raspberry Pi supports EGL Version 1.4 and the following EGL extensions (links are to EGL registry documentation):
 * EGL_KHR_image
 * EGL_KHR_image_base
 * EGL_KHR_image_pixmap
 * EGL_KHR_vg_parent_image
 * EGL_KHR_gl_texture_2D_image
 * EGL_KHR_gl_texture_cubemap_image
 * EGL_KHR_lock_surface

Creating an on-screen EGL rendering surface requires you to to use the eglCreateWindowSurface function, which takes a EGLNativeWindowType parameter. On the Raspberry Pi, this is implemented as a EGL_DISPMANX_WINDOW_T struct, which is defined in eglplatform.h as:

typedef struct { DISPMANX_ELEMENT_HANDLE_T element; int width;  /* This is necessary because dispmanx elements are not queriable. */   int height; } EGL_DISPMANX_WINDOW_T;

Therefore, to create a EGL surface you must also use the Dispmanx library to get a dispmanx element handle. The hello_triangle source code provides example code to do this.

Samples
See Getting Started with EGL post by Jon Macey. Extended documentation available in the EGL Version 1.4 Specification and Simple Reference available in the EGL 1.4 API Quick Reference Card.

This function allows you to get the size of the display, and is often used in creating an EGL Surface. width and height are pointers that you pass in to where you want the response values to be written to, display number 0 is usually used for the Raspberry Pi. return value of >= 0 indicates success, <0 indicates failure.

This function is implemented in libbcm_host, and the implementation has been made public showing that it internally calls the vc_dispmanx_display_get_info function to get the width and height of the current mode.

Bugs present in early versions of the firmware caused this function to always return 1920x1080, however this can easily be fixed by updating the firmware with Hexxeh's rpi-update (see the Updating firmware page for help).

OpenGL ES
The Raspberry Pi supports OpenGL ES 2.0 with the following extensions: TODO: Sample of how to use/initalise OpenGL ES
 * GL_OES_compressed_ETC1_RGB8_texture
 * GL_OES_compressed_paletted_texture
 * GL_OES_texture_npot GL_OES_depth24
 * GL_OES_vertex_half_float
 * GL_OES_EGL_image
 * GL_OES_EGL_image_external
 * GL_EXT_discard_framebuffer
 * GL_OES_rgb8_rgba8
 * GL_OES_depth32
 * GL_OES_mapbuffer
 * GL_EXT_texture_format_BGRA8888
 * GL_APPLE_rgb_422
 * GL_EXT_debug_marker

OpenVG
The Raspberry Pi supports OpenVG 1.1 for hardware-accelerated two-dimensional vector and raster graphics. See the full specification here or the API quick reference card.

Samples

 * Code sample: Drawing a triangle by using OpenVG (Java implentation for RIM devices, however should be able to get a grasp on how it works and be able to port it to the Raspberry Pi)

OpenMAX
See /documentation/ilcomponents on GitHub firmware repository for official documentation.

TODO: Sample of how to use OpenMAX to decode audio/video/images, etc.
 * Programming the Raspberry Pi's GPU is a free ebook by Jan Newmarch covering several of the VideoCore APIs including OpenMAX, OpenVG and OpenGL ES

Bindings for Other Programming Languages
Python Java
 * RPi_Vid_Core - Python bindings of EGL, OpenGL ES, OpenVG and bcm_host for Raspberry Pi implemented using Cython
 * JogAmp JOGL - ''JogAmp JOGL Java bindings for EGL and OpenGL ES for Raspberry Pi implemented using Gluegen.
 * Raspberry Pi autodetection is included in JogAmp JOGL RC11 and later. http://forum.jogamp.org/Release-v2-0-rc11-td4026703.html
 * LWJGL - http://www.raspberrypi.org/phpBB3/viewtopic.php?p=211087 Raspberry Pi community port by spsn

Open Source Software compatible with the VideoCore APIs

 * omxplayer - a command line a/v player for the Raspberry Pi utilising the OpenMAX APIs and ffmpeg
 * XBMC - a media center, omxplayer's "big brother" 
 * piNGL - port of the NGL graphics library to Raspberry Pi
 * Qt on Pi - Raspberry Pi port of the Qt toolkit