Raspberry Pi VideoCore APIs

The Raspberry Pi contains a Broadcom VideoCore IV GPU providing OpenGL ES 1.1, OpenGL ES 2.0, hardware-accelerated OpenVG 1.1, Open EGL, OpenMAX and 1080p30 H.264 high-profile decode. There are 24 GFLOPS of general purpose compute and a bunch of texture filtering and DMA infrastructure. Eben worked on the architecture team for this and the Raspberry Pi team are looking at how they can make some of the proprietary features available to application programmers.

Currently C header files and libraries for many of the Broadcom APIs are located in  and   respectively, or available from GitHub within the same directory structure. Some documentation is contained within comments in the header files, as well as documentation for the OpenMAX IL components in the documentation directory on Github, however it is severely lacking and difficult to understand in a general sense for people wanting to experiment with the device due to it being proprietary. As such, we encourage you to edit this page to help us build a much better open documentation.

I would like to add a picture of http://en.wikipedia.org/wiki/File:Raspberrypi_video_API_03.svg here but the stupid Wiki system keeps crashing on me so you'll have to follow the link. Fucking varnish.

Quirks
(See this sample code which does initialisation for vc_tvservice_* APIs)
 * The Raspberry Pi requires that the  function is called first before any GPU calls can be made.
 * Before calling any of the vc_* functions, you need to initialise vcos and vchi, and make a vchi connection, before then calling the corresponding vc_*_init function for the part of the library you want to use:,  ,   or.
 * is a broadcom-specific function

Sample Programs
The Raspberry Pi comes with 7 sample programs and two helper libraries in the /opt/vc/src/hello_pi directory showing some of the system's capabilities and sample code.

hello_video
This sample decodes h264 video using the OpenMAX APIs and the ilclient library, and is distributed with a short scene of Big Buck Bunny

ilclient helper library
This library is described as:

libbcm_host
bcm_host is the Broadcom hardware interface library.

vc_dispmanx_*
Dispmanx is a windowing system in the process of being deprecated in favour of OpenWF (or similar), however dispmanx is still used in all API demos and it's replacement may not yet be available. (confirmation required?)

vc_tvservice_*
These APIs allow controlling the HDMI and SDTV video outputs of the Raspberry Pi.

TODO: Describe better

vc_cec_*
TODO

vc_gencmd_*
TODO

EGL
EGL (Native Platform Graphics Interface) is an interface between Khronos rendering APIs such as OpenGL ES or OpenVG and the underlying native platform. The Raspberry Pi supports EGL Version 1.4 and the following EGL extensions (links are to EGL registry documentation):
 * EGL_KHR_image
 * EGL_KHR_image_base
 * EGL_KHR_image_pixmap
 * EGL_KHR_vg_parent_image
 * EGL_KHR_gl_texture_2D_image
 * EGL_KHR_gl_texture_cubemap_image
 * EGL_KHR_lock_surface

Samples
See Getting Started with EGL post by Jon Macey. Extended documentation available in the EGL Version 1.4 Specification and Simple Reference available in the EGL 1.4 API Quick Reference Card.

This function allows you to get the size of the display, and is often used in creating an EGL Surface. width and height are pointers that you pass in to where you want the response values to be written to, display number 0 is usually used for the Raspberry Pi. return value of >= 0 indicates success, <0 indicates failure.

This function is implemented in libbcm_host, and the implementation has been made public showing that it internally calls the vc_dispmanx_display_get_info function to get the width and height of the current mode.

Bugs present in early versions of the firmware caused this function to always return 1920x1080, however this can easily be fixed by updating the firmware with Hexxeh's rpi-update (see the Updating firmware page for help).

OpenGL ES
The Raspberry Pi supports OpenGL ES 2.0 with the following extensions: TODO: Sample of how to use/initalise OpenGL ES
 * GL_OES_compressed_ETC1_RGB8_texture
 * GL_OES_compressed_paletted_texture
 * GL_OES_texture_npot GL_OES_depth24
 * GL_OES_vertex_half_float
 * GL_OES_EGL_image
 * GL_OES_EGL_image_external
 * GL_EXT_discard_framebuffer
 * GL_OES_rgb8_rgba8
 * GL_OES_depth32
 * GL_OES_mapbuffer
 * GL_EXT_texture_format_BGRA8888
 * GL_APPLE_rgb_422
 * GL_EXT_debug_marker

OpenMAX
See /documentation/ilcomponents on GitHub firmware repository for official documentation.

TODO: Sample of how to use OpenMAX to decode audio/video/images, etc.

Other Programming Languages
Python
 * RPi_Vid_Core - python bindings of EGL, OpenGL ES, OpenVG and bcm_host for Raspberry Pi implemented using Cython

Programs designed to work with the Raspberry Pi's VideoCore APIs

 * omxplayer - a command line a/v player for the Raspberry Pi utilising the OpenMAX APIs and ffmpeg
 * XBMC - a media center, omxplayer's "big brother" 
 * piNGL - port of the NGL graphics library to Raspberry Pi
 * Qt on Pi - Raspberry Pi port of the Qt toolkit