Please note that User Registration has been temporarily disabled due to a recent increase in automated registrations. If anyone needs an account, please request one here: RequestAccount. Thanks for your patience!--Wmat (talk)
Please email User:Wmat if you experience any issues with the Request Account form.

Difference between revisions of "Raspberry Pi VideoCore APIs"

From eLinux.org
Jump to: navigation, search
(Bindings for Other Programming Languages: LWJGL)
(Updated image to reflect the current state of affairs)
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{Template:RPi_Software}}
 
{{Template:RPi_Software}}
  
[[File:Raspberrypi_video_API_03.svg|500px|thumb|Source: http://elinux.org/images/b/be/Raspberrypi_video_API_03.svg]]
+
[[File:VideoCore-Architecture-and-Source.png|500px|thumb|Source: http://www.raspberrypi.org/wp-content/uploads/2012/10/Architecture-and-Source.png]]
  
 
The Raspberry Pi contains a Broadcom [http://en.wikipedia.org/wiki/Videocore VideoCore] IV GPU providing OpenGL ES 1.1, OpenGL ES 2.0, hardware-accelerated OpenVG 1.1, Open EGL, OpenMAX and 1080p30 H.264 high-profile decode. There are 24 GFLOPS of general purpose compute and a bunch of texture filtering and DMA infrastructure. Eben worked on the architecture team for this and the Raspberry Pi team are looking at how they can make some of the proprietary features available to application programmers.  
 
The Raspberry Pi contains a Broadcom [http://en.wikipedia.org/wiki/Videocore VideoCore] IV GPU providing OpenGL ES 1.1, OpenGL ES 2.0, hardware-accelerated OpenVG 1.1, Open EGL, OpenMAX and 1080p30 H.264 high-profile decode. There are 24 GFLOPS of general purpose compute and a bunch of texture filtering and DMA infrastructure. Eben worked on the architecture team for this and the Raspberry Pi team are looking at how they can make some of the proprietary features available to application programmers.  
Line 16: Line 16:
 
== Built-in Sample Programs==
 
== Built-in Sample Programs==
 
The Raspberry Pi comes with 7 sample programs and two helper libraries in the /opt/vc/src/hello_pi directory showing some of the system's capabilities and sample code.  
 
The Raspberry Pi comes with 7 sample programs and two helper libraries in the /opt/vc/src/hello_pi directory showing some of the system's capabilities and sample code.  
 +
===hello_encode===
 +
This  sample is a GPU-assisted  h.264 encoder using the OpenMAX APIs , and currently
 +
works with raw frames. It outputs a raw h.264 file.
 +
 
===hello_video===
 
===hello_video===
 
This sample decodes h264 video using the OpenMAX APIs and the ilclient library, and is distributed with a short scene of [http://www.bigbuckbunny.org/ Big Buck Bunny]
 
This sample decodes h264 video using the OpenMAX APIs and the ilclient library, and is distributed with a short scene of [http://www.bigbuckbunny.org/ Big Buck Bunny]
Line 37: Line 41:
  
 
== Libraries ==
 
== Libraries ==
 +
{| border="1"
 +
| ''' The userland libraries have been [http://www.raspberrypi.org/archives/2221 open-sourced] !  Code can be found at [https://github.com/raspberrypi/userland github.com/raspberrypi/userland]'''
 +
|}
 +
 
=== libbcm_host ===
 
=== libbcm_host ===
 
bcm_host is the Broadcom hardware interface library.
 
bcm_host is the Broadcom hardware interface library.

Revision as of 15:51, 10 March 2013

Back to the Hub.


Software & Distributions:

Software - an overview.

Distributions - operating systems and development environments for the Raspberry Pi.

Kernel Compilation - advice on compiling a kernel.

Performance - measures of the Raspberry Pi's performance.

Programming - programming languages that might be used on the Raspberry Pi.

The Raspberry Pi contains a Broadcom VideoCore IV GPU providing OpenGL ES 1.1, OpenGL ES 2.0, hardware-accelerated OpenVG 1.1, Open EGL, OpenMAX and 1080p30 H.264 high-profile decode. There are 24 GFLOPS of general purpose compute and a bunch of texture filtering and DMA infrastructure. Eben worked on the architecture team for this and the Raspberry Pi team are looking at how they can make some of the proprietary features available to application programmers.

Currently C header files and libraries for many of the Broadcom APIs are located in /opt/vc/include and /opt/vc/lib respectively, or available from GitHub within the same directory structure. Some documentation is contained within comments in the header files, as well as documentation for the OpenMAX IL components in the documentation directory on Github, however it is severely lacking and difficult to understand in a general sense for people wanting to experiment with the device due to it being proprietary. As such, we encourage you to edit this page to help us build a much better open documentation.

Quirks

  • The Raspberry Pi requires that the bcm_host_init() function is called first before any GPU calls can be made.
  • Before calling any of the vc_* functions, you need to initialise vcos and vchi, and make a vchi connection, before then calling the corresponding vc_*_init function for the part of the library you want to use: vc_vchi_dispmanx_init, vc_vchi_tv_init, vc_vchi_cec_init or vc_vchi_gencmd_init.
    (See this sample code which does initialisation for vc_tvservice_* APIs)
  • Linking against the EGL library (-lEGL) also requires you to link against the GLESv2 library (-lGLESv2).
  • graphics_get_display_size() is a broadcom-specific function

Built-in Sample Programs

The Raspberry Pi comes with 7 sample programs and two helper libraries in the /opt/vc/src/hello_pi directory showing some of the system's capabilities and sample code.

hello_encode

This sample is a GPU-assisted h.264 encoder using the OpenMAX APIs , and currently works with raw frames. It outputs a raw h.264 file.

hello_video

This sample decodes h264 video using the OpenMAX APIs and the ilclient library, and is distributed with a short scene of Big Buck Bunny

hello_audio

This sample plays a sine wave for ten seconds using the ??? APIs.

hello_triangle

This sample shows how to get a OpenGL ES context on the Raspberry Pi.

ilclient helper library

This library is described as: This API defines helper functions for writing IL clients.

*
* This file defines an IL client side library.  This is useful when
* writing IL clients, since there tends to be much repeated and
* common code across both single and multiple clients.  This library
* seeks to remove that common code and abstract some of the
* interactions with components.  There is a wrapper around a
* component and tunnel, and some operations can be done on lists of
* these.  The callbacks from components are handled, and specific
* events can be checked or waited for.
*/

Libraries

The userland libraries have been open-sourced ! Code can be found at github.com/raspberrypi/userland

libbcm_host

bcm_host is the Broadcom hardware interface library.

vc_dispmanx_*

Dispmanx is a windowing system in the process of being deprecated in favour of OpenWF (or similar), however dispmanx is still used in all API demos and it's replacement may not yet be available. [1](confirmation required?)

vc_tvservice_*

These APIs allow controlling the HDMI and SDTV video outputs of the Raspberry Pi, as well as allowing the user to query the supported HDMI resolutions and audio formats and turn on/off copy protection. The Raspberry Pi can only have one output active at a time - turning on the HDMI automatically turns off the SDTV and vice-versa. However either can be turned off using the vc_tv_power_off() function.

The header file vc_tvservice.h documents the functions quite well via comments above each prototype function, however before any of these functions will succeed you must initialise the connection to the host API like so:

void tvservice_init() {
   VCHI_INSTANCE_T vchi_instance;
   VCHI_CONNECTION_T *vchi_connections;
   
   // initialise bcm_host
   bcm_host_init();
   
   // initialise vcos/vchi
   vcos_init();
   if (vchi_initialise(&vchi_instance) != VCHIQ_SUCCESS) {
       fprintf(stderr, "failed to open vchiq instance\n");
       exit(-2);
   }
   
   // create a vchi connection
   if ( vchi_connect( NULL, 0, vchi_instance ) != 0) {
       fprintf(stderr, "failed to connect to VCHI\n");
       exit(-3);
   }
   
   // connect to tvservice
   if ( vc_vchi_tv_init( vchi_instance, &vchi_connections, 1) != 0) {
       fprintf(stderr, "failed to connect to tvservice\n");
       exit(-4);
   }
}

TODO: Deinitialization

Samples

  • rpi-output-swapper Uses tvservice apis to power on HDMI or SDTV display in specific or preferred mode.

vc_cec_*

The vc_cec APIs allow programs to interface with the Consumer Electronics Control (CEC) host middleware on the system to send CEC packets, set strings for the middleware to auto-respond to certain requests or override the middleware handling by adding custom callbacks.

TODO: Samples/Howto

vc_gencmd_*

TODO

EGL

EGL (Native Platform Graphics Interface) is an interface between Khronos rendering APIs such as OpenGL ES or OpenVG and the underlying native platform. The Raspberry Pi supports EGL Version 1.4 and the following EGL extensions (links are to EGL registry documentation):[2]


Creating an on-screen EGL rendering surface requires you to to use the eglCreateWindowSurface function, which takes a EGLNativeWindowType parameter. On the Raspberry Pi, this is implemented as a EGL_DISPMANX_WINDOW_T struct, which is defined in eglplatform.h as:

 typedef struct {
   DISPMANX_ELEMENT_HANDLE_T element;
   int width;   /* This is necessary because dispmanx elements are not queriable. */
   int height;
 } EGL_DISPMANX_WINDOW_T;

Therefore, to create a EGL surface you must also use the Dispmanx library to get a dispmanx element handle. The hello_triangle source code provides example code to do this.

Samples

See Getting Started with EGL post by Jon Macey. Extended documentation available in the EGL Version 1.4 Specification and Simple Reference available in the EGL 1.4 API Quick Reference Card.

Function Documentation

int32_t graphics_get_display_size( const uint16_t display_number, uint32_t *width, uint32_t *height);

This function allows you to get the size of the display, and is often used in creating an EGL Surface. width and height are pointers that you pass in to where you want the response values to be written to, display number 0 is usually used for the Raspberry Pi. return value of >= 0 indicates success, <0 indicates failure.

This function is implemented in libbcm_host, and the implementation has been made public showing that it internally calls the vc_dispmanx_display_get_info function to get the width and height of the current mode.

Bugs present in early versions of the firmware caused this function to always return 1920x1080, however this can easily be fixed by updating the firmware with Hexxeh's rpi-update (see the Updating firmware page for help).[3][4]

OpenGL ES

The Raspberry Pi supports OpenGL ES 2.0 with the following extensions: [2]

  • GL_OES_compressed_ETC1_RGB8_texture
  • GL_OES_compressed_paletted_texture
  • GL_OES_texture_npot GL_OES_depth24
  • GL_OES_vertex_half_float
  • GL_OES_EGL_image
  • GL_OES_EGL_image_external
  • GL_EXT_discard_framebuffer
  • GL_OES_rgb8_rgba8
  • GL_OES_depth32
  • GL_OES_mapbuffer
  • GL_EXT_texture_format_BGRA8888
  • GL_APPLE_rgb_422
  • GL_EXT_debug_marker

TODO: Sample of how to use/initalise OpenGL ES

OpenVG

The Raspberry Pi supports OpenVG 1.1 for hardware-accelerated two-dimensional vector and raster graphics. See the full specification here or the API quick reference card.

Samples

OpenMAX

See /documentation/ilcomponents on GitHub firmware repository for official documentation.

TODO: Sample of how to use OpenMAX to decode audio/video/images, etc.

Bindings for Other Programming Languages

Python

  • RPi_Vid_Core - python bindings of EGL, OpenGL ES, OpenVG and bcm_host for Raspberry Pi implemented using Cython

Java

Open Source Software compatible with the VideoCore APIs

  • omxplayer - a command line a/v player for the Raspberry Pi utilising the OpenMAX APIs and ffmpeg
  • XBMC - a media center, omxplayer's "big brother"
  • piNGL - port of the NGL graphics library to Raspberry Pi
  • Qt on Pi - Raspberry Pi port of the Qt toolkit

References

  1. http://www.raspberrypi.org/phpBB3/viewtopic.php?t=5621&p=77981
  2. 2.0 2.1 eglinfo, http://www.raspberrypi.org/phpBB3/viewtopic.php?p=35942#p35942 , http://www.raspberrypi.org/phpBB3/viewtopic.php?p=36318#p36318
  3. http://www.raspberrypi.org/phpBB3/viewtopic.php?f=68&t=8252
  4. https://github.com/raspberrypi/firmware/issues/14