BeagleBoard/GSoC/2010 Projects/XBMC - Revision history

Peter Huewe: Added to category BeagleBoard and GSOC and spellcheck

2011-04-13T05:38:29Z

Added to category BeagleBoard and GSOC and spellcheck

Show changes

Topfs2: /* Interesting patches for beagleboard */

2010-10-06T16:43:56Z

‎Interesting patches for beagleboard

Topfs2 at 20:33, 16 August 2010

2010-08-16T20:33:37Z

Show changes

Topfs2 at 20:26, 16 August 2010

2010-08-16T20:26:51Z

Topfs2: /* Embedded and Beagle Board specific */

2010-08-11T22:57:51Z

‎Embedded and Beagle Board specific

Topfs2: /* Embedded and Beagle Board specific */

2010-08-11T22:02:42Z

‎Embedded and Beagle Board specific

Topfs2: /* How does it work? */

2010-08-11T21:57:04Z

‎How does it work?

Topfs2: /* Documentation */

2010-08-11T20:38:32Z

‎Documentation

Topfs2: /* Interesting patches for beagleboard */

2010-08-11T08:52:41Z

‎Interesting patches for beagleboard

Topfs2: /* Build Instructions (native) */

2010-08-11T08:51:06Z

‎Build Instructions (native)

@@ Line 44: / Line 44: @@
 To enable dirty region based rendering add this as advancedsettings.xml (~/.xbmc/userdata/advancedsettings.xml)
-<advandedsettings>
+<advancedsettings>
 	<gui>

@@ Line 13: / Line 13: @@
 Latest blog entries:
-[http://xbmc.org/topfs2/2010/08/04/weekly-report-10/ Weekly report 10]
+[http://xbmc.org/topfs2/2010/08/16/weekly-report-12/ Weekly report 12]
-[http://xbmc.org/topfs2/2010/07/26/weekly-report-9/ Weekly report 9]
+[http://xbmc.org/topfs2/2010/08/09/weekly-report-11/ Weekly report 11]
-[http://xbmc.org/topfs2/2010/07/18/weekly-report-8/ Weekly report 8]
+[http://xbmc.org/topfs2/2010/08/04/weekly-report-10/ Weekly report 10]
-−
-[http://xbmc.org/topfs2/2010/07/12/weekly-report-7/ Weekly report 7]
-−
-[http://xbmc.org/topfs2/2010/07/05/weekly-report-6/ Weekly report 6]
 [http://xbmc.org/topfs2/2010/06/30/what-to-do-when-you-have-the-dirty-regions/ What to do when you have the dirty regions]
@@ Line 34: / Line 30: @@
 == Documentation ==
+==Abstract==
 === Dependencies ===
@@ Line 102: / Line 99: @@
 MACHINE=beagleboard bitbake xbmc
 ==Optimizing skins for Beagle Board==
 There are a number of ways to optimize skins, some of which are general and some are specifically target for embedded and GLES both will be covered in the following section.
@@ Line 136: / Line 128: @@
 The overlay reads data from the framebuffers found in /dev/fbX. Multiple overlays can read from the same framebuffer and an overlay can be told to read only from a part of the framebuffer. The display area of the overlay can be larger or smaller then that  found in the framebuffer and the overlay will then scale it accordingly. Since the overlay is able to read only part of the data from a framebuffer its possible to create a larger framebuffer than the data its meant to hold to create a front- and backbuffer and switch the location the overlay should read from before presenting.
 Overlay 1 (video layer) is able to read YUV422 (or RGB) which will be transformed, scaled and positioned before presenting and all is done in hardware. Normally X and SGX is rendering to fb0 which is linked to overlay 0 (graphics layer). Graphics layer is above video layer but interestingly enough by default the display manager sets final alpha to zero on graphics layer when mixing making video layer apearing over graphics. This can be changed however, making it possible to render with alpha to framebuffer 0 and having the ability to choose what will and what will not be above video layer. Note that this is only possible if the framebuffer is set to 32bit BGRA, which it should be by default in Ångström.
-==Finite==
+===Using the Overlay===
-Looking back to the beginning of this summer XBMC didn't even compile on Ångström and now its possible to atleast use XBMC and Beagle Board as a fully working SD box. While there still probably exist lots of rendering optimizations and there still is a need to create a really optimized skin I would like to say I am happy with the result. Before xbmc ran at about 10fps (with default skin) which now with dirty region based rendering turned on it runs at almost 20fps (default skin), . The POC optmized skin which looks essentially the same as the default skin runs at 30fps with dirty region based rendering turned on it hits the goal I was aiming for 720p!. Its also clear that the greatest bottleneck is still the GPU which will be significantly faster on the Beagle Board xM it might just be a very viable HD choice for XBMC.
+The overlay is available for use by any user in the group 'video'. Its control and used by posix ioctl calls and by filling up the framebuffer with the data. While its perfectly possible to displaying video using singlebuffering doublebuffering will generate prettier and sturdier code. If we only have singlebuffering we would need to make sure we never write something that the overlay haven't read, and it can be very tricky keeping read and write from the different processes in sync. Doublebuffering does need a larger memory but its usually no problem since we are dealing with YUV. Since we want to limit unnecessary copying a smart solution is to use just one framebuffer for doublebuffering. The framebuffer will be twice as large framebuffer and we only let the overlay ever display half of that framebuffer. This means that we dedicate one part of the framebuffer as frontbuffer and the other part as backbuffer. Just before we are stating that we are ready for displaying we switch the offset the overlay will read from, alas we have switched the front and backbuffers.

@@ Line 115: / Line 115: @@
 ===Embedded and Beagle Board specific===
 Up to and including C4 version of Beagle Board the GPU is the harsh bottleneck and to optimize rendering its worth understanding what takes most resources.
-On embedded platforms bandwith is usually very limited, this is true both in uploading data to RAM/GPU and the bandwith used to manipulate the backbuffer. To create a fluid skin on an embedded plattform its thus vital to limit the needed textures to present the skin, this will limit loading time but also rendering. As a suggestion its very wise to use GIMP or Photoshop to create the mockups since using layers is very useful in the development stage but when the skin is meant to be used on an embedded plattform unnecessary layers can take quite a lot of resources to render. This holds true especially when multiple layers are used to form one static image, its alot faster to use GIMP or Photoshop to merge down the layers and only use the one in XBMC. Every layer that could be removed will help alot hence it is worth considering skipping dynamic backgrounds for embedded usage and merge together overlays and backgrounds to a single layer for the skin. On the beagle board showing a few buttons will yield about 60fps in 720p but also having one non-blended background will make the speed 20fps, and this is only with one merged background! Since the SGX graphics core is designed to be used mostly in low resolution situations the textures used are generally small. If a texture is to big the SGX core might not be able to use the texture cache properly, this means that using textures above 512x512 is significantly slower, regardless of the rendering area. So skinners need to think long and hard before using larger textures and if a large texture is required splitting it into pieces with multiple controls can be alot faster. Also usually using power of two textures are faster so as an example using a 1280x720 texture rendered over the same area as 4 512x256 textures will be 20fps against 30fps, the resulting picture is the same to the user but with 50% performance increase!.
 Using textures with alpha are a very common practice to create nice looking skins, its worth knowing that rendering with blending enable will use more than twice the amount of bandwith since it needs to read the backbuffer, blend and then write it back. Without blend it only needs to write. Note that the needed bandwith is on a per pixel basis so avoiding alpha on big images is key, for example backgrounds. XBMC isn't extremely smart at understanding if the image has alpha or not so for non-alpha images use JPG as its guaranteed to work.

@@ Line 115: / Line 115: @@
 ===Embedded and Beagle Board specific===
 Up to and including C4 version of Beagle Board the GPU is the harsh bottleneck and to optimize rendering its worth understanding what takes most resources.
-On embedded platforms bandwith is usually very limited, this is true both in uploading data to RAM/GPU and the bandwith used to manipulate the backbuffer. To create a fluid skin on an embedded plattform its thus vital to limit the needed textures to present the skin, this will limit loading time but also rendering. As a suggestion its very wise to use GIMP or Photoshop to create the mockups since using layers is very useful in the development stage but when the skin is meant to be used on an embedded plattform unnecessary layers can take quite a lot of resources to render. This holds true especially when multiple layers are used to form one static image, its alot faster to use GIMP or Photoshop to merge down the layers and only use the one in XBMC. Every layer that could be removed will help alot hence it is worth considering skipping dynamic backgrounds for embedded usage and merge together overlays and backgrounds to a single layer for the skin. On the beagle board showing a few buttons will yield about 60fps in 720p but also having one non-blended background will make the speed 20fps, and this is only with one merged background!
+On embedded platforms bandwith is usually very limited, this is true both in uploading data to RAM/GPU and the bandwith used to manipulate the backbuffer. To create a fluid skin on an embedded plattform its thus vital to limit the needed textures to present the skin, this will limit loading time but also rendering. As a suggestion its very wise to use GIMP or Photoshop to create the mockups since using layers is very useful in the development stage but when the skin is meant to be used on an embedded plattform unnecessary layers can take quite a lot of resources to render. This holds true especially when multiple layers are used to form one static image, its alot faster to use GIMP or Photoshop to merge down the layers and only use the one in XBMC. Every layer that could be removed will help alot hence it is worth considering skipping dynamic backgrounds for embedded usage and merge together overlays and backgrounds to a single layer for the skin. On the beagle board showing a few buttons will yield about 60fps in 720p but also having one non-blended background will make the speed 20fps, and this is only with one merged background! Since the SGX graphics core is designed to be used mostly in low resolution situations the textures used are generally small. If a texture is to big the SGX core might not be able to use the texture cache properly, this means that using textures above 512x512 is significantly slower, regardless of the rendering area. So skinners need to think long and hard before using larger textures and if a large texture is required splitting it into pieces with multiple controls can be alot faster. Also usually using power of two textures are faster so as an example using a 1280x720 texture rendered over the same area as 4 512x256 textures will be 20fps against 30fps, the resulting picture is the same to the user but with 50% performance increase!.
 Using textures with alpha are a very common practice to create nice looking skins, its worth knowing that rendering with blending enable will use more than twice the amount of bandwith since it needs to read the backbuffer, blend and then write it back. Without blend it only needs to write. Note that the needed bandwith is on a per pixel basis so avoiding alpha on big images is key, for example backgrounds. XBMC isn't extremely smart at understanding if the image has alpha or not so for non-alpha images use JPG as its guaranteed to work.
 ==Dirty region rendering==
 Most of my time during this google summer of code was spent on making the rendering system in XBMC being able to only render what had changed. This is a very common tecniuqe when there is little available bandwith and the interface rarely changes over big areas. In theory a dirty region based rendering should allow quite a big performance increase if the interface is close to static. In XBMC this is generally true but given its meant for TV usage and viewed from a distance its quite normal that rather large parts are changing when something happens. Thus the hard limit for performance increase is up to the skinner if he or she can design a skin which only changes in small areas at one time, this could be accomplised if animations are few or small.

@@ Line 119: / Line 119: @@
 ==Dirty region rendering==
 Most of my time during this google summer of code was spent on making the rendering system in XBMC being able to only render what had changed. This is a very common tecniuqe when there is little available bandwith and the interface rarely changes over big areas. In theory a dirty region based rendering should allow quite a big performance increase if the interface is close to static. In XBMC this is generally true but given its meant for TV usage and viewed from a distance its quite normal that rather large parts are changing when something happens. Thus the hard limit for performance increase is up to the skinner if he or she can design a skin which only changes in small areas at one time, this could be accomplised if animations are few or small.
-==How does it work?==
+===How does it work?===
 When you use XBMC every view or intendt you see is called a window. A skin has the ability to form a window how they see fit. A window is made up of controls or groups of controls. A control could be an inanimate object such as a background image or something you interact with, for example a button. This means the skinner have full control what and how everything is displayed in every window.
 Under the hood XBMC will process each control and if something changes it will mark said controls covering area as dirty. When every control have had the opportunity to mark, XBMC will try to optimize how it should render the possibly overlapping dirty regions. Note that since opengl is normally flipping back and front buffers by pointer, i.e. The backbuffer or drawing board is not the same as the one just presented, XBMC needs to track dirty regions from the presenting buffer as well. Normally opengl is doublebuffered and thus we track dirty regions for 2 renderings.
 ===Leverage dirty region based rendering as a skinner===
 The core problem dirty region based rendering tries to solve is to only render what has changed. This is only effective if the changing parts are keept small. While without dirty regions a skinner could create a minimal skin which is rather fast, with dirty region based rendering a skinner have the possibility to present a complex skin with the illusion of fast framerate. What is key here is that if a skinner is able to confine changes to small areas the rendering would be incremental over the entire screen. During google summer of code a cost reduction algorithm was devised to allow for small areas being parted over vast distances to occur. For example if a small change happens in the lower left corner and the upper right corner, the entire screen wouldn't be redrawn.
 ==OMAP Overlay==
 In the desktop segment XBMC utilizes the GPU to do the YUV to RGB conversion, both to offload these calculations from the CPU but also to limit the uploading bandwith. This is a very optimized approach if the GPU is powerfull enough to do this while still presenting the GUI, on the Beagle board this is sadly not the case.

← Older revision		Revision as of 20:38, 11 August 2010
Line 102:		Line 102:

	MACHINE=beagleboard bitbake xbmc		MACHINE=beagleboard bitbake xbmc
		+
		+	Google summer of code 2010 – Rendering optimizations of XBMC for the Beagle Board
		+
		+	==Abstract==
		+
		+	==Optimizing skins for Beagle Board==
		+	There are a number of ways to optimize skins, some of which are general and some are specifically target for embedded and GLES both will be covered in the following section.
		+	===General optimizations===
		+	On desktop its very rare that the GPU is limiting when rendering XBMC but lower resources could mean less heat generation and slower fanspeeds but also less power requirements which could mean longer battery life or cheaper electric bill. With that said some slower built in graphic cards could still use the performance boost in the higher resolutions.
		+	Generally speaking more controls being rendered equals more performance needed thus it makes sense to limit redundant controls if possible.
		+	A fluid skin is not just something which runs at a fast framerate when loaded but also a skin which loads each window quickly. Its worth considering that a harddrive might not be as fast on the HTPC as it is on the workstaion and by far the thing which takes longest to load are images. While it is possible to background load images to get a fast initial load of a window its not very nice if it takes tens of seconds to get it fully finalized. Scaling and positioning textures in accelerated rendering systems is usually almost for free which means using the border tag in skinning to limit the size of a texture could significantly lower the needed load.
		+	===Embedded and Beagle Board specific===
		+	Up to and including C4 version of Beagle Board the GPU is the harsh bottleneck and to optimize rendering its worth understanding what takes most resources.
		+	On embedded platforms bandwith is usually very limited, this is true both in uploading data to RAM/GPU and the bandwith used to manipulate the backbuffer. To create a fluid skin on an embedded plattform its thus vital to limit the needed textures to present the skin, this will limit loading time but also rendering. As a suggestion its very wise to use GIMP or Photoshop to create the mockups since using layers is very useful in the development stage but when the skin is meant to be used on an embedded plattform unnecessary layers can take quite a lot of resources to render. This holds true especially when multiple layers are used to form one static image, its alot faster to use GIMP or Photoshop to merge down the layers and only use the one in XBMC. Every layer that could be removed will help alot hence it is worth considering skipping dynamic backgrounds for embedded usage and merge together overlays and backgrounds to a single layer for the skin. On the beagle board showing a few buttons will yield about 60fps in 720p but also having one non-blended background will make the speed 20fps, and this is only with one merged background!
		+	Using textures with alpha are a very common practice to create nice looking skins, its worth knowing that rendering with blending enable will use more than twice the amount of bandwith since it needs to read the backbuffer, blend and then write it back. Without blend it only needs to write. Note that the needed bandwith is on a per pixel basis so avoiding alpha on big images is key, for example backgrounds. XBMC isn't extremely smart at understanding if the image has alpha or not so for non-alpha images use JPG as its guaranteed to work.
		+	==Dirty region rendering==
		+	Most of my time during this google summer of code was spent on making the rendering system in XBMC being able to only render what had changed. This is a very common tecniuqe when there is little available bandwith and the interface rarely changes over big areas. In theory a dirty region based rendering should allow quite a big performance increase if the interface is close to static. In XBMC this is generally true but given its meant for TV usage and viewed from a distance its quite normal that rather large parts are changing when something happens. Thus the hard limit for performance increase is up to the skinner if he or she can design a skin which only changes in small areas at one time, this could be accomplised if animations are few or small.
		+	==How does it work?==
		+	When you use XBMC every view or intendt you see is called a window. A skin has the ability to form a window how they see fit. A window is made up of controls or groups of controls. A control could be an inanimate object such as a background image or something you interact with, for example a button. This means the skinner have full control what and how everything is displayed in every window.
		+	Under the hood XBMC will process each control and if something changes it will mark said controls covering area as dirty. When every control have had the opportunity to mark, XBMC will try to optimize how it should render the possibly overlapping dirty regions. Note that since opengl is normally flipping back and front buffers by pointer, i.e. The backbuffer or drawing board is not the same as the one just presented, XBMC needs to track dirty regions from the presenting buffer as well. Normally opengl is doublebuffered and thus we track dirty regions for 2 renderings.
		+	===Leverage dirty region based rendering as a skinner===
		+	The core problem dirty region based rendering tries to solve is to only render what has changed. This is only effective if the changing parts are keept small. While without dirty regions a skinner could create a minimal skin which is rather fast, with dirty region based rendering a skinner have the possibility to present a complex skin with the illusion of fast framerate. What is key here is that if a skinner is able to confine changes to small areas the rendering would be incremental over the entire screen. During google summer of code a cost reduction algorithm was devised to allow for small areas being parted over vast distances to occur. For example if a small change happens in the lower left corner and the upper right corner, the entire screen wouldn't be redrawn.
		+	==OMAP Overlay==
		+	In the desktop segment XBMC utilizes the GPU to do the YUV to RGB conversion, both to offload these calculations from the CPU but also to limit the uploading bandwith. This is a very optimized approach if the GPU is powerfull enough to do this while still presenting the GUI, on the Beagle board this is sadly not the case.
		+	Thankfully the OMAP plattform beagle board is based on have a hardware accelerated display driver which is capable of doing this conversion, scaling and positioning in hardware. The result meant offloading both CPU and GPU from the strain to do this coversion and in turn being able to do this up to 720p resolutions. Sadly the player in xbmc isn't optimized enough to give 720p playback on beagleboard using just the CPU for decoding. The driver is also capable of blending the resulting RGB with the other layers to produce the illusion of having rendered stuff over the resulting video. Which is needed when displaying for example volume changes or on screen menus for control the video.
		+	===Technicals===
		+	The overlay reads data from the framebuffers found in /dev/fbX. Multiple overlays can read from the same framebuffer and an overlay can be told to read only from a part of the framebuffer. The display area of the overlay can be larger or smaller then that found in the framebuffer and the overlay will then scale it accordingly. Since the overlay is able to read only part of the data from a framebuffer its possible to create a larger framebuffer than the data its meant to hold to create a front- and backbuffer and switch the location the overlay should read from before presenting.
		+	Overlay 1 (video layer) is able to read YUV422 (or RGB) which will be transformed, scaled and positioned before presenting and all is done in hardware. Normally X and SGX is rendering to fb0 which is linked to overlay 0 (graphics layer). Graphics layer is above video layer but interestingly enough by default the display manager sets final alpha to zero on graphics layer when mixing making video layer apearing over graphics. This can be changed however, making it possible to render with alpha to framebuffer 0 and having the ability to choose what will and what will not be above video layer. Note that this is only possible if the framebuffer is set to 32bit BGRA, which it should be by default in Ångström.
		+	==Finite==
		+	Looking back to the beginning of this summer XBMC didn't even compile on Ångström and now its possible to atleast use XBMC and Beagle Board as a fully working SD box. While there still probably exist lots of rendering optimizations and there still is a need to create a really optimized skin I would like to say I am happy with the result. Before xbmc ran at about 10fps (with default skin) which now with dirty region based rendering turned on it runs at almost 20fps (default skin), . The POC optmized skin which looks essentially the same as the default skin runs at 30fps with dirty region based rendering turned on it hits the goal I was aiming for 720p!. Its also clear that the greatest bottleneck is still the GPU which will be significantly faster on the Beagle Board xM it might just be a very viable HD choice for XBMC.

@@ Line 50: / Line 50: @@
 	<gui>
-		<usedirtyregions>true</usedirtyregions>
+		<algorithmdirtyregions>1</algorithmdirtyregions>
 	</gui>
 </advancedsettings>
 === Build Instructions (native) ===

@@ Line 66: / Line 66: @@
 ./bootstrap.angstrom
-./configure --enable-gles --prefix=/usr --sysconfdir=/etc --cache=config.cache --disable-optical-drive
+./configure --enable-gles --enable-omap-overlay --prefix=/usr --sysconfdir=/etc --cache=config.cache --disable-optical-drive
 make