Difference between revisions of "BeagleBoard/GSoC/2021 Proposal/bela on bbai"

From eLinux.org
Jump to: navigation, search
m (Proposed method of Implementation and Timeline)
m (Proposed method of Implementation)
 
(27 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
[[Category: GSoCProposal2021]]
 
[[Category: GSoCProposal2021]]
  
=[[BeagleBoard/GSoC/ProposalTemplate|ProposalTemplate]] =
+
=[[BeagleBoard/GSoC/ProposalTemplate|Proposal-Bela support for BBAI]] =
{{#ev:youtube|https://www.youtube.com/watch?v=aVLRUyPBBJk}} <!--  -->
+
[https://www.youtube.com/watch?v=aVLRUyPBBJk Youtube Video] <br>
About
+
About<br>
 
''Student'': [https://elinux.org/User:DhruvaG2000 Dhruva Gole]<br>
 
''Student'': [https://elinux.org/User:DhruvaG2000 Dhruva Gole]<br>
''Mentors'': [http://elinux.org/User:giuliomoro Giulio Moro]<br>
+
''Mentors'': [https://github.com/giuliomoro Giulio Moro], Stephen Arnold and Robert Manzke<br>
 
''Code'': https://github.com/BelaPlatform/Bela<br>
 
''Code'': https://github.com/BelaPlatform/Bela<br>
''Wiki'': https://learn.bela.io/get-started-guide/say-hello-to-bela/#what-is-bela<br>
+
''Wiki'': https://forum.beagleboard.org/t/bela-support-for-bbai-later-ti-chips/29257/7<br>
''GSoC'': [https://summerofcode.withgoogle.com/archive/2016/projects/4885400476712960/ GSoC entry]<br>
+
''GSoC'': [https://summerofcode.withgoogle.com/projects/#5697403266531328 GSoC entry]<br>
 
<div style="clear:both;"></div>
 
<div style="clear:both;"></div>
  
 
=Status=
 
=Status=
This project is currently just a proposal.
+
This project proposal has been accepted as part of GSoC 2021.
  
 
=Proposal=
 
=Proposal=
Please complete the requirements listed on the [http://bbb.io/gsocideas ideas page] and fill out this template.
+
Bela is a hardware and software system for creating beautiful interaction with sensors and sound. This project proposes to provide restructuring and improvement of existing Bela Software Code to allow for compatibility and easier transition to newer TI chips (like in the BBAI).  
 
 
 
==About you==
 
==About you==
 
''IRC'': dhruvag2000<br>
 
''IRC'': dhruvag2000<br>
Line 30: Line 29:
 
==About your project==
 
==About your project==
 
''Project name'': Bela support for the BeagleBone AI<br>
 
''Project name'': Bela support for the BeagleBone AI<br>
 
+
''My Blog'': My [https://dhruvag2000.github.io/bbai-notes/#Bela blog] related to this project and my findings. <br>
 +
''Logs'' : I maintain weekly progress updates [https://dhruvag2000.github.io/bbai-notes/logs.html here ].
 
===Description===
 
===Description===
 
;What is Bela?  
 
;What is Bela?  
Line 37: Line 37:
 
Both Bela systems use the same Bela software. It uses a customised Debian distribution which - most notably - uses a '''Xenomai kernel''' instead of a stock kernel. ''Xenomai'' is ''co-kernel'' for Linux which allows to achieve hard '''real-time performance''' on Linux machines (http://xenomai.org/). It thus takes advantage of features of the BeagleBone computers and can achieve extremely fast audio and sensor processing times.
 
Both Bela systems use the same Bela software. It uses a customised Debian distribution which - most notably - uses a '''Xenomai kernel''' instead of a stock kernel. ''Xenomai'' is ''co-kernel'' for Linux which allows to achieve hard '''real-time performance''' on Linux machines (http://xenomai.org/). It thus takes advantage of features of the BeagleBone computers and can achieve extremely fast audio and sensor processing times.
 
Although the proposal Title mentions support for AI, I will try to develop a standardized setup that allows an easy jump across all TI chips.  
 
Although the proposal Title mentions support for AI, I will try to develop a standardized setup that allows an easy jump across all TI chips.  
 +
 
;Applications of Bela:  
 
;Applications of Bela:  
 
Bela is ideal for creating anything interactive that uses sensors and sound. So far, Bela has been used to create:
 
Bela is ideal for creating anything interactive that uses sensors and sound. So far, Bela has been used to create:
Line 44: Line 45:
 
#interactive sound installations
 
#interactive sound installations
 
and many more applications that are listed [https://learn.bela.io/get-started-guide/say-hello-to-bela/#what-is-bela here]
 
and many more applications that are listed [https://learn.bela.io/get-started-guide/say-hello-to-bela/#what-is-bela here]
 +
 
;Why add support for BBAI/newer TI chips?  
 
;Why add support for BBAI/newer TI chips?  
 
The Beagle Black and PocketBeagle are getting outdated by the day and also it would be better to have a more standardised setup that allows to more easily jump across TI chips, soon newer boards with different and more efficient chips like the AM5X and the TI C66x digital-signal-processor (DSP) cores in the BBAI are coming up that will need compatiblity with the Bela Software and Hardware. As part of the process we will try to offload peripheral initialisation to the mcasp and spi driver to in order to make it more portable for porting to other boards in the family, at present or future.  
 
The Beagle Black and PocketBeagle are getting outdated by the day and also it would be better to have a more standardised setup that allows to more easily jump across TI chips, soon newer boards with different and more efficient chips like the AM5X and the TI C66x digital-signal-processor (DSP) cores in the BBAI are coming up that will need compatiblity with the Bela Software and Hardware. As part of the process we will try to offload peripheral initialisation to the mcasp and spi driver to in order to make it more portable for porting to other boards in the family, at present or future.  
 
;Programming languages and tools to be used:  
 
;Programming languages and tools to be used:  
C, PRU, dtb, uboot, make
+
C, C++, PRU, dtb, uboot, GNU Make, ARM Assembly
  
===Proposed method of Implementation and Timeline===
+
===Proposed method of Implementation===
  
Current status:
+
The Bela cape is normally used in combination with the TI AM3358 chip on the BeagleBone Black. The hardware was partially working on the BBAI only using ALSA and the SPI driver [https://github.com/giuliomoro/beaglebone-ai-bela], so the pinout and pin settings are somewhat okay, but the Bela real-time code on ARM and PRU is not running on BBAI yet. During this project I will:
The Bela cape is normally used in combination with the TI AM3358 chip on the BeagleBone Black. The hardware is currently working on the BBAI only using ALSA and the SPI driver [https://github.com/giuliomoro/beaglebone-ai-bela], so the pinout and pin settings are OK, but the Bela real-time code on ARM and PRU is not running on BBAI yet. During this project I will:
 
  
# create a device tree overlay for the BBAI using the Cape Compatibility layer
+
# Create a device tree overlay using Cape Compatibility layer for the [https://github.com/beagleboard/bb.org-overlays/blob/master/src/arm/BB-BONE-AUDI-02-00A0.dts BB-BONE-AUDI] which worked but with a few frequency issues.
 +
# create a device tree overlay for the BELA Cape to work on the BBAI using the Cape Compatibility layer
 
# update the Bela code to use the McASP, GPIO and McSPI on the AM5729 SoC of the BBAI
 
# update the Bela code to use the McASP, GPIO and McSPI on the AM5729 SoC of the BBAI
 
# adapt the Bela PRU and ARM code and workflow to use the PRU via remoteproc instead of uio_pruss
 
# adapt the Bela PRU and ARM code and workflow to use the PRU via remoteproc instead of uio_pruss
# install a Xenomai kernel and run the full Bela stack
+
# install a Xenomai patched kernel &#10003; and run the full Bela stack.
  
 
The project will involve dealing with pinmuxing, uboot, PRU assembly, C++ for Linux user space applications, the Technical Reference Manual for the Sitara family of SoCs. ([https://elinux.org/BeagleBoard/GSoC/Ideas-2021#Beagle_Bone_Audio_Applications ref. here])  
 
The project will involve dealing with pinmuxing, uboot, PRU assembly, C++ for Linux user space applications, the Technical Reference Manual for the Sitara family of SoCs. ([https://elinux.org/BeagleBoard/GSoC/Ideas-2021#Beagle_Bone_Audio_Applications ref. here])  
Line 67: Line 69:
 
# in the Makefile, update the workflow to build the PRU code for remoteproc
 
# in the Makefile, update the workflow to build the PRU code for remoteproc
 
# in core/PRU.cpp, use remoteproc instead of the libprussdrv API
 
# in core/PRU.cpp, use remoteproc instead of the libprussdrv API
# in pru/pru_rtaudio.p, the hardcoded McASP constants should be replaced with board-dependent ones
+
# in pru/pru_rtaudio.p, the hardcoded McASP constants should be replaced with board-dependent ones.
# in order to progress via intermediate steps, I may want to test the code first running on vanilla Linux without Xenomai. This requires replacing the ''__wrap_pthread_{...}'' calls (in core/RTAudio.cpp, core/AuxiliaryTask.cppm include/xenomai_wraps.h) with the corresponding ''pthread_{...}'' functions.
 
# the BBAI can be added to the BelaHw enum in ''include/Bela.h'' and used in ''Bela_detectUserHw()'' and ''Bela_checkHwCompatibility()'' in ''core/board_detect.cpp''.
 
  
 
All these changes will be made so that the same code base can run on all supported boards (e.g.: BBAI, BBB) with build- or runtime- checks. To explain in short how I aim to establish a common code base for supporting all boards, I will start off by enumerating different boards once we detect which board it is. Then in the libraries, the constants (like pin numbers and their symlinks) will be set accordingly and the rest of the code base will become much easier to use without the need to hard code anything.  
 
All these changes will be made so that the same code base can run on all supported boards (e.g.: BBAI, BBB) with build- or runtime- checks. To explain in short how I aim to establish a common code base for supporting all boards, I will start off by enumerating different boards once we detect which board it is. Then in the libraries, the constants (like pin numbers and their symlinks) will be set accordingly and the rest of the code base will become much easier to use without the need to hard code anything.  
 
<br>
 
<br>
 +
 
; PRU:
 
; PRU:
 
# The current Bela core code uses ''pasm'' to build the PRU assembly ''pru/pru_rtaudio.p'' and uses libprussdrv, which binds to the uio_pruss kernel driver to load the firmware to the PRU and handle access to the PRU RAM. Both pasm and libprussdrv are now deprecated, replaced by the clpru toolchain and remoteproc driver respectively.
 
# The current Bela core code uses ''pasm'' to build the PRU assembly ''pru/pru_rtaudio.p'' and uses libprussdrv, which binds to the uio_pruss kernel driver to load the firmware to the PRU and handle access to the PRU RAM. Both pasm and libprussdrv are now deprecated, replaced by the clpru toolchain and remoteproc driver respectively.
# The PRU firmware contains hardcoded values for the address of the McASP, McSPI and GPIO peripherals. These addresses will change for the BBAI, so these constants need to become conditional.
+
# The PRU firmware contains hardcoded values for the address of the McASP, McSPI and GPIO peripherals. These addresses will change for the BBAI, so these constants need to become conditional. &#10003;(yet to be tested, but base address for the McASP of the bbai has been updated)
 
# As the Bela PRU firmware is written in assembly for pasm, instead of rewriting or updating it in such a way that it will stop working on the current Bela images, we will use the workflow detailed below.
 
# As the Bela PRU firmware is written in assembly for pasm, instead of rewriting or updating it in such a way that it will stop working on the current Bela images, we will use the workflow detailed below.
  
Line 81: Line 82:
 
The workflow below works as a proof of concept, but needs further testing.
 
The workflow below works as a proof of concept, but needs further testing.
 
# Build the .p file as is with pasm with ''-V2 -b''. This generates a .bin file that contains the assembled program
 
# Build the .p file as is with pasm with ''-V2 -b''. This generates a .bin file that contains the assembled program
# I will be using the disassembler Giulio Moro put together hacking the one that was inside prudebug. (Find it [https://github.com/giuliomoro/prudebug/tree/disassembler here](built with ''gcc da.c -o prudis'') .
+
# I will be using the disassembler Giulio Moro put together hacking the one that was inside prudebug. (Find it [https://github.com/giuliomoro/prudebug/tree/disassembler here](built with ''gcc da.c -o prudis'') . ('''Note:''' ''A disassembler'' is a computer program that translates ''machine language'' into ''assembly language'')
 
# Process the bin through the disassembelr and make it ready to be included inside an ''__asm__'' directive (i.e.: add quotes and prepend a space at the beginning of each line):
 
# Process the bin through the disassembelr and make it ready to be included inside an ''__asm__'' directive (i.e.: add quotes and prepend a space at the beginning of each line):
 
     prudis file.bin | sed 's/^\(.*\)$/" \1\\n"/' > included_assembly.h
 
     prudis file.bin | sed 's/^\(.*\)$/" \1\\n"/' > included_assembly.h
# have the following in main.c:
+
4. have the following in main.c:
 
  void main()
 
  void main()
 
  {
 
  {
Line 92: Line 93:
 
       );
 
       );
 
  }
 
  }
# build main.c with the regular clpru toolchain
+
5. build main.c with the regular clpru toolchain
  
I will need to change these MCASP addresses in pru/pru_rtaudio.p :
+
6. I will also need to change these MCASP addresses in pru/pru_rtaudio.p :
  
 
  #define MCASP0_BASE 0x48038000
 
  #define MCASP0_BASE 0x48038000
Line 102: Line 103:
 
  #define MCASP1_DATAPORT 0x46400000
 
  #define MCASP1_DATAPORT 0x46400000
  
Additionally,  I will need to compare the McASP and McSPI sections of the AM5729 and AM3358 to verify that all the registers of the McASP and McSPI peripherals kept the same meaning and offsets between the two chips, or adjust the code accordingly.
+
Additionally,  I will need to compare the ''McASP'' and ''McSPI'' sections of the AM5729 and AM3358 manuals to verify that all the registers of the McASP and McSPI peripherals kept the same meaning and offsets between the two chips, or adjust the code accordingly.
  
 
; PINMUXING
 
; PINMUXING
 
I am yet to figure out pinmuxing to make sure the relevant pins are set to the correct function (they are NOT by default). I will see here https://elinux.org/BeagleBoard/GSoC/2020_Projects/Cape_Compatibility and/or ask on the BB slack about for the best way to go about it.
 
I am yet to figure out pinmuxing to make sure the relevant pins are set to the correct function (they are NOT by default). I will see here https://elinux.org/BeagleBoard/GSoC/2020_Projects/Cape_Compatibility and/or ask on the BB slack about for the best way to go about it.
 +
I have so far created a basic AUDIO overlay (named BBAI-AUDI-02-00A0.dts) that has been merged into the  [https://github.com/beagleboard/BeagleBoard-DeviceTrees/blob/v4.19.x-ti-overlays/src/arm/overlays/BBAI-AUDI-02-00A0.dts beagleboard / BeagleBoard-DeviceTrees repository].
  
 
; PRU->ARM INTERRUPTS
 
; PRU->ARM INTERRUPTS
Line 118: Line 120:
  
 
; XENOMAI kernel
 
; XENOMAI kernel
I would need to install a xenomai kernel through the default procedure to update kernel. A Xenomai kernel is already available, however it is untested. If any issues arise, I may have to rebuild the kernel. I will also need to install the user-space Xenomai library as well. That MAY be available through apt-get (Xenomai >= 3.0), or it can be easily built from source.
+
Xenomai kernel (v4.19.94-ti-xenomai-r64) has been built and tested for the BBAI. I have installed the xenomai kernel and [https://dhruvag2000.github.io/Blog-GSoC21/xenomai/install.html libraries] through the default procedure to update kernel. I have also managed to successfully build the entire Bela core code on the BBAI so far.
  
 
; Hardware required:
 
; Hardware required:
Line 124: Line 126:
 
# [https://www.digikey.com/product-detail/en/seeed-technology-co-ltd/102110362/1597-102110362-ND/10492208 BeagleBone AI].
 
# [https://www.digikey.com/product-detail/en/seeed-technology-co-ltd/102110362/1597-102110362-ND/10492208 BeagleBone AI].
 
# Bela cape: [https://shop.bela.io/bela The original Bela board].
 
# Bela cape: [https://shop.bela.io/bela The original Bela board].
#
 
  
;Timeline
+
===Timeline===
  
 
{| class="wikitable"
 
{| class="wikitable"
 
| Mar 29 || Applications open || Students register with GSoC, work on proposal with mentors.
 
| Mar 29 || Applications open || Students register with GSoC, work on proposal with mentors.
 
|-
 
|-
| Apr 13 || Proposal complete || Submitted to https://summerofcode.withgoogle.com  
+
| Apr 13 || Proposal complete || Submitted to https://summerofcode.withgoogle.com &#10003;
 
|-
 
|-
 
| May 17 || Proposal accepted or rejected ||
 
| May 17 || Proposal accepted or rejected ||
 +
# [https://summerofcode.withgoogle.com/projects/#5697403266531328 Proposal Accepted] &#10003;
 
# Community Bonding Period and discussion on the project and resources available.  
 
# Community Bonding Period and discussion on the project and resources available.  
# Setting up beaglebone-ai i.e flashing [https://beagleboard.org/latest-images up to date Linux image] and connect to local area network (LAN) via either Ethernet or WiFi and try to run bare_minimum code from this [https://github.com/jadonk/librobotcontrol/tree/master/examples repository] to test basic stuff is working.
+
# Learn about embedded linux structure &#10003;
 +
# Setting up beaglebone-ai i.e flashing [https://beagleboard.org/latest-images up to date Linux image] and connect to local area network (LAN) via either Ethernet or WiFi and try to run example codes from this [https://elinux.org/BeagleBoard/GSoC/2021_Proposal/bela_on_bbai repository] to test basic stuff is working.
 
|-
 
|-
 
| Jun 07 || Pre-work complete, Coding officially begins! ||  
 
| Jun 07 || Pre-work complete, Coding officially begins! ||  
# All the boards and additional hardware will be available to me at this period of time and I should have set up my BeagleBone Black and Ai boards i.e flashing up to date Linux image and Testing user-led blink code :)
+
# All the boards and BELA Cape will be available to me at this period of time and I should have set up my BeagleBone Black and Ai boards i.e flashing up to date Linux image.
# Initial checks for hardware like audio ports and other peripheral devices will be completed and all the soldering related work will be done.
+
# Initial checks for hardware like audio ports and other peripheral devices will be completed.
# A detailed spreadsheet will be created if not already available for cape pin mapping and referencing for further use during BeagleBone AI software development ease.
+
# A detailed spreadsheet will be created if not already available for cape pin mapping and referencing for further use during BeagleBone AI software development ease. (one exists [https://docs.google.com/spreadsheets/d/1h-oUVhZXogOkKJkq73dM1JPOzcslBfcdpxTx4fZ-Cg0/edit#gid=227990209 here])
(above ref. from [https://elinux.org/index.php?title=BeagleBoard/GSoC/2020_Projects/Cape_Compatibility here] )  
+
(some points above are ref. from [https://elinux.org/index.php?title=BeagleBoard/GSoC/2020_Projects/Cape_Compatibility here] )  
  
 
|-
 
|-
 
| Jun 17 || Milestone #1, ||  
 
| Jun 17 || Milestone #1, ||  
# Introductory YouTube video
+
# Introductory YouTube video &#10003;
# Setting pinmux values appropriately and fix the dtb to get the correct clock on the McASP MCLK pin.
+
# Setting pinmux values appropriately and fix the dtb to get the correct clock on the McASP MCLK pin. &#10003;
# verify it works running with ALSA using the BB-BONE-AUDI overlay.
+
# Writing the [https://github.com/beagleboard/BeagleBoard-DeviceTrees/blob/v4.19.x-ti-overlays/src/arm/overlays/BBAI-AUDI-02-00A0.dts BBAI-AUDI-02-00A0.dts] overlay to port the old BB-BONE-AUDI overlay using the CCL to work on the AI. &#10003;
 +
# verify that it works running with ALSA. &#10003;
 
|-
 
|-
 
| June 24 || Milestone #2 ||  
 
| June 24 || Milestone #2 ||  
# test and debug running Bela's PRU code on BBAI
+
# Writing the overlay for Bela Cape.
 +
# Study about the remote processor framework and rpmesg, and how I can integrate it into the PRU.cpp code
 +
# Test and debug the workflow for building the existing pasm PRU code with clpru, (or possibly reconsider/change it)
 
|-
 
|-
 
| June 30 || Milestone #3 ||  
 
| June 30 || Milestone #3 ||  
# Try to run Bela without Xenomai (This requires changing a few calls to ''__wrap_pthread_...()'' in the Bela core code to ''pthread_...()'' and minor changes.)
+
# Try to implement the PRU code using ''rproc''.
# run Bela with Xenomai, without rtdm(Real-time data monitoring) ie. generate (PRU->ARM interrupts)
+
# Try to test and debug running Bela's PRU code on BBAI.
# Document everything done so far and make READMEs for undocumented but pre-existing codes(for eg: [https://github.com/BelaPlatform/rtdm_pruss_irq this]) .
+
# make READMEs for undocumented but pre-existing codes (for eg: [https://github.com/BelaPlatform/rtdm_pruss_irq this]) .
 
|-
 
|-
| July 12 18:00 UTC || Milestone #4 ||
+
| July 12 18:00 UTC || Milestone #4 ||  
 +
# Try to run Bela examples with Xenomai, but without rtdm(Real-time data monitoring) ie. generate (PRU->ARM interrupts)
 +
# Start documenting everything done so far.
 +
# Make demo videos for proving whatever that has been done so far works on actual-hardware.
 
Mentors and students can begin submitting Phase 1 evaluations
 
Mentors and students can begin submitting Phase 1 evaluations
 
|-
 
|-
Line 164: Line 173:
 
| July 23 || Milestone #5 ||
 
| July 23 || Milestone #5 ||
 
# run Bela with Xenomai, with (PRU->ARM interrupts)
 
# run Bela with Xenomai, with (PRU->ARM interrupts)
# Try to run all [https://learn.bela.io/tutorials/ the stock tutorials]  
+
# Try to run [https://learn.bela.io/tutorials/ the stock tutorials]  
 
|-
 
|-
 
| July 30 || Milestone #6 ||
 
| July 30 || Milestone #6 ||
# Validate existing work with [https://github.com/BelaPlatform/Bela/tree/master/examples/Extras/cape-test some functional tests].
+
# Try to update the *.p files to suit the new gcc-pru supported syntax.
# Test and debug if any errors occur while running the stock tutorials and functional tests.
+
# Validate existing work with [https://github.com/BelaPlatform/Bela/tree/master/examples/ some functional tests and examples].
# Start organizing the codes so far by adding comments and making variable readable.  
+
# Test and debug if any errors occur while running the stock tutorials and functional tests.  
 
|-
 
|-
 
| Aug 06 || Milestone #7 ||
 
| Aug 06 || Milestone #7 ||
Line 200: Line 209:
  
 
# Derek Molloy's [http://derekmolloy.ie/beaglebone beagle bone guide] provides all the information needed for getting up and running with my beagle.
 
# Derek Molloy's [http://derekmolloy.ie/beaglebone beagle bone guide] provides all the information needed for getting up and running with my beagle.
# Content on [https://e-ale.org/ e-ALE.org] is very useful for information that ranges from tutorials and walkthroughs to the level of very advanced stuff.
+
# [https://www.kernel.org/doc/html/latest/staging/remoteproc.html remoteproc]
# https://credentiality2.blogspot.com/2015/09/beaglebone-pru-gpio-example.html
+
# [https://docs.google.com/spreadsheets/d/1h-oUVhZXogOkKJkq73dM1JPOzcslBfcdpxTx4fZ-Cg0/edit?usp=sharing BBAI vs BBB pin headers Google Sheet]
 +
# [https://credentiality2.blogspot.com/2015/09/beaglebone-pru-gpio-example.html beaglebone pru gpio example]
 +
# [https://learn.bela.io/tutorials/ official BELA website]
 +
# Ask on the [https://forum.beagleboard.org/t/bela-support-for-bbai-later-ti-chips/29257/4 BeagleBoard.org forum]
  
 
===Benefit===
 
===Benefit===
Line 207: Line 219:
 
<br>
 
<br>
  
By going through the steps needed to have the Bela environment running on BBAI, we will go through refactoring and rationalisation, using mainline drivers and APIs where possible. This will make Bela easier to maintain and to port to new platforms, benefiting the project's longevity and allowing it to expand its user base.
+
'' By going through the steps needed to have the Bela environment running on BBAI, we will go through refactoring and rationalisation, using mainline drivers and APIs where possible. This will make Bela easier to maintain and to port to new platforms, benefiting the project's longevity and allowing it to expand its user base.'' <br>
 
-Giulio Moro
 
-Giulio Moro
  

Latest revision as of 04:47, 25 July 2021


Proposal-Bela support for BBAI

Youtube Video
About
Student: Dhruva Gole
Mentors: Giulio Moro, Stephen Arnold and Robert Manzke
Code: https://github.com/BelaPlatform/Bela
Wiki: https://forum.beagleboard.org/t/bela-support-for-bbai-later-ti-chips/29257/7
GSoC: GSoC entry

Status

This project proposal has been accepted as part of GSoC 2021.

Proposal

Bela is a hardware and software system for creating beautiful interaction with sensors and sound. This project proposes to provide restructuring and improvement of existing Bela Software Code to allow for compatibility and easier transition to newer TI chips (like in the BBAI).

About you

IRC: dhruvag2000
Github: https://github.com/DhruvaG2000
School: Veermata Jijabai Technological Inst.
Country: India
Primary language : English, Marathi, Hindi
Typical work hours: 10AM - 7PM Indian Standard Time
Previous GSoC participation: I find embedded and IOT pretty interesting, given I have experience with ESP32, SIMCOM, ESP8266, Atmega, and many other microcontroller devices and I think I will be able to excel in this project. This is the first time i am participating in GSoC.

About your project

Project name: Bela support for the BeagleBone AI
My Blog: My blog related to this project and my findings.
Logs : I maintain weekly progress updates here .

Description

What is Bela?

As given on the official website, Bela is a hardware and software system for creating beautiful interaction with sensors and sound. Bela consists of a Bela cape on top of a BeagleBone Black computer (uptil now). Bela has a lot of analog and digital inputs and outputs for hooking up sensors and controlling other devices, and most importantly Bela has stereo audio i/o allowing you to interact with the world of sound.
Both Bela systems use the same Bela software. It uses a customised Debian distribution which - most notably - uses a Xenomai kernel instead of a stock kernel. Xenomai is co-kernel for Linux which allows to achieve hard real-time performance on Linux machines (http://xenomai.org/). It thus takes advantage of features of the BeagleBone computers and can achieve extremely fast audio and sensor processing times. Although the proposal Title mentions support for AI, I will try to develop a standardized setup that allows an easy jump across all TI chips.

Applications of Bela

Bela is ideal for creating anything interactive that uses sensors and sound. So far, Bela has been used to create:

  1. musical instruments and audio effects
  2. kinetic sculptures
  3. wearable devices
  4. interactive sound installations

and many more applications that are listed here

Why add support for BBAI/newer TI chips?

The Beagle Black and PocketBeagle are getting outdated by the day and also it would be better to have a more standardised setup that allows to more easily jump across TI chips, soon newer boards with different and more efficient chips like the AM5X and the TI C66x digital-signal-processor (DSP) cores in the BBAI are coming up that will need compatiblity with the Bela Software and Hardware. As part of the process we will try to offload peripheral initialisation to the mcasp and spi driver to in order to make it more portable for porting to other boards in the family, at present or future.

Programming languages and tools to be used

C, C++, PRU, dtb, uboot, GNU Make, ARM Assembly

Proposed method of Implementation

The Bela cape is normally used in combination with the TI AM3358 chip on the BeagleBone Black. The hardware was partially working on the BBAI only using ALSA and the SPI driver [1], so the pinout and pin settings are somewhat okay, but the Bela real-time code on ARM and PRU is not running on BBAI yet. During this project I will:

  1. Create a device tree overlay using Cape Compatibility layer for the BB-BONE-AUDI which worked but with a few frequency issues.
  2. create a device tree overlay for the BELA Cape to work on the BBAI using the Cape Compatibility layer
  3. update the Bela code to use the McASP, GPIO and McSPI on the AM5729 SoC of the BBAI
  4. adapt the Bela PRU and ARM code and workflow to use the PRU via remoteproc instead of uio_pruss
  5. install a Xenomai patched kernel ✓ and run the full Bela stack.

The project will involve dealing with pinmuxing, uboot, PRU assembly, C++ for Linux user space applications, the Technical Reference Manual for the Sitara family of SoCs. (ref. here)

Syntax Analysis

The places within the Bela core code that will require intervention are:

  1. in the Makefile, update the workflow to build the PRU code for remoteproc
  2. in core/PRU.cpp, use remoteproc instead of the libprussdrv API
  3. in pru/pru_rtaudio.p, the hardcoded McASP constants should be replaced with board-dependent ones.

All these changes will be made so that the same code base can run on all supported boards (e.g.: BBAI, BBB) with build- or runtime- checks. To explain in short how I aim to establish a common code base for supporting all boards, I will start off by enumerating different boards once we detect which board it is. Then in the libraries, the constants (like pin numbers and their symlinks) will be set accordingly and the rest of the code base will become much easier to use without the need to hard code anything.

PRU
  1. The current Bela core code uses pasm to build the PRU assembly pru/pru_rtaudio.p and uses libprussdrv, which binds to the uio_pruss kernel driver to load the firmware to the PRU and handle access to the PRU RAM. Both pasm and libprussdrv are now deprecated, replaced by the clpru toolchain and remoteproc driver respectively.
  2. The PRU firmware contains hardcoded values for the address of the McASP, McSPI and GPIO peripherals. These addresses will change for the BBAI, so these constants need to become conditional. ✓(yet to be tested, but base address for the McASP of the bbai has been updated)
  3. As the Bela PRU firmware is written in assembly for pasm, instead of rewriting or updating it in such a way that it will stop working on the current Bela images, we will use the workflow detailed below.
Workflow for building the existing pasm PRU code with clpru

The workflow below works as a proof of concept, but needs further testing.

  1. Build the .p file as is with pasm with -V2 -b. This generates a .bin file that contains the assembled program
  2. I will be using the disassembler Giulio Moro put together hacking the one that was inside prudebug. (Find it here(built with gcc da.c -o prudis) . (Note: A disassembler is a computer program that translates machine language into assembly language)
  3. Process the bin through the disassembelr and make it ready to be included inside an __asm__ directive (i.e.: add quotes and prepend a space at the beginning of each line):
   prudis file.bin | sed 's/^\(.*\)$/" \1\\n"/' > included_assembly.h

4. have the following in main.c:

void main()
{
     __asm__ __volatile__
     (
#include "included_assembly.h"
     );
}

5. build main.c with the regular clpru toolchain

6. I will also need to change these MCASP addresses in pru/pru_rtaudio.p :

#define MCASP0_BASE 0x48038000
#define MCASP1_BASE 0x4803C000
#define MCASP0_DATAPORT 0x46000000
#define MCASP1_DATAPORT 0x46400000

Additionally, I will need to compare the McASP and McSPI sections of the AM5729 and AM3358 manuals to verify that all the registers of the McASP and McSPI peripherals kept the same meaning and offsets between the two chips, or adjust the code accordingly.

PINMUXING

I am yet to figure out pinmuxing to make sure the relevant pins are set to the correct function (they are NOT by default). I will see here https://elinux.org/BeagleBoard/GSoC/2020_Projects/Cape_Compatibility and/or ask on the BB slack about for the best way to go about it. I have so far created a basic AUDIO overlay (named BBAI-AUDI-02-00A0.dts) that has been merged into the beagleboard / BeagleBoard-DeviceTrees repository.

PRU->ARM INTERRUPTS

Bela can work with a PRU->ARM interrupt, which is default these days, but requires an rtdm driver, which is another layer of complications. As an intermediate step to avoid further complications, I will try to run it without the PRU->ARM interrupt by adding BELA_USE_DEFINE=BELA_USE_POLL to my make command line. After building the Xenomai kernel, I will build the rtdm_ kernel and revert to using PRU->ARM interrupts.

PRU transitioning from libprussdrv to rproc

The Remote Processor Framework (rproc) allows different platforms/architectures to control (power on, load firmware, power off) those remote processors while abstracting the hardware differences, so the entire driver doesn't need to be duplicated. I will need to change the initialization code in PRU.cpp that is currently relying on lib prussdrv and move to rproc . Not sure if rproc provides some functionalities to access the PRU's RAM the way prussdrv_map_prumem() used to, but that essentially gives access to a previously mmap'ed area of memory , so it should be easy to replace, as long as I am able to find the correct addresses in the TRM. On the latest Bela code there's a Mmap class which can make this somehow simpler ref. here.

XENOMAI kernel

Xenomai kernel (v4.19.94-ti-xenomai-r64) has been built and tested for the BBAI. I have installed the xenomai kernel and libraries through the default procedure to update kernel. I have also managed to successfully build the entire Bela core code on the BBAI so far.

Hardware required

The hardware listed below will be necessary for testing if my code implementation works correctly on the hardware.

  1. BeagleBone AI.
  2. Bela cape: The original Bela board.

Timeline

Mar 29 Applications open Students register with GSoC, work on proposal with mentors.
Apr 13 Proposal complete Submitted to https://summerofcode.withgoogle.com
May 17 Proposal accepted or rejected
  1. Proposal Accepted
  2. Community Bonding Period and discussion on the project and resources available.
  3. Learn about embedded linux structure ✓
  4. Setting up beaglebone-ai i.e flashing up to date Linux image and connect to local area network (LAN) via either Ethernet or WiFi and try to run example codes from this repository to test basic stuff is working.
Jun 07 Pre-work complete, Coding officially begins!
  1. All the boards and BELA Cape will be available to me at this period of time and I should have set up my BeagleBone Black and Ai boards i.e flashing up to date Linux image.
  2. Initial checks for hardware like audio ports and other peripheral devices will be completed.
  3. A detailed spreadsheet will be created if not already available for cape pin mapping and referencing for further use during BeagleBone AI software development ease. (one exists here)

(some points above are ref. from here )

Jun 17 Milestone #1,
  1. Introductory YouTube video ✓
  2. Setting pinmux values appropriately and fix the dtb to get the correct clock on the McASP MCLK pin. ✓
  3. Writing the BBAI-AUDI-02-00A0.dts overlay to port the old BB-BONE-AUDI overlay using the CCL to work on the AI. ✓
  4. verify that it works running with ALSA. ✓
June 24 Milestone #2
  1. Writing the overlay for Bela Cape.
  2. Study about the remote processor framework and rpmesg, and how I can integrate it into the PRU.cpp code
  3. Test and debug the workflow for building the existing pasm PRU code with clpru, (or possibly reconsider/change it)
June 30 Milestone #3
  1. Try to implement the PRU code using rproc.
  2. Try to test and debug running Bela's PRU code on BBAI.
  3. make READMEs for undocumented but pre-existing codes (for eg: this) .
July 12 18:00 UTC Milestone #4
  1. Try to run Bela examples with Xenomai, but without rtdm(Real-time data monitoring) ie. generate (PRU->ARM interrupts)
  2. Start documenting everything done so far.
  3. Make demo videos for proving whatever that has been done so far works on actual-hardware.

Mentors and students can begin submitting Phase 1 evaluations

July 16 18:00 UTC Phase 1 Evaluation deadline
July 23 Milestone #5
  1. run Bela with Xenomai, with (PRU->ARM interrupts)
  2. Try to run the stock tutorials
July 30 Milestone #6
  1. Try to update the *.p files to suit the new gcc-pru supported syntax.
  2. Validate existing work with some functional tests and examples.
  3. Test and debug if any errors occur while running the stock tutorials and functional tests.
Aug 06 Milestone #7
  1. Create detailed Documentation about the above implementation.
  2. Provide guides on how to provide support for upcoming TI processors into the existing code base.
August 10 Milestone #8
  1. Writing the overall project summary and outcomes.
  2. Start making the Completion YouTube video.
August 16 - 26 18:00 UTC Final week: Students submit their final work product and their final mentor evaluation
August 23 - 30 18:00 UTC Mentors submit final student evaluations

Experience and approach

  1. I have used C++, C and Python programming languages over the past 3 years in a variety of projects involving embedded systems using the ESP32, Arduino UNO, ESP8266 and am also well-versed with freeRTOS.
  2. I have an aptitude for writing good reports and blogs, and have written a small blog on how to use a debugger.
  3. I recently did a project using ESP32, in which I used the DHT11 sensor to display humidity and temperature on a local HTML server . Other than that I have worked on developing hardware and making documentation for a 3 DOF arm based on an ESP32 custom board.
  4. I also interned at an embedded device startup where I
    1. Interfaced ADS1115 ADC with the ESP32 and used it to read battery voltage.
    2. Used UART for ESP32 and SIMCOM SIM 7600IE communication to gain LTE support.
    3. Published local sensor data to the cloud via LTE.
  5. I actively contribute to open source (most recently, I contributed to the ADS1115 library for ESP32 on the unclerus repo and can be seen here).
  6. Currently I am working on designing a Development board for the Raspberry Pico (RP2040) using KiCAD.
  7. I also do a lot of mini projects throughout the year, you can find my several more interesting projects at my github page

Contingency

I believe that if I get stuck on my project and my mentor isn’t around, I will use the resources that are available to me. Some of those information portals are listed below.

  1. Derek Molloy's beagle bone guide provides all the information needed for getting up and running with my beagle.
  2. remoteproc
  3. BBAI vs BBB pin headers Google Sheet
  4. beaglebone pru gpio example
  5. official BELA website
  6. Ask on the BeagleBoard.org forum

Benefit

If successfully completed, this project will add support for the Bela cape + Xenomai + PRU on the BeagleBone AI, and also the code will be easier to port to other Texas Instruments systems-on-chip.

By going through the steps needed to have the Bela environment running on BBAI, we will go through refactoring and rationalisation, using mainline drivers and APIs where possible. This will make Bela easier to maintain and to port to new platforms, benefiting the project's longevity and allowing it to expand its user base.
-Giulio Moro

Misc

Completed all the requirements listed on the ideas page.
The code for the cross-compilation task can be found here submitted through pull request #149.