Student: Jyothirmayee Donthineni
Mentors: Jason Kridner
GSoC: GSoC entry
This project is currently just a proposal.
This project will be implemented in 3 phases.
- Install the Flite program for text to speech generation.
- Install Pocketsphinx libraries for speech recognition.
- Build Level 1 puzzle: Implement the basic 'spell the world' puzzle.
- Build Level 2 puzzle: Fill the missing letters in a word within specified time (This will be implemented as a dynamic game )
- Improving the Chatbot functionality in terms of giving feedback after every puzzle.The puzzles are invoked by giving commands through voice.It will :also be able to give suggestions to improve user's performance.It will be developed as a teaching aid.
- Implement the complete model by adding a keyboard,audio in/out and an OLED display.
- Bug fixing and Documentation of the project.
|Refine the weekly plan with suggestions from the mentor and community.|
Familiarise myself with the processor that is to be used for the project.
|1||Install flite,start developing an efficient voice recognition system|
|2||Implementing the voice recognition system on pocketbeagle|
|3||Implement basic "spell the word" game in python|
|4||Implement level 2 game and documenting the finished code for Phase-1 evaluation|
|5||Improve the code based on feedback after phase-1 evaluation.|
Implement level 2 game.
|6||Develop level 2 game|
|7||Build Chatbot with feedback feature|
|8||Develop the Chatbot into a Teaching aid and document the finished work for phase-2 evaluation|
|9||Improve the Chatbot features based on feedback after phase-2 evaluation|
|10||Make a demo adding a keyboard,OLED display and an audio IN/OUT|
|11||BUG fixing and documentation|
|12||BUG fixing and documentation for final evaluation(phase-3)|
School: National Institute of Technology,Surathkal
Primary language (We have mentors who speak multiple languages): English
Typical work hours (We have mentors in various time zones): 2:30 AM to 11.30 AM UTC
Previous GSoC participation: No previous experience but excited to join the open source community.
About your project
Project name: Modern "Speak&Spell" using PocketBeagle
Task Completion: A "Hello World" application has been created,cross-compiled using GCC-ARM tool chain and executed using QEMU.A Pull request has been generated for the same.Please find the link here.
This project motto is to implement an updated "Speak&Spell"using PocketBeagle for modern day preschoolers with improved games/Puzzles and better hardware interfaces like I2C gpio expander.Previously used VF display can be replaced with an OLED touchscreen display and an improved keypad can
Current Proposal for the project aims at building an Open Source reproducible Linux Application that can be implemented by downloading the code anywhere for its real time implementation so that it can be promoted for usage in commercial purposes. Since the current Speak&Spell's basic features are not sufficent for present day preschoolers,the updated puzzles will cater their present needs in improving their verbal skills.Puzzles will be implemented in python in 2 levels.For text to voice generation,a program called 'Flite' will be used.Flite is specially built for embedded systems and has an added advantage of changing to multiple accents and languages.For adding the speech recognition feature (instead of typing the words on keyboard),Pocketsphinx libraries will be used which are light and compatible for Pocketbeagle.
Experience and approach
I have done projects on Raspberry Pi before and therefore will be comfortable enough to implement this project on a Pocketbeagle.I am quite new to Speech recognition,so I have been doing required background work to implement these functionalities on Pocketbeagle and therefore can assure you that I will be able to complete the Phase-1 in planned time.I had only done some basic projects on python before and therefore planning to spend most of Phase-2 to build the games.Most of the time in phase-3 will be spent on documenting the whole project as it is very essential for it to be reproducible later .
What will you do if you get stuck on your project and your mentor isn’t around?
I will ensure that probability of this happening is less by planning things in advance.I will also check with the community whether they have a backup mentor provision.I will communicate the coding issues with the community or students working on similar projects to see if someone can help. Else I will contact the organisation administrator to talk about the issue and make sure the work is not delayed in the mean time by documenting the finished work.
If successfully completed, what will its impact be on the BeagleBoard.org community? Include quotes from BeagleBoard.org community members who can be found on http://beagleboard.org/discuss and http://bbb.io/gsocchat.
"Speak&Spell" was more than just a popular kids toy,it should be better described as a blueprint for the devices we use today.Until its invention, real time speech synthesis was defined to be impossible.Implementing and Updating its functionality for the present day kids is a perfect way to celebrate its importance in the development of Signal processing technology.
Is there anything else we should have asked you?
I am currently pursuing my Bachelor's degree in Electronics and Communcation Engineering and I am good in C and Python programming.I have no other commitments this summer and can dedicate my entire time for the project.