BeagleBoard/GSoC/SynchronousDataCollectionPRU

=BeagleBone synchronous data collection=

Easily set up a serial terminal server on a BeagleBone, configure and manage attached serial devices through an UI and read past I/O logs.

Student: Nishant Pani Mentors: Hunyue Yau Code: Wiki: http://elinux.org/BeagleBoard/GSoC/SynchronousDataCollectionPRU GSoC: [GSoC entry]

=Status= This project is currently just a proposal. I am collecting Ideas and thoughts on my blog and preparing for the project on GitHub

=Proposal= I made a pull request to the gsoc-application repo

About me
IRC: NP95 Github:https://github.com/NP95 School: Birla Institute of Technology and Science,Pilani Hyderabad Campus (BPHC) Country: India Primary language: English,Hindi Typical work hours: 9 AM - 5 PM(IST)

About my project
Project name: BeagleBone-based Serial Terminal Server

Synopsis
There are many commercial solutions for serial terminal servers such as those provided by Cisco or Moxa. Their products allow easily accessing serial devices connected to a server from remote and include features like buffering of past I/O, remote configuration of serial devices through an UI, interconnecting serial devices over a network or setting up rules for automation. However, these products are often expensive and exceed the budget of makers, small developer studios or hobbyists. This project aims to provide an open source solution which includes most of these features, is easy to set up and runs on the BB.

--verbose
Let's step back a bit: What is this all about and what is already possible? Let's assume that you have a laser cutter and that your laser cutter provides a UART interface for logging its head position and accepting shutdown-messages. As you want to see what's really going on you connect a BB to the laser cutter. Now you could log into the BB via SSH (from your PC) and read out all the messages from the terminal you used to open that SSH connection. But configuring the serial port from the command line and reading all the logs continuously sounds like a time consuming task. This is where my project comes to the rescue: Instead of doing all of this manually, get the Debian image which this project will be packaged into. After you hooked up everything, you just open the provided website in your browser and use the UI to configure the serial ports you need and to enable logging. Now all I/O will be written to a log file and logging will go on even if you disconnect your PC from the BB. After some time you decide to send a shutdown-message through the web interface to the laser cutter. After the shutdown, all the I/O logs are still available on the BB and you can read them whenever you want.

Project Structure
The project can be divided into three main components which expose interfaces to other components. These components and interfaces describe the deliverables and will be provided to users through an image:
 * C Library for accessing serial devices via UART: Direct access to the serial devices will be done inside a C-library like component. It will enable the use of all four serial ports and handle communication with connected devices, keep track of all the configurations for each serial port and map the logging files to the appropriate serial port. The project was supposed to use already existing software like Screen or Minicom, but they are more appropriate for interactive tasks. Expect can be useful for scripted interaction, but the actual device access will happen through device nodes and might utilize the termios API. I'm aware of the requirement to track all I/O in a way that no inputs are missed. I examined UART and found that even with high baud rates and continuously spamming on the serial port we have 1041,6µs until the 16Byte input FIFO fills up. That should be enough to write a file to DRAM so that this library can handle the file writing in time and thus prevents FIFO overflows. However, I talked to m_w about modifying the serial device driver. According to him DMA is enabled by default and could be configured to write to a file. I will test these approaches before GSoC starts with the help of my Arduino as serial device. The library is accessed through an interface written in C. Stretch Goals: Also include SPI, I2C etc.
 * Library Interface: This interface allows other applications, e.g. the front end, to access the library's features. Stretch Goal: Due to the project's architecture a CLI can be implemented on top of the library interface with less effort
 * NodeJS Bindings: As the terminal server component will be implemented with NodeJS I also need bindings to talk to my library. This component behaves like a wrapper.
 * Terminal Server: This provides a runtime for the web interface. The first goal for this component is to just delegate calls from the web interface to the binding. Stretch Goals: Autostart the web server on boot. For future use this component can be turned into a web server which exposes the HTML interface to the whole network. Note that functionality can be extended here without needing to modify the C library. This comes in handy when features like automation, action buttons, interconnection of serial devices, scripted interaction (etc) are added.
 * HTML Interface: Provides the web interface and other applications with an HTML interface. The scope of this component depends on the features implemented in the terminal server component. To allow access to new features this interface will have to be extended accordingly.
 * Web Interface: Any functionality added to the terminal server component can be included into the UI and will rely on the HTML interface. Implementation will happen with ReactJS unless I find an appropriate framework/template to start from. The UI provides the user with...
 * Configuration of UART communication (parity bit, baud rate)
 * Enable/Disable logging of I/O to files
 * Read logs and current messages
 * Send messages to a serial device
 * Handle multiple serial devices comfortably

Timeline
I will implement some samples and experiments useful for this project before and during the community bonding period. During that time I will also look out for existing software useful for this project which I have not seen in my research yet.

TODO recheck as soon as scope is approved...

2017-06-06: C library configures all four serial ports 2017-06-13: C library has read and write calls, port configurations can be saved to and restored from a file 2017-06-20: I/O is logged to files; possibly introducing multi threading to avoid blocking the caller (depending on tests) 2017-06-27: library interface is added; C library is completed 2017-07-04: Arduino scripts for testing as well as test cases are improved; left days serve as reserve for completing the C part 2017-07-11: Node Binding Stub is created; NodeJS Server runs on BB and has all methods required for delegating calls to binding 2017-07-18: Binding calls c interface; HTML interface exposes serial configuration, setting log files, reading log files 2017-07-25: HTML interface provides sending messages and reading current messages; web interface started 2017-08-01: Basic web interface with required UI components 2017-08-08: Wiring UI to HTML interface; styling UI 2017-08-15: Packaging everything into an image; left days serve as reserve Last week will serve as reserve or, otherwise, will be used for improving the existing code base in terms of code-style, docs and additional tests or implementing stretch goals

Experience and approach
I developed strong skills in Java and after creating some PC pograms and Android apps in Java I got involved in web development. I wrote a web back end for an Android app in PHP and proceeded with writing small JavaScript games. You can find some of my projects on my website or GitHub profile. In my latest project I created an app in a team of five people, the code can be seen in this repo. I also experimented with C# and Unity5. However, after finding out about Arduino I decided to give embedded systems a closer look and created a DIY smart watch some time ago which offloads processing intensive tasks to an Android phone. My university requires me to know basics in Assembler and some robust C skills. At the moment, I'm developing my C skills even further in order to prepare for GSoC (and my C exam) and I'm confident that I will improve my C coding until GSoC starts. Im learning how drivers, serial communication and web servers work on the BB. Implementing experiments and samples for how to set up a web server with NodeJS on the BB, access serial devices and checking the DMA approach (see above) will also help me leveraging my JS and C skills before project start.

Contingency
BeagleBone is an open source project, but even more important, its an open source community. If I get stuck on a problem and my mentor is not available I will ask the community for help and try to find a solution that way. Consulting the Docs is also one option for me or, if there still is no solution or there is something only my mentor knows, I can also proceed with different tasks until he is available again.

Benefit
According to nerdboy, this project will make "a nice remote management tool" besides "enhancing the project sphere with serial control of anything with serial interface to motors/sensors/pumps". maciejjo also thinks that "anyone who has >1 target with serial debug can benefit from turning BBB into terminal server, especially if presented with good configuration interface. It can help setting up small board farm without need for expensive device, so it is well suited for home use among embedded enthusiasts".

After implementing above features, end user will be able to set up a BeagleBone-based serial terminal server faster, to configure serial ports without needing to use the command line and to manage serial devices remotely. This will lead to more people regarding BeagleBoards as a true alternative to costly commercial products. Developers can build applications which access serial devices straight through the HTML interface, which will make including serial device interaction easier. As the project evolves (stretch goals, future developments) it can move more towards a serial control application, equipped with useful API's which, at some point, might allow accessing serial devices through the Internet. This project will lay the foundation of making such a serial control application along with a feature rich API possible.

Suggestions
I intend to dedicate most of my time to this project. This means that I will not attend university lectures or work elsewhere during the coding period except from one course which I will have to attend. That will require about 7h a week; I will make sure to provide up to 40h/week for GSoC and use the weekends for preparing that course.

I aim to specialize on embedded systems during my studies and regard GSoC as a great opportunity to dive into this topic; I enjoy working on projects I'm interested in and, in these cases, am comfortable with steep learning curves