BeagleBoard/GSoC/BeagleBone-based serial terminal server

=BeagleBone-based Serial Terminal Server Proposal=

Easily set up a serial terminal server on a BeagleBone, configure and manage attached serial devices through a UI and read past I/O logs. - This project was not accepted for GSoC 2017

Student: Giorgio Gross Mentors: Michael Welling Code: https://github.com/ordsen/BeagleBone-based-Serial-Terminal-Server Wiki: http://elinux.org/BeagleBoard/GSoC/BeagleBone-based_serial_terminal_server GSoC: [GSoC entry]

=Status= This project is currently just a proposal. I am collecting Ideas and thoughts on my blog and preparing for the project on GitHub

=Proposal= I made a pull request to the gsoc-application repo

About me
IRC: ordsen Github: https://github.com/ordsen School: Karlsruhe Institute of Technology (KIT) Country: Germany Primary language: German Typical work hours: 8AM-4PM UTC (I’ll also be on IRC around 8PM UTC)

About my project
Project name: BeagleBone-based Serial Terminal Server

Synopsis
There are many commercial solutions for serial terminal servers such as those provided by Cisco or Moxa. Their products allow easily accessing serial devices connected to a server from remote and include features like buffering of past I/O, remote configuration of serial devices through a UI, interconnecting serial devices over a network or setting up rules for automation. However, these products are often expensive and exceed the budget of makers, small developer studios or hobbyists. This project aims to provide an open source solution which includes most of these features, is easy to set up and runs on the BeagleBone.

--verbose
Let's step back a bit: What is this all about and what is already possible? Let's assume that you have a laser cutter and that your laser cutter provides a UART interface for logging its head position and accepting shutdown-messages. As you want to see what's really going on you connect a BeagleBone to the laser cutter. Now you could log into the BeagleBone via SSH (from your PC) and read out all the messages from the terminal you used to open that SSH connection. But configuring the serial port from the command line and reading all the logs continuously sounds like a time consuming task. This is where my project comes to the rescue: Instead of doing all of this manually, get the Debian image which this project will be packaged into. After you hooked up everything, you just open the provided website in your browser and use the UI to configure the serial ports you need and to enable logging. Now all I/O will be written to a log file and logging will go on even if you disconnect your PC from the BeagleBone. After some time you decide to send a shutdown-message through the web interface to the laser cutter. After the shutdown, all the I/O logs are still available on the BeagleBone and you can read them whenever you want.

Project Structure
The project can be divided into three main components which expose interfaces to other components. These components and interfaces describe the deliverables and will be provided to users through an image:
 * C Library for accessing serial devices via UART: Direct access to the serial devices will be done inside a C-library like component. It will enable the use of all four serial ports and handle communication with connected devices, keep track of all the configurations for each serial port and map the logging files to the appropriate serial port. The project was supposed to use already existing software like Screen or Minicom, but they are more appropriate for interactive tasks. Expect can be useful for scripted interaction, but the actual device access will happen through device nodes and will utilize the termios API. I'm aware of the requirement to track all I/O in a way that no inputs are missed. I examined UART and found that even with high baud rates and continuously spamming on the serial port we have 1041,6µs until the 16Byte input FIFO fills up. That should be enough to write a file to DRAM so that this library can handle the file writing in time and thus prevents FIFO overflows. I made some experiments and hooked up my Arduino to the Serial port of the BeagleBone to test the performance of my attempt. It turned out that both receiving inputs and writing them to a file synchronously do not lead to any issues, so I'm confident that I/O buffering can be implemented in user space. However, it's always good to have a plan B: If I encounter the problem that messages are dropped I can try to make the I/O communication and buffering asynchronous. I also talked to m_w about modifying the serial device driver. According to him DMA is enabled by default and could be configured to write to a file. Stretch Goals: Also include SPI, I2C etc.
 * Library Interface: The library is accessed through an interface written in C. This interface allows other applications, e.g. the front end, to access the library's features. Stretch Goal: Due to the project's architecture a CLI can be implemented on top of the library interface with less effort
 * NodeJS Bindings: As the terminal server component will be implemented with NodeJS I also need bindings to talk to my library. This component behaves like a wrapper.
 * Terminal Server: This provides a runtime for the web interface. The first goal for this component is to just delegate calls from the web interface to the binding. I managed to run a basic NodeJS web server on the BeagleBone as m_w encouraged me to try out the technologies I aim to use on the BeagleBone. Stretch Goals: Autostart the web server on boot. For future use this component can be turned into a web server which exposes the HTML interface to the whole network. Note that functionality can be extended here without needing to modify the C library. This comes in handy when features like automation, action buttons, interconnection of serial devices, scripted interaction, etc are added.
 * HTML Interface: Provides the web interface and other applications with an HTML interface. The scope of this component depends on the features implemented in the terminal server component. To allow access to new features this interface will have to be extended accordingly.
 * Web Interface: Any functionality added to the terminal server component can be included into the UI and will rely on the HTML interface. Implementation will happen with ReactJS unless I find an appropriate framework/template to start from. The UI provides the user with...
 * Configuration of UART communication (parity bit, baud rate)
 * Enable/Disable logging of I/O to files
 * Read logs and current messages
 * Send messages to a serial device
 * Handle multiple serial devices comfortably

In addition I will add documentation in form of comments to the code. All interfaces and their usage will also be described on GitHub so that others can use my code easier. C code will be tested with the help of my Arduino. JS code will be tested with unit tests where appropriate, but I will focus on integration tests here.

Timeline
I will implement some samples and experiments useful for this project before and during the community bonding period. This includes collecting more experience with ReactJS and exploring what is possible with NodeJS on the BeagleBone, but also getting a feel of how capable user space I/O communication is. During that time I will also look out for existing software useful for this project which I have not seen in my research yet.

2017-06-06: C library configures all four serial ports: This means selecting the baud rate and parity bits for each serial port. 2017-06-13: C library has working read and write calls; port configurations can be saved to and restored from a file 2017-06-20: I/O is logged to files; possibly introducing multi threading or asynchronous I/O and file writing to avoid blocking the caller and dropping messages (depending on tests) 2017-06-27: library interface is added; C library is completed: The C library can now be used from any other program and works as described in the deliverables. 2017-07-04: Arduino scripts for testing as well as test cases are improved; left days serve as reserve for completing the C part 2017-07-11: Node Binding Stub is created; NodeJS Server runs on BeagleBone and has all methods required for delegating calls to the binding 2017-07-18: Binding calls C interface; HTML interface exposes serial configuration, setting log files, reading log files 2017-07-25: HTML interface provides sending messages and reading current messages; web interface started: At this point the HTML interface supports all functionality implemented in the C-library and can be used by other programs 2017-08-01: Basic web interface with required UI components: This will be a rather functional UI which can be improved later, but presents all the basic functions to the user 2017-08-08: Wiring UI to HTML interface; styling UI: Reading the logs and reading, writing and configuring serial devices can be done using the UI 2017-08-15: Packaging everything into an image; left days serve as reserve Last week will serve as reserve or, otherwise, will be used for improving the existing code base in terms of code-style, docs and additional tests or implementing stretch goals

Experience and approach
I developed strong skills in Java and after creating some PC pograms and Android apps in Java I got involved in web development. I wrote a web back end for one of my Android apps in PHP and proceeded with writing small JavaScript games. You can find some of my projects on my website or on my GitHub profile. In my latest project I created an app in a team of five people, the code can be seen in this repo. I also experimented with C# and Unity5. However, after finding out about Arduino I decided to give embedded systems a closer look and created a DIY smart watch some time ago which offloads processing intensive tasks to an Android phone. My university requires me to know basics in Assembler and some robust C skills. At the moment, I'm developing my C skills even further in order to prepare for GSoC (and my C exam) and I'm confident that I will improve my C coding until GSoC starts. I'm learning how drivers, serial communication and web servers work on the BeagleBone and implementing experiments and samples for how to use these technologies will also help me leveraging my JS and C skills before project start.

Contingency
BeagleBone is an open source project, but even more important, its an open source community. If I get stuck on a problem and my mentor is not available I will ask the community for help and try to find a solution that way. Consulting the Docs or searching online is also one option which often leads to a solution for me. If I still cannot solve my problem or there is something only my mentor knows, I can also proceed with different tasks until he is available again.

Benefit
According to nerdboy, this project will make "a nice remote management tool" besides "enhancing the project sphere with serial control of anything with serial interface to motors/sensors/pumps". maciejjo also thinks that "anyone who has >1 target with serial debug can benefit from turning BBB into terminal server, especially if presented with good configuration interface. It can help setting up small board farm without need for expensive device, so it is well suited for home use among embedded enthusiasts".

After implementing above features, end user will be able to set up a BeagleBone-based serial terminal server faster, to configure serial ports without needing to use the command line and to manage serial devices remotely. This will lead to more people regarding BeagleBoards as a true alternative to costly commercial products. Developers can build applications which access serial devices straight through the HTML interface, which will make including serial device interaction easier. As the project evolves (stretch goals, future developments) it can move more towards a serial control application, equipped with useful API's which, at some point, might allow accessing serial devices through the Internet. Thus, this project will also lay the foundation of making such a serial control application along with a feature rich API possible.

Suggestions
I enjoy working on projects I'm interested in and, in these cases, am comfortable with steep learning curves. I intend to dedicate most of my time to GSoC. This means that I will not attend university lectures or work elsewhere during the coding period except from one course which I will have to attend. That will require about 7h/week within the first 45 days of the coding period; I will make sure to provide up to 40h/week for GSoC and use the weekends for preparing that course.

I think that open source is a unique opportunity for giving back to society. Although I have open sourced some of my projects, being part of an open source project with such a great community and impact will be a new experience for me. GSoC is a chance which allows me to collect these experiences while learning new things, improving my knowledge and developing and adapting my skills. I aim to specialize on embedded systems during my studies and regard GSoC also as an opportunity to dive into this topic. That is why I would like to be a part of it.