ECE497 Project Remote Web Cam Viewer

From eLinux.org
Revision as of 14:59, 14 November 2012 by Yoder (talk | contribs)
Jump to: navigation, search

thumb‎ Embedded Linux Class by Mark A. Yoder


Team members: Alexander W. Drane, John Lobdell

Grading Template

I'm using the following template to grade. Each slot is 10 points. 0 = Missing, 5=OK, 10=Wow!

00 Executive Summary
00 Installation Instructions 
00 User Instructions
00 Highlights
00 Theory of Operation
00 Work Breakdown
00 Future Work
00 Conclusions
00 Demo
00 Late
Comments: I'm looking forward to seeing this.

Score:  10/100

(Inline Comment)

Executive Summary

The goal of this project is to create a simple webcam based security system that allows a user to remotely view the devices web cam by navigating to a webpage. Then using buttons the user can change what camera they are viewing.

Currently we have the ability to select a camera and view the stream in a remote webpage. This view uses the browser's VLC Plugin. We have tested this with 4 webcams on the xM board.

Operating with four webcams hooked to the board causes many failures of the system to stream back the video information. We determined that this was due to the physical cameras and the overhead of the devices by testing the system with only two we cams where the issues experienced regularly with the four camera setup greatly reduced in frequency.

Investigation of how to accomplish this task took up the bulk of our time and effort largely due to the infancy of HTML5 Related standards(sockets.io and MediaStream). In the end we decided to utilize the vlc plugin when the HTML5 proved unfeasible at its current level of development. Implementing the four cameras did seem to cause some device issues that caused the webpage to not sync with the incoming feed, but this appears to be hardware related only and not from the implementation (a further explanation is later on).

Installation Instructions

Extra Hardware Required:

  • Webcam 1+ (or Playstation Eye's) Max 4 without other supplementary hardware. Best performing build currently is 2 web cams on the board. 4 makes the XM Slow down too much.
  • Connected to the internet.

First Download the git Repository, it is located here:WebcamFeed

xm$ mkdir WebcamFeed
xm$ git clone https://github.com/jtlobdell/bbWebcamFeed.git WebcamFeed
xm$ cd WebcamFeed

Install Node.js

xm$ opkg update
xm$ opkg install nodejs

Verify gstreamer is available and up to date

xm$ opkg update
xm$ opkg install gstreamer

One last thing must be done on the user computer. VLC Media Player Plugin must be available in your browser of choice. This can be done by downloading the software here: [[1]] and making sure during installation to install the browser plugin. Many modern browsers will guide you through this if you run into a page that requires a missing plugin.

Now for reference type the following to get the xM's IP address. It is needed quite soon

xm$ ifconfig

User Instructions

Move to the source directory

xm$ cd ./WebcamFeeds/

The current code does not automatically direct the VLC plugin to play from the correct IP address; it relies instead on passing a hard-coded IP address to the client in the WebcamFeed.html file. Modify the following code in the file to use your xM's IP address:

<embed
         type="application/x-vlc-plugin"
         name="videoplayer"
         autoplay="yes" loop="no" hidden="no"
         width="320" height="240"
         target="tcp://[YOUR_BEAGLE_IP]:8080" />

Once the installation is done, the program is ready. Run it using:

xm$ node WebcamFeed.js

Now navigate on the user computer using a web browser to your computers IP Address. Once there click on the link to the video feed page. The program will start and video should play.

NOTE: When changing video feeds the XM will randomly not load and play the feed. This we believe is due to the physical limitations of the hardware. We tested this by attaching 4 webcams to the board and found that the failure rate went up drastically. When the extra 2 webcams were removed the failure rate returned to where it was before. Since the program only ever calls one feed at a time(it changes which device it targets) this has to be a memory or hardware related issue and not due to the software.

At the top of the screen is a text box which can be entered the desired video device. Each camera is added sequentially starting at 0. Below the Set Cam button are four buttons for convenience. These function identically as entering the number's 0 though 3 into the text box and clicking Set Cam for each number.

Highlights

A single video is captured from a web cam and streamed to a web page. The video capture device can be selected from the same page.

This program implements node.js and GStreamer working together to accomplish the video output and control over which video device to be viewed.

The software allows for as many video capture devices (webcams) as can be connected to the device. In the case of the Beagle Board xM, up to four cameras can be used. We only access one webcam at any given time and so the only increasing overhead on the system is that which is associated with the individual USB webcam's.

Here is a YouTube video demo of the four camera operation. A picture of the test setup is shown below:

BbWebFeed demo setup.jpg

This video starts out very slow and illustrates some of the hardware related issues that were encountered. While the most effective implementation currently is the two camera version we wished to show that four camera feeds could be accessed on the xM.

Theory of Operation

The project uses GStreamer to capture video from a webcam, convert it to video/ogg format, and stream it through Transmission Control Protocol (TCP). A diagram of the GStreamer pipeline is shown below.

Web Cam Feed - GStreamer Pipeline

The client system connects to a node.js server running on the xM. The server embeds the video stream into the web page and also contains controls for selecting a webcam for the stream.

When the server begins running the node.js script, it launches the GStreamer pipeline discussed above. When the client connects, they receive a web page containing controls for the video source and the embedded video that gets played using the VLC web plugin.

When the client uses the controls to change the video source, the server kills the current GStreamer process and launches a new one using the new video source. It then makes the client's page refresh to make the embedded video player use the new stream.

A block diagram of the system is shown below.

Web Cam Feed Block Diagram

Work Breakdown

  • Research- Research took up the bulk of our time and effort on this project due to the obscure nature of video streaming on the web using HTML5 and related tools. Most sources online advise you to use a plugin or third party software and treat the idea of using Javascript of HTML5 for this task as a taboo subject.
    • GStreamer Implementation- John Lobdell investigated using GStreamer to accomplish this project. This implementation in the end proved effective and the easiest to implement.
    • HTML5/JavaScript Implementation - Alex Drane investigated using a pure HTML5 implementation in order to not require anything to be done by the user to view the video streams. From initial research this appeared to be something that could be done quite easily. After considerable research wading through the draft standards that have been developed I found very little sample code or examples that pertained to remote video viewing. The easiest implementation I found was using it to create a 1-to-1 chat application. This is not suitable for our needs.
  • Program Creation- John Lobdell used his research to get GStreamer to stream a video or camera input onto port 8080 that could then be read by the VLC Media Player on a remote computer. The further integration of this into a program was done with the introduction of the VLC-plugin in an HTML document. The final program was jointly compiled by both John and Alex.

The program creation went quite fast after a working method was identified by John. Then it was a matter of putting the pieces together. The program consists of a node.js client server using the example provided in the examples/node.js folder, which provides the HTML server and controls for which video to stream. We embedded the VLC viewer into the page and pointed it to the location where the video would be streamed by an external function call in the .js file. This allowed for a simplified approach to running the needed utilities without having to port code into the Javascript language.

Future Work

Adding video capture would add true security camera capabilities to this system. In addition adding motion detection into the video selection stream would be interesting. Though this might be too much for the xM's hardware.

Having a webpage that displayed all of the video feeds at once would also be an interesting feature but this could over tax the XM's hardware or internet connection.

Figuring out how to implement this project with only HTML5 and make it so that no plugin is required by the end user was Alex's original plan but the maturity of the knowledge base for HTML5 is not adequate to accomplish this task at this time. But in the future when many of the draft standards are finished this project could be implemented without the VLC plugin.

Solving the interesting problem with the random stream failure. This issue arises when switching between streams. Occasionally the video receiver does not receive the stream even though the stream is playing across the network. From experimentation we found that as cameras were added the performance of the entire program greatly decreased. Since we only access one camera at a time the only explanation we could find was that the OS was devoting resources to the devices when they were connected thus creating a much greater overhead as cameras were added. When the cameras were disconnected the performance returned to their previous levels. With four cameras operating a noticeable increase in stream failures was observed. With only two cameras operating about 1/5 of attempts would be unsuccessful. With four cameras the failure rate was significantly higher (as much as 1/3 of attempts would fail). One possible explanation of this is the overhead causing a slow down in setup time for the video out thread compared to the load up time of the browser. While this is set up to go first if the system hanged due to lack of resources this could cause the video stream to start after the HTML file stopped looking for it. Occasionally the camera just won't turn on. We have found that signaling it again usually fixes the problem without much effort. After trying to figure out what was causing the problem we could not definitively pin down the root cause of the problem and we found no simple or effective solution for it but would like to see it fixed.

The server should automatically detect its external IP and then use it to direct the client's VLC plugin to the video stream.

The video should stream through UDP rather than TCP. UDP would have better performance, and data loss is not a major concern for low-resolution video.

Audio can be added to the video stream.

Conclusions

This project was quite surprising by how difficult it is to do using the new HTML5 standards. Through the research we learned that there were currently no simple implementations of what we wished to do with this project. We had originally assumed that something as common as video streaming would have a direct implementation in the new standard but in fact it is far more restricted then we expected. By splitting each of our focuses during the research stage we were able to research the use of the bleeding edge of development and the use of the common implementation and then whoever found an implementation that worked first would allow the project to progress. In addition our parallel research gave us greater insight into the potential ways of accomplishing this task then if we had both focused on the HTML5 implementation and did not allow the project to reach a dead end as it would have if we had both focused on the HTML5 method.

In the end we did settle on using the vlc-plugin to capture the outgoing stream that John setup using gstreamer. This worked exactly as we wished it to without all of the syntax fighting that took place when trying to port the HTML5 sample code into code we had the codex's for(i.e. websocket to socket.io syntax Note: websocket wanted a Microsoft related XML file which after searching for could not be found when we tried to just use websocket).

Many of the ideas listed in Future Work would be amazing addon's to the project that would greatly expand and enhance this projects implementation.




thumb‎ Embedded Linux Class by Mark A. Yoder