RPi-Cam-Web-Interface

RPi Cam Web Interface is a web interface for the Raspberry Pi Camera module that can be opened on any browser (smartphones included) and contains the following features:


 * View, stop and restart a live-preview with low latency and high framerate. Full sensor area available.
 * Control camera settings like brightness, contrast, ... live
 * Record full-hd videos and save them on the sd-card packed into mp4 container while the live-preview continues
 * Take single or multiple (timelapse) full-res pictures and save them on the sd-card (live-preview holds on for a short moment)
 * Preview, download and delete the saved videos and pictures, zip-download for multiple files
 * Trigger captures by motion detection
 * Trigger captures by many scheduling-possibilities
 * Circular buffer to capture the last actions afterwards
 * Control Pan-Tilt or Pi-Light
 * Shutdown/Reboot your Pi from the web interface
 * Show annotations (eg timestamp) on live-preview and taken images/videos
 * Supports selection from 2 cameras when used with a compute module

IMPORTANT NOTE: This is for the Raspberry Pi camera only. It does NOT support USB cameras. It's been programmed by silvanmelchior as a client for RaspiMJPEG in 2013. Since then, thanks to the help of many other programmers, it has become the best interface to control the RPi Cam over your browser.

Remember, anyone can create an account on here and add to this wiki.

= Installation Instructions =

Basic Installation
Warning: The installer will replace various files, so backup all your data.

Step 1: Install Raspbian on your RPi

Step 2: Attach camera to RPi and enable camera support (http://www.raspberrypi.org/camera)

Step 3: Update your RPi with the following commands:

Occasionally if camera core software updates have been done then a sudo rpi-update may be used to benefit from these before they become available as standard.

Step 4:

For Jessie Lite run sudo apt-get install git

Clone the code from github and enable and run the install script with the following commands:

5 separate scripts are provided to do separate installation and maintenance functions.

The scripts are
 * install.sh main installation as used in step 4 above
 * update.sh check for updates and then run main installation
 * start.sh starts the software
 * stop.sh stops the software
 * remove.sh removes the software


 * To run these scripts make sure you are in the RPi_Cam_Web_Interface folder then precede the script with a ./
 * E.g. To update an existing installation ./upgrade.sh
 * E.g. To start the camera software ./start.sh
 * E.g. To stop the camera software ./stop.sh

The main installation always does the same thing to simplify its logic. It gathers all user parameters first in one combined dialog and then always applies the parameters as it goes through the process. A q (quiet) parameter may be used to skip this and give an automatic install based on config.txt All parameters are always in the config.txt file, a default version is created if one doesn't exist and is then changed just once after the initial user dialog. The installation always tries to upgrade the main software components and then functionally goes through the configuration steps for each area like apache, motion start up.

After the setup finishes it offers to start the camera system. It will also start on a reboot if autostart was configured.

Step 5: Use it

Open up any browser on any computer in your network and enter the url to access the camera web site. This will be http://ipAddress:port/subfolder. If the port had been left at default 80 during install then this may be left out. Similarly the subfolder (default html) can be left out if that was cleared during the install. So for a port 80, no subfolder install the url becomes http://ipAddress

Original Installation method
The original installation used just 1 combined script. It was set up originally for wheezy and has had some changes to make it Jessie compatible but these are not complete.

Follow Steps 1 to 3 of the install method above then

The script by default will install in the normal web root (/var/www). It supports installing in a subfolder of /var/www. When run it will ask a question as to what folder to use. Either hit enter to use the default or enter just the subfolder name to be used (e.g. rcam will install in /var/www/rcam). You can also edit the script and put the subfolder name in directly.

After the setup finishes, you should be able to run the ./start.sh script. If there any problems then reboot the Pi. Now just open up any browser on any computer in your network and enter the IP of the RPi as URL.

When you want to update an existing install of the software to a later version first run the script with 'update' instead of 'install'. Then if differences are detected then you should run the script a second time using 'install'

Jessie: With the introduction of the Jessie version of the Raspberry OS the original installer is no longer working by default. This is primarily due to changes on the Apache side and also to the www-data user set up. Apache now has a default web file location of /var/www/html and the www-data has nologin set up. Also the default Apache site-locations file has changed.

To make it work I set the subdirectory in the install to html to make it compatible with the Apache default. I then edited the /etc/passwd file. Some of these changes have subsequently been added to the script.

www-data:x:33:33:www-data::/usr/sbin/nologin changed to www-data:x:33:33:www-data:/var/www:/bin/bash

I then separately installed libav-tools

sudo apt-get install libav-tools

With these changes it was working OK for me.

If you always get blank page in your browser, you can edit /etc/nginx/sites-enabled/rpicam, add

fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;

below "include fastcgi_params;", then restart nginx.

The installer is being worked on to try to make it work OK for both Wheezy and Jessie and to remove the need for these adjustments.

Startup Behaviour
To change the default startup-settings, edit the config-file /etc/raspimjpeg. If you want to disable autostart completely, navigate back to the directory, where you cloned the git-repo in Step 4 and run one of the following command: ./RPi_Cam_Web_Interface_Installer.sh autostart_no --> the interface doesn't start at startup, you need to run a command to use it (commands below) ./RPi_Cam_Web_Interface_Installer.sh autostart_yes --> the interface starts at startup and takes control over the camera (standard)

To temporarily start/stop or deinstall: Navigate back to the directory, where you downloaded the installer in Step 4. If you want to stop the interface temporarily, run "./RPi_Cam_Browser_Control_Installer.sh stop". To restart it, run "./RPi_Cam_Browser_Control_Installer.sh start". If you want to remove the interface completely, run "./RPi_Cam_Browser_Control_Installer.sh remove". Attention: It removes all files in /var/www.

!! it seems that we have to use "./RPi_Cam_Web_Interface_Installer.sh" for "stop" and "start" instead of "./RPi_Cam_Browser_Control_Installer.sh stop"

You may be have to do sudo chmod 755 /etc/rc.local and restart

Links
Project itself: https://github.com/silvanmelchior/RPi_Cam_Web_Interface

RaspiMJPEG: https://github.com/roberttidey/userland/tree/master/host_applications/linux/apps/raspicam

Forum: http://www.raspberrypi.org/forums/viewtopic.php?f=43&t=63276

Special thanks
To btidey for many, many new features

To jarrah31 for this wiki

To jonar for the github-repo

To James Cooke for the styling

To slabua for many fixes/improvements

To everybody else who helps on github, in the forum or here to develop the RPi Cam Web Interface

Ps: If you like Silvan Melchior's work and have some extra-money ;) : Bitcoin: 398h4RVQhptrgN7eT9abAStCe1yu7zxBoV PayPal: https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=HN25F6V378FB4&lc=CH&item_name=Private&currency_code=CHF&bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted

PPs: If you like Bob Tidey's work and have some extra-money ;) : PayPal: https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=FWX7UL9NSNM6L

= Usage =

Basic usage
Navigating to the main web site brings up a medium resolution live view of the camera, access to a number of control buttons, and access to camera settings and system controls. On all screens the top navigation bar steps back one view.



The basic controls allow for capturing single images, videos, or time lapse sequences. It can also be put into a motion detection state where motion will trigger activity like a video recording.

The buttons under the main controls allow for Downloading and previewing any captures, plus access to motion detection and scheduling set up.

Below this are Camera settings and a system control bar.

Clicking on the image will toggle between normal and full screen mode. BY default it starts up in normal mode but this can be altered by using the config variable fullscreen.

Clicking on Camera settings gives access to a wide range of camera set up controls from controlling annotation through to exposure set ups and image and video formats.

The system control bar allows for selection of streaming mode, shutdown and reboot of the system, clearing settings and selection of custom styles. By default the system generates the live preview as a continuous set of captured images as this has maximum browser compatibility. MJPEG streaming may be selected but this may not work on all browsers. This is a per browser setting.

The custom style bar shows any extra styles that have been added. As installed there is the normal white default look, and a dark night mode.

Download Clicking the Download Videos and images button shows all previous captures as a series of thumbnails.



These may be selected individually or as a set and either deleted or zip downloaded.

Clicking on an individual thumbnail brings it up in a larger preview pane and videos may be played from here. Next and previous buttons step forwards or backwards through the files. The file number in each thumbnail is also a link which will open the file in a new tab in the browser.



The selected file may be deleted or downloaded as a single file. If it is a timelapse sequence it will still be zipped as it contains multiple files.

The size of thumbnails, size of main preview, image sort order, image type and a time filter may be updated and this will be remembered as a per browser setting. Sort, Type and Filter take effect when the setting is changed. Sizes need the update button to be pressed after new values are entered.

Scheduler
The same program file schedule.php is used both as the web facing settings interface and the run-time daemon which does the automation. All the settings used are contained within the schedule.json config file. Important The scheduler daemon will normally be stopped and started by regular update methods. If you copy a new scheduler.php in as a patch then the old scheduler will still be running and maybe out of sync. So Stop and Start the scheduler if you do this to make sure that the daemon is the new copy.



Different rows will be shown according to the day mode selected. For example if Fixed Times is selected then 6 time based rows appear.



The Scheduler will normally be started on boot up and can be left running all the time. A Stop button shows when the background scheduling program is running and turns into a start button if it is not. Normally it should just be left running. Note that if it is stopped then motion start stop triggers will not be actioned as they pass through the scheduler. If the software is updated without rebooting then stop and start scheduler to make sure the new version is running.

Save Settings saves the settings to schedule.json. Changes will be passed to the background program and take immediate effect. These settings may be backed up and restored.


 * scheduleLog is the file where scheduling activity is recorded. The contents are displayed when the Show Log button is used and the log may be cleared from there.
 * Fifo_In defines the named pipe that schedule monitors. It should be the same as where motion sends its commands. Fifo_Out is where scheduler sends its commands to raspimjpg Do not change these without good reason. Other configs are relying on these and would need to be changed as well.
 * Cmd_Poll is how often the scheduler checks the Fifo_In for incoming commands. It should be kept quite short to avoid unnecessary delays.
 * Mode_Poll is how often the scheduler checks for changes between the 4 main daily periods. Setting it to 10 means there might be a 10 second delay in determining when day starts for example.
 * Max_Capture determines the maximum capture period. If motion sends a start command and doesn’t send a stop command then the scheduler will automatically stop the capture after this interval. This can be used to make all recordings this length by configuring motion never to send stop commands. The scheduler will then always time out and stop the recording. A value of 0 means no timeout.
 * Management_Interval causes Scheduler to run a task every Interval (seconds).  This is an optional user command plus automated purging.
 * Management_Command is a macro facility which will be run every Management_interval. macros are commands held in the macros folder in the web site typically shell scripts. This is a change from previous operation where any direct system command could be executed. The system installer must put any commands in the macros folder and give them execution rights. Take care when using this facility; test out any macros first.
 * Purge settings. See below.
 * Latitude, and Longtitude define where the camera is and allow the sunrise and sunset times to be calculated throughout the year.
 * GMTOffset adjusts for sunrise / sunset calculation. Set in hours or TimeZone string
 * DawnStart_Minutes through to DuskEnd_Minutes divide the day up into 4 periods based on sunrise and sunset. Dawn starts at sunrise + DawnStart_Minutes so would normally be negative to make Dawn start before sunrise. Similarly for the other 3 numbers. Night is from DuskEnd to DawnStart the next day.
 * DayMode – This provides 3 different types of scheduling.
 * Sun based splits the day up into 4 periods (Night, Dawn,Day, Dusk) based on sunrise and sunset calculations and the offsets.
 * All Day uses just the Day settings for the whole 24 hour period.
 * Fixed Times splits the Day up into up to 6 periods based on fixed times. The commands in force are those for the time just less than current time. These do not have to be in order.
 * AutoCapture_interval - If non zero sets the scheduler to issue capture starts at this interval. This is used in conjunction with Max_Capture to set up a repeating sequence of fixed length recordings of maxcapture duration at the autocapture interval. Note that Max_Capture should be set a little less than the AutoCapture_interval to make sure the video is stopped before the next is due to start. 1 second less should be sufficient.
 * AutoCapture_interval - If non zero then this sets the system up to only run the camera system when there is an active browser client attached. The interval determines how long the camera stays on after the browser stops watching the stream. The camera will automatically start when the browser next views the stream. Only use in situations where you expect the browser to be used normally. Scheduling and motion detection will not work in this mode if no browser is attached.

The period table allows mode changes and start and end commands to be different in each of the daily periods. Times are to the nearest minute and there can be an additional delay of Mode_Poll seconds before a period change is detected.
 * Motion Start are used to start captures when a motion trigger start is received. If left blank (e.g. at night) then no capture happens when motion is detected.


 * Motion Stop are used to end captures when a motion trigger stop.
 * Period Start are used to send in commands at the start of each daily period. So, for example they may be used to control motion detection and change camera settings. Changing to night mode for example extends the usefulness of the camera in dusk and dawn periods. They may also be used to start and stop recordings at the beginning of each period.

Note the scheduler is calculating the day periods based on local time conditions. The raspberry pi should have the appropriate time zone set in raspi-config. This may be checked by issuing a date +%R command line which should show local time. It is also displayed on the schedule settings page.

Example: Enable motion detection all day

When scheduler starts up (or if the camera system is restarted) then it detects which period it is in and performs the commands configured in the period start for that period.
 * 1) In All Day mode then there is only 1 period so any commands in period start are always sent. md 1 will therefore start motion detection
 * 2) In Sun based or Fixed Times mode then the period start commands for the current period are sent. Say you are in Sun based and put md 1 in Day and md 0 in Dawn, Night, Dusk. After a reboot if the time is in a day period then motion will be started, else it will wait till next day period starts.

Example: Send an email when motion is detected If you want the email sent when a capture is done then use the job macro facilities. For example, a video capture will trigger end_vid.sh if a file of this name is in the macros folder. The file should have execute permissions and preferably be owned by www-data. Job macros are passed the capture file name so there is no need for wild cards. This name is actually the capture file so if you want the thumbnail then the script needs to convert it. E.g. list=( $1*.th.jpg ) thumb="${list[-1]}" $1 is the parameter passed to the script and $thumb will be the thumbnail name.
 * 1) !/bin/bash

Purging

The scheduler has facilities for removing old files automatically. There are two mechanisms which are both checked and used at each Management_Interval. First the 3 PURGE_HOURS (one per file type) are checked and any file older than this is removed. So if PurgeVideo_Hours is set to 98 then any video captured more than a week ago will be removed. Setting the values to 0 disables this. This type of purging is useful when there is plenty of storage but you just want to remove old material. The second mechanism purges based on filing system space available or media space used. The detailed mechanism is selected with PurgeSpace_Mode which can be Off, Min Space % or Max Usage %, Min Space GB, Max Space GB. Min Space means that the available remaining space is checked and older files are removed until there is at least the level of free space set available on the filing system. Max Usage means that older files are removed so that the total space used by the media is less than the level set. The level in either case may be set in GB or as a % of the total filing system size depending on which Mode was selected. Note that this is different to the previous scheme where % could be put in as a unit. % will now be ignored if entered in the level.

Logging A button Show Log shows scheduler and also raspimjpeg activity (if raspimjpeg log is set the same as scheduler). The log may be downloaded and cleared.

The scheduler has a facility for log file maintenance. This is controlled by log_size which may be set under the camera settings. If this is 0 then no logging is done and no maintenance is needed. Otherwise log_size sets the maximum number of lines in the log. This is checked every management interval and any excess old log lines are removed.

[] are scheduler events, {} are raspimjpeg events

Motion
The motion screen gives access to the motion config settings. Motion detection must be on for this to work.

Not all motion settings are relevant here. A filtered list is shown and the full list can be accessed if required. Settings can be saved backed up and restored. Saving does tell motion to start using the new settings.



Internal Motion
v6 has a built in motion detection scheme. It is activated by selecting the motion detect mode under camera settings to be Internal. When this is done the original motion settings button disappears and a new Motion Settings accordion control appears on main page.

The settings here are likely to change as the detection algorithm is worked on. Currently there is a vector preview mode which allows the basic vector data to be seen in the live window. This does not work in all browsers and it is recommended to use mjpeg stream mode to minimise problems.

The current detection parameters are Noise Level, Threshold, Mask Image, Change frames to start and still frames to stop. The detection is working at full video frame rate (e.g. 25 fps) so one may want to use a fairly large still frame count to avoid early stop. These parameters are very likely to change in future versions.

The motionmask is like the external motion mask file mechanism. It is a grey scale pgm image file. It is used here as a binary mask. Changes in areas where the mask is non zero (not pure black black) are included. The mask file must be the same resolution as the motion vectors. This is essentially /1/16 of the video but rounded up by 1 in width ;121x68 for a 1920x1080, 82x61 for a 1296x972, 49x36 for 768x576. It is best created by grabbing an example cam.jpg and then using an photo editor to first change it into a grey scale black white mask where white is the area of interest. Then resize it to the right dimensions and save as a pgm file.

You can see what is going on with internal motion detection by including %c and %f in the annotation string (the date/time info bar at the top of the image). %c will show the filtered changes occurring by frames. When this is above the 'threshold' value then the change frame counter increments (%f) and when this exceeds the start frame count then a capture is triggered. Lower the value of the threshold to increase the sensitivity to small changes.

There are some new pipe commands to control the internal motion detect. Those will get added to the documentation when the parameters finalises a bit more.

Alternate algorithm
A second algorithm is available. To select this use Noise parameter values >=1000. The vector change data is first 2d filtered to remove isolated single change values within a frame. Next the accumulated changes in a frame are clipped so that a huge flash change in data does not contribute excessively to the detection. The clipping is based on the threshold value and the motion_clip factor determines the clip ceiling. So a motion_clip factor of 3 means that a single frame cannot contribute more than 3 times the threshold. Using higher values of clip factor allows individual frames to contribute more and could be useful if small threshold values are being used. The Noise parameter controls a frame based moving average. Any value of this above 999 triggers the alternate algorithm. The averaging is done over 'Noise' - 999 frames so a value of 1009 smooths vector data over 10 frames. If this averaged value exceeds the threshold for 'start'successive frames then a trigger is generated. Similarly the average must be below the threshold for 'stop'successive frames to trigger a stop.

A really good detailed explanation was given by Robert here (copied below too):

https://www.raspberrypi.org/forums/viewtopic.php?p=1020905&sid=2d571cf720f615789b2815098bef3440#p1020905

I use the new > 1000 internal motion detection for all my needs.

The older internal method (<1000) counts vector changes in each frame which are above the 'noise setting' and if the total is above the threshold then it is a changed frame. If there is a sequence of changed frames more than the 'change frames to start' then a start trigger occurs. Reverse applies to generate a stop trigger.

The newer method (>1000) has quite a few optimisations. First it applies a simple 2d filter to changes so that only larger moving objects contribute to the detection. Second, the noise level now controls a moving average time filter to all the vector changes to smooth out the changes. Any frame which now has the smoothed vector change greater than the threshold now is a changed frame. The trigger logic on change frames is similar to the older method.

Starting from defaults (Second algorithm internal) decreasing the noise level (but still above 1000) lowers the amount of smoothing applied. This makes it more responsive to quick changes but can lead to unnecessary 'false' triggers. Similarly increasing this will apply more smoothing making it more necessary for a change to be sustained before a trigger occurs.

Increasing or decreasing the threshold changes the sensitivity to when a particular change level is regarded as a changed frame.

Change frames to start or stop then apply a secondary filter before triggers occur. Higher start numbers mean movement must be sustained longer before a start is generated. Higher stop numbers mean stillness must be sustained longer before a stop is generated. It is normally good to have a highish number for the stop as that means if there is a pause in movement and it restarts then the capture continues in one recording.

Use the %c and %f variables in the annotation string to see what is going on. %c is showing the filtered change level. When it exceeds the threshold then the frame counter %f ticks up until it exceeds the start frame trigger point. Similarly it ticks up for values below the threshold when trying to detect the end of motion.

Extra explanation regarding Noise: When noise level is 1000 or above it kicks in a different algorithm. Firstly a simple 2d filter is applied to the vector changes so that isolated single vector block changes are removed. The 'noise' level then controls a temporal moving filter average to the vector changes found. 999 is subtracted from the noise level and this gives the frame averaging factor.

Moving_Average_n+1 = Moving_Average_n * (Factor - 1) / Factor + Changes_n+1 / Factor

So if Noise = 1000 then Factor = 1 and effectively no averaging is done as the previous average is ignored. If Noise = 1009 then Factor = 10 and 90% comes from the current average and only 10% from new changes. High Noise factors then give 'smoother' changes and make it less susceptible to false triggering but also mean that changes need to be sustained for longer before a trigger occurs.

For the mask generation.

Grab a still image at the same aspect ratio of the video format you are using. E.g. if using 1296 x 972 video then grab an image at the same resolution (temporarily modify a still image resolution to match the video res to grab the image).

Download the jpg (using scp on Linux or WinSCP on Windows) and load it into an image editor (PaintShopPro, Irfanview, Gimp). Resize the image to be the same as the mask image (Log will show that). So for 1296x972 the mask is 82x61. Now use the image editor to paint the areas looking for change to be white and the areas to be ignored to be black. Convert the image to 8bit gray scale and save as a portable greymap file (*.pgm).

Copy this pgm file to the root folder of the webpage, eg. /var/www/html/ and then include the full path to the filename in the Mask Image field. e.g.

/var/www/html/mask-file.pgm

= Configuration =

Configuration Scheme
A number of configuration files control how the overall system operates. The web pages give browser access to some of these.

raspimjpeg file in /etc is read whenever the raspimjpeg process starts up including if it is stopped and started from the browser. It contains basic paths and information to allow raspimjpeg to do its job plus the camera settings that will be used by default.

uconfig file in the /var/www folder is used to hold any changes from the defaults in raspimjpeg applied from the web browser. To start with it doesn’t exist but new values will be added if changes are made from the browser. Changes apply directly because the cmd_pipe process has fed those into the raspimjpeg process, but they are also written by raspimjpeg to the uconfig file. When raspimjpeg starts it reads the factory defaults file first and then overwrites any settings that are in the uconfig file. You can also clear the uconfig file from the web browser to effectively return to factory settings. This doesn’t hold real-time commands like capture start/end commands or motion enable.

The main index page also reads raspimjpeg config (via a link) and the uconfig file so that it can show the current settings. When a setting is changed here this triggers a cmd_pipe command into raspimjpeg. That updates uconfig and the web page reloads the config files to show the change.

motion.conf in the /etc/motion folder is read by motion to determine its operating characteristics. As motion is being used here in a failry simple mode of motion detection then many of its parameters are irrelevant. The primary ones of interest are those setting the motion detection characteristics like mask files, thresholds, noise levels. motion provides a web api to view and edit these settings and this is used by the motion.php page to show and allow altering the settings.

schedule.json in the /var/www folder is used by the scheduling process to determine the characteristics of the automation. It is read and edited using the browser based schedule.php. It is also read by the same file running in command line mode as a background daemon. If changes are made then the scheduler daemon must be started and stopped via theweb page to allow it to see the new settings.

uPresets.html is an optional file in the /var/www folder which if present controls the video and image format presets present on the main page. If not present then internal values are used. The file is just a list of html option fields. An example uPresetsV2.html is included which can be used as a starting point for creating a custom list.

Naming Scheme
raspimjpeg uses a name scheme for captures and annotation that are controlled by the settings in the config files. These have a number of %X substitution parameters which will be filled in as follows. X must be one of the the following characters. They can be put in any order and repeated if required for a maximum total of 16 substitutions. Any other characters are passed through, but do not use any ‘illegal’ filename characters or the subdir_char (@). Extra path separators may be put in below the media folder level.

So a config image_path of /var/www/media/im_%i_%Y%M%D_%h%m%s.jpg

will give a name like im_0005_20150309_093057.jpg

Lapse Index starts at 1 for a particular Time Lapse set and increments.

Thumbnails are named with the base capture name appended with ‘. [vit]IndexNumber.th.jpg’ where [vit] is a single character for video, image and lapse recordings. This allows cross referencing back to the real capture files independent of the actual base name.

The %i IndexNumber is shared between thumbnails and images. Vidoes have their own IndexNumber %v. raspimjpeg when starting scans the highest number in each category and uses these as the base. A pipe command (sc) can also be used to trigger a scan and set the indexes.

Image Quality and Compression
There are 2 settings to control quality and compression. quality (1-100) controls the quality of the live image preview, and image_quality (1-100) controls quality of still image captures. The number does not match normal jpeg Q factor as the Raspberry camera compression software Q factor is quite non linear. The default 10 is approximately equivalent to 75 in normal JPEG usage and gives a decent trade-off between quality and file size. 7 is about Q 50, and 20 is about Q 95.

Raspberry compression discussion

How do I change the path for the video images and pictures?
There are separate path settings for video, images and time lapse in the /etc/raspimjpeg config file.

These allow changing the naming scheme of the files and separating out into sub-folders and even including date parameters for these sub-folders.

If you change the 'root' from the media folder in the web install then you also have to change that setting in the web side config.php.

Instead use standard linux link/bind (fstab) facilities to 'relocate these to onto other locations like USB sticks, disk drives and network storage locations. When you do that bear in mind the linked location must have read/write permissions for user www-data

I recommend using doing it that way and leaving the logical location root location as media under the web install as this avoids having to change the web side config and also means that if the software is upgraded then nothing else needs to change each time.

Is there any way to have the "UPTIME" displayed on the Web Cam Interface?
You can use the user annotation variable %a to put any other info in the annotation. The info is picked up from /dev/shm/mjpeg/user_annotate.txt each time the annotation is generated. So use a CRON job or something similar to update this file with a string representing UPTIME.

have you considered adding some additions to this program to monitor the temperature?
Adding a temperature sensor (e.g. DS18B20) should be reasonably straightforward. There are lots of articles on how to connect and read theses on the Pi. It is more a question of how to display it. One way that could be used even with the current software would be to use a program to regularly read the temperature and write it to /dev/shm/mjpeg/user_annotate.txt. The %a annotation variable can then be used to incorporate this in the video annotation. See wiki for the annotation variables.

= Additions & Tricks = This section will be updated to contain links to posts with useful information and answers to popular questions. Anyone can add to this, so please create an eLinux account and help contribute!

Pre-Event Motion Capture Buffer
There is a video_buffer setting in /etc/raspimjpeg (default 0) and it can also be set from the camera settings (buffer). The value is a time duration estimate in milliseconds of the circular buffer.

If the value is 0 then operation is as it were originally; a video capture starts capturing the h264 camera stream to a file. With the various latencies using motion detection then the trigger and the start of a capture can be a second or so after motion really started, so the initial action is not recorded.

If the value is set to say 3000 (3 seconds) then raspimjpeg captures video data all the time into a RAM memory circular buffer sized to contain nominally 3 seconds, but in practice normally more as static video compresses well. When a video capture then starts, most[*] of the contents of this buffer is then prepended into the video file which is now capturing data. This means that file recording actually contains video from before the trigger. You control the amount before by adjusting the buffer.

The thumbnail image is still taken from the trigger moment so it helps show the cause. You will also notice that the thumbnail has a time stamp later than the start of the video. The difference is effectively the size of the buffer.

[*]Due to h264 interframe compression (eg. next frame may depend on previous frame), a video file must start at a "stand-alone" I-frame, which may occur only once every 60 video frames. Buffer sizes below 3000 are not recommended because an I-frame may not be found in the buffer if it is too small.

Pan-Tilt or Pi-Light
Information about Pi-Pan and Pi-Light: http://www.mindsensors.com/12-rpi

Needed Hardware for this addition:
 * Pi-Pan (MindSensors)
 * Pi-Light (MindSensors)
 * Pi-Case to mount Pi-Pan recommended (e.g. MindSensors)

In case you want an alternative minimal hardware (cheaper) option go to the end of this section.

With these code-changes and additions, it's possible to control Pi-Pan and Pi-Light from the RPi Cam Web Interface. Just follow these steps:


 * Assembly and install Pi-Pan: http://www.mindsensors.com/content/27-assembly-instructions-for-pi-pan-kit
 * Check if pipan and pilight work with the scripts as described on the website
 * Add the file "pipan_pipe.py" to the pipan-files with the following content:
 * Edit the line with /var/www/FIFO_pipan if using a subfolder. E.g. to /var/www/html/FIFO_pipan


 * Navigate to "/var/www" (or to /var/www/subfolder) and add a named pipe with the following commands:


 * Edit "/etc/rc.local": add the following line above the exit-command (change the path to the directory where you extracted the pipan-files):


 * Go to /var/www and rename the file "pipan_off" to "pipan_on" and "pilight_off" to "pilight_on"


 * Reenable camera-support in raspi-config

That's it. After rebooting your Pi, you should be able to control Pi-Pan with the new Buttons "Up", "Down", "Left" and "Right" or on the keyboard with "W", "S", "A" and "D". The Pi-Light can be controled in the settings-table or on the keyboard with "F". If you have a touch-device (Android or iOS), you can pan/tilt by dragging the preview-image around. To change the minimum/maximum pan/tilt angles, edit the settings in /var/www/pipan.php.

Pi-Pan uses servoblaster to generate the signal for pwm. However, it might be that servoblaster will interfere with 3.5mm jack audio output. If so, edit /etc/init.d/servoblaster.sh and add the option --pcm to the servod-command on line 17.

If you get the error "IOError: [Errno 2] No such file or directory", run raspi-config and disable device tree in advanced options and then reboot.


 * If you extract the pi-pan files in a different directory, the camera will not reset to neutral position on startup. To fix, verify the file "servoblaster.sh" in the pi-pan and the etc/init.d folders correctly references the pi-pan directory:

Minimal hardware (cheaper) option;

Servoblaster
There is also the possibility to just use ServoBlaster and an own servo-construction to move the camera; more information here: https://github.com/skalad/RPi_Cam_Web_Interface_ServoBlaster_pan_tilt. (But please note that this is now outdated. Use the functionality within this git as described below.)

Equivalent functionality can be turned on by renaming the file in the web folder servo_off to servo_on. Make sure that pipan_on is not also present as that takes precedence. Rename it to pipan_off if used before.

The co-ordinates and some set up information is preserved in the servo_on file in json format. This may be edited to change the behaviour. In particular the buttons can be mapped onto different servo directions, and the min, max and step rates can be changed for each axis. This file may be edited with a text editor. Only change the values, not the key names. The code uses the default servoblaster pins. You can set them using "sudo ./servod" from the user folder inside the ServoBlaster directory.

Remote access to website with User/Pass and changing port
1) Set up security on your camera by installing or re-installing with username:password and optionally choose a different port for access other than the default 80. Changing the port is not necessary but can be helpful in avoiding conflicts with other web uses and provides a bit of extra security against web scanners. It also allows multiple cameras by using different ports on each one.

2) Test security locally e.g. http://cameraip:port and make sure login is OK.

3) Login to your router and

a) Find its WAN IP address. This will be needed to get access from internet.

b) Set up port forward so that the port selected is redirected to the cameraip

c) The way to do this is obviously dependent on the actual router model. Check out http://www.howtogeek.com/66214/how-to-forward-ports-on-your-router/

4) To use a regular name from the WAN rather than the WAN IP of your router then set up a dynamic DNS account at a service like http://freedns.afraid.org

Central Motion Detection from Multiple Cams using iSpy Connect
http://www.raspberrypi.org/forums/viewtopic.php?p=518500#p518500

Now that an MJPEG stream has been implemented, you can configure iSpy with the following:

MJPEG URL tab within Video Source:

View video stream on an iDevice / Smartphone
Credit goes to Oke for the original post. A few more tweaks added for iPhone app. http://www.raspberrypi.org/phpBB3/viewtopic.php?p=507756#p507756

(note - cam.jpg created during the installation now)


 * Download "IP Cam View Pro" (I used IP Cam View Lite on the iPhone - can upgrade to Pro)
 * Select the menu icon
 * Press Manage Cameras
 * Select Add Camera then scroll down to find Raspberry, and then select RPi Cam Web Interface
 * Give your cam a name
 * Enter your URL/IP, e.g. 192.168.0.1
 * Enter user/pass if configured
 * Ch.# is the folder name, such as html
 * Press Test
 * If it works, press Save

Simple live-preview only page
A file min.php is included in the install which gives a basic live preview only. To use just browse to http://yourcamerip/min.php

The minimal code to embed the preview:

php:

js:

The size can be changed either as parameter in /etc/raspimjpeg or with CSS. To add further features (change settings, record images/videos), study the existing homepage.

Adjusting Live Preview bandwidth usage
The live video on the main screen is produced by a mjpeg feed either as a true stream or as succession of jpegs. Either way this can consume quite a bit of bandwidth. This section describes how this may be adjusted.

There are a number of factors affecting the live previews; width, divider, quality. These can all be controlled from the web interface. In addition you can choose between default streaming where it is fetching a sequence of jpegs or a true mjpeg stream. This doesn't fundamentally change the bandwidth but the mjpeg stream has less to and fro with the web server so may be smoother.

Width controls the size of the mjpeg images generated and will have a significant impact on bandwidth. It also controls the size on the display. By default it is 512px. If you can live with say 384 then that should halves the bandwidth.

Quality is the jpeg compression factor (0-100). Lower numbers give better compression at the expense of quality. It is set to 25 by default which gives pretty good compression and not too much degradation. You can try lowering it to say 15 which will significantly lower sizes further but you will start to see some degradation. Lowe than this it will start to get bad.

Divider is the rate at which new data is fetched. It is the video fps (default 25) divided by 'Divider'. Nominally that would mean it is trying to update at 25fps but in practice other delays lower this a bit. Increasing the divider lowers the fetch frame rate so 3 would give a nominal rate of 8fps. This will also have a dramatic effect on bandwidth.

Move saved images and videos to another folder
Detailed instructions on how to map a network NFS share to /var/www/media - http://www.raspberrypi.org/forums/viewtopic.php?p=531344#p531344

When to do the move - http://www.raspberrypi.org/forum/viewtopic.php?p=515967#p515967

Mounting a share - http://www.raspberrypi.org/phpBB3/viewtopic.php?p=513781#p513781

Mounting a Windows Share - http://www.stuffaboutcode.com/2012/05/raspberry-pi-connect-nas-windows-share.html

If you've mounted a network share to something other than /var/www/media, such as /mnt/myshare, you can bind the two together using this command:

Overlay/Composite two images together
One way to add an image over the top of another is with ImageMagick. I've constructed a crude example below, and it works! But it bogs down the refresh rate, drives the load level to 1.5 and is barely acceptable. I'd like to toggle it on/off with a new button, and improve it's performance, so please edit this article better.

h264 to mp4
To convert a h264 video file to mp4, the interface uses MP4Box. If you set MP4Box Off in /etc/raspimjpeg, perhaps so that the interface is more responsive to frequent start stop recording requests, you can later use MP4Box to convert any saved h264 videos file to MP4. The bash command is

Add extra users to security
The install.sh sets up the web security to allow one user to login with a password. If you want to add more users with different passwords then use the following command.

= Architecture = The overall functionality is quite complex but centres around the raspimjpeg process which accesses the camera data. The following diagram shows the major components. Blue lines indicate data flows. Red lines indicate control.



Data Flow
In normal monitoring mode raspimjpeg makes a connection to the camera (MMAL) and generates a continuous stream of preview jpeg captures in the /dev/shm/mjpeg directory all with the same name cam.jpg. This folder is RAM memory based so does not strain the SD card memory.

These captures may be accessed via URLs on the Apache Web server. cam_pic.php process returns the latest one and cam_pic_new.php merges them into a mjpeg stream. When a browser has logged into the web server then the main page (index.php) will use cam_new_pic.php to give a moving representation of the camera output. If motion detection is active then the motion process also accesses these images via cam_pic.php in order to analyse differences between successive frames and detect motion.

If raspimjpeg is put into a capture mode (described below) then the flow of preview images is maintained but an extra recording is made of either a single image, a time lapse sequence of single images, or a full video recording which can be at any format the camera can support including HD normal frame rates. This data is stored in the media folder. For the images and tile lapse images these are stored directly in jpg format. The video is initially stored as a raw h264 stream from the camera but can be automatically formatted into mp4 when the recording ends. The boxing mode (MP4Box) controls this. Three options are provided; false leaves the files in raw h264, true converts them inline but no further recording is possible till this completes, background spawns a separate process to do the boxing operation. Normally the raw h264 captures and box operations take place within the same folder. This can cause quite a lot of network traffic and potential problems if the media folder has been remotely mounted. A config option (boxing_path) may be defined as a separate local folder. If that is used then the capture is to the boxing_path folder and the boxing operation is from the boxing_path to the final destination. This limits the network traffic to one pass and also takes it out of the real time capture operation.

When raspimjpeg initiates any of these sequences it also grabs the current cam.jpg from the preview stream and stores it in the media folder with a thumbnail name tied to the captured data. These thumbnails are used by the preview_php process to give a representation of each capture when the download button is pressed.

Control Flow
Raspimjpeg can be controlled by sending in commands into a named FIFO pipe in the /var/www folder.

The commands give access to most of the camera settings plus stopping and starting the capture processes. Commands can come from effectively 3 sources but one is directed through a secondary process for scheduling purposes.


 * Commands can come from the web browser via the cmd_pipe.php web page. This is used to allow changing the camera settings and manually starting and stopping captures.
 * Commands can come from the scheduling process (scheduler.php – daemon) which can be used to change various modes, camera settings at different times based on sunrise and sunset.
 * Commands can also come from motion and these are used to start and stop captures based on motion detection. In the original scheme these commands went straight to raspimjpeg. In the modified scheme they are sent through the scheduling process daemon so that it may change the nature of stop and start based on the daily periods. To achieve this motion now sends its commands to a secondary named FIFO pipe (FIFO1). The scheduler is monitoring this for motion start / stops and then sends in translated commands to the main raspimjpeg process.

Config file
This is held in /etc/raspimjpeg and consists of keyword value lines. It is read at start up and when the samera is restarted. There is a uconfig file held in the web folder which allows user overrides to many of these values.

Pipe
The following commands are supported by raspimjpeg when received from the pipe. Many are equivalents of values in the config file. Others are associated with several parameters. They are sent in as a serial stream as a 2 character command, space, and space separated parameters.

The original scheme did not use LF terminators. LF terminators are now supported and recommended to improve reliability in command recognition. The old scheme is still supported. To send from a command line use something like (where this may need to be adjusted according to the install folder.

The main Pipe is FIFO which is used by the web interface and scheduler. Commands may also be sent to extra Pipes named FIFO11 up to FIFO19. All commands are treated the same no matter which pipe is used. This can be useful to make sure that commands from other sources don't get mixed up. The installer makes FIFO11 as an input; if others are required then they should be made manually and will then be recognised automatically next time the camera system is started.

Job Macros
Raspimjpeg supports calling macros for every capture job and error handling. These are shell scripts held in the macros folder. They must be created and stored there and given suitable ownership and permissions. Typically they should be owned by www-data and have execution privileges by that user (e.g. 764)

start_img.sh if present is called just before starting to capture a still images. It is passed a parameter containing the image file name. By default it is configured as a synchronous macro which means it will complete before the image is captured.

end_img.sh if present is called at the completion of capturing still images. It is passed a parameter containing the image file name.

start_vid.sh if present is called at the start of raw video capture. It is passed a parameter containing the mp4 video file name.

end_vid.sh if present is called at the end of raw video capture. It is passed a parameter containing the h264 video file name. Note that if boxing is being done then this file will disappear after boxing is complete.

end_box.sh if present is called at the end of any boxing command. It is passed a parameter containing the mp4 file prodcued by boxing.

error_hard.sh if present is called if any fatal error occurs. It is passed a parameter containing the error string.

error_soft.sh if present is called if any non-fatal error occurs. It is passed a parameter containing the error string.

do_cmd.sh if present is called if some selected pipe commands are processed. It is passed a parameter containing the command string. Currently only called by motion detect (md) commands.

= Trouble Shooting = This section contains trouble shooting tips.

General principle is that it is always easier to troubleshoot starting from an operational system and finding what breaks it rather than starting from a broken system and finding what makes it work. So install basic system leaving all config as default as this is most likely to work. Then make any changes you want one at a time.

Loading
Web interface starts up but just shows 'Loading'. There are a number of causes for this but most are caused by the core raspimjpeg process not running or not running correctly. First, Have you enabled autostart during installation and rebooted? If not then either run the install again to enable it or run the install script with the keyword start on the end.

Check that raspimjpeg process is running (ps -A) should show 2 instances of the process. Check that a thumbnail is being produced at the location /dev/shm/mjpeg There should be a file called cam.jpg which is being regularly updated. Check also the scheduler log available from the scheduler log pagae to see if any errors are being logged.

Motion Detection
Motion detection relies on a chain of events so trouble-shooting is deciding where the chain has broken down. First the motion process must be running, second it must get a continuous feed of updated cam.jpg images to analyse, third it must detect motion based on its settings, fourth it must issue start and stop commands, fifth the commands must get picked up by the scheduler and translated to raspimjpeg capture commands, and sixth raspimjpeg must action those commands. Working on these in turn and using a terminal session (e.g. ssh)
 * When motion detection is started then a ps –A cmd should show a motion process in the list. Similarly when motion detection is stopped then there should be no motion process in the list.
 * Motion picks up the images via the netcam_url setting. This defaults to http://localhost/cam_pic.php which should work with default set up. If the web site port has changed that it needs the port added and if a password has been added then the netcam_userpass setting must be changed. Test by browsing to http://cameraIPaddress/cam_pic.php . You shoud see the latest preview image. Change something in fron to of the camera and refresh to check t is getting fresh images.
 * For the motion detection and event triggers the best way to debug is to temporarily change the on_event_start from echo -n '1' > /var/www/FIFO1 to echo 'start' >> ~/motion.txt and on_event_end echo -n '0' > /var/www/FIFO1 to echo 'start' >> ~/motion.txt This will then write start and stop into that text file if motion triggers. Change threshold and noise settings in motion and check for triggers occurring.
 * Restore the motion settings and now check that scheduler is receiving and processing triggers. Check that scheduler is running (Schedule screen should show a stop button, if it says start then start it). For test purposes set Scheduler to All Day mode and make sure ca 1 is in Motion Start and ca 0 is in Motion stop. Now manually send a trigger by issuing a echo -n '1' > /var/www/FIFO1 command. Scheduler log should show start trigger received and should send on the command to raspimjpeg which should start the recording.
 * You can also manually check raspimjpeg itself by echo -n 'ca 1' > /var/www/FIFO which should start a video recording and echo -n 'ca 0' > /var/www/FIFO which should stop it.

Saving Motion Values
Configuration values for the external motion program are maintained via a http interface supported by motion. This means that motion must be running for this to work. Normally this should just work and you should see current values, be able to edit them and save the settings back. If this is not working then try the following steps.
 * Check that motion is running as the correct user by issuing a ps -Af command. motion should show up as running under user www-data
 * Check that the motion.conf program has the correct ownership and permissions. A sudo ls -l /etc/motion/motion.conf should show the file belonging to www-data and with rw user permissions.

You can try manually fixing any incorrect permissions as follows

Network speed / choppy video
This section contains hints on optimising the set up for lower network usage of the video viewing primarily of the live preview. Network bottlenecks particularly on wifi links can cause choppy video.
 * Default or MJPEG streaming. The default streaming is to fetch a series of jpegs from the server to give the equivalent of a live video feed. Each fetch is produced by a request from the client at a particular frame rate. An alternative method called MJPEG streaming may be selected under the system settings. Here a single request is made and the server responds with an MJPEG stream of data at the particular frame rate. Although the volume of data being sent from the server is approximately the same in both cases. The MJPEG stream avoids the multiple requests and can help smooth the flow. MJPEG stream should be used whenever possible but can cause problems with some browsers.
 * Preview Size. Controlled by a camera setting. It determiens the width in pixels of the jpeg images prodcued for the live video feed. The height is scaled from this to give the right aspect ratio. Decreasing the width will make the jpeg data smaller and lower the volume which needs to be sent to the browser. Halving the width from its default 512 will cut the volume of data almost by a factor of 4.
 * Quality. Controlled by a camera setting this determines the jpeg compression factor used in producing the live preview images. If th e quality is lowered then the data size of the jpegs is reduced. The default is 25 to give high quality images. It can be reduced to say 15 before significant degradation in quality takes place. This can halve the data size
 * Divider. Controlled by a camera setting this determines the frame update rate of the live preview relative to the frame rate of the video capture. So a divider of 1 means the preview attempts to be the same as video frame rate (e.g. 25 fps). A divider of 5 lowers the preview frame rate to 5 fps. A low frame rate will obviously tend to a more jerky view but lowering it a bit can help if the network is congested.

Taken together these 3 factors can be used to lower the network bandwidth requirement by an order of magnitude.

Choppiness when viewing video captures cannot be directly optimised as these are high quality videos with their own bandwidth requirements. One can slight lower the data rate by redusing the Video bit rate down from its maximum default to say 2Mbits but this is not a direct reduction as the video encoder is already doing a good compression job. One can change the capture resolution set up or the frame rate to lower the speed requirement or download the material for local playback on the client machine.

Buttons don't work
Web interface is showing video OK but the action buttons like record video don't work or are disabled.

The most common cause of this is a problem getting the status of the raspimjpeg process which then controls the state of the buttons.

The status of raspimjpeg is maintained in /run/shm/mjpeg/status_mjpeg.txt which in turn is linked to by a status_mjpeg.txt in the web install folder. The web interface retrieves the status via the link. The link should have been set up correctly during the installation process providing the install.sh method was used but earlier versions sometimes didn't set the link correctly.

First check the link file exists in the web install folder e.g. ls -l /var/www/html There should be an entry status_mjpeg.txt shown as a link to the real file.

If there is not then either re-run the install.sh making sure it is the most recent version or create the link manually by

replace the subfolder or leave it out according to where the install is.

You can also check the contents of the file by cat /var/www/subfolder/status_mjpeg.txt It should show 1 word typically somethink like ready for a running but idle system.

If the status file looks OK and is linked properly then the next check is to activate developer tools in the browser e.g. Chrome. You should then see this status being retrieved about every 5 seconds under the network tab.