Difference between revisions of "BeagleBoard/GSoC/BeagleBoneSonicAnemometerBasicWeatherStation"

From eLinux.org
Jump to: navigation, search
(Remote Connectivity & Software End)
(Remote Connectivity & Software End)
Line 189: Line 189:
 
Some of the approaches pictured below  
 
Some of the approaches pictured below  
  
Deployment Approach one
+
                                                    [[File:Localserverremote.png|450px|thumb|center]]
 +
                                                   
 +
                                                    [[File:Maildumpremote.png|450px|thumb|center]]
  
                                                     [[File:Beaglebone_base_station.png|Network Diagram|450px]]  
+
                                                     [[File:Lanlocal.png|450px|thumb|center]]
  
 +
                                                    [[File:Beaglebone_base_station.png|Network Diagram|450px|thumb|center]]
  
Deployment Approach Two using small nodes connected via rf to the main beagle acting as a server thus saving on remote comm link hardware costs
+
                                                         [[File:Fileforf.png|Network 2|450px]|thumb|center]]
 
 
                                                         [[File:Fileforf.png|Network 2|450px]]
 
  
 +
  
  

Revision as of 07:21, 25 March 2018


ProposalTemplate

Working prototype sonic anemometer using BeagleBone, high speed DAC (eg, PRUDAQ), and ultrasonic sensors. Analyze methods for accuracy/sensitivity (eg, time-of-flight vs. phase difference) and implement "best" method. Use PRUs for independent real-time control of ultrasonics.

Student: Manas Mangaonkar
Mentors: Stephanie Lockwood-Childs, Steve Arnold
Code: https://github.com/Pac23/Gsoc_SonicAnemometerBasicWeatherStation
Wiki: http://elinux.org/BeagleBoard/GSoC/ProposalTemplate

Status

This Proposal for Sonic Anemometer was accepted in Gsoc 2016,but the chosen ADC proved to be inefficient for sampling at high frequency.

In 2017 again the project was accepted and PRUDAQ was used which solved the sampling and creating trigger for ultrasonic pulse problems.

However Since the final idea is to have a working prototype of a basic weather station with every possible thing ranging from Humidity,temperature to air speed and maybe even pressure.

I discuss what all i plan to implement below.

Proposal

I have completed the tast required as described on the ideas page, and created a pull request, as listed here

About Me

IRC: Pac23 || Pac23xyz
Github: Pac23
School: vesit Which is a campus of and falls under University of Mumbai
Country: India
Primary language: English,Hindi
Typical work hours: 9:30 - 23:00 IST GMT/EST/PST to Ist Adjusted Time
Previous GSoC participation: First GSoc Participation, i am excited to be a part of gsoc,since contributing to open source means a lot to me and gsoc will be an special achievement. I got interested in IoT as a teenager,the fact that i could control things to do stuff for me with a single click of a button remotely over the internet or Lan was fascinating and makes life somewhat comfortable.Remotely controlled Air conditioners are a good example of the later,on a hot summer day by the time one reaches home the room is already cooled to perfection since the user activated it before leaving work/school.

Skill(proficient): C/C++,Python,HTML & CSS

Tools(proficient) : Bash,Git,Docker,aws,Linux,ESXI

Experience : C,C++,Python,Java,HTML & CSS,Php,CAD,RaspberryPi

Hardware Skills : Rpi,Lots of Pc builds You can see my Pc builds here

Sonic Anemometer / Basic Weather Station

Project name: BeagleBone Sonic Anemometer / Basic Weather Station

Description

Get the Anemometer/Basic weather station prototype working,by implementing the best methods to counter operational Environment Disturbances.

My approach to this problem shall be more software,usability and added sensor's and less anemometer since most of the anemometer code has been taken care of in gsoc 2017.

Sonic AnemoMeter / Basic Weather station

Anemometer's are used world wide by Meterologists,Researchers and Scientists to calculate Three Dimensional Windspeed and turbulence. Sonic Anemometers are however utilised since they aren't affected by their operational environmental Conditions.

I am not going to harp too much on the anemometer part since that was achieved last year with in depth coverage of how it functions Here rather would focus on the whole weather station package and software end of it,making it functional and user friendly is a important aspect of a working station.

Sonic Anemometers and other weather station devices cost a lot since majority of them are commercially manufactured. What i plan to implement is on a beagleboard using a single python module to control all of the senors to collect the data from not just the anemometer but also the temperature,pressure,rain and humidity sensor.

From a practical perspective a meteorologist/researcher should be able to deploy the station remotely and get data via the internet,and only access it to change the battery bank/pack maybe even run it on solar(Something that can be done with efficient and large enough panels).


Principle of Sonic Anemometer

When wind flows in the same direction as sound,the velocity of sound increases and when in opposite direction it reduces.Along a axis two transducers are placed,Time of flight is calculate in both directions.These transducers act alternately as transmitters and receivers.Time of flight is measured.

For proper measurement the anemometer should have atleast 2 axis.In a 2D anemometer the time taken for ultrasonic pulse of sound to travel from North Transducer to south transducer.Similarly times are compared between west to east.

2D axis

The speed of sound is calculated is independent of factors such as temperatures. Using this device wind speed as well as temperature can be calculated.Even though temperature calculation is not the most accurate and a dedicated sensor must be used.

the speed of sound and the air speed along the axis of the transducers as follows:

T = L/(c+v)

Where T is time, L is the distance between transducers, c is the speed of sound, and v is the air speed along the transducer axis.

The speed of sound in air is dependant on temperature, pressure and suspended contaminants such as dust and fog. In order to obtain the air speed between the transducers, each transducer alternates as transmitter and receiver so that pulses travel in both directions between them. The air speed is calculated from the pulse times in each direction as follows:

v = 0.5L(1/t1-1/t2)

The speed of sound, from which an estimate of air temperature may be derived may be calculated from the pulse times as follows:

c = 0.5L(1/t1+1/t2)

By arranging three pairs of transducers on three different axis, three dimensional wind speed and hence direction and wind angle is obtained.

Requirement Analysis

Following hardware could be used to start with:-

  • Beaglebone Black/Green(1)
  • PRUDAQ/ADC having rate higher than 400ksps per channel(1/2)
  • Transducers(40Khz,Waterproof)(4)
  • MCP9808 High prescision Temperature sensor
  • MQ-XX Sensors (individual reaction)
  • BMP180 Air pressure sensor
  • BME680 - Temperature, Humidity, Pressure and Gas Sensor (optional)
  • USB 4G LTE Modem (Huawei E303 Modem works perfectly)[This should be left to the user]
  • Breadboard
  • JumperCables

As you may see there are some sensors that overlap each other's functionality and there's a valid reason for that below.

DAQ are the Data Acquisition devices which are used to acquire data from sensors and acts as a bridge between sensors and host computer.

PRUDAQ is a fully open source 40MSPS Data Acquisition (DAQ) cape for the BeagleBone Black or Green designed by Jason Holt and his team at Google Research with a software collaboration from the BeagleLogic creator, Kumar Abhishek(one of the mentors this year).PRUDAQ was created to address a need not currently addressed by the market for a portable and low-cost DAQ system that doesn't compromise on performance.

After looking into the previous years code and the work done it was determined that as of now temperature is taken through LM-35 using PRU and PRUDAQ,This year i plan to use the MCP9808 from microchip instead even though it will require me to rewrite some code but that wont take too long(a day Max) nothing more than that,since the sensor offers additional modularity which the LM-35 does not offer and since a working proper prototype is to be achieved this approach seems the best to me.

Since linux is non-preemptive Os it can't be used for realtime measurements,instead the built in PRU would be used for realtime measurements.It can be programmed with assembly instructions and hence can be customised as per our requirements and can transfer information with user space programs.The PRU1 for reading the data from ADC and transfer sampled data to BBB host system, however during the reading and transferring , it downsamples the data. Following diagram illustrates the data flow.PRU0 Switches input channels for higher sampling.

Since work would be done upon previous years code as of now the 2D axis program provides result every 2s.The 2D anemometer works but has a buffer time since the actual output can be achieved at 0.800s but the buffer exists for future 3 axis additions.This will be changed so that faster results are achieved.

Another change is to change the pru assembly code so that control PWM with pru and collect samples only around the time, when wave was sent over the axis,although i only plan on changing this after rest of the sensors and remote config module has been tested and working.

I did discuss with the last years contributor and other than changing some typos the circuit as of now works fine.

                                                      Circuit.png 


The MCP9808 comes with user-programmable registers that provide flexibility for temperature sensing applications. The registers allow user-selectable settings such as Shutdown or Low-Power modes and the specification of temperature Alert window limits and critical output limits. When the temperature changes beyond the specified boundary limits, the MCP9808 outputs an Alert signal. The user has the option of setting the Alert output signal polarity as an active-low or active-high comparator output for thermostat operation, or as a temperature Alert interrupt output for microprocessor-based systems. The Alert output can also be configured as a critical temperature output only. The MCP9808 has some ready to deploy python libraries so that makes it even better and quicker to implement. This also is cheaper than the DHTXX sensors which are not precise.

The MQ-XX Air Quality Sensor will be used to determine the gases and air quality. These sensor can detect a wide range of Volatile Organic Compounds (VOCs it will return a Total Volatile Organic Compound (TVOC) reading and an equivalent carbon dioxide reading (eCO2) over I2C.The only benefit of this sensor over the BMP680 is that it can detect Alcohols, Aldehydes, Ketones, Organic Acids, Amines, Aliphatic and Aromatic Hydrocarbons all of this Individually.The problem though is that this sensor may require I2C stretching which can be gotten around with. With each individual sensor only detecting a certain gas.ADC would be used to do this.

For Barometric pressure sensing i plan on using the BMP180,its cheap,small and operates in various environments. Also easier to program and deploy.

Optional Sensor

This sensor does everything under the hood,is quick to deploy and program for only problem is it is limited to a certain range of temperature -40*C ~ 85*C whereas all of the sensors above operate -50 ~ 150+ *c and offer certain low level additional controls.The only advantage of this its small and has ready modules.

The BMP680 sensor will be used to measure the air pressure,temperature,humidity and gas since the final aim is to build a complete weather station.The sensor combines all of the small specific application sensors into a single package that roughly costs the same as all the senors individually.Given its small footprint it would be easy to manage manage it the only downside being loosing the modularityoffered by individual sensors.Humidity and temperature Sensors can be found for cheaper but they often dont offer the reliability and precision.Bosch as of now provides a api,but the data sheet isn't the most complete thing although there are couple of python libraries available. I have included this sensor as a fallback/optional thing that can be used instead of the ccs811 gas sensor.

Below i have described generalised calculations for sensors.

Humidity

A humidity sensor (or hygrometer) senses, measures and reports both moisture and air temperature. The ratio of moisture in the air to the highest amount of moisture at a particular air temperature is called relative humidity. Relative humidity becomes an important factor, when looking for comfort.This is going to be important for proper anemometer data graph adjustments.

To calculate relative humidity, you only need one simple equation:

                                                       Relative-humidity-equation-600x85.png


However, there is more here than immediately meets the eye. The saturation vapor density relies on both temperature and pressure data, and can be a very complicated thing to measure in reality. Luckily, some modern sensors, like capacitive humidity sensors, can measure relative humidity directly.Some sensors require baking in the sensor(bosch BMP680). I plan on using a cheap BMP280 sensor since even though they aren't the best for temperature,they get the job done for humidity. The humidity value inside the sensor does not fluctuate rapidly and does not require low pass filtering. However, the environmental pressure is subject to many short-term changes, caused e.g. by slamming of a door or window, or wind blowing into the sensor. To suppress these disturbances in the output data without causing additional interface traffic and processor work load,IIR filtering will be utlised using the formula

                                                    Datafilterformula.png 

Data_filtered_old is the data coming from the current filter memory, and data_ADC is the data coming from current ADC acquisition. Data_filtered is the new value of filter memory and the value that will be sent to the output registers. The IIR filter can be configured to different filter coefficients, which slows down the response to the sensor inputs.

Gas Detection

Since global warming is a thing,Gas detectors are a must thing for every weather station.Gas sensors that not only detect the gas but also measure the ppm.Gases like Methane,Carbon Monoxide are important in determining the air quality index.Most of the standard (MQ) sensors require ADC conversion which can be done using the prudaq. Now as far as the formula for detection goes every sensor has a different thing so that shall be decided as per the sensor.The mq sensors react to different gases but calculation has to be done to derive the ppm of individual gases. The prudaq's adc will be used to convert the analog signal given out by these sensors to digital for processing.There's the BMP 680 which also does the same but since it reacts to lot of different gases using individual MQ-X sensors that react to individual gases is a better way of measurement.Target gases to be detected are CO2,CO3,Methane

Pressure

Air pressure varies by weather,season,air currents so this is something that every weather station should have to predict future weather.The BMP180 get the job done well,has ready modules available on github which only have to be integrated with the main code.The BME280 will be used to measure humidity,The sensor has a I2C interface thus integrating it wont be a problem and can be directly connected to the I2C pin on the beaglebone, several oversampling options exist. The pressure measurement is controlled by the osrs_p[2:0] setting For the pressure measurement, oversampling is possible to reduce the noise. The resolution of the pressure data depends on the IIR filterand the oversampling setting When the IIR filter is enabled, the pressure resolution is 20 bit. When the IIR filter is disabled, the pressure resolution is 16 + (osrs_p – 1) bit, e.g. 18 bit when osrs_p is set to ‘3’. This will allow the user to get more precise data.

Remote Connectivity & Software End

Since the station should be deployed remotely,data transfer back to the main server is a very important aspect.Cellular usb modem/satellite internet can be used to beam the data back.

The raw data shall be logged to a json file,at a set interval defined by the user the data shall then be beamed over .The user shall then choose the factors to automatically generate the graphs using matplotlib and numpy ,to see the data of each sensor individually,all of the heavy lifting shall be done at the client side gui,since that offers more control and modularity.

The simplest and cheapest approach would be to setup a server locally to which the data will be sent over to,although this wont be advised as even if the user has access to a static ip,unless the user has backup(power,internet and machine) incase the main server or any connection fails due to any reason.Practically the data could be sent to a cloud hosted database,or the data upload can be programmed in such a way on the station that it will continuously ping the server for status.Whenever the server is found to be live the data shall be sent in.For dynamic addresses a dns resolving service shall be required.

The more advance but expensive approach will be to setup a server & database to which the data will be logged on to.There are couple of disadvantages to it though since if the self hosted server faces downtime all of the users will be affected.Hosting it in on a cloud service will have recurring costs given the amount of data that will sent over,another web based approach to this would require all of the costs of the domain,web hosting,dns etc.

There are various approaches to this problem which i wish to discuss with the mentor

A stretch goal shall be to add functionality to individually configure/calibrate all of the sensors remotely,have a web interface.

The front end gui described will be done in gtk or pyqt,most of the software end would be done in python and C,with majority code being in python.The user shall have the option to generate the graph as per their liking with each factor and read individual sensors history.The gui will also have a lan mode.The software needs to be cross platform linux & windows.As far as gsoc is concerned Linux will be prioritised.Although a browser based gui would be even better.

Data retriving/uploading

Over Lan For getting data over lan as of now i think integrating it into the client end gui would be the quickest option given gsoc's timeline so as to not derail the major aim that is remote working prototype since that part would require work.My opinion is to store the data at a given location hardcode it into the client side software,acess it over ssh&ftp(sftp) and retrieve it. Adafruits pi finder is a great example of this,something along those lines will be implemented. The traditional approach would be to setup a local web based dash enabled server but time constraints

Some of the remote methods

  • Data uploaded via email(SMTP) to be processed at the client gui
  • Server with DDNS for a dashboard on site
  • Cloud interface where all the data is uploaded and the client sees it,web based ui/dashboard

Remote methods have been discussed with the mentors and more brainstorming shall be done during the community bonding period if the proposal is accepted.

Some of the approaches pictured below

Localserverremote.png
Maildumpremote.png
Lanlocal.png
Network Diagram
450px]



Cellular Connectivity

Remote deployments would require this,as of the gsoc timeline i would be hardcoding the data transfer but running a standalone server on cellular/satellite is expensive given the data transfer rates,rather sending data to a remote server with a hardcoded ip would be way better and cheaper.Also there are various celllular modules that can be integrated with the beaglebone, aliexpress.Any generic Usb cellular modem that runs on linux and supports 4G LTE shall get the job done.A generic driver will be packaged into the main software for gsoc but the final what modem to use implementation shall be left to the user.Using addon boards could be done for cellular but they just hog up pins and often power and drivers are limited to certain chips. Usb modules with generic drivers for certain chipsets are modular solution. Huawei E303 Modem is a good match for the beaglebone.

Reusing Older Code and time constraints

As mentioned above the anemometer part is working and rewriting code for it would be a waste of time,energy and money.I have looked into older code and as of now the only place that things need to be changed are in the process.py file where the temperature accusation and passing functionality will be changed a bit.Rest of the files wont be touched in the repo,only newer code will be added to it for data accusation from other sensors.

Since rebuilding the 3 axis physical anemometer in a portable package will consume a lot of time,i plan on starting to build it as soon as i get my acceptance mail from google.I do have ready acess to a metal workshop in my college and the instructor there is knowledgeable and cooperative so building the anemometer wont take time. Most of the gsoc official period would be dedicated to the software side of things.My workstation,testing station and other stuff is already setup since i usually tinker around with things.

Transfer/Transitioning

Transitioning the project from one team to another sometimes leads to loss of time(things not working out for reasons).I have taken care of that aspect by discussing with the previous years contributor what works,what does not and where things need to be polished so as to emulate the same environment to make sure things work out without being stuck in the troubleshooting cycle.My testing lab / homelab as we fondly call on reddit has all the physical things required for quick setup(kvm,switches,extensions,dc powersuply). I will also be in contact with the above mentioned contributor and do have acess to college labs that have some magnificent electronics testing/powering equipment.

Timeline:

I have my Finals till may 30th,after that i can commit 55+ hours a week to this project.During finals there's a gap of 4~5 days between each subject exam so during that time i can commit around 35 hours a week.

I will be starting work on this project asap once accepted by reading as much documentation i can get on my hands about beaglebone and the sensors.

Acceptance Mail

Commence building of the physical anemometer to save time,order stuff.

Week 1:

During this week i will try out the older code and the basic portable anemometer build.Study all the pinout diagrams,spec sheets and test all the hardware and sensors.Transition the older code base,make sure things are working for additions to begin.

Week 2 - 3

By now i should know all of technical stuff and armed with that i shall begin writing the framework for rest of the addon modules(barometer,temp, humidity).This sound less time for sensors to work but imo this shouldn't take much time two weeks shall be more than enough.

Deliverables : All of the core modules for the added sensors shall be ready and tested individually and a working anemometer

Week 4- 6

Begin developing the remote connection module,test the networking side of things and write the basic framework required to upload data.

Week 6 - 9

Test the remote connectivity module by isolating it on my homelabs virtual network,start work on the user end gui.Add individual control functionality by end of week 9 with a database setup to store the data locally.

Deliverable : Successful connectivity remotely and Designed prototype of the user end gui,database setup.

Week 10

Add graph generation functionality and connect to a local database to store the graphs.add setup scripts and begin polishing for final package installation.

This is going to be a fun week since i plan to go to my mud trail where i bike on everyday.Its a swamp with high humidity and temperature. i plan to deploy it and camp out Here the whole day to test the remote connectivity and sensors.

Deliverable's : Testing of package code,deployed beaglebone in a swamp location and testing of over 5 hours.

Week 11

Buffer week

Week 12-13

Test,bugfixing merging

Week 14

Prepare the final presentation slides and video.

Final Goals:

A Working Weather station prototype that can be deployed quickly in any environmental condition. The weather station shall be able to gather Wind speed,Air pressure,temperature,humidity,gas detection(methane for global warming) and whatever the user decides to add to it further.The station shall be capable of sending the data to a remote server and the user shall be able to access the data from multiple machines.This isn't the actual final goal but rather a gsoc 2018 goal.

Future Goals

More complex web based interface that has more functionality,user groups.This can be sold as a commercial kit,with the front end software being opensource and free to download.

  • switching modes of sensors from tx to rx and rx to tx.This has been listed as a future goal even in 2017 to find TOF for both directions along a axis, in that way we obtain more accurate wind speed and sound speed but after lot of research it has been concluded that it would require hardware redesign,writing assembly code from scratch for a certain chip from TI and another manufacturer.As far as gsoc goes this won't fit into the timeline without disturbing other things.
  • Rf i have listed the rf connected beagle modules in the network diagram,but given gsoc time constraints this is something i would love to work on personally after gsoc ends given my rtlsdr hobby.

Product Lifecycle Support

I will be contributing to further development of this prototype,to make sure its production ready,even after gsoc ends and i am not saying this for the sake of this but because i love weather charts,satellites and stuff that acquires data related to weather,adsb so you can count on me.

Experience and Approach

As a part of my graduation program of Computer Science Engineering,i am familiar with the concepts of physics and chemistry that would be involved in the project.Have developed a system for automatic watering of plants using a raspberry pi.

As far as software skills are concerned. I am proficient in C,Python and C++.I usually program in c and python since my college curriculum is completely based on programming in C and i use python for all of my personal projects.Having worked with various frameworks like django,flask and various api's i think i do have the knowledge required for implementing a remote connectivity module.

Projects

  • Smart CryptoPriceTracker : Used couple of api's and the requests framework to keep track of various crypto currencies set by the user.The user was able to set the minimum and maximum targets for the specific user defined currencies and whenever the target set was achieved. A sms,mail and call with all of the information would be sent to the user. The program has no limit on amount of currencies that can be tracked in realtime. It's basically a bot that does all the tracking work for you,you just have to tell it what currency track.
  • Automatic Water Dispenser : Indian summers are hot and harsh thus using a rpi and soil moisture detectors a script was developed that would dispense water whenever required.[software implementation not original]being honest about this.

Participations

  • Won the first place in sysadmin/Devops contest held at my college.

Approach

During majority of the gsoc i won't be having any academic duties to fulfil since the summer break would be pretty long.Thus i can devote my whole time to the project.This shall also help me enhance my database managment and remote networking knowledge which would be required for the remote connectivity part of the project.

Since much of the groundwork has already been laid i am going to jump directly to adding the sensors and completing the software part of it which had been promisied in both previous gsocs but was never achieved.

The reason i believe i should be chosen,is my interest for iot and love for tinkering around with hardware.As i have mentioned above i have a lot of experience in sysadmin/devop side of things.These things came naturally to me as a teenager and and i would enjoy working on this project and pour in all of the hardwork and passion required.I do like reading weather charts,graphs and radar data so this was the first gsoc idea that caught my eye.

Contingency

As mentioned above i am proficient in most of the concepts that are going to be utilised in this project.In case i cant contact my mentor due to some reason,i can always turn to my experienced faculty of my college and my family members since everyone in the extended family is a software engineer.Reddit and stackexchange are always there to help,i have learned a lot from both sites and lastly google.

There are things that can go wrong like sensors not working,mentor being busy and medical emergencies like food poising.Thus my plan is to start asap once accepted than to wait for later.Sensors going kaput can be replaced quickly since i have check on my local amazon and ebay and almost everything seems to be available and since i live in major city of india express next day delivery wont be a problem.

Benefit

The weather station shall be beneficial to students,teachers and researchers who are currently plagued by the cost of commercial hardware available.Even if researchers with large amount of funds do have acess to equipment they still have to spend more money to retrieve it after deployment.Where as given the cost,size and portability of this project it can be airdropped,used and forgotten about.Transporting,storing it would be easy on aircraft's,ships that already have limited space and often researchers have to sacrifice supplies for equipment.Can also be strapped to drones to be deployed since the weight is negligible.


What community members speak -

   Great project for the Aspiring Weather Scientist.
                           Michael Welling(m_w) 
   A working/documented FOSS implementation would be a huge
   plus for both the open source and weather communities.
                              Steve Arnold(nerdboy) 

and the best one award goes to...

   I'll be tempted to build one!
         Jonathan Cameron(jic23)
      • Mentors!!! please add your Quote here.***

Suggestions

Is there anything else we should have asked you?