Kevin Key's Blog

Saturday, November 23, 2013

Bridge reader: App that reads data from Acu-Link bridge and posts to Weather Underground, WeatherBug, and PWS is now available for download

I've finished developing my Acu-Link to Weather Underground and WeatherBug app.  Unlike my earlier app, this one reads the data directly from the Acu-Link instead of scraping the data from Acu-Link.com.  Note that this app requires a PC - either a laptop with a wired and wireless network capability or a desktop with two Ethernet ports - that can be left on 24/7.



Quick recap from my earlier post. I recently ordered and received the Acu-Link Internet Bridge and couldn't wait to set it up. I also set up an account on wundergroud.com.  I was disappointed to find out that Acu-Link.com only posts data to wunderground.com every 15 minutes. I wanted the data to get posted to Weather Underground much more frequently.  So, I developed an app to scrape the data form Acu-Link.com and post it to Weather Underground.

Now I've come up with a better idea: read the data directly from the Acu-Link device. I make use of WinPcap and the PcapDotNet libraries to sniff the packets directly from the Acu-Link hardware. While my other app was coded in VB.NET, I decided to develop this one in C#.  The bridges receives and outputs sensor data (wind speed, wind direction, temperature, humidity, and rain) as often as every 18 seconds - and my app immediately passes this data along to Weather Underground and WeatherBug. My app also calculates dewpoint, rainfall total over the past hour, and rainfall total for the day.

Some code snippets.

Posting the data to Weather Underground.

Processing the packet data from the Acu-Link device.

Parsing and converting the wind speed, temperature, and humidity data.

Calculating the dew point. 

Calculating wind gust.

Other people have developed code for Linux devices, but I believe my app is the first of this nature for the PC.  I've made it available for you to download here and enjoy.

Latest build is 2018.06.05.2122 and includes the following changes:

  • Fixed the ERROR 400 problem. Also, data now transmitted via a secure URL (https). 

Latest build is 2016.10.13.0004 and includes the following changes:

  • Additional fix for Settings form sometimes not closing after clicking the Save button.
  • Fix for rare problem where the app writes numerous entries to Debug_Log.txt due to not being able to acquire time from AcuRite device.

Latest version is 2016.10.09.1231 and includes the following changes:

  • Rain in the last 24 hours and rain in the last hour values finally work properly. Acu-Rite resets these values at midnight and every hour on the hour, but some of the weather networks want the rain in the past hour ongoing and rain in the past 24 hours ongoing values. The day in the past day value is still included as well. NOTE: Be sure that you have the correct location of your weather station specified on Acu-Rite's website.
  • Removed the "You're using the old firmware" check. Some false positives were reported and I figured that this logic wasn't really necessary after all.
  • When you first start the app, it will tell you what data it's waiting to acquire before it begins posting to the weather networks. This should take less than a minute.
  • NOTE: The three DLL files included in the ZIP have been updated - be sure to replace your existing DLLs with the new ones.
  • Fixed issues with WindData.xml and RainData.xml

Latest version is 2016.09.27.2112 and includes the following changes:

  • (Hopefully) fixed the rare, but reported, bug where the app sometimes crashes when clicking the "Save" button on the Settings form. I have not been able to personally reproduce this bug, but I was able to narrow down the cause to the code that refreshes the main screen when one saves and exits the Settings form. 
  • Additional improvements to error handling and reporting.
  • New/experimental: The ability to post temperature and humidity data directly to a Vera / Mios / Micasaverde / OpenHAB home automation controller. If this proves to be a popular feature, I will expands the options here.

Latest version is 2016.09.25.0229 and includes the following changes:

  • Brought back the "p" value for CWOP. This reports the cumulative rain for the past 24 hours.
  • Improved error handling and reporting.
  • Insignificant network errors will no longer be reported in the "Errors" value near the bottom of the screen. This value only increments when my app encounters trouble posting your weather data to one of the weather networks. NOTE: It is normal to see around 0 - 10 or so errors here in a 24 hour period. The weather networks don't have 100% uptime. 

Latest version is 2016.09.19.2137 and includes the following changes:

  • Waits for WinPcap to come online (service name is npf.sys). Sometimes WinPcap takes some time to start when a PC is first booted up. Previously, the weather app would report an error regaring WinPcap if WinPcap wasn't already running. With this fix, the app will keep waiting and trying until WinPcap starts. Also, should WinPcap shut down on its own, the weather app will keep trying to resume until WinPcap starts up again.
  • Fix for not properly identifying sensor IDs when filtering by sensor ID is specified. Acu-Rite decided to append leading zeroes to the sensor IDs with the latest firmware update. My app will now allow you to specify sensor IDs the old way (without the leading zeros) or new way (with the leading zeros) and filer on the correct sensor ID regardless.
  • Fix for random crash when clicking the Save button in the Settings screen while the app is processing data in the background.
  • Settings form can now be accessed via a single click. I got rid of the "Edit" menu since "Settings" was the only submenu item on it.
  • FUTURE ENHANCEMENT COMING SOON: Move the settings file to the same directory as the application - this way you won't have to keep reentering your settings when you upgrade.

Latest version is 2016.09.10.0203 and includes the following changes:

  • Exclusively supports the new My AcuRite smartHUB / Acu-Link with the latest My AcuRite smartHUB firmware
  • CWOP / APRS users can now enter a custom comment to be included in their weather data transmission
  • Vastly enhanced Debug Mode to aid in troubleshooting
  • Fix for Settings not saving
-----

Latest version is 2015.11.9.1951  and includes the following new features/fixes:

  • Much improved error handling of corrupted WindData and RainData files. The app will now automatically delete and recreate these files when they are corrupted - no more endless error messages popping up.
  • Adjustable time delay for posting to CWOP
  • CWOP now posts data via port 14580 for better performance
  • Added the ability to post to CWOP via ARPS ham (amateur) radio network
  • Main screen contains an error counter to keep track of posting errors. NOTE: Do not be concerned with this value unless it's really high - sometimes one or more weather servers are simply unavailable or bogged down for a moment or two.
Fixes and enhancements from previous versions:

  • Fix for temperatures between -1 and -9 being incorrectly formatted for CWOP
  • Fix for Wind Speed Offset
  • Fix for CWOP rain data reporting incorrectly
  • Fix for CWOP rain data not reporting
  • Truly fixed the memory leak issue. Removed auto-restart option since it's no longer needed.
  • App now minimizes to the system tray
  • Support for posing to CWOP (Citizen Weather Observer Program)
  • Support for posting to OpenWeatherMap
  • Ability to specify offset value for soil/water temperature
  • Have an extra sensor and want to monitor your pool, attic, garage, barn, or other location's temperature and humidity? You can now specify that a senor report it's temperature data as "Soil Temp" and humidity as "Soil Moisture".  To see these values on Weather Underground, drill down to the screen for your individual station (not the forecast screen).  NOTE: Set "Global Filter on Sensor" values to blank when using this option.
  • (Hopefully) fix for rainfall data when more than one sensor is used. Please report if you still find a problem, however.
  • Added Temperature Offset option to allow compensation for inaccurate thermometer readings.  Set this value to 0 if no adjustment is necessary.
  • Added Wind Offset option to allow compensation for inaccurate wind speed readings. This is a percentage value. Set to 100% if no adjustment is necessary, 50% to slow down reading, 200% to speed up, etc.
  • Have more than one sensor?  Now you can post specific data from multiple sensors to the weather sites. For example: Post the temperature from a tower sensor in the shade while posting wind, rain, and humidity data from a 5-in-1 sensor.
  • Customize the path and filename of the CSV output file
  • Bug fix for globally filtering on sensor type or sensor ID
  • (In previous update) Support for 5-in-1, 3-in-1, tower, and rain sensors

Download

My Weather Underground station (new location and new station ID)


If you really enjoy my app, donations are always appreciated.  You can donate to me via PayPal


-Kevin


Keywords: Acu-Link aculink weather weather underground rapid fire faster real-time CWOP  wunderground rapid-fire weatherbug anything weather PWS

Saturday, November 09, 2013

Scraper App: Weather and Coding Geek Project: Kevin's Acu-Link to Weather Underground Rapid Fire Updater

Scraper App: Weather and Coding Geek Project:  Kevin's Acu-Link to Weather Underground Rapid Fire Updater

Latest version: 2015-03-17 20:06

I recently ordered and received the Acu-Link Internet Bridge and couldn't wait to set it up. It was very easy to configure with my 5-in-1 (temperature, humidity, rain, wind speed, and wind direction) sensor.  Barometric pressure is reported by the internet bridge.  I then went to Acu-Link.com and could view my own weather report online.  I also installed the Acu-Link Android app.




The next thing I did was set up a free account on Weather Underground. I went back to Acu-Link.com, entered my Weather Underground station ID and password, and began observing my weather data on Weather Underground.  Click the station ID to view my weather station data in real time: KCARAMON46


The sensor transmits data ever 18 seconds. Weather Underground has a feature named Rapid Fire which refreshes real-time weather data every few seconds. I soon discovered, however, that I was only seeing updates on WU every 15 MINUTES.  I thought that something must be wrong, so I searched around the acurite.com forums and found out that Acu-Rite only posts the weather data to Weather Underground every 15 minutes.  How disappointing. People on the forums have been asking for more frequenting refreshing of the data for over a year now.

I knew not to expect Acu-Rite to increase the update frequency anytime soon, so I decided to come up with my own solution: write an app that would scrape the data from AcuLink.com and post it to Weather Underground.


First, I had to scrape the data from Acu-Link's website.  I wrote a VB.NET app that makes use of HTML Agility Pack libraries and good ol' RegEx to scrape the data from AcuLink.com.  Much of the data parsing involves simply scraping the text values from HTML. Wind direction is represented graphically, but fortunately the JPG images are named appropriately. In order to make the wind direction data compatible with Weather Underground, I had to convert the direction (N, S, SW, SSW, etc.) into degrees.  Also, the decimal value for temperature is represented in a different set of tags than the integer value.



In order to calculate rainfall rate (vs. cumulative rainfall for the last 24 hours), I had to save the total rainfall for the past hour - and keep repeating.  I made use of a DataTable for this.  I did something similar for calculating wind gusts (highest wind speed in the past 10 minutes).



Finally, I need to post the data to Weather Underground.  The upload protocol is surprisingly simple.  There's a wiki page that explains it all.

And I made use of a timer control to refresh the data at regular intervals.
So now I have the data posting to Weather Underground every 15 seconds instead of every 15 minutes.  If you'd like a copy of the app, let me know.




2/3/2014: Latest version now includes bug fix that allows you to use alarms on Acu-Link without causing the app to crash.

Download the latest version 2015-03-17 20:06 This version includes added support for the metric system and other improvements.