Scraper App: Weather and Coding Geek Project: Kevin's Acu-Link to Weather Underground Rapid Fire Updater
I recently ordered and received the
Acu-Link Internet Bridge and couldn't wait to set it up. It was very easy to configure with my 5-in-1 (temperature, humidity, rain, wind speed, and wind direction) sensor. Barometric pressure is reported by the internet bridge. I then went to Acu-Link.com and could view my own weather report online. I also installed the Acu-Link Android app.
The next thing I did was set up a free account on Weather Underground. I went back to Acu-Link.com, entered my Weather Underground station ID and password, and began observing my weather data on Weather Underground. Click the station ID to view my weather station data in real time:
KCARAMON46
The sensor transmits data ever 18 seconds. Weather Underground has a feature named Rapid Fire which refreshes real-time weather data every few seconds. I soon discovered, however, that I was only seeing updates on WU every 15 MINUTES. I thought that something must be wrong, so I searched around the acurite.com forums and found out that Acu-Rite only posts the weather data to Weather Underground every 15 minutes. How disappointing. People on the forums have been asking for more frequenting refreshing of the data for over a year now.
I knew not to expect Acu-Rite to increase the update frequency anytime soon, so I decided to come up with my own solution: write an app that would scrape the data from AcuLink.com and post it to Weather Underground.
First, I had to scrape the data from Acu-Link's website. I wrote a VB.NET app that makes use of HTML Agility Pack libraries and good ol' RegEx to scrape the data from AcuLink.com. Much of the data parsing involves simply scraping the text values from HTML. Wind direction is represented graphically, but fortunately the JPG images are named appropriately. In order to make the wind direction data compatible with Weather Underground, I had to convert the direction (N, S, SW, SSW, etc.) into degrees. Also, the decimal value for temperature is represented in a different set of tags than the integer value.
In order to calculate rainfall rate (vs. cumulative rainfall for the last 24 hours), I had to save the total rainfall for the past hour - and keep repeating. I made use of a DataTable for this. I did something similar for calculating wind gusts (highest wind speed in the past 10 minutes).
Finally, I need to post the data to Weather Underground. The upload protocol is surprisingly simple. There's a
wiki page that explains it all.
And I made use of a timer control to refresh the data at regular intervals.
So now I have the data posting to Weather Underground every 15 seconds instead of every 15 minutes. If you'd like a copy of the app, let me know.
2/3/2014: Latest version now includes bug fix that allows you to use alarms on Acu-Link without causing the app to crash.
Download the latest version 2015-03-17 20:06 This version includes added support for the metric system and other improvements.