Using WUnderground.com to integrate Personal Weather Stations

@Gene. OK. Once again I’ve been away but I’ve just switched back. The v2 seems to have been working fine in the meantime - even the dew point readings settled down to something sensible after an hour or so. Only odd thing is that normally cron-job sends a message if a transfer fails. I’ve had none in the last 24 hours but inspecting the PWS data there is a mixture of 5 and 10 minute gaps between readings (see for instance 15 June 03.40am GMT 05.30am GMT etc) even though cron-job is set to 5 minute intervals.

Hi buddy,

Just FYI, I’m still testing two new approaches. I’ll see which one deserves a further study.

So far V2 provided unreliable, but not completely useless. I’ll see if I can integrate it as a double-check scenario
Test data available here: https://www.pwsweather.com/obs/WUSTATION.html
it is based on https://www.wunderground.com/personal-weather-station/dashboard?ID=KFLBOCAR130

I’m also testing V3 which uses a different method, also independent of API
Test data available here: https://www.pwsweather.com/obs/TSTSTATION.html
it is based on https://www.wunderground.com/personal-weather-station/dashboard?ID=KFLFORTP31

If any of these provide promising, I’ll send you an invite. Sorry I’m keeping address secret for now (/v2/ points to old code), but as you can see I’m testing several options.

Cheers,
Gene

3 Likes

Thanks @Gene, you’re a top man

Hi Gene, I’ve had your solution running for a few weeks now. Just about every day I get maybe one to five alerts from cron that there was a failure; avg twice per day. If I look soon enough at cron-job.org, I see it’s a 503 “server not available” error.
It doesn’t tell me any more than that, unfortunately.
And it typically succeeds on the next attempt.
So this isn’t urgent, for sure; but if it’s a sign of a growing load on your server, stuff like that, I figured I should alert you to it.

I doubt it’s useful, but just to be complete here are links to two screenshots of the logs:
https://drive.google.com/open?id=0B8Elw9AiB7ttSDZJc0hNc0dtNlk
https://drive.google.com/open?id=0B8Elw9AiB7ttMGVLVV9mNWpoX2s

Thanks for all you’re doing!

@knwpsk Thanks for letting me know, I’m so sorry that you are experience these persistent errors.

One of the issues of cron-job.org is that I have pretty much all of the users hit the server within a few seconds every 5 minutes, similar to your setup.

I’m currently trying to work something out with the programmers of cron-job.org:
https://github.com/pschlan/cron-job.org/issues/16

As a backup plan, I’m working on introducing internal load balancing and having everyone configure the cron-jobs to “every minute or two” setup.

For now feel free to setup yours to run “every 2 minutes”, wunderground API seems to be able to handle this without issues.

Cheers,
Gene

1 Like

Gene, you’re a hero for building this service in the first place! No need to apologize for my “experience” ! Thanks for being a playa.

OK Gene (or anyone) i’ve got a question now.
How much does the reporting interval matter to the Rachio controller?
That is, the controller looks for rainfall data as one input when it decides whether to water or delay watering. If I have the controller set up to look at my personal weather station (PWS) on the PWSWeather.com, using Gene’s script, does the data on PWSWeather include all of the cumulative info in between the times that we run the script? If I run the script every 10 mins, or every hour, will PWSWeather and my controller know how much rain fell in between the run times?

From what I know, Rachio is currently uses a daily summary data, similar to what you see via Rachio app whereas there is one value for all of the rainfall during a particular day. I know that Rachio is at least considering other weather data sources which would allow a more granular access to data.

While Rachio themselves do not make use of the individual update data-points, the weather service Rachio is using does have that access and probably uses it for Forecast evaluation. Theoretically the more data you can provide, the better everything will work. For example going from 15 minute updates to 10 minute updates may result in larger gain than from 10 minutes to 5, and going from 5 minutes to 2 probably doesn’t make much difference besides making sure that there will not be a 10 minute gap in case one of the updates fail.

Cheers,
Gene

I don’t think I asked my question very clearly. Retry:

If it’s raining 1" per minute for 20 minutes,
And I run the cron job > Gene’s script to copy data from Wunderground to PWSWeather every 10 minutes, so it runs 2x during the rain storm,
Will PWSWeather think it rained a total of 2", or 20" ?

If the answer is 2", then it matters a whole lot that I run the script more often.
If the answer is 20", meaning that each update is cumulative since the last update… then it doesn’t matter if I run the script once a minute or once a day, Rachio controller will end up with the same information.

Unlike temprature, pressure, etc… Rain data from wunderground is sent/recorded on an hourly basis and daily basis.

It should be sufficient to do one transfer at 1 minute to midnight to capture all of the rain data for that day as far as Rachio can use / display.

OK now I get it. Thanks again, Gene.
I set my cron to 2 mins, and it’s running fine for several hours now.

1 Like

Howdy. I’m seeing two very different numbers in my data. The Rachio app shows .09 inches of rain. However the weather station shows .69 (which is accurate). Any idea what the discrepancy is?

It seems that you’ve switched your PWS from Norwood to Buckingshop at or near the 30th, that is the only reason I could think of why a difference may be present.

You can try adjusting the moisture level to “Fill” all of your zones in order to account for missed rain data, let us know if something similar ever happens again.

Cheers,
Gene

1 Like

I’m using the wupws from @johnny2678, but I periodically during rain events get false data from WU and that affects my Rachio calculations. This is what I see from pwsweather - glitches in the total rain fall.

You can see that it reports 0.38" for one cycle, then back to 0.05". Unfortunately, Rachio appears to latch on to the bad data and use that. The station is here: https://www.pwsweather.com/obs/KCOTIMNA4.html.

@Gene, is this a case that WUFYI can avoid? If so I’ll likely switch.

@jcloudm It is hard to tell without access to debug data, as there are several reasons why such false data can occur, but seeing many spikes within your data does indicate that you are not protected from the most common issue with wunderground API: old data records.

I’ve implemented a fix in place for wufyi on March 26th (link), so you will definitely see a benefit from switching.

That being said, I am aware of a different issue which could result in false rain readings, though this is far more seldom and only a few people experience it, a fix for that issue is coming soon (I’m still testing it). I do not think that you are experiencing this particular issue, as it tends to effect several readings in a row.

So to answer your question, I’m pretty sure that wufyi is already avoiding this issue, but if it doesn’t, a solution is actively being developed.

Cheers,
Gene

Thanks! I’ve pulled it down from github and am just running it via php locally on a raspberry pi.

1 Like

Hi @Gene. I switched to your code overnight, but I am still seeing the same behavior:

I have all the logs from the cron, but there’s nothing other than the standard “The latest data from 4 seconds ago was transfered to PWS weather station KCOTIMNA4”.

Could it be something about the weather station itself reporting bad daily rain totals?

If you don’t mind, switch to wufyi for now. I have a debug capture running there so I’ll be able to better assert why the issue is occurring.

All set, running via cron-job.org. Thanks!

1 Like

It happened again about 50 minutes ago, if that helps you find the debug log: