PWS Station total precipitation question


#1

Hi there! For Christmas I received an acurite 5-in-1 weather station that I’ve been itching to link to my gen 2 rachio. Since I live in Edmonton, the winter delayed my fun, but yesterday was my first true test of rachio and PWS working together.

Background:
Since acurite doesn’t upload to PWS, I need some middle ground. I am uploading to wunderground and then using a python script that runs every 10 minutes to grab the .json data and push to PWSweather.com.

As you can see from the chart on the bottom of PWS, in total yesterday my station received 1.63 in of rain. The very last reading is 1.16 in which is odd.

This morning I woke up and was exited to see my rachio match some of these numbers, but alas I this is what I saw:

Where does 0.43 in come from? For a SCADA junkie like me, I would love to see more logging under the hood of the values it is pulling from PWS/Aeris. I keep a log of all the reads and writes with my python script so I can share that if there is something there.

Thanks for helping me through this fun journey!

Update
Now that my rain delay is over, my flex daily schedule has calculated and it has the same incorrect information. Where did 0.43in come from??


#2

Hey there fellow developer :slight_smile: I’m working on a similar fix using PHP (link). It is a work in progress, but may help you avoid some of the issues I had to deal with regarding data from WeatherUnderground (such as the spike within your data on May 22nd).

I can’t comment on how the data from pwsweather was interpreted by Rachio’s data source, perhaps someone from rachio’s team can be of more assistance.

Cheers,
Gene


#3

Oh man, that data spike really frustrates me. I have some logic to see if it is NULL or is ridiculously high and then I toss it out. Next steps were to calculate rate of change and see if the value is much higher or lower than the previous values.

I will take a look at your PHP, that is awesome that others are working on this as well!


#4

That data spike is due to old data being reported by weather underground. The fix I came up with was to check the age of the data, and filter out overly old data (anything older than 33 minutes).

I’ve seen weather underground reporting data anywhere from a little over an hour in the past to almost 58 days in the past.
Look for the fix on the lines 39 and 41.

Cheers,
Gene


#5

Ohh! I see it now. I have all the .json files stored and the unix epoch time at May 22 01:00 does not match the actual time. Way to go wunderground

-rw-r–r-- 1 admin administ 2.5k May 22 01:00 wu.json.1495411637
-rw-r–r-- 1 admin administ 2.5k May 22 00:00 wu.json.1495432801
-rw-r–r-- 1 admin administ 2.5k May 22 00:10 wu.json.1495433393
-rw-r–r-- 1 admin administ 2.5k May 22 00:20 wu.json.1495433985
-rw-r–r-- 1 admin administ 2.5k May 22 00:30 wu.json.1495434577
-rw-r–r-- 1 admin administ 2.5k May 22 00:40 wu.json.1495435169
-rw-r–r-- 1 admin administ 2.5k May 22 00:50 wu.json.1495435798
-rw-r–r-- 1 admin administ 2.5k May 22 01:10 wu.json.1495436982

Nice. Thanks… now to figure out where I get 0.43in and why the last value of the day on pwsweather is different.

I should adjust my cron to not occur at 00:00 perhaps. Maybe every 10 minutes on the 5. (23:05… 23:55)


#6

Seems that data, which is almost 7 hours old, is as good as new :wink:


#7

By the way, @tydollasign what is your setup? You mentioned python, what hardware are you running it on?

I’ve developed my PHP code to run on a generic web host, I also host wufyi (link), available to others.

Cheers,
Gene

P.S. administ is perhaps the best admin group name I’ve ever seen. :sunglasses:


#8

Data would make sense if Racho’s 4AM corresponded to 10AM on the data. At 09:59 AM rainfall was 0.43 in


#9

Haha, “administ” is cut off representation of administrators group. That is running on my QNAP NAS as a python cron job.

I see that .43 matches with 9:59AM which is interesting. I’m looking at past days data and a lot of it seems off with what Rachio thinks.

For example: This is last week. See the precip on May 19th is 0.07in

Yet the precip on May 19 according to pwsweather.com is 0.00 in. The day before, May 18th, was 0.07. It’s like Rachio seems to be grabbing 24 hours in different increments? The rain on May 18th didn’t start until 5PM MDT


#10

We are uploading the data to pws with &dateutc. I’m assuming that observation_epoch is in utc. And then when you calc the diff, your time() function also returns utc?

Here is my log from 10:00:02 MDT (15:59 UTC)

Wed May 24 10:00:02 MDT 2017
Grabbing JSON using the following URL: http://api.wunderground.com/api/xx/conditions/q/pws:IEDMONTO33.json
PWSDATEUTC=2017-05-24+15%3A59%3A50
PWSEPOCH=1495641590
PWSWINDDIR=0
PWSWINDSPEEDMPH=4
PWSWINDGUSTMPH=0
PWSTEMPF=49.2
PWSRAININ=0.00
PWSDAILYRAININ=0.43
PWSBAROMIN=29.10
PWSDEWPTF=49
PWSHUMIDITY=98

And here it is from the end of the day:

Wed May 24 23:50:01 MDT 2017
Grabbing JSON using the following URL: http://api.wunderground.com/api/xxx/conditions/q/pws:IEDMONTO33.json
PWSDATEUTC=2017-05-25+05%3A49%3A27
PWSEPOCH=1495691367
PWSWINDDIR=225
PWSWINDSPEEDMPH=1
PWSWINDGUSTMPH=0
PWSTEMPF=48
PWSRAININ=0.00
PWSDAILYRAININ=1.63
PWSBAROMIN=29.58
PWSDEWPTF=46
PWSHUMIDITY=96

It did not rain today. I’m curious to see if tomorrow I have 1.63-0.43 = 1.2 in for May 25th.

UPDATE
Nope, fail. This is frustrating


#11

@franz. I noticed you were able to assist with another PWSweather.com issue. Can you please look into this one?

On Wednesday May 24th, my PWS reported 1.63in of rain:
https://www.pwsweather.com/archivewx.php?id=EDMRHAT&d=20170524&t=day

Yet the next day, my flex schedule identified 0.43in only.

In that other article you mentioned to only focus on the summary total, which I did and it also says 1.63in. Are you able to share the json for that day for my PWS?

Thanks,


#12

I think I figured out where the issue might be, you were on the right track before @Gene ! For some reason, the Aeris/PWS weather API doesn’t set my city correctly:

This is my PWS: EDMRHAT
And this is the API call from Aeris to get data from that PWS: JSON blob

Now, this is another PWS from my city. Notice how the LAT/LONG are pretty much the same: ALBERTA290
Now this is the API results from Aeris to get data from this one: JSON blob

As you can see, it thinks I live in Russia!! In fact, it thinks my PWS is at POSITIVE 113 Longitude, not NEGATIVE 113. The time in Bagdarin Russia is +8, where as my time is -6. So when it is midnight in Bagdarin, it is 10AM my time.

I will attempt to adjust my lat/long to get the results I expect and not Russia. THen I’ll report back.


Linking Netatmo to Rachio
#13

Good catch, and definitely something to lookout for. Way to use a free Aeris developer account :slight_smile:


#14

I could not change it back to Edmonton, so I opened a ticket with pwsweather.com (Aeris) and they found the issue. When I first created the station, I neglected to put the negative in there. When I fixed it shortly after, it was too late. The system doesn’t update unless you force it.

Aeris has forced an update and now the API is showing correct results. Once it rains next (Thursday?) I’ll update this thread if my problem is solved!


Weather station accuracy?