Scraping In The Name Of!

Data & Observations

Everything is looking great! We now have a single page we can load that combines 4 pages worth of data. There is no need to load ads or images, which saves time if loading from a mobile device. Now we know what the weather will be like in two locations, and it looks pretty darn fancy.

One thing to consider while scraping any website is the fact that certain variables (such as time of day in our case) may change the structure or content of what is displayed on the site. For instance, changes the text and structure of the HTML for the forecast page once the high has already been observed.

Forecast Elements Before High Observed
Forecast Elements After High Observed

Also, if the wind is calm, it does not display a wind speed value but rather the string "calm" in a completely different HTML element. Luckily, we looked at the site many times over the course of the day and caught these changes and handled them in our application. =D