Thursday, 14 May 2015

Generate a kml file from Google results


Is it possible to generate a kml file of searched items in Google Maps. For example:



  • Search for highschools in Virginia

  • Return the results as a kml file with the schoolname address and location




Answer



I can't quite figure out what the second set of coordinates are in the search string Mapperz provided, but if you look at the page source you can see urls for subsequent pages in the maps search


so for the first 10 results


http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=high+schools+in+Virginia&sll=37.0625,-95.677068&sspn=40.409448,92.109375&ie=UTF8&cd=1&hq=high+schools&hnear=Virginia&sa=N&start=0


results 11 to 20


http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=high+schools+in+Virginia&sll=37.0625,-95.677068&sspn=40.409448,92.109375&ie=UTF8&cd=1&hq=high+schools&hnear=Virginia&sa=N&start=10


results 21 to 30


http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=high+schools+in+Virginia&sll=37.0625,-95.677068&sspn=40.409448,92.109375&ie=UTF8&cd=1&hq=high+schools&hnear=Virginia&sa=N&start=20


etc.



You could build a scraper with this (just add the &output=kml to the end of the string as Mapperz suggested, it appears the coordinates are only in the kml file, not in the original url source) but the problem is that the query itself only returns at max 200 results. You could add to this a spatially varying search term (i.e. plus or minus the latitude and longitude) to cover the whole state. (Also I thought at first you could just save all of the data if you used Google Earth directly, but this is not the case and it is exactly the same as using the Google Maps search.) I wouldn't be surprised if there is more magic in the url that you could use to your advantage in building a scraper that I am unaware of.


This map comparing density of grocery stores to bars uses data collected from Google, so maybe you could ask the makers how they pulled the data.


No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...