Rank119
Idea#56
I wrote a python script to download all the census tract data for four different counties, parse it and save it as csv to upload to our database.
it works great! except that sometimes only the first and fourth queries return data.
Is there limit on how fast we can fire off queries to the API?
I haven't actually tested the response, I'm using python's urllib for the first time, and I haven't played around with it's responses yet. I'll let you know on Tuesday, when I work on this next!
PS any word on whether the 2011 data will be available through the API on Dec 6th?
:-)
For me the issue dealt with multiple query functions running as a single batch function using XDomainRequests. F12 debug showed all requests to the remote server would abort but the last. Running with alert(query) before each query worked. So I ended up moving my XDR invocation (e.g. new window.XDomainRequest) inside the loop so a new instance would be created for each query to the remote server. Each result is inserted into an different input box via getElementById(id).value. With a delay timer added I can see it runs sequentially filling in each value now with no issues. Without delay it is virtually instantaneous.
it turns out that the the order that I run the queries in matters!
if I run them in one order, I only get data for the first and fourth counties.
If I change the order so that the last county is run first, I get data for all four counties.
Weird!
I'll be happy to post my python code, if you'd like to test it yourself.
here's the list of queries in my python script that get appended to the baseUrl:
counties = ['B19013_001E,B19013_001M&for=tract:*&in=state:53+county:011',
'B19013_001E,B19013_001M&for=tract:*&in=state:41+county:005',
'B19013_001E,B19013_001M&for=tract:*&in=state:41+county:051',
'B19013_001E,B19013_001M&for=tract:*&in=state:41+county:067'
]
