JSONDecodeError: Expecting value: line 1 column 1 (char 0), when requesting data! the code works for a while then it breaks

I am trying to request geocoding data from the census bureau. The code runs and got more than 1450 records so far (my total is about 60K records), but then it breaks and returns this error :

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

My data looks like this: enter image description here

Here is my function:

def get_fips(df):  
    
    num=len(ven_lst)
    for i,e in df.itertuples(index=False):
        if e not in repostory_lst:
            try:
                num+=1
                address=i
                vendor=e
                link="https://geocoding.geo.census.gov/geocoder/geographies/onelineaddress?address={0}&benchmark=Public_AR_Census2020&vintage=Census2020_Census2020&layers=10&format=json".format(address)
                reponse = requests.get(link).text
                reponse_1=json.loads(response)
                x=reponse_1['result']['addressMatches'][0]['geographies']['Census Blocks'][0]['GEOID'][:11]

                fields=[num,e,x]
                with open(r'fibs&ven.csv', 'a',newline='') as f: #because my data is big a save all the #data into this csv incase the code breaks
                    writer = csv.writer(f)
                    writer.writerow(fields)
            except (RuntimeError, TypeError, NameError,IndexError):
                pass
        elif e in repostory_lst:
            pass
    #df_result=pd.DataFrame(columns=['Vendor Code','fips'],index=range(len(fips_lst)))
    #df_result['Vendor Code']=ven_lst
    #df_result['fips']=fips_lst
    #x.to_csv('fibs&ven.csv', mode='a', header=False)
    return None

1 answer

  • answered 2021-04-22 07:12 Nishant Patel

    Normally if the API on the server side is well written and you have specified format=JSON, it should strictly return JSON data only. But as a fail safe you should add a header to your get a json response.

    Also instead of text you can use json() and you need not to do json.loads

    Request like

     headers = {'Accept': 'application/json'}
     reponse_1 = requests.get(link, headers=headers).json()
     # reponse_1=json.loads(response)