Discussions

I previously downloaded data from this product last month that went up to May 2025 (max partition key of 202505). Now the latest partition key is Feb 2025. I downloaded the most files and confirmed that the latest DATE_RANGE_START is indeed Feb 2025.

Perhaps the more recent files still exist but something broke with how the get_file_list function runs? Here are steps to replicate:

Running:

import deweydatapy as ddp
apikey = '__KEY__'
pp = "https://app.deweydata.io/api/v1/external/data/fldr_wmhsgd6xbhuhzogpx"
ddp.get_file_list(apikey, pp, start_date = '2025-03-01')

gives the error:

File ~/.venvs/global_env/lib/python3.13/site-packages/deweydatapy/download.py:183, in get_file_list_full(apikey, product_path, start_page, end_page, start_date, end_date, meta, print_info)
    175 if(print_info == True):
    176     print(f"Collecting files information for page {res_json['page']}/{res_json['total_pages']}...")
    178 pages_meta = pd.concat([
    179     pages_meta,
    180     pd.DataFrame({
    181         'page': [res_json['page']],
    182         'number_of_files_for_page': [res_json['number_of_files_for_page']],
--> 183         'avg_file_size_for_page_MB': [res_json['avg_file_size_for_page'] / 1000000],
    184         'partition_column': [res_json['partition_column']]
    185     })], ignore_index=True)
    187 page_files_df = pd.DataFrame(res_json['download_links'])
    189 page_files_df.insert(loc=0, column='page', value=res_json['page'])

TypeError: unsupported operand type(s) for /: 'NoneType' and 'int'