Discussions

Hi,

I just downloaded all the job records and job descriptions data from Dewey using ux. After examining the data, I found that there are a lot of empty or small Paraquet files. Could you verify these are corrupt files? If so, how should I proceed? Should I just re-download all the files from scratch again?

job-records: Out of 250 Parquet files, 28 are corrupt/empty placeholders (size ~4KB, missing parquet magic bytes), leaving 222 valid files.
job-descriptions: Out of 2,251 Parquet files, 1,658 are empty placeholders, leaving 593 valid files.

Thanks,

Jing