"distributed.worker - WARNING - Compute Failed" on simple example

[Atreyagaurav]

Hello.

I tried to use the library, the idea seems really nice. But I couldn't make it run.

What I tried was a simple example:

xarray_data = xarray_data_accessor.get_xarray_dataset(
    data_accessor_name='CDSDataAccessor',
    dataset_name='reanalysis-era5-single-levels',
    variables=['total_precipitation'],
    start_time='2023-03-01T00:00:00',
    end_time='2023-03-31T23:00:00',
    shapefile=basin,
)

Where basin is a GeoDataFrame created from HUC 05 in Watershel Boundary Dataset. But looking at the log below, it only has a bounding box so you can put that to test.

And I got multiple warning like this from all workers:

2024-02-05 16:03:59,655 - distributed.worker - WARNING - Compute Failed
Key:       _get_api_response-dc28667366caa21242385ca14afe7319
Function:  _get_api_response
args:      ({'year': ['2023'], 'month': ['03'], 'day': ['08', '09', '10', '11', '12', '13', '14'], 'time': ['00:00', '01:00', '02:00', '03:00', '04:00', '05:00', '06:00', '07:00', '08:00', '09:00', '10:00', '11:00', '12:00', '13:00', '14:00', '15:00', '16:00', '17:00', '18:00', '19:00', '20:00', '21:00', '22:00', '23:00'], 'product_type': 'reanalysis', 'variable': 'total_precipitation', 'format': 'netcdf', 'grid': [0.25, 0.25], 'area': [35.31331634622836, -89.2663719739644, 42.4498337184844, -77.83937040107634], 'index': 1})
kwargs:    {}
Exception: "TypeError('expected str, bytes or os.PathLike object, not int')"

No data is returned from it, and the python call errors out. If you have experienced it let me know. I'm using the copernicus data, and I have my account info into the rc file.

I did install the library using pip install -e . not conda, so that could be something different. And I saw in the readme that the "support pip" is not checked out.

Is this something that you guys have seen before? And is this the reason the pip install is not ticked? Meaning, I should use conda. I don't use conda on anything, so if you guys want to make it work with pip, and there are issues, I can help with trying to figure that out as well.

But this looks like an issue of calling functions with different type parameter. Not an issue that's from pip vs conda (unless there is different versions of libraries causing issues). Only int in that args is index at the end, and the error could mean that the library was supposed to send a string or path there. The API changed? or the worker function changed and they are out of sync with the library function calling it. So, I made this issue.