Unable to download ERA5 data when running 03_Extracting_forcing_data.ipynb

[yfhuang7]

Generic Issue

• ravenpy version: 0.20.0
• Python version: python3.12
• Operating System: iOS

Description

I'm going through the notebook examples. I stuck in 03_Extracting_forcing_data.ipynb when downloading ERA5. It seems I can't link to Wasabi/Amazon S3. Do you have any suggestions for how I can continue? Thank you!

What I Did

# Get the ERA5 data from the Wasabi/Amazon S3 server.
catalog_name = "https://raw.githubusercontent.com/hydrocloudservices/catalogs/main/catalogs/atmosphere.yaml"
cat = intake.open_catalog(catalog_name)
ds = cat.era5_reanalysis_single_levels.to_dask()```

What I Received

---------------------------------------------------------------------------
gaierror                                  Traceback (most recent call last)
File ~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1562, in TCPConnector._create_direct_connection(self, req, traces, timeout, client_error)
   1558 try:
   1559     # Cancelling this lookup should not cancel the underlying lookup
   1560     #  or else the cancel event will get broadcast to all the waiters
   1561     #  across all connections.
-> [1562](https://file+.vscode-resource.vscode-cdn.net/Users/yfhuang/Downloads/RavenPy-main/docs/notebooks/~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1562)     hosts = await self._resolve_host(host, port, traces=traces)
   1563 except OSError as exc:

File ~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1178, in TCPConnector._resolve_host(self, host, port, traces)
   1177 try:
-> [1178](https://file+.vscode-resource.vscode-cdn.net/Users/yfhuang/Downloads/RavenPy-main/docs/notebooks/~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1178)     return await asyncio.shield(resolved_host_task)
   1179 except asyncio.CancelledError:

File ~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1562, in TCPConnector._create_direct_connection(self, req, traces, timeout, client_error)
   1558 try:
   1559     # Cancelling this lookup should not cancel the underlying lookup
   1560     #  or else the cancel event will get broadcast to all the waiters
   1561     #  across all connections.
-> [1562](https://file+.vscode-resource.vscode-cdn.net/Users/yfhuang/Downloads/RavenPy-main/docs/notebooks/~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1562)     hosts = await self._resolve_host(host, port, traces=traces)
   1563 except OSError as exc:

File ~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiohttp/connector.py:1153, in TCPConnector._resolve_host(self, host, port, traces)
   1152 try:
...
--> [284](https://file+.vscode-resource.vscode-cdn.net/Users/yfhuang/Downloads/RavenPy-main/docs/notebooks/~/miniforge3/envs/CIROH_FFF/lib/python3.12/site-packages/aiobotocore/httpsession.py:284)     raise EndpointConnectionError(endpoint_url=request.url, error=e)
    285 except asyncio.TimeoutError as e:
    286     raise ReadTimeoutError(endpoint_url=request.url, error=e)

EndpointConnectionError: Could not connect to the endpoint URL: "https://s3.wasabisys.com/era5/world/reanalysis/single-levels/zarr/timeseries_real_time/.zmetadata"

[Zeitsperre]

Hi @yfhuang7!

I'm not certain as to why it wouldn't be working. As far as I know, that endpoint is openly-accessible, and all the necessary dependencies are listed in order to run the notebook. My only theory would be that your IP may be geoblocked. With a fresh installation, I have no errors.

Can you try performing a wget/curl request to https://s3.wasabisys.com/era5/world/reanalysis/single-levels/zarr/timeseries_real_time/.zmetadata to see what happens?