Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: 'extra_metadata' when running example "setup_images_to_stack.ipynb" #36

Open
byee4 opened this issue Feb 28, 2024 · 1 comment

Comments

@byee4
Copy link

byee4 commented Feb 28, 2024

🐛 Bug

KeyError: 'extra_metadata' when running example "setup_images_to_stack.ipynb"

To Reproduce

Steps to reproduce the behavior:

  1. Install punctatools
  2. Run example notebook "setup_images_to_stack.ipynb"
The following code returns the number of stacks that were found
If executing this cell results in an error, make sure the channel and z-position codes are provided correctly. Note that the codes are case-sensitive.
#################################
#  Don't modify the code below  #
#################################
​
n = get_number_of_stacks(input_dir, channel_code=channel_code, z_position_code=z_position_code)
n
/opt/conda/lib/python3.9/site-packages/pandas/core/dtypes/common.py:1645: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  npdtype = np.dtype(dtype)
/opt/conda/lib/python3.9/site-packages/pandas/core/dtypes/common.py:1645: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  npdtype = np.dtype(dtype)
/opt/conda/lib/python3.9/site-packages/pandas/core/dtypes/common.py:1645: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  npdtype = np.dtype(dtype)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Input In [7], in <cell line: 5>()
      1 #################################
      2 #  Don't modify the code below  #
      3 #################################
----> 5 n = get_number_of_stacks(input_dir, channel_code=channel_code, z_position_code=z_position_code)
      6 n

File /opt/conda/lib/python3.9/site-packages/punctatools/lib/convert.py:130, in get_number_of_stacks(input_dir, channel_code, z_position_code)
    109 """
    110 Return the number of stacks in the dataset.
    111 
   (...)
    127 
    128 """
    129 src = __get_source(input_dir, channel_code, z_position_code)
--> 130 n_stacks = intake_io.imload(src, metadata_only=True)['npartitions']
    131 return n_stacks

File /opt/conda/lib/python3.9/site-packages/intake_io/io.py:50, in imload(uri, partition, metadata_only, **kwargs)
     48 if isinstance(uri, intake.DataSource):
     49     if metadata_only:
---> 50         return uri.discover()
     51     out = _to_xarray(uri, partition=partition)
     52     if not isinstance(out, xr.Dataset):

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:155, in DataSourceBase.discover(self)
    153 def discover(self):
    154     """Open resource and populate the source attributes."""
--> 155     self._load_metadata()
    157     return dict(
    158         dtype=self.dtype,
    159         shape=self.shape,
    160         npartitions=self.npartitions,
    161         metadata=self.metadata,
    162     )

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:84, in DataSourceBase._load_metadata(self)
     82 """load metadata only if needed"""
     83 if self._schema is None:
---> 84     self._schema = self._get_schema()
     85     self.dtype = self._schema.dtype
     86     self.shape = self._schema.shape

File /opt/conda/lib/python3.9/site-packages/intake_io/source/filepattern.py:255, in FilePatternSource._get_schema(self)
    254 def _get_schema(self) -> Schema:
--> 255     header = self._files.get_file_metadata()
    256     metadata = header["metadata"]
    257     shape = self._set_shape_metadata(self._files.axes, self._files.shape, metadata["spacing"], metadata["spacing_units"], self._files.coords)

File /opt/conda/lib/python3.9/site-packages/intake_io/source/filepattern.py:188, in FilePattern.get_file_metadata(self)
    186 file = self.files.iloc[0]["file"]
    187 if self.source is None:
--> 188     with autodetect(file) as src:
    189         return src.discover()
    190 else:

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:245, in DataSourceBase.__enter__(self)
    244 def __enter__(self):
--> 245     self._load_metadata()
    246     return self

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:88, in DataSourceBase._load_metadata(self)
     86 self.shape = self._schema.shape
     87 self.npartitions = self._schema.npartitions
---> 88 self.metadata.update(self._schema.extra_metadata)

File /opt/conda/lib/python3.9/site-packages/intake/source/base.py:18, in Schema.__getattr__(self, item)
     17 def __getattr__(self, item):
---> 18     return self[item]

KeyError: 'extra_metadata'

Expected behavior

image

Environment

  • Operation System: Linux
  • conda version (conda --version): conda 23.1.0
  • pip version (pip --version): pip 22.0.4 from /opt/conda/lib/python3.9/site-packages/pip (python 3.9)
  • punctatools version: 0.2.0
  • Any other relevant information:
    environment.txt

Additional context

@lsxucsf
Copy link

lsxucsf commented Mar 8, 2024

I'm also receiving this error, albeit in the setup_roi_segmentation.ipynb. This error is also occurring when I use the example data stacks, so I imagine it's not intrinsic to my data. Any suggestions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants