Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cached namespaces during validation #531

Open
wants to merge 4 commits into
base: dev
Choose a base branch
from

Conversation

stephprince
Copy link
Contributor

Motivation

Fix #528.

Recent Zarr-related updates switched the inspector from using pynwb.validate(paths=paths) to pynwb.validate(io=io), resulting in the inspector not using cached namespaces during validation. We will temporarily revert this change, since pynwb validation of Zarr NWB files is not yet fully supported anyway due to pending issues that need to be addressed in hdmf/pynwb.

Since this is more of a bug on the pynwb side that using the io argument ignores the use_cached_namespaces=True setting in the validator (and will be updated in the next major pynwb release), I'm not sure if there are really any specific tests to add here.

While addressing this issue, I added a couple of other small updates to the Zarr support here:

  1. Passing the path of an NWB file Zarr folder to the CLI (instead of the parent folder) would result in 0 files being inspected, due to some conditionals in inspect_all that detected whether the path provided was a directory of NWB files or an NWB file.
  2. The test_inspect_all_parallel tests had been indented and were not running

@@ -126,7 +126,9 @@ def inspect_all(
if progress_bar_options is None:
progress_bar_options = dict(position=0, leave=False)

if in_path.is_dir():
if in_path.is_dir() and (in_path.match("*.nwb*")) and (in_path / ".zgroup").exists():
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if (in_path / ".zgroup").exists() is the appropriate check here for a Zarr NWB file or if it is even necessary

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stephprince
Copy link
Contributor Author

@rly you mentioned you had a related test file, can you let me know if this solves the issue or share the file?

@stephprince stephprince marked this pull request as ready for review October 24, 2024 00:32
@stephprince stephprince requested a review from rly October 24, 2024 00:32
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.71%. Comparing base (c26924b) to head (3efd7b8).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #531      +/-   ##
==========================================
+ Coverage   82.77%   86.71%   +3.94%     
==========================================
  Files          47       47              
  Lines        1504     1506       +2     
==========================================
+ Hits         1245     1306      +61     
+ Misses        259      200      -59     
Files with missing lines Coverage Δ
src/nwbinspector/_nwb_inspection.py 87.31% <100.00%> (+4.73%) ⬆️

... and 4 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: PyNWB validator not using cached schema
3 participants