Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[skip ci] Interpolate missing nominal values during Averaging #246

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

sjperkins
Copy link
Member

  • Tests added / passed

    $ py.test -v -s africanus

    If the pep8 tests fail, the quickest way to correct
    this is to run autopep8 and then flake8 and
    pycodestyle to fix the remaining issues.

    $ pip install -U autopep8 flake8 pycodestyle
    $ autopep8 -r -i africanus
    $ flake8 africanus
    $ pycodestyle africanus
    
  • Fully documented, including HISTORY.rst for all changes
    and one of the docs/*-api.rst files for new API

    To build the docs locally:

    pip install -r requirements.readthedocs.txt
    cd docs
    READTHEDOCS=True make html
    

@sjperkins sjperkins changed the title [skip ci] Interpolation missing nominal values during Averaging [skip ci] Interpolate missing nominal values during Averaging Apr 12, 2021
@sjperkins
Copy link
Member Author

@bennahugo would you consider giving this a quick review, mostly of the new test cases: test_interpolation and test_interpolation_edge? I'm less interested in review of the refactor, although you're welcome to provide feedback here too. Regardless I think the new test cases illustrate how the code now interpolates missing rows, as well as how it deals (imperfectly) with missing edge data.

Note that this doesn't change the BDA averaging code, yet. It only modifies the standard time and channel averaging code because it's far easier to reason about including X seconds worth of data in the bin. If you're happy with the standard case, I'll implement the same in the BDA code.

@sjperkins
Copy link
Member Author

Discussion Points

  • Regular/Irregular grid
  • Missing/Non-missing data

10 seconds bin
3 seconds
X is missing (not even flagged)
O is present (flagged/unflagged)

O O O X X X X O

| O O O | X X X X O
| O O O | X X X | X O | -- split samples across multiple bins

| O O O | X X X X O | -- current strategy, samples completely in/out of a bin

| O O O | X X X X | O |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant