-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: extract trend from signal #8
base: main
Are you sure you want to change the base?
Conversation
…dsl into vebjorn/add-hilbert-huang
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #8 +/- ##
===========================================
- Coverage 91.22% 70.03% -21.19%
===========================================
Files 103 104 +1
Lines 3770 3958 +188
Branches 815 851 +36
===========================================
- Hits 3439 2772 -667
- Misses 207 1056 +849
- Partials 124 130 +6
|
from .simple_filters import status_flag_filter | ||
from .wavelet_filter import wavelet_filter | ||
|
||
|
||
TOOLBOX_NAME = "Filter" | ||
|
||
__all__ = ["wavelet_filter", "status_flag_filter"] | ||
__all__ = ["hilbert_huang_transform", "wavelet_filter", "status_flag_filter"] | ||
|
||
__cognite__ = ["wavelet_filter", "status_flag_filter"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add it here as well
|
||
|
||
def generate_synthetic_signal(): | ||
wave = sine_wave( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add the tests that you removed, they are testing different types of signals
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The algorithm should work with different signals
|
||
# DROP_SENTINAL is used to get the smallest (most negative) | ||
# number that can be represented with a 32-bit signed integer. | ||
DROP_SENTINAL = np.iinfo(np.int32).min |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you rename it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And use it directly in the code
error_tolerance: float = 0.05, | ||
return_trend: bool = True, | ||
) -> pd.Series: | ||
r"""Perform the Hilbert-Huang Transform (HHT) to find the trend of a signal. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can look at the description from removed function to get some idea how to write documentation.
You dont need to explain what different type of signals are...
Trend extraction by the use of the Hilbert-Huang Transform.
PROBLEM: Some of the tests fail. This could be because we're calculating the Hilbert spectrum and not the Hilbert marginal spectrum.
Description
This function extracts the trend of a signal by the use of the Hilbert-Huang Transform. This is a powerful tool in the sense that it can take any non-stationary and non-linear time series and find the trend even though there's lots of noise in the signal. The package PyEMD is used which does not have a function to find the Hilbert spectrum of the transformed signal. In order to compensate for this, a manual calculation of the Hilbert spectrum is performed. The algorithm is defined in https://wwaw.researchgate.net/publication/261234992_Trend_extraction_based_on_Hilbert-Huang_transform and it goes as follows:
Motivation and Context
It will be used to develop methods required to measure data quality.
How Has This Been Tested?
Generated synthetic signal with nonuniform timestamps, Gaussian noise and gaps, and compared with the obtained trend.
Screenshots:
Types of changes
Contributor Checklist:
fix: <description>
,feat: <description>
, etc.Reviewer Checklist for Charts compliant functions: