Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Numeric ranges #477

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Numeric ranges #477

wants to merge 7 commits into from

Commits on Jan 21, 2021

  1. clicktest: add a test for the clicktest script 

    None of the existing ones test anything related to regexes. I'm about
to make a bunch of changes and wanted to make sure I don't break the
`%expect -w` behaviour for regexes.

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    17e0f58 View commit details
    Browse the repository at this point in the history

  2. clicktest: parse nested double braces (roughly a no-op) 

    This change prepares for my next commit in which I'll be making use of
nested double braces. The biggest part of the change is that it
replaces regex based matching with explicit matching. I don't think
it's possible to implement this based on a single somewhat readable
regex.

I also added a unit test which can be run using `clicktest
--self-test-mode`.

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    f173566 View commit details
    Browse the repository at this point in the history

  3. clicktest: add support for integer range expressions 

    This change adds support for three types of integer range expressions
in %expect and %ignore sections. Here is an example of each type:

{{~ 5 - 15}}    : allow integers from 5 to 15 (inclusive)
{{~ 100 +- 5}}  : allow integers from 95 to 105 (inclusive)
{{~ 100 +- 5%}} : allow integers approximately 5% from 100

These can be used bare as well as inside double-braced regexes (and
similarly inside %expectx and %ignorex sections) such as:
  {{ \d+ buckets? for {{~10 +- 10%}} euros }}

I am hoping this will help developers write tests that are robust
against noise by making it easy to express ranges of permitted values.
Without this, developers need to handcraft equivalent regular
expressions, which is cumbersome and hard to read, e.g.:
{{ 9[5-9]|10[0-5] }}

The current implementation only supports non-negative integers. A
future change could extend this to negative numbers and floats.

I added an algorithm that converts a numeric range to a regex.
Ideally we'd be able to check ranges without using regexes but that
would require a significant rework of clicktest. It was easier to write
the algorithm. The algorithm runs in log time.

Measured performance impact of this and the previous change combined
by running the set of Meraki click tests that take less than 0.2
seconds each (according to parallel's joblog) in serial. I expect
these to have more overhead from the clicktest script than longer
running tests. As follows:

The following alternates between a revert of this commit and this commit:
```
git reset --hard 6a62887e4bd
while :; do
  git rv 6a62887e4bd b23972e0b1d
  git log --oneline HEAD~..HEAD >>before.txt
  (time for t in `perl -ane 'print if $F[3] <0.2' joblog.sorted.txt | awk '{print $11}'`; do ./scripts/clicktest $t; done) 2>&1 1>&2 |grep ^real >>before.txt
  git reset --hard HEAD~~
  git log --oneline HEAD~..HEAD >>after.txt
  (time for t in `perl -ane 'print if $F[3] <0.2' joblog.sorted.txt | awk '{print $11}'`; do ./scripts/clicktest $t; done) 2>&1 1>&2 |grep ^real >>after.txt
done
```

After 328 iterations, I measured an average runtime of 2.633s (before this
commit) and 2.655s (after), with a standard deviation of ~6.5%. This represents
an increase of 0.84%.

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    4762a48 View commit details
    Browse the repository at this point in the history

  4. clicktest: vary_from_index: abbreviate single-digit regex ranges 

    E.g. '[3-3]' can be replaced by just '3'. Making this change to make
generated regexes a bit more readable.

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    391478c View commit details
    Browse the repository at this point in the history

  5. clicktest: add algorithm to convert floating point ranges to regexes 

    ..in preparation for handling floating point range expressions (next commit).

This is mostly a no-op but not quite: integer ranges are handled as a special
case of numeric (float/int) ranges, which support leading zeros. As a result
integer range now accept leading zeros as well (as they should have before).

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    4deba27 View commit details
    Browse the repository at this point in the history

  6. clicktest: add support for floating point range expressions 

    Previously, I added integer range expressions. In this commit, I expand
them to "numeric range expressions", which support integer and float
ranges. The change in format is pretty minimal:  if any of the numbers
in the format contains a decimal point, floats and integers are
accepted; otherwise only integers are accepted (as before). Here is an
example of the float variant of each type:
  {{~ 5.0 - 15 }}   # note: equivalent to {{~ 5 - 15.0 }}
  {{~ 100 +- 5.0 }}
  {{~ 100 +- 5.0% }}

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    b82b962 View commit details
    Browse the repository at this point in the history

  7. clicktest: update doc for numeric ranges 

    I manually updated the pod documentation inside the clicktest script and
then generated clicktest.1 using `pod2man -d '' -c ''` as specified in
doc/Makefile.in. A bigger diff resulted than I expected, but I believe
this is the correct procedure.

Reviewed-by: Yoann Desmouceaux <[email protected]>
    Patrick Verkaik committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    53edcf9 View commit details
    Browse the repository at this point in the history