Skip to content
This repository has been archived by the owner on May 4, 2021. It is now read-only.

Ticket30406 refactor header constants #357

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions sbws/lib/stem_bandwidth_file.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
"""Stem's Bandwidth file parser part.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is this the right way to do this? Just include a file from stem? Doesn't this mean we need to keep track of stem's revisions to this file? :/

Also if this is the right way to do this, then perhaps we should mention which revision of this file this is (commit id). Also, why does it say "To be removed on Stem's release 1.8.0"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is this the right way to do this? Just include a file from stem? Doesn't this mean we need to keep track of stem's revisions to this file? :/

Only until the next release, i think there're stem's releases every ~6 months.
And so far this file has only changed when after adding more headers to sbws i create an issue in stem so it gets added too.
With this change, instead i could directly create a PR in stem and duplicate it in sbws until the next release.

Not sure is there's a better way, but i definitely don't want to wait 8 months again to do a sbws release waiting for a stem release and it's not a solution to change the dependency to stem's git master because that does not work for system packages (Debian).

Also if this is the right way to do this, then perhaps we should mention which revision of this file this is (commit id).

Do you think it would be better i just add the stem's bandwidth_file.py as it is currently?.
I can add the commit id (in v3bwfile if i add the stem's bandwidth_file.py as it's)

Also, why does it say "To be removed on Stem's release 1.8.0"?

Because that's the next Stem's release, i can add this comment somewhere. Where it would fit better?.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i add the file as it is, i also need to copy the __init__.py
Thinking more about this, i think it's better to leave only that part, to follow better what i substitute.
I could also do a try import and if there's no bandwidth_file.py in stem then leave the old code + the refactor.
But i think duplicating all that code when as i said, the file won't change unless someone open a ticket, and there'll be a new stem release soon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See small changes in fixup

To be removed on Stem's release 1.8.0."""

# Converts header attributes to a given type. Malformed fields should be
# ignored according to the spec.

def _str(val):
return val # already a str


def _int(val):
return int(val) if (val and val.isdigit()) else None


def _date(val):
try:
return stem.util.str_tools._parse_iso_timestamp(val)
except ValueError:
return None # not an iso formatted date


def _csv(val):
return map(lambda v: v.strip(), val.split(',')) if val is not None else None


# mapping of attributes => (header, type)

HEADER_ATTR = {
# version 1.1.0 introduced headers

'version': ('version', _str),

'software': ('software', _str),
'software_version': ('software_version', _str),

'earliest_bandwidth': ('earliest_bandwidth', _date),
'latest_bandwidth': ('latest_bandwidth', _date),
'created_at': ('file_created', _date),
'generated_at': ('generator_started', _date),

# version 1.2.0 additions

'consensus_size': ('number_consensus_relays', _int),
'eligible_count': ('number_eligible_relays', _int),
'eligible_percent': ('percent_eligible_relays', _int),
'min_count': ('minimum_number_eligible_relays', _int),
'min_percent': ('minimum_percent_eligible_relays', _int),

# version 1.3.0 additions

'scanner_country': ('scanner_country', _str),
'destinations_countries': ('destinations_countries', _csv),

# version 1.4.0 additions

'time_to_report_half_network': ('time_to_report_half_network', _int),

'recent_stats.consensus_count': ('recent_consensus_count', _int),
'recent_stats.prioritized_relay_lists': ('recent_priority_list_count', _int),
'recent_stats.prioritized_relays': ('recent_priority_relay_count', _int),
'recent_stats.measurement_attempts': ('recent_measurement_attempt_count', _int),
'recent_stats.measurement_failures': ('recent_measurement_failure_count', _int),
'recent_stats.relay_failures.no_measurement': ('recent_measurements_excluded_error_count', _int),
'recent_stats.relay_failures.insuffient_period': ('recent_measurements_excluded_near_count', _int),
'recent_stats.relay_failures.insufficient_measurements': ('recent_measurements_excluded_few_count', _int),
'recent_stats.relay_failures.stale': ('recent_measurements_excluded_old_count', _int),
}
3 changes: 3 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ python =
3.5: py35, inst, setup, unit, integration, lint, doc
3.6: py36, inst, setup, unit, integration

[flake8]
exclude = sbws/lib/stem_bandwidth_file.py

[testenv]
# install_command can be removed when --process-dependency-links is not
# needed anymore, and this section commented
Expand Down