Tb_DAH_scenarios #1032
Closed
Tb_DAH_scenarios #1032
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… methods pkgs in RF (#807) * co: get items from pkgs * add check for items not recogised. * improve message on failure of assert * test_co: test_contraception_coverage_with_use_healthsystem - if item codes not recognised warn & calculate average availability for the remaining item(s), check there are some to calculate it * RF_Consumables_Items_and_Packages based on orig code in contraception.py with some changes * RF_Consumables_Items_and_Pkgs: item 'Syringe, Autodisable SoloShot IX' back with 0 units to fix some tests (test_alri, ...) --------- Co-authored-by: Tim Hallett <[email protected]>
… to make robust to seed (#854)
…nted person to calibrate the use of U5Malnutr appointment (#831)
…regated reporting rates (#843)
Co-authored-by: Tim Hallett <[email protected]>
Co-authored-by: Tim Hallett <[email protected]> Co-authored-by: Asif Tamuri <[email protected]>
…d possibly result in non-emergency care (#776) Co-authored-by: Matt Graham <[email protected]> Co-authored-by: Bingling <[email protected]> Co-authored-by: Asif Tamuri <[email protected]>
…t issues with queue ordering (#883)
Co-authored-by: Matt Graham <[email protected]> Co-authored-by: Tim Hallett <[email protected]>
…mment-token step always runs (#921)
…QUEUE (#881) * Allow for HealthSystem to enact priority policy * Enforce priority policy and (optionally) fast-tracking * Invert topen and priority in queue to sort it faster * Priority and topen swapped in order. Added a rand_queue int for tie-break, followed by the queue_counter (for safery). This is creating problems with the NewBorn outcome modules so rand_queue is set equal to queue_counter for now. Some of the tests have been modified as no longer relevant, but double check this. * Style changes * Added tests. Events below priority threshold through never_ran(). Polished priority policy. * . * Update tests/test_alri.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update tests/test_healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update tests/test_healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Update src/tlo/methods/healthsystem.py Co-authored-by: Tim Hallett <[email protected]> * Optimised enforce_priority_policy by converting PriorityRank dataframe into dictionary, and by gathering attributes and fast-tracking channels that can be considered given the simulation set up (i.e. modules included) upon initialisation. Added a test specific to fast tracking routes in test_healthsystem.py. Addressed other minor comments to previous commit. * Ensure priority_policy returns valid priority even if fast tracking option is turned off * Merged ListFTAttributes and ListFTChannels into one list_fasttracking * Add TB test to check order with random queue is not important, fix test in healthsystem, modify check on priority value * Style adjustments to ordering of parameters and names * refactor PR -> pr for style guide * use list comprehension * use explicit name in special case for 'age' * elaborate docstring * Apply suggestions from code review * simplify construction of dict * Update test_tb.py add further description to test * Notes if want to reverse priority/topen order in queue * Reverse priority/topen order in queue back to original * Fixed style errors * test for determinism of the healthsystem (incl. consumables) * adding test_determinism_of_hsi_that_run_and_consumables * adding test_determinism_of_hsi_that_run_and_consumables and fix to use a different RNG for randomising the queue * change back to earlier test to original configuration now that issue has been solved * additional comments/stringency on tests * remove unused kwarg `priority_rank_dict` * seperate-out use of RNG for Consumables and DxTests * Fix flake8 errors * Make randomisation of queue false by default --------- Co-authored-by: Margherita Molaro <[email protected]> Co-authored-by: Tim Hallett <[email protected]> Co-authored-by: Tara <[email protected]>
* allow control of spurious_symptoms via parameter * add resourcefile * remove using kwargs in full model to determine spurious symptoms * set default of spurious_symptoms to False to preserve original behaviour * change the default for module_kwargs to be an empty dict * introduce `get_parameters_for_status_quo` to provide parameters that describe the SQ scenario * update comment * adapt `long_run_all_diseases.py` to use `get_parameters_for_status_quo` (and not kwargs to fullmodel) * create test
…ing up the number of IPA admission events affecting Bingling's calibration (#981)
…s_expected` (#991) * make bigger population size * remove `test_hsi_events_that_run_with_and_without_randomisation_are_as_expected`
* Make non-emergency care seeking occur at different levels and introduce parameter to healthcare seeking module to allow this to be determined probabilistically. * name chnages in other modules * linting * better description of parameter * add default for facility_level to enable legacy tests to pass * use of string in the resourcefile causes problems with read in of bool, use pd.eval * refactor for new names * introduce new test to protect new behaviour * update test in arli to make all non-emergency care start at level 0, for the purpose of the tests * make all non-emergency care go to level '0' to preserve original behaviour * make bigger population size * remove `test_hsi_events_that_run_with_and_without_randomisation_are_as_expected`
…#984) * Add an additional day to the t-close values of the two key labour HSIs * Set all 'treatment_effect_modifier...' parameters to 1.0 to have no effect * update t-close values for maternal PNC HSIs * update t-close values for maternal PNC HSIs * Remove the separate HSI which represented newborn resuscitation as it logically occurs within intrapartum care which is already a HSI * update test files to ensure new resuscitation logic is tested * fixed error in newborn_outcomes.py * fixed error in test_maternal_health_helper_and_analysis_functions.py * add new error message to newborn_outcomes.py, ensure only women who seek care first for birth could have their newborn resuscitated
…ent failure (#995) * create the parameter, with value 0.3, for probability of followup after treatment failure * fix typo * fix typo * update test_alri * Update src/tlo/methods/alri.py --------- Co-authored-by: Tim Hallett <[email protected]>
…Disorders` Module (#993)
* add up Delivery and Csection usage to Delivery to be mapped with tlo output * create a temporary scale run to check hsi calibration result * update temporary scale run scr * update temporary scale run scr * update residual_prob_caesarean value to improve Csection usage ratio * delete the temporary scale run file
… seeking (#973) * create the function frame * update function frame * create the resource file template * rename file * reformat in manner discussed with bingling * correct paths * reformat import * finish the resource file of params list * strengthen `test_switch_parameters` through combining with `test_get_parameter_functions` * refactor name of function for improved clarity * refactor filename to match name of function * refactor to move file out of healthsystem folder * introduce strong test for dtypes and shapes of dataframes/series * update excel file using 0.0001 for '0.0' and 0.9999 for '1.0' to prevent coercion to int by pd.read_excel and some other things picked up * options for bespoke handling of multiindex (neccesitated by the Shisto module) * save excel file again! * linting * explicitly set service availability in `get_parameters_for_status_quo` * define scenrios using new functions that serve parameters to change * helper function to mix scenarios together * typing hint * update scenarios to be run in scenario_impact_of_healthsystem.py * annotate excel file * linting * rename scenario * introduce the `ScenarioSwitcher` module to allow updating of parameters without passing pd.Series or pd.DataFrame through draws in the Scenario class. * introduce test_scenario_switcher * linting * add spotcheck * update plotting of healthcare system scenarios * CardioMetabolicDisorders: prob_care_provided_given_seek_emergency_care --> 1.0 * Alri: prob_for_followup_if_treatment_failure --> 1.0 * remove trailing space --------- Co-authored-by: Tim Hallett <[email protected]>
* apply the probs of health care seeking at different levels and submit a 10-year scale run * set the probs of health care seeking in resource file * delete the temporary scale run script
* set up batch runs * remove parameter mal_inc as unused * remove worksheet "incidence" from resourcefile_malaria * streamline malaria treatment HSIs, now 2 HSI types for non-complicated and severe malaria * edit referral to new treatment HSIs in HSI_Malaria_rdt, previously severe cases directed straight to treatment, clinical cases referred for additional rdt (with imperfect sensitivity) and asymptomatic cases not treated Edited: RDT performed once for all, if diagnosed and severe refer for HSI_Malaria_Treatment_Complicated, if diagnosed and clinical/asymptomatic refer for HSI_Malaria_Treatment This ensures people with malaria parasitaemia but fever due to other causes will be treated (which is correct in practice although missing the true cause of illness). * add blank footprint to HSI_rdt if person already on malaria treatment * move counter reset to treatment logger (records clinical episodes and treatment events each year) * remove on_hsi_alert * update malaria treatment referrals in hsi_generic_first_appts remove separate treatment route for children aged 5-15 (age-dependent treatment now decided in HSI_Malaria_Treatment) * update adults treatment referrals in hsi_generic_first_appts.py change priority of treatment from 1 to 0 * change treatment referral in emergency appt in hsi_generic_first_appts.py * remove mockitis and chronic-syndrome symptom checks * change logger to start on day 0 * move clinical counter in malaria_poll2() to include asym infections as these will also be treated * change schedule rdt to include only malaria infected people who are not on treatment * keep malaria clinical counter for clinical and severe cases only asym cases would not seek care as no symptoms (unless co-infection present) weight rdt scheduled testing to clinical and severe cases * remove schedule_rdt function - occurring each month so only selects those iwth current untreated malaria infection. Average duration clinical infection is 6 days so will miss many cases add symptom malaria fever with high OR for healthcare-seeking add condition fever or malaria_fever for rdt testing in hsi_generic_first_appts.py * remove symptom malaria_fever add symptome severe_malaria to replace coma, acidosis etc which all have same healthcare-seeking properties but occur in different frequencies. These individual symptoms are not tracked or used for anything so can be simplified use one function to add symptoms for clinical, severe and infections in pregnancy symptom onset one week after infection date infection date scattered across month rdt scheduled at symptom onset with probability testing_adj death occurs in severe cases 1-7 days after symptom onset * remove MalariaScheduleTesing event * make sure new symptoms and infection dates are assigned correctly for new clinical cases who were asymptomatic in previous timeperiod * add checks to ensure symptoms assigned on correct dates after infection ensure all new infections assigned symptoms (clinical cases can be drawn from previous asym infections and dates must be over-written) * some consumables constraints - items 163 (rdt) and 164 (treatment) not available increase demand for testing (testing_adj) to compensate * remove additional rdt from malaria treatment package * edit consumables required: antimalarials required for treatment; paracetamol, gloves, cannula etc become optional items * change logging to end of year * add diagnostic tracker for malaria cases * update severe malaria symptoms in hsi_generic_appts add diagnosis rates to malaria logger * move clinical counter to clinical_symptoms * add test to check rdts scheduled for every clinical case * add condition in hsi_generic_first_appts.py if malaria registered and not on malaria treatment then run dx test * move symptom onset and rdt scheduling to regular daily event (with cure event) otherwise properties ma_infected and ma_inf_type are set before person has symptom onset clinical counter also occurs on symptom onset not infection * develop scripts for updated plots * add plot rdt yield * fix flake8 errors * clean up code * fix flake8 * add mockitis/chronic syndrome back into hsi_generic_first_appts.py * isort checks * refactor to put logic into the malaria module, avoid repetition for adults and children, and correct the check on symptoms (replacing == with in) for the emergency presentation * remove commented-out lines * do you let infection happen same day to ensure the following assert line works * correct presumed typo * make the symptom list into a list * refactor for brevity * uses set to join to lists * random draw for rdt should resepct df.is_alive * update comment * linting * finish refactor * Update tests/test_malaria.py Co-authored-by: Tim Hallett <[email protected]> * make changes following review * remove json file * add NMCP commodities data * add NMCP commodities data * check rdt usage from NMCP * flake8 checks * add rdt as separate consumable to add in to treatment package * add rdt as optional item in all malaria treatment packages * change consumables declaration to quantity=1 (default) as item code will return expected_units_per_case which will include the full course of treatment - not individual tablets * update treatments with rdt - format optional item codes * add set of malaria-related symptoms to qualify person for malaria rdt in hsi_generic_first_appts.py for non-emergency appt at level 0 change name of function to reflect change of conditions (def do_for_suspected_malaria_case) * add rdt for general population run this every month within MalariaPollingEventDistrict probability related to numbers of rdts dispensed each year in NMCP report change parameter name for testing probability of malaria cases * add malaria rdt to occur at facility level 0 general population testing will prompt for community rdt a positive result will prompt referral to level 1a for confirmatory test and treatment * correct HIV test scheduler which could potentially allow an error where tclose occurs before topen * include community rdt testing scheduled each month for the general population scaled rdt usage to match WHO estimates from 2010 * remove variable df not used in function * push fixes to failing tests caused by change of parameter name * check priority values for each HSI (priority=0 for severe malaria only) * edit consumables request for iptp for pregnant women - this is now an integer item code and not a dict * scale rdt usage reports to reflect consumables availability * add variable level for RDT HSI * add in logger to record where rdts are being delivered additional plot * revert to master version of ResourceFile_HIV.xlsx * run checks * add to rdt log to include age and presence of fever * add to rdt log to include age and presence of fever * edit rdt distribution plots * rdt facility level tests * Make non-emergency care seeking occur at different levels and introduce parameter to healthcare seeking module to allow this to be determined probabilistically. * name chnages in other modules * linting * better description of parameter * add default for facility_level to enable legacy tests to pass * use of string in the resourcefile causes problems with read in of bool, use pd.eval * refactor for new names * introduce new test to protect new behaviour * update test in arli to make all non-emergency care start at level 0, for the purpose of the tests * edit plots * faster way of checking symptoms for malaria in hsi_generic_first_appts.py * change probability healthseeking behaviour by facility level to [0, 1a, 1b, 2] = [0.14, 0.61, 0.10, 0.15] edit malaria plots * edit plots * update `prob_non_emergency_care_seeking_by_level` values to those in `master` --------- Co-authored-by: Tim Hallett <[email protected]>
* Changes to contraception for missing footprint * linting
…t (and avoid attempting to look up date past the end of the data) (#1004) * set up batch runs * add fix to rdt_testing_rates to read in data up to 2020, any future dates past 2020 will use the 2020 values * use 2023 values as latest date for rdt rates * edit ResourceFile_malaria.xlsx to include the rdt rate data up to 2023 * roll back change to resource file and cap year of testing to the last year for which data is provided * refactor to cope with the last row of test_rates being all NA apart from year --------- Co-authored-by: Tim Hallett <[email protected]>
…t squeeze is more balanced (#979) * pool capabilities of facilities 1b and 2 (into level 2), AND redirect HSI for level '1b' into '2' * to the redirect of the HSI's facility level in initialise * fix alri tests that make an explicit expectation about the level of facility at which HSI run. * more checks and correcting an errror that caused capabilities at above level 3 to be lost * temporarily make scale-run last 10 years * Revert "temporarily make scale-run last 10 years" This reverts commit 67140f3. * merge consumables between '1b' and '2' * update `test_non_emergency_first_appt_can_be_levels_0_1a_1b_2` * declare AVAILABILITY_AT_MERGED_LEVELS_1B_AND_2 = ['1b'] ... so that availability at '1b'/'2' is the same as as in '1b' * make updating function flexible and default to being the same as at '1b' * linting * renaming functions (so as to be ambiguous about what the renmaed merged facility really is) * merge level 1b and 2 in hsi calibration plots * merge level 1b and 2 for figure "Usage of Healthcare Worker Time (Average)" * merge level 1b and 2 for healthsystem logger 'Capacity'/'summary_by_officef' and figure "Usage of Healthcare Worker Time (Average)" --------- Co-authored-by: Bingling <[email protected]>
…ceased to be on antiepilpetics (#1022)
* update DCSA count and capabilities in funded plus scenario * update funded_plus hcw capabilities resource file * fix format check failure
…-epileptics if medicine continue to be not available (#1023) * Made consumables to be requested in the follow-up appointment (refactoring the logic of the best available consumables out of the initiation HSI event). Added logic to the follow-up event so that if the medicine are not available, then repeated attempts are made (with modified appt footprint) before defaulting to being off anti-epileptics (after certain number of times). * add test to cause lots of HSI
…pond to availability of resources and consumables (#1024)
* plot errorbar for model vs real usage * update errorbar plot * re-structure the script * add real usage with mean, 25% percentile and 75% percentile * update adjustment for MentalAll usage data (average annual to annual) * rename plot names * update coding to make sure that any aggregation usage of appts and of levels is calculated before confidence level calculation, for Simulation * update coding to make sure that any aggregation usage of appts and of levels is calculated before confidence level calculation, for Real * plot Model vs Data with either usage having 95% CI * plot Model vs Data with fraction for each level * get unadjusted real usage data * upload unadjusted real usage data * update text * update text * plot simulation vs adjusted and unadjusted real average annual usage * plot model vs unadjusted real data with 95% CI, refactor coding * upload the correct version of unadjusted real data (the Discharges appt is only for paper use, not for TLO, as IPAdmission already include Discharges) * update the plot considering uncertainty of Model data and adjustment of real data * update the plot considering fraction by level * adjust yticks * adjust position of bar * correct format of fraction_by_level plot * plot uncertain simulation against uncertain real, update plot names * fix failed checks * fix typo * correct typo * recover older plots * plot simulation with 95% CI vs adjusted real * delete alternative plots - hard to justify the combined uncertainty * add an appendix plot that compare simulation with 95% CI with Adjusted and Unadjusted real * update text * update color of facility level to be consistent across different plots * replace "Real" by "Data" * update bar plot: log scale for y axis and bar from y=1.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here is the new PR and hopefully works out. Files as follows: tb_DAH_scenarios and analysis_tb_DAH_scenarios