Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(app): #148 Export data from scraped files #150

Merged
merged 4 commits into from
Feb 23, 2024
Merged

feat(app): #148 Export data from scraped files #150

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2d2c72f
feat(app): #148 Export data from scraped files
AlexAxthelm Feb 18, 2024
f74c51f
Merge branch 'main' into feat/148-export-scraped-inputs
cjyetman Feb 22, 2024
94fe89f
Merge branch 'main' into feat/148-export-scraped-inputs
cjyetman Feb 23, 2024
60dd009
Merge branch 'main' into feat/148-export-scraped-inputs
cjyetman Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ default:
data_prep_outputs_path: "" # a single, absolute or relative path to a directory where the outputs will be saved
asset_impact_data_path: "" # a single, absolute or relative path to a directory that contains the AI inputs (can be the same as `factset_data_path`)
factset_data_path: "" # a single, absolute or relative path to a directory that contains the FactSet inputs (can be the same as `asset_impact_data_path`)
preflight_data_path: "" # a single, absolute or relative path to a directory that contains a previous "currencies.rds" file and/or will be used to save a copy of pre-flight data files generated on-the-fly (by default [""], this is set the same path as `data_prep_outputs_path`)
masterdata_ownership_filename: "" # a single filename of the intended masterdata_ownership CSV (must exist in the `asset_impact_data_path` directory)
masterdata_debt_filename: "" # a single filename of the intended masterdata_debt CSV (must exist in the `asset_impact_data_path` directory)
ar_company_id__factset_entity_id_filename: "" # a single filename of the intended ar_company_id__factset_entity_id CSV (must exist in the `asset_impact_data_path` directory)
Expand Down
13 changes: 13 additions & 0 deletions run_pacta_data_preparation.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,16 @@ factset_manual_pacta_sector_override_path <-

# pre-flight filepaths ---------------------------------------------------------

preflight_data_path <- config$preflight_data_path
if (preflight_data_path == "") {
preflight_data_path <- data_prep_outputs_path
}

currencies_preflight_data_path <- file.path(preflight_data_path, "currencies.rds")
currencies_data_path <- file.path(data_prep_outputs_path, "currencies.rds")
index_regions_data_path <- file.path(data_prep_outputs_path, "index_regions.rds")

index_regions_preflight_data_path <- file.path(preflight_data_path, "index_regions.rds")

# computed options -------------------------------------------------------------

Expand Down Expand Up @@ -157,10 +164,16 @@ if (update_currencies) {
currencies <- pacta.data.scraping::get_currency_exchange_rates(
quarter = imf_quarter_timestamp
)
saveRDS(currencies, currencies_preflight_data_path)
} else {
logger::log_info("Using pre-existing currency data.")
# This requires the preflight path to be defined in the config
currencies <- readRDS(currencies_preflight_data_path)
}

logger::log_info("Scraping index regions.")
index_regions <- pacta.data.scraping::get_index_regions()
saveRDS(index_regions, index_regions_preflight_data_path)

logger::log_info("Fetching pre-flight data done.")

Expand Down