Develop vm #146
Closed
Develop vm #146
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Define the prerequisite steps prior to running data prep
Everything works up through creating and connecting to VM
Add a helper script to mount Azure File Shares
Mounting an Azure File Share to a linux OS via SMB defaults to read/write access. The change to the mount script default to read-only with file permissions (`0555`) Update Docs accordingly.
Reflect actual Azure FIles structure
don't touch config in this PR.
|
Tested as of 78f5c4a. Ran into an error exporting Note, memory consumption during 2022Q4 prep currently peaking at ~17GiB, though the process started with 700MiB consumed. Start: Peak Consumption Comparing to the docker container logs, it looks like (as expected) |
Create an input_filepaths object, rather than use individual filepaths
…lls-to-write_manifest
adjust code to account for changes made in #150
create a unique directory for outputs, and warn if it already exists Closes: #151
Export a zip archive of all files specified as inputs and the contents of the output directory Closes: #152
Use explicit filepath vectors for archive export process defined in #153
Establish a working draft of a 2023Q4 config that build sucessfully. Closes: #144
Only change from 2022Q4 is data sources, in hope of getting a functional build, while RMI-PACTA/pacta.scenario.preparation#83 is in process.
|
Closing PR as superseded by #163 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Not intended for Merging
This PR tracks the status of the
develop-vmbranch, which combines:Config Changes:
build/143-2022q4-config(Add current working config for 2022q4 #145)harmonize-output-filepath-for-docker(refactor(deploy): Harmonize docker mount points #154)Docs & Deploy Scripts
docs/how-to-run-on-vm(Run on VM #142)Behavior Changes: PRs in this section should be reviewed and merged in approximately this order, since there are dependencies between them. Hopefully nothing is too controversial, but most of the behavior can be reverted back to independence is that's desired.
feat/148-export-scraped-inputs(feat(app): #148 Export data from scraped files #150)feat/151-isolate-outputs(feat(app): #151 Put outputs into unique timestamped directory #153)feat/147-update-calls-to-write_manifest(Feat/147 update calls to write manifest #149)into a complete working whole, while allowing those branches to stay focused.
Additional changes:
Temporary updates to README to include directions for testing with this branch.