Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IT-3964] Update synapse client usage #129

Merged
merged 2 commits into from
Dec 14, 2024
Merged

[IT-3964] Update synapse client usage #129

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7b04135
[IT-3964] Update synapse client usage
zaro0508 Dec 14, 2024
e3783c7
update doc
zaro0508 Dec 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/main.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,9 @@ jobs:
# use older checkout version due to https://github.com/dawidd6/action-download-artifact/issues/261
- uses: actions/checkout@v2
- name: Import Synapse Data
run: ./import-data.sh $BRANCH $SYNAPSE_USERNAME $SYNAPSE_PASSWORD $DB_HOST $DB_USER $DB_PASS
run: ./import-data.sh $BRANCH $SYNAPSE_PASSWORD $DB_HOST $DB_USER $DB_PASS
env:
BRANCH: ${{ github.ref_name }}
SYNAPSE_USERNAME: ${{ secrets.SYNAPSE_USERNAME }}
SYNAPSE_PASSWORD: ${{ secrets.SYNAPSE_PASSWORD }}
DB_HOST: ${{ secrets.DB_HOST }}
DB_USER: ${{ secrets.DB_USER }}
Expand Down
7 changes: 3 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,9 @@ The following secrets need to be setup in Github for the scripts to deploy datab

Global secrets:

| Variable | Description | Example |
|----------------------|-----------------------------------|-----------------------------|
| SYNAPSE_USERNAME | The Synapse service user | syn-service-user |
| SYNAPSE_PASSWORD | The Synapse service user password | supersecret |
| Variable | Description | Example |
|----------------------|----------------------------------|----------------------------------|
| SYNAPSE_PASSWORD | Synapse service user token (PAT) | glY4283tLQHZ...0eXAiOi...JKV1QiL |


Context specific secrets for each environment that corresponds to a git branch (develop/staging/prod):
Expand Down
15 changes: 7 additions & 8 deletions import-data.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,10 @@
set -e

BRANCH=$1
SYNAPSE_USERNAME=$2
SYNAPSE_PASSWORD=$3
DB_HOST=$4
DB_USER=$5
DB_PASS=$6
SYNAPSE_PASSWORD=$2
DB_HOST=$3
DB_USER=$4
DB_PASS=$5

CURRENT_DIR=$(pwd)
WORKING_DIR=$CURRENT_DIR
Expand All @@ -26,19 +25,19 @@ TEAM_IMAGES_ID=$(cat $WORKING_DIR/data-manifest.json | grep team-images-id | hea
echo "$BRANCH branch, DATA_VERSION = $DATA_VERSION, manifest id = $DATA_MANIFEST_ID"

# Download the manifest file from synapse
synapse -u $SYNAPSE_USERNAME -p $SYNAPSE_PASSWORD get --downloadLocation $DATA_DIR -v $DATA_VERSION $DATA_MANIFEST_ID
synapse -p $SYNAPSE_PASSWORD get --downloadLocation $DATA_DIR -v $DATA_VERSION $DATA_MANIFEST_ID

# Ensure there's a newline at the end of the manifest file; otherwise the last listed file will not be downloaded
# echo >> $DATA_DIR/data_manifest.csv

# Download all files referenced in the manifest from synapse
cat $DATA_DIR/data_manifest.csv | tail -n +2 | while IFS=, read -r id version; do
echo Downloading $id,$version
synapse -u $SYNAPSE_USERNAME -p $SYNAPSE_PASSWORD get --downloadLocation $DATA_DIR -v $version $id ;
synapse -p $SYNAPSE_PASSWORD get --downloadLocation $DATA_DIR -v $version $id ;
done

# Download team images
synapse -u $SYNAPSE_USERNAME -p $SYNAPSE_PASSWORD get -r --downloadLocation $TEAM_IMAGES_DIR/ $TEAM_IMAGES_ID
synapse -p $SYNAPSE_PASSWORD get -r --downloadLocation $TEAM_IMAGES_DIR/ $TEAM_IMAGES_ID

echo "Data Files: "
ls -al $WORKING_DIR
Expand Down
Loading