From 5d7e88ebfe7af9453e556a8a738708beb370f70d Mon Sep 17 00:00:00 2001 From: Harunobu Ishii Date: Tue, 20 Aug 2024 16:31:07 -0400 Subject: [PATCH 01/28] setup --- setup.md | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 setup.md diff --git a/setup.md b/setup.md new file mode 100644 index 0000000..8852b43 --- /dev/null +++ b/setup.md @@ -0,0 +1,2 @@ +## Setup Guide + From b6d76a8948e2701f7b7a8ceb0f9be46295e6e7af Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Tue, 20 Aug 2024 17:29:57 -0400 Subject: [PATCH 02/28] Update setup.md --- setup.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 1 deletion(-) diff --git a/setup.md b/setup.md index 8852b43..cb7ce8a 100644 --- a/setup.md +++ b/setup.md @@ -1,2 +1,88 @@ -## Setup Guide +# Secrecy OpenMPI Setup Guide + +This document is a step-by-step guide to setting up Secrecy Server and OPEN MPI implementation. + +## 1) Create VPC + +- Select VPC and more +- Need 1 zone and public and private subnet + +image + +image + +## 2) Launch three EC2 instances +image + +- Pick appropriate size of instance +- Generate a key pair if you haven't and save the key in your local machine +image + +- Hit Edit in Network settings and pick the VPC you've just created in step 1 +image +image + +- Repeat this process till you have created three instances + +## 3) Open ports in instances + +##### 1. Go to EC2 instance dashboard +##### 2. Click a line of your instance (Mines are named Secrecy-node#) +##### 3. Save Public and Private IP addresses +image + +- Open Security tab +image + +- Hit Security Groups, which opens a new browser tab +image + +- Hit the value under Security group ID +image + +- Under the inbound rules , open Edit inbound rules button +image + +- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the private IP addresses of other two instances +image + +- Repeat this process till you have applied these configurations to all three instances + +## 4) Access instance + +##### With these steps so far, you should be able to access the EC2 instance and are ready to launch the secrecy app +Try running the command below in your terminal: +``` +ssh -i /path/to/your/key.pem ec2-user@public-IP +``` + +##### If you successfully ssh into your instance, run below: + +``` +sudo yum groupinstall -y "Development Tools" \ +&& sudo yum install -y cmake libsodium libsodium-devel openmpi openmpi-devel pkg-config git openssh-server nano \ +&& git clone https://github.com/multiparty/Secrecy.git \ +&& mkdir -p Secrecy/include/external-lib \ +&& git clone https://github.com/mfaisal97/sql-parser.git Secrecy/include/external-lib/sql-parser \ +&& echo 'export PATH=$PATH:/usr/lib64/openmpi/bin' >> ~/.bashrc \ +&& export PATH=$PATH:/usr/lib64/openmpi/bin \ +&& which mpicc && which mpicxx \ +&& cd Secrecy \ +&& mkdir build && cd build && cmake .. && make \ +&& echo -e "\n\n" > hostfile.txt +``` + +##### Repeat these steps in the other two instances. + +Run below + +``` +ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa +``` + +``` +ssh-copy-id -i ~/.ssh/id_rsa.pub ec2-user@ +``` + + From bb8180e2fb3386e3fee983a69dab388d482a83c4 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 21 Aug 2024 11:42:39 -0400 Subject: [PATCH 03/28] Update setup.md --- setup.md | 103 +++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 66 insertions(+), 37 deletions(-) diff --git a/setup.md b/setup.md index cb7ce8a..e219691 100644 --- a/setup.md +++ b/setup.md @@ -1,62 +1,89 @@ # Secrecy OpenMPI Setup Guide -This document is a step-by-step guide to setting up Secrecy Server and OPEN MPI implementation. +This document is a step-by-step guide to setting up Secrecy Server and OPEN MPI implementation on AWS EC2 instances. + +## Introduction + +In this guide, you will learn how to create a Virtual Private Cloud (VPC), launch and configure EC2 instances, and set up Secrecy with OpenMPI. This guide assumes you have a basic understanding of AWS services and SSH. + +### Prerequisites +- AWS Account +- Basic knowledge of EC2 and VPC +- SSH client installed on your local machine ## 1) Create VPC -- Select VPC and more -- Need 1 zone and public and private subnet +1. Navigate to the AWS VPC console. +2. Select **VPC and more**. +3. Create a VPC with **1 zone** and both **public** and **private subnets**. + + + + + + +
VPC SetupVPC Subnet Configuration
+ +## 2) Launch Three EC2 Instances + +EC2 Instances + +1. Pick an appropriate instance size. +2. Generate a key pair if you haven't and save the key to your local machine. + + Key Pair -image +3. Hit **Edit** in Network settings and pick the VPC you've just created in step 1. -image + + + + + +
Network SettingsVPC Selection
-## 2) Launch three EC2 instances -image +4. Repeat this process until you have created three instances. -- Pick appropriate size of instance -- Generate a key pair if you haven't and save the key in your local machine -image +## 3) Open Ports in Instances -- Hit Edit in Network settings and pick the VPC you've just created in step 1 -image -image +1. Go to the EC2 instance dashboard. +2. Click the line of your instance (e.g., Secrecy-node#). +3. Save the Public and Private IP addresses. -- Repeat this process till you have created three instances + EC2 Dashboard -## 3) Open ports in instances +4. Open the **Security** tab. -##### 1. Go to EC2 instance dashboard -##### 2. Click a line of your instance (Mines are named Secrecy-node#) -##### 3. Save Public and Private IP addresses -image + Security Tab -- Open Security tab -image +5. Hit **Security Groups**, which opens a new browser tab. -- Hit Security Groups, which opens a new browser tab -image + Security Groups -- Hit the value under Security group ID -image +6. Click the value under **Security group ID**. -- Under the inbound rules , open Edit inbound rules button -image + Security Group ID -- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the private IP addresses of other two instances -image -- Repeat this process till you have applied these configurations to all three instances +- Under the **inbound rules**, open the **Edit inbound rules** button. -## 4) Access instance + Edit Inbound Rules + +- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the private IP addresses of the other two instances. + + Inbound Rules Configuration + +- Repeat this process until you have applied these configurations to all three instances. + +## 4) Access Instance + +With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. Try running the command below in your terminal: -##### With these steps so far, you should be able to access the EC2 instance and are ready to launch the secrecy app -Try running the command below in your terminal: ``` ssh -i /path/to/your/key.pem ec2-user@public-IP ``` -##### If you successfully ssh into your instance, run below: +If you successfully SSH into your instance, run the following commands: ``` sudo yum groupinstall -y "Development Tools" \ @@ -71,10 +98,12 @@ sudo yum groupinstall -y "Development Tools" \ && mkdir build && cd build && cmake .. && make \ && echo -e "\n\n" > hostfile.txt ``` - -##### Repeat these steps in the other two instances. + +Repeat these steps on the other two instances. + +Generate an SSH key pair on each instance by running the following command: + -Run below ``` ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa From 9d16d44ea1597777435a9a2035bcfec60c0f672b Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 21 Aug 2024 11:45:39 -0400 Subject: [PATCH 04/28] Update setup.md --- setup.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/setup.md b/setup.md index e219691..c0d3f9f 100644 --- a/setup.md +++ b/setup.md @@ -104,14 +104,12 @@ Repeat these steps on the other two instances. Generate an SSH key pair on each instance by running the following command: - ``` ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa ``` +And register the public key to remote servers (two other instances) + ``` ssh-copy-id -i ~/.ssh/id_rsa.pub ec2-user@ ``` - - - From d81b92cd44192946f3592a8d4b1ac7e0386fb106 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 21 Aug 2024 15:44:15 -0400 Subject: [PATCH 05/28] Update setup.md --- setup.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/setup.md b/setup.md index c0d3f9f..f15d5f4 100644 --- a/setup.md +++ b/setup.md @@ -1,6 +1,6 @@ # Secrecy OpenMPI Setup Guide -This document is a step-by-step guide to setting up Secrecy Server and OPEN MPI implementation on AWS EC2 instances. +This document is a step-by-step guide to setting up a Secrecy Server and OPEN MPI implementation on AWS EC2 instances. ## Introduction @@ -15,12 +15,13 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 1. Navigate to the AWS VPC console. 2. Select **VPC and more**. -3. Create a VPC with **1 zone** and both **public** and **private subnets**. +3. Name your VPC +4. Create a VPC with **1 zone** and **public** and zero **private subnets**. - - + +
VPC SetupVPC Subnet ConfigurationVPC SetupVPC Subnet Configuration
From 7249a813551becc0cd3caac56fe27118f19ff699 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 21 Aug 2024 16:02:40 -0400 Subject: [PATCH 06/28] Update setup.md --- setup.md | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/setup.md b/setup.md index f15d5f4..36811c1 100644 --- a/setup.md +++ b/setup.md @@ -25,25 +25,31 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc -## 2) Launch Three EC2 Instances +## 2) Launch EC2 Instance EC2 Instances 1. Pick an appropriate instance size. -2. Generate a key pair if you haven't and save the key to your local machine. +2. Name Instance and select Amazon Linux + + image + +4. Generate a key pair if you haven't and save the key to your local machine. Key Pair -3. Hit **Edit** in Network settings and pick the VPC you've just created in step 1. +5. Hit **Edit** in Network settings and pick the VPC you've just created in step 1 + - **Create new subnet** + - Enable **Auto-assign public IP** + - Select **Create security group** - +
Network SettingsVPC Selectionimage
-4. Repeat this process until you have created three instances. ## 3) Open Ports in Instances @@ -74,17 +80,18 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc Inbound Rules Configuration -- Repeat this process until you have applied these configurations to all three instances. - ## 4) Access Instance -With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. Try running the command below in your terminal: +With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. +Exchange public IP addresses with other participants. + +Try running the command below in your terminal: ``` -ssh -i /path/to/your/key.pem ec2-user@public-IP +ssh -i /path/to/your/key.pem ec2-user@your-public-IP ``` -If you successfully SSH into your instance, run the following commands: +If you successfully SSH into your instance, run the following commands, after replacing with actual public IP addresses of your and other participants: ``` sudo yum groupinstall -y "Development Tools" \ @@ -97,20 +104,20 @@ sudo yum groupinstall -y "Development Tools" \ && which mpicc && which mpicxx \ && cd Secrecy \ && mkdir build && cd build && cmake .. && make \ -&& echo -e "\n\n" > hostfile.txt +&& echo -e "\n\n" > hostfile.txt ``` -Repeat these steps on the other two instances. - Generate an SSH key pair on each instance by running the following command: - ``` -ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa +ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa && cat ~/.ssh/id_rsa.pub ``` -And register the public key to remote servers (two other instances) +Exchange public keys with other participants. +Paste their IP addresses in authorized_keys, which can be opened by: ``` -ssh-copy-id -i ~/.ssh/id_rsa.pub ec2-user@ +nano ~/.ssh/authorized_keys ``` + +and paste by **"command + v "**. From d76d54e5f32efd3125de97b5942cbde35882704f Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 21 Aug 2024 16:04:35 -0400 Subject: [PATCH 07/28] Update setup.md --- setup.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/setup.md b/setup.md index 36811c1..781b3dc 100644 --- a/setup.md +++ b/setup.md @@ -13,7 +13,10 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc ## 1) Create VPC -1. Navigate to the AWS VPC console. +1. Navigate to the AWS VPC console and hit **Create VPC**. + +image + 2. Select **VPC and more**. 3. Name your VPC 4. Create a VPC with **1 zone** and **public** and zero **private subnets**. From 040a41b4268e3c9154f413b4e4a1e1221940a2a8 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 21 Aug 2024 17:07:55 -0400 Subject: [PATCH 08/28] Update setup.md --- setup.md | 50 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/setup.md b/setup.md index 781b3dc..b6ff32a 100644 --- a/setup.md +++ b/setup.md @@ -32,16 +32,17 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc EC2 Instances -1. Pick an appropriate instance size. -2. Name Instance and select Amazon Linux +1. Name Instance and select Amazon Linux image -4. Generate a key pair if you haven't and save the key to your local machine. +2. Pick an appropriate instance size. + +3. Generate a key pair if you haven't and save the key to your local machine. Key Pair -5. Hit **Edit** in Network settings and pick the VPC you've just created in step 1 +4. Hit **Edit** in Network settings and pick the VPC you've just created in step 1 - **Create new subnet** - Enable **Auto-assign public IP** - Select **Create security group** @@ -58,18 +59,14 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 1. Go to the EC2 instance dashboard. 2. Click the line of your instance (e.g., Secrecy-node#). -3. Save the Public and Private IP addresses. +3. Save the Public IP address and exchange it with other participants. EC2 Dashboard -4. Open the **Security** tab. +4. Open the **Security** tab and Hit **Security Groups**, which opens a new browser tab. Security Tab -5. Hit **Security Groups**, which opens a new browser tab. - - Security Groups - 6. Click the value under **Security group ID**. Security Group ID @@ -79,13 +76,14 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc Edit Inbound Rules -- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the private IP addresses of the other two instances. +- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the public IP addresses of the other two instances. Inbound Rules Configuration ## 4) Access Instance -With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. +With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. + Exchange public IP addresses with other participants. Try running the command below in your terminal: @@ -106,8 +104,7 @@ sudo yum groupinstall -y "Development Tools" \ && export PATH=$PATH:/usr/lib64/openmpi/bin \ && which mpicc && which mpicxx \ && cd Secrecy \ -&& mkdir build && cd build && cmake .. && make \ -&& echo -e "\n\n" > hostfile.txt +&& mkdir build && cd build && cmake .. && make ``` Generate an SSH key pair on each instance by running the following command: @@ -124,3 +121,28 @@ nano ~/.ssh/authorized_keys ``` and paste by **"command + v "**. + +## 5) Make a hostfile and run MPI program + +If you are initiating the MPI process, make a host file in the build directory by running the following command: + +``` + echo -e "\n\n" > hostfile.txt +``` + +or manually by: + +``` +nano hostfile.txt +``` + +The resulting hostfile should look like this: + +image + +The initiating party should be able to run the Secrecy algorithm by running the following command: + +``` +mpirun -np 3 --hostfile hostfile.txt ./test_join_sail ./../sample1.json ./../sample2.json +``` + From e50fa9e9ef3b2cc66104d512915ab701e4e7e040 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Fri, 23 Aug 2024 13:06:07 -0400 Subject: [PATCH 09/28] Update setup.md --- setup.md | 73 ++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 52 insertions(+), 21 deletions(-) diff --git a/setup.md b/setup.md index b6ff32a..382debc 100644 --- a/setup.md +++ b/setup.md @@ -43,9 +43,9 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc Key Pair 4. Hit **Edit** in Network settings and pick the VPC you've just created in step 1 - - **Create new subnet** - Enable **Auto-assign public IP** - - Select **Create security group** + - Select **Create security group** if the other two parties haven't created one + - Alternatively **Select existing security group** if one of the other two parties have created one @@ -59,15 +59,17 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 1. Go to the EC2 instance dashboard. 2. Click the line of your instance (e.g., Secrecy-node#). -3. Save the Public IP address and exchange it with other participants. +3. Save the Public and Private IP addresses. Share these IP addresses with an initializing party. EC2 Dashboard +Steps 4 and 5 are required **ONLY IF** you are the first one to create an EC2 instance. + 4. Open the **Security** tab and Hit **Security Groups**, which opens a new browser tab. Security Tab -6. Click the value under **Security group ID**. +5. Click the value under **Security group ID**. Security Group ID @@ -76,23 +78,21 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc Edit Inbound Rules -- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the public IP addresses of the other two instances. +- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the private IP addresses of the other two instances. Inbound Rules Configuration -## 4) Access Instance +## 4) Access Instance and Network Configuration With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. -Exchange public IP addresses with other participants. - -Try running the command below in your terminal: +1. Try running the command below in your terminal: ``` ssh -i /path/to/your/key.pem ec2-user@your-public-IP ``` -If you successfully SSH into your instance, run the following commands, after replacing with actual public IP addresses of your and other participants: +2. If you successfully SSH into your instance, run the following commands: ``` sudo yum groupinstall -y "Development Tools" \ @@ -107,40 +107,71 @@ sudo yum groupinstall -y "Development Tools" \ && mkdir build && cd build && cmake .. && make ``` -Generate an SSH key pair on each instance by running the following command: +3. Generate an SSH key pair on each instance by running the following command: + +``` +ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa +``` + +and below command will show your pubic key just generated: ``` -ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa && cat ~/.ssh/id_rsa.pub +cat ~/.ssh/id_rsa.pub ``` -Exchange public keys with other participants. -Paste their IP addresses in authorized_keys, which can be opened by: +4. Exchange public keys with other participants. To grant them access, add their public keys to your authorized_keys file. ``` nano ~/.ssh/authorized_keys ``` -and paste by **"command + v "**. +Then, paste their public keys by pressing Command + V (on macOS) or the appropriate paste command for your operating system. + +5. In order to enable agent forwarding and disable strict host key checking for all SSH connections, you can add the following configuration to your SSH config file by editing it with: + +``` +nano ~/.ssh/config + +``` + +Then add: + +``` +Host * + ForwardAgent yes +Host * + StrictHostKeyChecking no +``` +This allows you to use the SSH keys stored on your local machine to authenticate to other remote servers through an intermediary server, without having to copy your private key to the intermediary. + + +Make files accessible by: + +``` +chmod 600 ~/.ssh/id_rsa +chmod 600 ~/.ssh/config +``` -## 5) Make a hostfile and run MPI program +## 5) Initiate MPI program -If you are initiating the MPI process, make a host file in the build directory by running the following command: +If you're initiating the MPI process, you'll need to create a host file in the build directory. You can do this automatically by running the following command: ``` echo -e "\n\n" > hostfile.txt ``` +Make sure you replace , before running the command. -or manually by: +Alternatively, you can create it manually by opening the file in a text editor: ``` nano hostfile.txt ``` -The resulting hostfile should look like this: +The resulting hostfile should resemble the following format (The order matters): -image +image -The initiating party should be able to run the Secrecy algorithm by running the following command: +Once the host file is prepared, the initiating party can run the Secrecy algorithm with the command: ``` mpirun -np 3 --hostfile hostfile.txt ./test_join_sail ./../sample1.json ./../sample2.json From d861f4cc4b7088edd9fee96261749d1739ac7687 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Fri, 23 Aug 2024 13:07:15 -0400 Subject: [PATCH 10/28] Update setup.md --- setup.md | 1 - 1 file changed, 1 deletion(-) diff --git a/setup.md b/setup.md index 382debc..39b0e29 100644 --- a/setup.md +++ b/setup.md @@ -8,7 +8,6 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc ### Prerequisites - AWS Account -- Basic knowledge of EC2 and VPC - SSH client installed on your local machine ## 1) Create VPC From 051e476fe3b12f0b2754a6a70d86604641be5b0f Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Fri, 23 Aug 2024 13:12:28 -0400 Subject: [PATCH 11/28] Update setup.md --- setup.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/setup.md b/setup.md index 39b0e29..e0a5fd7 100644 --- a/setup.md +++ b/setup.md @@ -10,6 +10,9 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc - AWS Account - SSH client installed on your local machine +### Terminology +In this guide, we call a party who initiates/runs the mpirun command **initializing party**. + ## 1) Create VPC 1. Navigate to the AWS VPC console and hit **Create VPC**. From 7fee413916590e949aa2d6724a215903731b8697 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Fri, 23 Aug 2024 13:21:48 -0400 Subject: [PATCH 12/28] Update setup.md --- setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.md b/setup.md index e0a5fd7..3b99fbd 100644 --- a/setup.md +++ b/setup.md @@ -63,7 +63,7 @@ In this guide, we call a party who initiates/runs the mpirun command **in 2. Click the line of your instance (e.g., Secrecy-node#). 3. Save the Public and Private IP addresses. Share these IP addresses with an initializing party. - EC2 Dashboard + EC2 Dashboard Steps 4 and 5 are required **ONLY IF** you are the first one to create an EC2 instance. From 6688b6faf988c7fdaa838a9ab53c9240d62c1fee Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Fri, 23 Aug 2024 13:24:51 -0400 Subject: [PATCH 13/28] Update setup.md --- setup.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/setup.md b/setup.md index 3b99fbd..af6c069 100644 --- a/setup.md +++ b/setup.md @@ -69,9 +69,9 @@ In this guide, we call a party who initiates/runs the mpirun command **in 4. Open the **Security** tab and Hit **Security Groups**, which opens a new browser tab. - Security Tab + image -5. Click the value under **Security group ID**. +6. Click the value under **Security group ID**. Security Group ID From 8672e69c21445563c60c1429b60e47cd1106521c Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Fri, 23 Aug 2024 14:12:21 -0400 Subject: [PATCH 14/28] Update setup.md --- setup.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/setup.md b/setup.md index af6c069..7a4edf9 100644 --- a/setup.md +++ b/setup.md @@ -80,10 +80,13 @@ In this guide, we call a party who initiates/runs the mpirun command **in Edit Inbound Rules -- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from the private IP addresses of the other two instances. +- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from **the private IP addresses** of the other two instances. Inbound Rules Configuration +* We add private IPs instead of public IPs because all communication between instances and services hosted in AWS uses AWS's private network, explained in AWS FAQ (Does traffic go over the internet when two instances communicate using public IP addresses, or when instances communicate with a public AWS service endpoint?). + + ## 4) Access Instance and Network Configuration With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. From b89c55085045c5c003af74cd6bfa45bd529a651f Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Tue, 27 Aug 2024 17:11:57 -0400 Subject: [PATCH 15/28] Update setup.md --- setup.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/setup.md b/setup.md index 7a4edf9..87382b2 100644 --- a/setup.md +++ b/setup.md @@ -10,9 +10,6 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc - AWS Account - SSH client installed on your local machine -### Terminology -In this guide, we call a party who initiates/runs the mpirun command **initializing party**. - ## 1) Create VPC 1. Navigate to the AWS VPC console and hit **Create VPC**. @@ -158,8 +155,9 @@ chmod 600 ~/.ssh/config ``` ## 5) Initiate MPI program +Designate **initializing party**, and only the initializing party executes the following steps. -If you're initiating the MPI process, you'll need to create a host file in the build directory. You can do this automatically by running the following command: +You'll need to create a host file in the build directory to run the MPI process. You can do this automatically by running the following command: ``` echo -e "\n\n" > hostfile.txt From 3af241e607d3f624ab8bd1d9e6f5b547cc01da18 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Thu, 5 Sep 2024 17:40:04 -0400 Subject: [PATCH 16/28] Update setup.md --- setup.md | 91 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 58 insertions(+), 33 deletions(-) diff --git a/setup.md b/setup.md index 87382b2..76f06e8 100644 --- a/setup.md +++ b/setup.md @@ -9,6 +9,7 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc ### Prerequisites - AWS Account - SSH client installed on your local machine +- Decide on roles 1, 2, and 3 amongst three participating parties ## 1) Create VPC @@ -18,7 +19,8 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 2. Select **VPC and more**. 3. Name your VPC -4. Create a VPC with **1 zone** and **public** and zero **private subnets**. +4. Pick the IPv4 CIDR block: 10.0.0.0/16 for role-1, 10.1.0.0/16 for role-2, 10.2.0.0/16 for role-3. +5. Create a VPC with **1 zone** and **public** and zero **private subnets**.
@@ -53,38 +55,61 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc
- -## 3) Open Ports in Instances - -1. Go to the EC2 instance dashboard. -2. Click the line of your instance (e.g., Secrecy-node#). -3. Save the Public and Private IP addresses. Share these IP addresses with an initializing party. - - EC2 Dashboard - -Steps 4 and 5 are required **ONLY IF** you are the first one to create an EC2 instance. - -4. Open the **Security** tab and Hit **Security Groups**, which opens a new browser tab. - - image - -6. Click the value under **Security group ID**. - - Security Group ID - - -- Under the **inbound rules**, open the **Edit inbound rules** button. - - Edit Inbound Rules - -- Add rules so that Port 22 is accessible from your local machine and all ports (0-65535) are accessible from **the private IP addresses** of the other two instances. - - Inbound Rules Configuration - -* We add private IPs instead of public IPs because all communication between instances and services hosted in AWS uses AWS's private network, explained in AWS FAQ (Does traffic go over the internet when two instances communicate using public IP addresses, or when instances communicate with a public AWS service endpoint?). - - -## 4) Access Instance and Network Configuration +## 3) Setup Subnets +1. Go to the VPC Dashboard +2. In the left-hand navigation pane, select **"Subnets"**. +3. Click **"Create subnet"** +image +4. Select your VPC from the drop-down. Then the following settings pane pops up. + - Name subnet + - Set Availability Zone 'us-east-1a' + - IPv4 VPC CIDR block should be your VPC CIDR block + - IPv4 subnet CIDR block should be set according to: 10.0.16.0/20 if role-1, 10.1.16.0/20 if role-2, and 10.2.16.0/20 if role-3. + - Hit **"Create subnet"** + +## 4) Create VPC Peering Connection + +1. Go to the VPC Dashboard +2. In the left-hand navigation pane, select **"Peering Connections"**. +3. Click **"Create Peering Connection"** + image +4. Fill in parameters, + - Name: Give the Peering Connection a name + - Local VPC: Select your VPC + - Another VPC: Select 2 if you are role-1. Select 3 if you are role-2. Select 1 if you are role-3. +If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. +5. Click **"Create Peering Connection"** +image +6. Go back to the **"Peering Connections"** Dashboard. +7. Select your Peer Connection, click **"Actions"** at the right top, and hit **Accept request** + +## 5) Update Route Tables + +1. Go to the **Route Tables** section in the VPC Dashboard. +2. Select the Route Table associated with the subnets in each VPC from the list. +3. Click **Edit routes** in the Routes tab: + image +4. Click **"Add route"** to add a new route: + - **Destination**: The CIDR block of the peered VPC + - The CIDR block for each roles are: `10.0.0.0/16` for role-1, `10.1.0.0/16` for role-2, and`10.2.0.0/16` for role-3 + - Therefore, if you are role-1, you would want to add two routes for each role-2 and role-3, 10.1.0.0/16 and 10.2.0.0/16 respectively as Destination + - **Target**: Select the Peering Connection and select corresponding subnets (e.g., `pcx-xxxxxx`) + image +5. Click **Save routes**. + +## 6) Update Security Groups and Network ACLs +1. Go to the EC2 Dashboard. +2. Select your instance, choose the **Security** tab, and hit the pop-up link. + image +3. This will take you to Security Groups Dashboard. Click on the Security Group ID + image +4. Click **"Edit inbound rules"** +6. Add an inbound rule to allow traffic from the peered VPC’s CIDR block: + - **Type**: Select the desired traffic type (e.g., All traffic or specific ports). + - **Source**: Enter the CIDR block of the peered VPC (e.g., `10.0.0.0/16` for role-1, `10.1.0.0/16` for role-2, and`10.2.0.0/16` for role-3) + - If you are role-1, you would want to add two routes for each role-2 and role-3, 10.1.0.0/16 and 10.2.0.0/16 respectively + +## 7) Access Instance and Network Configuration With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. From bc2f847be14098fcf4f5c84c2641678063c81dde Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Tue, 10 Sep 2024 12:01:57 -0400 Subject: [PATCH 17/28] Update setup.md --- setup.md | 66 ++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 48 insertions(+), 18 deletions(-) diff --git a/setup.md b/setup.md index 76f06e8..824e6cb 100644 --- a/setup.md +++ b/setup.md @@ -9,7 +9,10 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc ### Prerequisites - AWS Account - SSH client installed on your local machine -- Decide on roles 1, 2, and 3 amongst three participating parties + + +## Before You Start +**Designate each party to roles 1, 2, and 3** ## 1) Create VPC @@ -19,8 +22,11 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 2. Select **VPC and more**. 3. Name your VPC -4. Pick the IPv4 CIDR block: 10.0.0.0/16 for role-1, 10.1.0.0/16 for role-2, 10.2.0.0/16 for role-3. -5. Create a VPC with **1 zone** and **public** and zero **private subnets**. +4. Pick the IPv4 CIDR block: + - role-1: 10.0.0.0/16 + - role-2: 10.1.0.0/16 + - role-3: 10.2.0.0/16 +6. Create a VPC with **1 zone** and **public** and zero **private subnets**. @@ -43,10 +49,7 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc Key Pair -4. Hit **Edit** in Network settings and pick the VPC you've just created in step 1 - - Enable **Auto-assign public IP** - - Select **Create security group** if the other two parties haven't created one - - Alternatively **Select existing security group** if one of the other two parties have created one +4. Hit **Edit** in Network settings, pick the VPC you've just created in step 1, enable **Auto-assign public IP**
@@ -60,26 +63,35 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 2. In the left-hand navigation pane, select **"Subnets"**. 3. Click **"Create subnet"** image + 4. Select your VPC from the drop-down. Then the following settings pane pops up. - Name subnet - Set Availability Zone 'us-east-1a' - IPv4 VPC CIDR block should be your VPC CIDR block - - IPv4 subnet CIDR block should be set according to: 10.0.16.0/20 if role-1, 10.1.16.0/20 if role-2, and 10.2.16.0/20 if role-3. + - IPv4 subnet CIDR block should be set according to your role: + - role-1: 10.0.16.0/20 + - role-2: 10.1.16.0/20 + - role-3: 10.2.16.0/20 - Hit **"Create subnet"** ## 4) Create VPC Peering Connection 1. Go to the VPC Dashboard 2. In the left-hand navigation pane, select **"Peering Connections"**. -3. Click **"Create Peering Connection"** +3. Click **"Create Peering Connection"**. Make sure that other parties have already created VPCs at this point image 4. Fill in parameters, - Name: Give the Peering Connection a name - - Local VPC: Select your VPC - - Another VPC: Select 2 if you are role-1. Select 3 if you are role-2. Select 1 if you are role-3. -If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. + - VPC ID(Requester): Select your VPC + - VPC ID(Accepter): + - If you are role-1: Select 2 + - role-2: Select 3 + - role-3: Select 1 + - If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. + 5. Click **"Create Peering Connection"** -image + image + 6. Go back to the **"Peering Connections"** Dashboard. 7. Select your Peer Connection, click **"Actions"** at the right top, and hit **Accept request** @@ -91,11 +103,24 @@ If your partners/other parties use a separate AWS account, select "another accou image 4. Click **"Add route"** to add a new route: - **Destination**: The CIDR block of the peered VPC - - The CIDR block for each roles are: `10.0.0.0/16` for role-1, `10.1.0.0/16` for role-2, and`10.2.0.0/16` for role-3 - - Therefore, if you are role-1, you would want to add two routes for each role-2 and role-3, 10.1.0.0/16 and 10.2.0.0/16 respectively as Destination + - The CIDR block for each role is: + - role-1: `10.0.0.0/16` + - role-2: `10.1.0.0/16` + - role-3: `10.2.0.0/16` + Therefore: + - If you are role-1, you would want to add two routes for each role-2 and role-3: + - 10.1.0.0/16 + - 10.2.0.0/16 + - If you are role-2: + - 10.0.0.0/16 + - 10.2.0.0/16 + - If you are role-3: + - 10.0.0.0/16 + - 10.1.0.0/16 + - **Target**: Select the Peering Connection and select corresponding subnets (e.g., `pcx-xxxxxx`) image -5. Click **Save routes**. +6. Click **Save routes**. ## 6) Update Security Groups and Network ACLs 1. Go to the EC2 Dashboard. @@ -107,7 +132,12 @@ If your partners/other parties use a separate AWS account, select "another accou 6. Add an inbound rule to allow traffic from the peered VPC’s CIDR block: - **Type**: Select the desired traffic type (e.g., All traffic or specific ports). - **Source**: Enter the CIDR block of the peered VPC (e.g., `10.0.0.0/16` for role-1, `10.1.0.0/16` for role-2, and`10.2.0.0/16` for role-3) - - If you are role-1, you would want to add two routes for each role-2 and role-3, 10.1.0.0/16 and 10.2.0.0/16 respectively + - If you are role-1, you would want to add two routes for each role-2 and role-3: + - 10.1.0.0/16 and 10.2.0.0/16 + - If you are role-2: + - 10.0.0.0/16 and 10.2.0.0/16 + - If you are role-3: + - 10.0.0.0/16 and 10.1.0.0/16 ## 7) Access Instance and Network Configuration @@ -202,6 +232,6 @@ The resulting hostfile should resemble the following format (The order matters): Once the host file is prepared, the initiating party can run the Secrecy algorithm with the command: ``` -mpirun -np 3 --hostfile hostfile.txt ./test_join_sail ./../sample1.json ./../sample2.json +mpirun -np 3 --hostfile hostfile.txt ./test_join_sail ./../sample1.csv ./../sample2.csv ``` From 5f0dd32170378f521343480d2640026e770db659 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 10:45:55 -0400 Subject: [PATCH 18/28] Update setup.md --- setup.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/setup.md b/setup.md index 824e6cb..f91595a 100644 --- a/setup.md +++ b/setup.md @@ -83,8 +83,8 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc 4. Fill in parameters, - Name: Give the Peering Connection a name - VPC ID(Requester): Select your VPC - - VPC ID(Accepter): - - If you are role-1: Select 2 + - VPC ID(Accepter): Select in the following way: + - role-1: Select 2 - role-2: Select 3 - role-3: Select 1 - If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. @@ -97,18 +97,16 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc ## 5) Update Route Tables +** Ensure that everyone has created a peering connection before implementing this step + 1. Go to the **Route Tables** section in the VPC Dashboard. 2. Select the Route Table associated with the subnets in each VPC from the list. 3. Click **Edit routes** in the Routes tab: image 4. Click **"Add route"** to add a new route: - **Destination**: The CIDR block of the peered VPC - - The CIDR block for each role is: - - role-1: `10.0.0.0/16` - - role-2: `10.1.0.0/16` - - role-3: `10.2.0.0/16` - Therefore: - - If you are role-1, you would want to add two routes for each role-2 and role-3: + - Add the CIDR blocks in the following way: + - If you are role-1: - 10.1.0.0/16 - 10.2.0.0/16 - If you are role-2: @@ -209,7 +207,7 @@ chmod 600 ~/.ssh/id_rsa chmod 600 ~/.ssh/config ``` -## 5) Initiate MPI program +## 8) Initiate MPI program Designate **initializing party**, and only the initializing party executes the following steps. You'll need to create a host file in the build directory to run the MPI process. You can do this automatically by running the following command: From 77223e3b774732a16c19977db3adf4044c8e56c5 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 10:55:03 -0400 Subject: [PATCH 19/28] Update setup.md --- setup.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/setup.md b/setup.md index f91595a..9f8a3ee 100644 --- a/setup.md +++ b/setup.md @@ -75,6 +75,11 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc - Hit **"Create subnet"** ## 4) Create VPC Peering Connection +**You are making only one connection** +Some people might get confused and try to create two connections because your instance communicates with two other instances. +However, you only need to create **ONE** connection, as one of your two peers will also establish a connection with you. + +The resulting connections will form a triangle, connecting all participants. 1. Go to the VPC Dashboard 2. In the left-hand navigation pane, select **"Peering Connections"**. @@ -141,13 +146,15 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. -1. Try running the command below in your terminal: +1. Exchange Private IP Address with other participants + +2. Try running the command below in your terminal: ``` ssh -i /path/to/your/key.pem ec2-user@your-public-IP ``` -2. If you successfully SSH into your instance, run the following commands: +3. If you successfully SSH into your instance, run the following commands: ``` sudo yum groupinstall -y "Development Tools" \ @@ -159,10 +166,10 @@ sudo yum groupinstall -y "Development Tools" \ && export PATH=$PATH:/usr/lib64/openmpi/bin \ && which mpicc && which mpicxx \ && cd Secrecy \ -&& mkdir build && cd build && cmake .. && make +&& mkdir build && cd build && cmake .. && make -j ``` -3. Generate an SSH key pair on each instance by running the following command: +4. Generate an SSH key pair on each instance by running the following command: ``` ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa @@ -174,7 +181,7 @@ and below command will show your pubic key just generated: cat ~/.ssh/id_rsa.pub ``` -4. Exchange public keys with other participants. To grant them access, add their public keys to your authorized_keys file. +5. Exchange public keys with other participants. To grant them access, add their public keys to your authorized_keys file. ``` nano ~/.ssh/authorized_keys @@ -182,7 +189,7 @@ nano ~/.ssh/authorized_keys Then, paste their public keys by pressing Command + V (on macOS) or the appropriate paste command for your operating system. -5. In order to enable agent forwarding and disable strict host key checking for all SSH connections, you can add the following configuration to your SSH config file by editing it with: +6. In order to enable agent forwarding and disable strict host key checking for all SSH connections, you can add the following configuration to your SSH config file by editing it with: ``` nano ~/.ssh/config @@ -208,7 +215,7 @@ chmod 600 ~/.ssh/config ``` ## 8) Initiate MPI program -Designate **initializing party**, and only the initializing party executes the following steps. +Designate one of two parties with a dataset as an initializing party, and only the initializing party executes the following steps. You'll need to create a host file in the build directory to run the MPI process. You can do this automatically by running the following command: From fa3c899a361d243abf3ca7732717591f2dfdc609 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 11:18:43 -0400 Subject: [PATCH 20/28] Update setup.md --- setup.md | 61 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/setup.md b/setup.md index 9f8a3ee..1b7a363 100644 --- a/setup.md +++ b/setup.md @@ -4,7 +4,7 @@ This document is a step-by-step guide to setting up a Secrecy Server and OPEN MP ## Introduction -In this guide, you will learn how to create a Virtual Private Cloud (VPC), launch and configure EC2 instances, and set up Secrecy with OpenMPI. This guide assumes you have a basic understanding of AWS services and SSH. +This guide navigates you through how to set up an AWS environment and get the MPC vehicle up and running. ### Prerequisites - AWS Account @@ -13,6 +13,7 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc ## Before You Start **Designate each party to roles 1, 2, and 3** +This step is particularly important to avoid confusion. You will see why in a second. ## 1) Create VPC @@ -21,12 +22,15 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc image 2. Select **VPC and more**. -3. Name your VPC -4. Pick the IPv4 CIDR block: +3. Name your VPC in the following manner: + - role-1: secrecy1 + - role-2: secrecy2 + - role-3: secrecy3 +4. Pick the respective IPv4 CIDR block as follows: - role-1: 10.0.0.0/16 - role-2: 10.1.0.0/16 - role-3: 10.2.0.0/16 -6. Create a VPC with **1 zone** and **public** and zero **private subnets**. +5. Create a VPC with **1 zone** and **public** and zero **private subnets**.
@@ -65,7 +69,10 @@ In this guide, you will learn how to create a Virtual Private Cloud (VPC), launc image 4. Select your VPC from the drop-down. Then the following settings pane pops up. - - Name subnet + - Name subnet as follows: + - role-1: SubScy1 + - role-2: SubScy2 + - role-3: SubScy3 - Set Availability Zone 'us-east-1a' - IPv4 VPC CIDR block should be your VPC CIDR block - IPv4 subnet CIDR block should be set according to your role: @@ -86,19 +93,26 @@ The resulting connections will form a triangle, connecting all participants. 3. Click **"Create Peering Connection"**. Make sure that other parties have already created VPCs at this point image 4. Fill in parameters, - - Name: Give the Peering Connection a name + - Name: Give the Peering Connection a name as below: + - role-1: secrecy12 + - role-2: secrecy23 + - role-3: secrecy31 - VPC ID(Requester): Select your VPC - VPC ID(Accepter): Select in the following way: - - role-1: Select 2 - - role-2: Select 3 - - role-3: Select 1 + - role-1: secrecy2 + - role-2: secrecy3 + - role-3: secrecy1 - If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. 5. Click **"Create Peering Connection"** image 6. Go back to the **"Peering Connections"** Dashboard. -7. Select your Peer Connection, click **"Actions"** at the right top, and hit **Accept request** +7. Select with a radio button your incoming Peer Connection request + - role-1: secrecy31 + - role-2: secrecy12 + - role-3: secrecy23 +8. Click **"Actions"** at the right top, and hit **Accept request** ## 5) Update Route Tables @@ -111,13 +125,13 @@ The resulting connections will form a triangle, connecting all participants. 4. Click **"Add route"** to add a new route: - **Destination**: The CIDR block of the peered VPC - Add the CIDR blocks in the following way: - - If you are role-1: + - role-1: - 10.1.0.0/16 - 10.2.0.0/16 - - If you are role-2: + - role-2: - 10.0.0.0/16 - 10.2.0.0/16 - - If you are role-3: + - role-3: - 10.0.0.0/16 - 10.1.0.0/16 @@ -132,15 +146,18 @@ The resulting connections will form a triangle, connecting all participants. 3. This will take you to Security Groups Dashboard. Click on the Security Group ID image 4. Click **"Edit inbound rules"** -6. Add an inbound rule to allow traffic from the peered VPC’s CIDR block: - - **Type**: Select the desired traffic type (e.g., All traffic or specific ports). - - **Source**: Enter the CIDR block of the peered VPC (e.g., `10.0.0.0/16` for role-1, `10.1.0.0/16` for role-2, and`10.2.0.0/16` for role-3) - - If you are role-1, you would want to add two routes for each role-2 and role-3: - - 10.1.0.0/16 and 10.2.0.0/16 - - If you are role-2: - - 10.0.0.0/16 and 10.2.0.0/16 - - If you are role-3: - - 10.0.0.0/16 and 10.1.0.0/16 +6. Add two inbound rules to allow traffic from the peered VPC’s CIDR blocks: + - **Type**: Select `All TCP` + - **Source**: Enter the CIDR block of the peered VPC + - role-1: + - 10.1.0.0/16 + - 10.2.0.0/16 + - role-2: + - 10.0.0.0/16 + - 10.2.0.0/16 + - role-3: + - 10.0.0.0/16 + - 10.1.0.0/16 ## 7) Access Instance and Network Configuration From bcb5083347f1a8fdbe2abfa7f95c959a09993575 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 11:33:18 -0400 Subject: [PATCH 21/28] Update setup.md --- setup.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/setup.md b/setup.md index 1b7a363..f8812f3 100644 --- a/setup.md +++ b/setup.md @@ -43,11 +43,14 @@ This step is particularly important to avoid confusion. You will see why in a se EC2 Instances -1. Name Instance and select Amazon Linux +1. Name Instance and select Amazon Linux: + - role-1: secrecy1 + - role-2: secrecy2 + - role-3: secrecy3 image -2. Pick an appropriate instance size. +2. Pick t2.micro as an instance size. 3. Generate a key pair if you haven't and save the key to your local machine. From b1e028fed6d5b8d5c19a06dd2d8ff15424710104 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 11:37:53 -0400 Subject: [PATCH 22/28] Update setup.md --- setup.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/setup.md b/setup.md index f8812f3..37aee23 100644 --- a/setup.md +++ b/setup.md @@ -43,20 +43,27 @@ This step is particularly important to avoid confusion. You will see why in a se EC2 Instances -1. Name Instance and select Amazon Linux: +1. Name Instance as follows: - role-1: secrecy1 - role-2: secrecy2 - role-3: secrecy3 image -2. Pick t2.micro as an instance size. +2. Select Amazon Linux -3. Generate a key pair if you haven't and save the key to your local machine. +3. Pick t2.micro as an instance size. + +4. Generate a key pair if you haven't and save the key to your local machine. Key Pair -4. Hit **Edit** in Network settings, pick the VPC you've just created in step 1, enable **Auto-assign public IP** +5. Hit **Edit** in Network settings, pick the VPC you've just created in step 1: + - role-1: secrecy1 + - role-2: secrecy2 + - role-3: secrecy3 + +7. Enable **Auto-assign public IP**
From fa2b8f16639276e312f99ab56d7f07ea5a39842c Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 12:02:16 -0400 Subject: [PATCH 23/28] Update setup.md --- setup.md | 54 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 17 deletions(-) diff --git a/setup.md b/setup.md index 37aee23..02c9ab0 100644 --- a/setup.md +++ b/setup.md @@ -47,6 +47,7 @@ This step is particularly important to avoid confusion. You will see why in a se - role-1: secrecy1 - role-2: secrecy2 - role-3: secrecy3 + image @@ -80,9 +81,9 @@ This step is particularly important to avoid confusion. You will see why in a se 4. Select your VPC from the drop-down. Then the following settings pane pops up. - Name subnet as follows: - - role-1: SubScy1 - - role-2: SubScy2 - - role-3: SubScy3 + - role-1: SubScy1 + - role-2: SubScy2 + - role-3: SubScy3 - Set Availability Zone 'us-east-1a' - IPv4 VPC CIDR block should be your VPC CIDR block - IPv4 subnet CIDR block should be set according to your role: @@ -107,7 +108,10 @@ The resulting connections will form a triangle, connecting all participants. - role-1: secrecy12 - role-2: secrecy23 - role-3: secrecy31 - - VPC ID(Requester): Select your VPC + - VPC ID(Requester): Select your VPC: + - role-1: secrecy1 + - role-2: secrecy2 + - role-3: secrecy3 - VPC ID(Accepter): Select in the following way: - role-1: secrecy2 - role-2: secrecy3 @@ -133,20 +137,36 @@ The resulting connections will form a triangle, connecting all participants. 3. Click **Edit routes** in the Routes tab: image 4. Click **"Add route"** to add a new route: - - **Destination**: The CIDR block of the peered VPC - - Add the CIDR blocks in the following way: - - role-1: - - 10.1.0.0/16 - - 10.2.0.0/16 - - role-2: - - 10.0.0.0/16 - - 10.2.0.0/16 - - role-3: - - 10.0.0.0/16 - - 10.1.0.0/16 - - **Target**: Select the Peering Connection and select corresponding subnets (e.g., `pcx-xxxxxx`) - image +
+ + + + + + + + + + + + + + + + + + + + + + + + +
roledestinationtarget
role-110.1.0.0/16, 10.2.0.0/16secrecy12, secrecy31
role-210.0.0.0/16, 10.2.0.0/16secrecy12, secrecy23
role-310.0.0.0/16, 10.1.0.0/16secrecy31, secrecy23
+ + image + 6. Click **Save routes**. ## 6) Update Security Groups and Network ACLs From 319085fcbbd50fb048c78c516560945f90ba5fa9 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Wed, 11 Sep 2024 15:14:30 -0400 Subject: [PATCH 24/28] Update setup.md --- setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.md b/setup.md index 02c9ab0..8aa7704 100644 --- a/setup.md +++ b/setup.md @@ -213,7 +213,7 @@ sudo yum groupinstall -y "Development Tools" \ && export PATH=$PATH:/usr/lib64/openmpi/bin \ && which mpicc && which mpicxx \ && cd Secrecy \ -&& mkdir build && cd build && cmake .. && make -j +&& mkdir build && cd build && cmake .. && make ``` 4. Generate an SSH key pair on each instance by running the following command: From cf5551b40a40de9aa7f1ccc01eb5e78abad398ee Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Thu, 12 Sep 2024 10:10:18 -0400 Subject: [PATCH 25/28] Update setup.md --- setup.md | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/setup.md b/setup.md index 8aa7704..80defab 100644 --- a/setup.md +++ b/setup.md @@ -103,20 +103,15 @@ The resulting connections will form a triangle, connecting all participants. 2. In the left-hand navigation pane, select **"Peering Connections"**. 3. Click **"Create Peering Connection"**. Make sure that other parties have already created VPCs at this point image -4. Fill in parameters, - - Name: Give the Peering Connection a name as below: - - role-1: secrecy12 - - role-2: secrecy23 - - role-3: secrecy31 - - VPC ID(Requester): Select your VPC: - - role-1: secrecy1 - - role-2: secrecy2 - - role-3: secrecy3 - - VPC ID(Accepter): Select in the following way: - - role-1: secrecy2 - - role-2: secrecy3 - - role-3: secrecy1 - - If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. +4. Select parameters per below: + +| You are..| Peering Connection Name | VPC ID (Requester) | VPC ID (Accepter) | +|----------|-------------------------|--------------------|-------------------| +| role-1 | secrecy12 | secrecy1 | secrecy2 | +| role-2 | secrecy23 | secrecy2 | secrecy3 | +| role-3 | secrecy31 | secrecy3 | secrecy1 | + + If your partners/other parties use a separate AWS account, select "another account" and enter their Account ID. 5. Click **"Create Peering Connection"** image @@ -136,7 +131,11 @@ The resulting connections will form a triangle, connecting all participants. 2. Select the Route Table associated with the subnets in each VPC from the list. 3. Click **Edit routes** in the Routes tab: image -4. Click **"Add route"** to add a new route: +4. Add route + - Click **"Add route"** + - Type in the CIDR Block + - Select 'Peering Connection' in the dropdown. + - It will pop up another dropdown. Select a Peer Connection per the table below (e.g. pcx-xxx (secrecyXY)) From 244883d6163ec493364c15e4aa3cabfa1e3ecd6f Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Thu, 12 Sep 2024 11:38:00 -0400 Subject: [PATCH 26/28] Update setup.md --- setup.md | 29 +++++------------------------ 1 file changed, 5 insertions(+), 24 deletions(-) diff --git a/setup.md b/setup.md index 80defab..0988755 100644 --- a/setup.md +++ b/setup.md @@ -73,26 +73,7 @@ This step is particularly important to avoid confusion. You will see why in a se
-## 3) Setup Subnets -1. Go to the VPC Dashboard -2. In the left-hand navigation pane, select **"Subnets"**. -3. Click **"Create subnet"** -image - -4. Select your VPC from the drop-down. Then the following settings pane pops up. - - Name subnet as follows: - - role-1: SubScy1 - - role-2: SubScy2 - - role-3: SubScy3 - - Set Availability Zone 'us-east-1a' - - IPv4 VPC CIDR block should be your VPC CIDR block - - IPv4 subnet CIDR block should be set according to your role: - - role-1: 10.0.16.0/20 - - role-2: 10.1.16.0/20 - - role-3: 10.2.16.0/20 - - Hit **"Create subnet"** - -## 4) Create VPC Peering Connection +## 3) Create VPC Peering Connection **You are making only one connection** Some people might get confused and try to create two connections because your instance communicates with two other instances. However, you only need to create **ONE** connection, as one of your two peers will also establish a connection with you. @@ -123,7 +104,7 @@ The resulting connections will form a triangle, connecting all participants. - role-3: secrecy23 8. Click **"Actions"** at the right top, and hit **Accept request** -## 5) Update Route Tables +## 4) Update Route Tables ** Ensure that everyone has created a peering connection before implementing this step @@ -168,7 +149,7 @@ The resulting connections will form a triangle, connecting all participants. 6. Click **Save routes**. -## 6) Update Security Groups and Network ACLs +## 5) Update Security Groups and Network ACLs 1. Go to the EC2 Dashboard. 2. Select your instance, choose the **Security** tab, and hit the pop-up link. image @@ -188,7 +169,7 @@ The resulting connections will form a triangle, connecting all participants. - 10.0.0.0/16 - 10.1.0.0/16 -## 7) Access Instance and Network Configuration +## 6) Access Instance and Network Configuration With these steps so far, you should be able to access the EC2 instance and are ready to launch the Secrecy app. @@ -260,7 +241,7 @@ chmod 600 ~/.ssh/id_rsa chmod 600 ~/.ssh/config ``` -## 8) Initiate MPI program +## 7) Initiate MPI program Designate one of two parties with a dataset as an initializing party, and only the initializing party executes the following steps. You'll need to create a host file in the build directory to run the MPI process. You can do this automatically by running the following command: From 7d111eef4afd0e8a2bead5abffb367607d331819 Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Thu, 12 Sep 2024 11:41:08 -0400 Subject: [PATCH 27/28] Update setup.md --- setup.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/setup.md b/setup.md index 0988755..bb75949 100644 --- a/setup.md +++ b/setup.md @@ -110,6 +110,13 @@ The resulting connections will form a triangle, connecting all participants. 1. Go to the **Route Tables** section in the VPC Dashboard. 2. Select the Route Table associated with the subnets in each VPC from the list. + + | You are..| Route Table Name | + |----------|--------------------| + | role-1 | secrecy1-rtb-public| + | role-2 | secrecy2-rtb-public| + | role-3 | secrecy3-rtb-public| + 3. Click **Edit routes** in the Routes tab: image 4. Add route From ea1073d397343220faf150071023424c2877140f Mon Sep 17 00:00:00 2001 From: Snafkin547 <62607343+Snafkin547@users.noreply.github.com> Date: Thu, 12 Sep 2024 15:39:15 -0400 Subject: [PATCH 28/28] Update setup.md --- setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.md b/setup.md index bb75949..b6716b5 100644 --- a/setup.md +++ b/setup.md @@ -106,7 +106,7 @@ The resulting connections will form a triangle, connecting all participants. ## 4) Update Route Tables -** Ensure that everyone has created a peering connection before implementing this step +**Ensure that everyone has created a peering connection before implementing this step** 1. Go to the **Route Tables** section in the VPC Dashboard. 2. Select the Route Table associated with the subnets in each VPC from the list.