Merge branch 'main' of https://github.com/walkeronwater/SOMAS-Final-R…

…eport into main
walkeronwater · Jan 15, 2021 · d5234fa · d5234fa
2 parents d22e697 + aef8aa0
commit d5234fa
Show file tree

Hide file tree

Showing 28 changed files with 283 additions and 41 deletions.
diff --git a/02_specification/index.tex b/02_specification/index.tex
diff --git a/03_gamedesign/index.tex b/03_gamedesign/index.tex
@@ -209,15 +209,14 @@ \subsubsection{Code testing}
 
 While Go is strongly typed and would mean that most errors can be detected at compile-time (or even at time of coding with its performant language server), having unit and integration tests greatly improved the maintainability and ease of development of the project--these issues were particularly important as this was a large-scale group software engineering project--developers stepping on each other's toes is a common occurrence in non-tested group project code. Henceforth, tests were required for non-trivial server-side code.
 
-\subsection{Peer Review}
+\subsubsection{Peer Review}
 
 Peer review was also setup via GitHub pull requests--each change required the approval of another member in the infrastructure team. This practice not only promoted consistent code design and implementation, it minimised mistakes and ensured that the code implementation meets design and implementation requirements set forth. Further, as there was variation in programming skill among code contributors, knowledge sharing was facilitated by code reviews. This was also a critical opportunity for engineers to learn more about the other code being contributed to the project.
 
-\subsection{Continuous Integration}
-\label{sec:GD:implementation:practices:CI}
+\subsubsection{Continuous Integration}
 
-Via GitHub Actions\footnote{https://github.com/features/actions}, continuous integration was set up. Each Pull Request (PR) was set up to trigger automated runs of written tests and a full simulation in addition to static code analysers such as linters. These checks must all pass for the PR to be merged into the main branch. Automated testing and code analysis saved time and facilitates regression testing, as all tests--existing and new--were run for each change. Running a full simulation also served as a good stress-test for the system to make sure that it does not crash on a similar full simulation. Further, automated tests were run on a reference system (Ubuntu 20.04 with Go 1.15.5 and Node 14), helping to prevent system-specific quirks or bugs from polluting the codebase.
+Via GitHub Actions\footnote{https://github.com/features/actions}, continuous integration was set up. Each Pull Request (PR) was set up to trigger automated runs of written tests and a full simulation in addition to static code analysers such as linters. These checks must all pass for the PR to be frequently merged into the main branch. Automated testing and code analysis saved time and facilitates regression testing, as all tests--existing and new--were run for each change. Running a full simulation also served as a good stress-test for the system to make sure that it does not crash on a similar full simulation. Further, automated tests were run on a reference system (Ubuntu 20.04 with Go 1.15.5 and Node 14), helping to prevent system-specific quirks or bugs from polluting the codebase.
 
-\subsection{Continuous Development}
+\subsubsection{Continuous Deployment}
 
-On receiving PR approval, passing automated tests and finally merging into the main codebase, the visualisation website is automatically rebuilt with the latest changes. This saved time as manual builds were not required. The builds were produced on a reference system (identical to that mentioned in~\ref{sec:GD:implementation:practices}), ensuring consistent builds free from system-specific quirks. Further, automated builds meant that the website always runs on the latest codebase.
+On receiving PR approval, passing automated tests and finally merging into the main codebase, the visualisation website is automatically rebuilt with the latest changes. This saved time as manual builds were not required. The builds were produced on a reference system (identical to that mentioned in the Continuous Integration section), ensuring consistent builds free from system-specific quirks. Further, automated builds meant that the website always runs on the latest codebase.
diff --git a/15_simulations/images/baseline_gifts.png b/15_simulations/images/baseline_gifts.png
diff --git a/15_simulations/images/baseline_iigo_roles_1.png b/15_simulations/images/baseline_iigo_roles_1.png
diff --git a/15_simulations/images/baseline_iigo_roles_2.png b/15_simulations/images/baseline_iigo_roles_2.png
diff --git a/15_simulations/images/baseline_iigo_tax_alloc_sanction.png b/15_simulations/images/baseline_iigo_tax_alloc_sanction.png
diff --git a/15_simulations/index.tex b/15_simulations/index.tex
@@ -40,63 +40,106 @@ \section{Metrics}
 \section{Baseline Simulation}
 \label{sec:Simulations:baseline}
 
-In order to make any meaningful analysis one must first have a baseline against which to compare other simulations. In our case, this baseline is a \emph{normal} set of environmental conditions which allows the agents to thrive for a significant period of time, chosen to be around 100 turns. From this baseline, we can then start adjusting the configuration of the simulation by either making disasters more frequent and deadly or making resources more sparse.
+In order to make any meaningful analysis one must first have a baseline against which to compare other simulations. In our case, this baseline is a \emph{normal} set of environmental conditions which allows the agents to thrive for a significant period of time, chosen to be around 100 turns. From this baseline, we can then start adjusting the configuration of the simulation by either making disasters more frequent and deadly or making resources more sparse. The full details of this baseline can be found in Appendix~\ref{sec:Config_Appendix:baseline}.
 
-Note: If a metric is not of interest to a certain area of exploration it will not be shown.
+The sections below illustrate how the metrics will be displayed. For more accuracy some values are aggregated while others that can't be aggregated, such as IIGO Roles, will have multiple graphs shown to display the full range of possibilites using that configuration. No analysis will be done, rather in later sections comparisons will be made against these values and graphs and analysis will be performed then. The reason for this being that the config was tuned to allow the islands to proser and this any analysis would just lead, circularly, back to that conclusion. All graphs shown are available on the website: \url{https://somas2020.github.io/SOMAS2020/\#/}. Here anyone can run their own simulations whilst also being able to choose their own configs.
 
 \subsection{Baseline Numeric Metrics}
 \label{subsec:Simulations:baseline:num_metrics}
+
+Table~\ref{table:Simulations:num_metric} shows the aggregate numeric metrics from 5 seperate simulations run with the baseline config. It should be noted, that due to the randomness of the simulation there are occasions where the metrics vary greatly due to the islands being hit by a large disaster however these are rare and were assumed to be anomolies for the sake of averaging values.
+
+Another point worthy of note is that for Archipelago Survivability and First Island Death they are only 100 because the maximum number of turns that we run the simulation for was 100, and thus island would likely survive much farther than this number if the simulation ran for longer.
 \begin{table}[htb]
     \centering
     \begin{tabular}{|l|c|}
     \hline
     \textbf{Metric}                     & \textbf{Value} \\ \hline
-    \textbf{Archipelago Survivability}  & 50     \\
-    \textbf{First Island Death}         & 50     \\
-    \textbf{Gini Index}                 & 50     \\
-    \textbf{Disasters Survived}         & 50     \\
-    \textbf{ADDM}                       & 50     \\
-    \textbf{AFS}                        & 50     \\ \hline
+    \textbf{Archipelago Survivability}  & 100     \\
+    \textbf{First Island Death}         & 100     \\
+    \textbf{Gini Index}                 & 0.39    \\
+    \textbf{Disasters Survived}         & 20     \\
+    \textbf{ADDM}                       & 341.7     \\
+    \textbf{AFS}                        & 1.92     \\ \hline
 \end{tabular}
 \caption{Numerical Metrics}
+\label{table:Simulations:num_metric}
 \end{table}
 
 \subsection{Baseline IFS}
 \label{subsec:Simulations:baseline:ifs}
+
+Similarly to the table in subsection \ref{subsec:Simulations:baseline:num_metrics}, Table~\ref{table:Simulations:ifs} shows the foraging statistics over 3 simulations run on the baseline config. Once again, these may vary drastically in some simulations of the baseline config where a much larger than average disaster was to hit the archipelago. Those results were not included in the averaging.
+
 \begin{table}[htb]
     \centering
         \begin{tabular}{|p{0.125\textwidth}|>{\centering\arraybackslash}p{0.125\textwidth}|>{\centering\arraybackslash}p{0.125\textwidth}|>{\centering\arraybackslash}p{0.125\textwidth}|}
         \hline
-        Island            & Investment  & Return      & Ratio      \\ \hline
-        \textbf{1: Total} & \textbf{50} & \textbf{50} & \textbf{50}\\
-        Deer              & 50          & 50          & 50         \\
-        Fish              & 50          & 50          & 50         \\ \hline
-        \textbf{2: Total} & \textbf{50} & \textbf{50} & \textbf{50}\\
-        Deer              & 50          & 50          & 50         \\
-        Fish              & 50          & 50          & 50         \\ \hline
-        \textbf{3: Total} & \textbf{50} & \textbf{50} & \textbf{50}\\
-        Deer              & 50          & 50          & 50         \\
-        Fish              & 50          & 50          & 50         \\ \hline
-        \textbf{4: Total} & \textbf{50} & \textbf{50} & \textbf{50}\\
-        Deer              & 50          & 50          & 50         \\
-        Fish              & 50          & 50          & 50         \\ \hline
-        \textbf{5: Total} & \textbf{50} & \textbf{50} & \textbf{50}\\
-        Deer              & 50          & 50          & 50         \\
-        Fish              & 50          & 50          & 50         \\ \hline
-        \textbf{6: Total} & \textbf{50} & \textbf{50} & \textbf{50}\\
-        Deer              & 50          & 50          & 50         \\
-        Fish              & 50          & 50          & 50         \\ \hline
+        Island            & Investment      & Return              & Ratio      \\ \hline
+        \textbf{1: Total} & \textbf{1515.5} & \textbf{5459.5} & \textbf{3.60}\\
+        Deer              & 1118.2           & 712.5           & 1.79        \\
+        Fish              & 397.2          & 4747.0           & 4.24         \\ \hline
+        \textbf{2: Total} & \textbf{17983.4} & \textbf{14935.8} & \textbf{0.83}\\
+        Deer              & 8162.7          & 11204.0       & 1.37         \\
+        Fish              & 9820.7          & 3731.8          & .38         \\ \hline
+        \textbf{3: Total} & \textbf{2343.5} & \textbf{5450.2} & \textbf{2.32}\\
+        Deer              & 831.35          & 2080.2          & 2.50         \\
+        Fish              & 1371.1          & 3370.0          & 2.23         \\ \hline
+        \textbf{4: Total} & \textbf{7005.16} & \textbf{8658.2} & \textbf{1.24}\\
+        Deer              & 3039.7          & 4846.5          & 1.59         \\
+        Fish              & 3965.4          & 4410.7          & 0.96         \\ \hline
+        \textbf{5: Total} & \textbf{2741.1} & \textbf{5625.8} & \textbf{2.05}\\
+        Deer              & 560.5          & 1214.9          & 2.16         \\
+        Fish              & 2180.6          & 4410.8          & 2.02         \\ \hline
+        \textbf{6: Total} & \textbf{5848.2} & \textbf{7445.6} & \textbf{1.27}\\
+        Deer              & 2079.1          & 3970.5          & 1.91         \\
+        Fish              & 3769.2          & 3475.1          & 0.92         \\ \hline
 \end{tabular}
 \caption{Island Foraging Statistics}
+\label{table:Simulations:ifs}
 \end{table}
 
 
 \subsection{Baseline IIGO Tax, Allocations and Sanctions}
 \label{subsec:Simulations:baseline:IIGO}
 
+Due to the similarities between these three metrics it made sense to group then into a single graph. Figure~\ref{fig:Simulations:IIGO_numbers} shows how much was expected to be paid by each island for each metric and how much they actually paid. These metrics are the ones to vary the most across the same config, in terms of absolute values. However the trends, such how much tax Team 1 pays relative to the other teams, tends to stay the same.
+
+\begin{figure}[H] 
+    \centering
+    \includegraphics[width=1\textwidth]{15_simulations/images/baseline_iigo_tax_alloc_sanction.png}
+    \caption{The tax, allocation and sanction metrics for each Island}
+    \label{fig:Simulations:IIGO_numbers}
+\end{figure} 
+
 
 \subsection{Baseline IIGO Roles}
 \label{subsec:Simulations:baseline:IIGO_roles}
 
+Figure~\ref{fig:Simulations:iigo_roles} show's two possibilites for the IIGO roles that occur when running with the base config. Both result in similar values for the other metrics.
+
+\begin{figure}
+    \centering
+    \subfigure[More Tyranical]{
+      \centering
+      \includegraphics[width=.4\linewidth]{15_simulations/images/baseline_iigo_roles_1.png}
+    }
+    \subfigure[More Inclusive]{
+      \centering
+      \includegraphics[width=.4\linewidth]{15_simulations/images/baseline_iigo_roles_2.png}
+    }
+    \caption{Two potential outcomes for role allocations in IIGO}
+    \label{fig:Simulations:iigo_roles}
+\end{figure}
+
 \subsection{Baseline Gifting}
 \label{subsec:Simulations:baseline:trading}
+
+\begin{figure}[H] 
+    \centering
+    \includegraphics[width=1\textwidth]{15_simulations/images/baseline_gifts.png}
+    \caption{A diagram representing the gifts between all islands}
+    \label{fig:Simulations:IITO_gifts}
+\end{figure} 
+
+Note: If a metric is not of interest to a certain area of exploration it will not be shown.
diff --git a/16_results/images/meme.jpg b/16_results/images/meme.jpg
diff --git a/16_results/index.tex b/16_results/index.tex
diff --git a/16_results_and_eval/disaster.tex b/16_results_and_eval/disaster.tex
diff --git a/16_results_and_eval/foraging.tex b/16_results_and_eval/foraging.tex
diff --git a/16_results_and_eval/iigo.tex b/16_results_and_eval/iigo.tex
diff --git a/02_specification/images/meme.jpg → 16_results_and_eval/images/meme.jpg b/02_specification/images/meme.jpg → 16_results_and_eval/images/meme.jpg
diff --git a/16_results_and_eval/index.tex b/16_results_and_eval/index.tex
@@ -0,0 +1,5 @@
+\chapter{Results \& Evaluation}
+
+\include{16_results_and_eval/iigo.tex}
+\include{16_results_and_eval/foraging.tex}
+\include{16_results_and_eval/disaster.tex}
diff --git a/17_evaluation/images/meme.jpg b/17_evaluation/images/meme.jpg
diff --git a/17_evaluation/index.tex b/17_evaluation/index.tex
diff --git a/19_appendix_config/json_configs/baseline_config.json b/19_appendix_config/json_configs/baseline_config.json
@@ -0,0 +1,79 @@
+{
+	"Config": {
+		"MaxSeasons": 100,
+		"MaxTurns": 100,
+		"InitialResources": 50,
+		"InitialCommonPool": 0,
+		"CostOfLiving": 10,
+		"MinimumResourceThreshold": 5,
+		"MaxCriticalConsecutiveTurns": 3,
+		"ForagingConfig": {
+			"DeerHuntConfig": {
+				"MaxDeerPerHunt": 5,
+				"IncrementalInputDecay": 0.9,
+				"BernoulliProb": 0.95,
+				"ExponentialRate": 0.3,
+				"InputScaler": 18,
+				"OutputScaler": 18,
+				"DistributionStrategy": "InputProportionalSplit",
+				"ThetaCritical": 0.97,
+				"ThetaMax": 0.99,
+				"MaxDeerPopulation": 20,
+				"DeerGrowthCoefficient": 0.4
+			},
+			"FishingConfig": {
+				"MaxFishPerHunt": 12,
+				"IncrementalInputDecay": 0.95,
+				"Mean": 1.45,
+				"Variance": 0.1,
+				"InputScaler": 18,
+				"OutputScaler": 18,
+				"DistributionStrategy": "EqualSplit"
+			}
+		},
+		"DisasterConfig": {
+			"XMin": 0,
+			"XMax": 10,
+			"YMin": 0,
+			"YMax": 10,
+			"Period": 5,
+			"SpatialPDFType": "Uniform",
+			"MagnitudeLambda": 1,
+			"MagnitudeResourceMultiplier": 85,
+			"CommonpoolThreshold": 200,
+			"StochasticPeriod": false,
+			"CommonpoolThresholdVisible": false,
+			"PeriodVisible": true,
+			"StochasticPeriodVisible": true
+		},
+		"IIGOConfig": {
+			"IIGOTermLengths": {
+				"Judge": 4,
+				"President": 4,
+				"Speaker": 4
+			},
+			"GetRuleForSpeakerActionCost": 2,
+			"BroadcastTaxationActionCost": 2,
+			"ReplyAllocationRequestsActionCost": 2,
+			"RequestAllocationRequestActionCost": 2,
+			"RequestRuleProposalActionCost": 2,
+			"AppointNextSpeakerActionCost": 2,
+			"InspectHistoryActionCost": 2,
+			"HistoricalRetributionActionCost": 2,
+			"InspectBallotActionCost": 2,
+			"InspectAllocationActionCost": 2,
+			"AppointNextPresidentActionCost": 2,
+			"DefaultSanctionScore": 2,
+			"SanctionCacheDepth": 3,
+			"HistoryCacheDepth": 3,
+			"AssumedResourcesNoReport": 100,
+			"SanctionLength": 5,
+			"SetVotingResultActionCost": 2,
+			"SetRuleToVoteActionCost": 2,
+			"AnnounceVotingResultActionCost": 2,
+			"UpdateRulesActionCost": 2,
+			"AppointNextJudgeActionCost": 2,
+			"StartWithRulesInPlay": true
+		}
+	}
+}
diff --git a/19_appendix_config/sim_configs.tex b/19_appendix_config/sim_configs.tex
@@ -0,0 +1,6 @@
+\chapter{Simulations Configs}
+
+\section{Baseline Config}
+\label{sec:Config_Appendix:baseline}
+
+\lstinputlisting[language=json]{19_appendix_config/json_configs/baseline_config.json}
diff --git a/20_appendix_roles/index.tex b/20_appendix_roles/index.tex
@@ -0,0 +1,28 @@
+\chapter{Roles}
+
+\section{Overall Group Responsibilities}
+\label{sec:roles_appendix:overall}
+
+\begin{table}[!h]
+    \centering
+    \begin{tabular}{|l|l|l|l|l|}
+    \hline
+    \multicolumn{1}{|c|}{\multirow{2}{*}{\textbf{Team}}} & \multicolumn{2}{l|}{\textbf{Design}} & \multicolumn{2}{l|}{\textbf{Infra}}                                     \\ \cline{2-5} 
+    \multicolumn{1}{|c|}{}                               & \textbf{Area}     & \textbf{Reps}    & \textbf{Area} & \textbf{Reps}                                           \\ \hline
+    1                                                    & IITO/IIFO         & Naim             & Core          & \begin{tabular}[c]{@{}l@{}}LH\\ Yusuf\end{tabular}      \\ \hline
+    2                                                    & Core              & Yannis           & IITO/IIFO     & Eirik                                                   \\ \hline
+    3                                                    & IIGO              & Ezgi             & IIGO          & \begin{tabular}[c]{@{}l@{}}Neelesh\\ Preet\end{tabular} \\ \hline
+    4                                                    & IIGO              & Rudolfs          & IIGO          & James K.                                                \\ \hline
+    5                                                    & Enviroment        & Asimina          & Enviroment    & James T.                                                \\ \hline
+    6                                                    & Voting            & Tae              & Voting        & Ning                                                    \\ \hline
+    \end{tabular}
+    \caption{List of roles across the entire class}
+    \label{table:roles_appendix:overall}
+\end{table}
+
+\input{20_appendix_roles/teams/team1.tex}
+\input{20_appendix_roles/teams/team2.tex}
+\input{20_appendix_roles/teams/team3.tex}
+\input{20_appendix_roles/teams/team4.tex}
+\input{20_appendix_roles/teams/team5.tex}
+\input{20_appendix_roles/teams/team6.tex}
diff --git a/20_appendix_roles/teams/team1.tex b/20_appendix_roles/teams/team1.tex
@@ -0,0 +1,36 @@
+\section{Team 1}
+\label{sec:roles_appendix:team1}
+
+\begin{table}[!h]
+    \centering
+    \begin{tabular}{|l|l|}
+    \hline
+    \textbf{Team Member} & \textbf{Areas}     \\ \hline
+    Lin H. Lee  & Infra, Vis, Report         \\
+    Darrick L.  & Infra, Vis                 \\
+    Olly L.     & Design, Vis, Report        \\
+    Naim G.     & Design, Infra, Report, Sim \\
+    Yusuf I.    & Infra                      \\
+    Ethan S.K.  & Vis                        \\
+    Emily M.    & Infra, Report, Sim, Agent  \\
+    Michalis P. & Infra, Report, Sim, Agent  \\ \hline
+\end{tabular}
+\caption{Areas of work team 1 members have contributed to}
+\label{sec:roles_appendix:team1}
+\end{table}
+
+\begin{table}[!h]
+    \centering
+    \begin{tabular}{|l|l|}
+    \hline
+    \textbf{Key}    & \textbf{Meaning}                               \\ \hline
+    Infra  & Worked on infrastructure code or assisted in code review \\
+    Design & Worked on the design of the game                         \\
+    Vis    & Worked on the visualisation/website                      \\
+    Report & Contributed to the reports                               \\
+    Sim    & Worked on the simulation and analysis                    \\
+    Agent  & Worked on team 1's agent                                 \\ \hline
+    \end{tabular}
+    \end{table}
+
+Note: the above tables says nothing about the amount of work a person put in merely the breadth of topics they contributed to.
diff --git a/20_appendix_roles/teams/team2.tex b/20_appendix_roles/teams/team2.tex
@@ -0,0 +1,2 @@
+\section{Team 2}
+\label{sec:roles_appendix:team2}
diff --git a/20_appendix_roles/teams/team3.tex b/20_appendix_roles/teams/team3.tex
@@ -0,0 +1,2 @@
+\section{Team 3}
+\label{sec:roles_appendix:team3}
diff --git a/20_appendix_roles/teams/team4.tex b/20_appendix_roles/teams/team4.tex
@@ -0,0 +1,2 @@
+\section{Team 4}
+\label{sec:roles_appendix:team4}
diff --git a/20_appendix_roles/teams/team5.tex b/20_appendix_roles/teams/team5.tex
@@ -0,0 +1,2 @@
+\section{Team 5}
+\label{sec:roles_appendix:team5}
diff --git a/20_appendix_roles/teams/team6.tex b/20_appendix_roles/teams/team6.tex
@@ -0,0 +1,2 @@
+\section{Team 6}
+\label{sec:roles_appendix:team6}
diff --git a/main.synctex(busy) b/main.synctex(busy)
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		\section{Team 2}
		\label{sec:roles_appendix:team2}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		\section{Team 3}
		\label{sec:roles_appendix:team3}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		\section{Team 4}
		\label{sec:roles_appendix:team4}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		\section{Team 5}
		\label{sec:roles_appendix:team5}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		\section{Team 6}
		\label{sec:roles_appendix:team6}