Merge remote-tracking branch 'origin/v1.0.0'

# Conflicts:
#	CONTRIBUTING.md

.gitignore

@@ -19,3 +19,4 @@ build/*
 *.egg-info/*
 __pycache__
 rfswarm_gui/logs/*
+!Doc/Images/*.html

CONTRIBUTING.md

@@ -1,15 +1,17 @@
 # All contributions are welcome
 
+
 ## Issues
-If you find an issue please check the [Issues](../../issues) page 
-- If it's a known issue please add any aditional information you have to the comments or use the thumbs up reaction on the issue so we know other people have this issue. 
+If you find an issue please check the [Issues](../../issues) page
+- If it's a known issue please add any additional information you have to the comments or use the thumbs up reaction on the issue so we know other people have this issue.
 - If you found a new issue please log it with as many details as you can so we can try to resolve the issue.
 
 ## Updates
-If you would like to work on updates to the code or documentation, please create your own branch to work on your changes and then submit a pull request as that branch. In the pull request and commit comments please reference any known issues you are addressing.
-
-Pull requests to master that have code changes will be rejected, and should be made to the issue branch, once verified will be merged into a release branch.
-
-Pull requests for documentation updates should be made to the branch that the documentation addresses, pull requests for documentation to master may be accepted if the documentation update relates to the current release.
 
+If you would like to work on updates to the code or documentation, we simply ask you follow this simple process:
+1. Have an [Issue](../../issues) registered, there may already be an [Issue](../../issues) created, if so add a comment that you'd like to work on fixing it, if not please create a new [Issue](../../issues) and add a comment you would like to work on it.
+1. Fork this project and create a branch for your changes, make sure to include the issue number in the branch name.
+1. Wait for your issue to get a release number assigned, this will show in the milestone section (if we don't know your working on the issue it might not have a release assigned see step above).
+1. when your changes are ready make a pull request for your branch to the release branch (it will have the same number as the milestone assigned to your issue)
 
+Pull requests to master that have code changes will be rejected, and should be made to a version numbered branch.

Doc/AUT_Monitoring.md

@@ -0,0 +1,207 @@
+[Index](README.md)
+
+## AUT Monitoring
+
+The ability to monitor your Application Under Test (AUT) servers and store the monitoring data in the test results has been available as part of the rfswarm Manager since version v0.6.3 and was introduced from feature request #72.
+
+With feature release v0.8.0 this became more useful as you could use the live graphs to monitor this data during a test, and now with release 1.0.0 you can easily include this information in your test reports.
+
+- [Overview](#Overview)
+- [Unix AUT Example](#Unix-AUT-Example)
+
+### Overview
+
+While it may not be obvious at first the process for monitoring and reporting the performance data of your AUT is quite simple and quite flexible.
+
+1. Create a robot framework test case that connects to your AUT server and gathers the performance data you are interested in collecting, as a minimum you will probably want to collect CPU, Memory and Disk IO Information, you may also want to collect network IO, and then depending on the type of server you are monitoring there may be other details you want to monitor.
+1. report the details back to the manager using the rfswarm API [POST /Metric](Agent_Communication.md#post-metric)
+1. to make this easier the variable `${RFS_SWARMMANAGER}` as documented in the [Swarm Manager](Preparing_for_perf.md#swarm-manager) section of [Useful Variables](Preparing_for_perf.md#useful-variables), can be used to avoid hard coding the manager details.
+
+### Unix (Linux) AUT Example
+
+The robot file below is an example of connecting to a unix (linux) AUT server via a SSH session using the robot framework SSH library, collecting a variety of statistics from the AUT server and then posting those details to the Manager API using the robot framework Requests Library.
+
+This example may work for you, or you may need to modify it to work with the OS that your AUT uses. It is not intended as a ready to use example, but rather a starting point to help you build a monitoring script for your AUT.
+
+ssh_example.robot
+```
+*** Settings ***
+
+Library                SSHLibrary
+Suite Teardown         Close All Connections
+
+Library					String
+Library					Collections
+Library					RequestsLibrary
+Library					json
+
+*** Variables ***
+${HOST}                Undefined
+${USERNAME}            Undefined
+${PASSWORD}            Undefined
+
+*** Test Cases ***
+Monitor Linux AUT
+	${HOST}=		Set Variable    AUT_Server
+	${USERNAME}=	Set Variable    AUT_System_User
+	${PASSWORD}=	Set Variable    AUT_System_Pass
+
+	# Connect SSH Session to AUT server
+	Open Connection And Log In	${HOST}	${USERNAME}	${PASSWORD}
+
+	# Gather the AUT Server statistics every 5 seconds
+	# 3600 / 5 = 720 - poll every 5 seconds for 1 hour
+	FOR 	${i}	IN RANGE	720
+		${epoch}=		Get Time	epoch
+		${stats}=	Collect Stats
+		Post AUT Stats		${HOST} 	AUT Web 	${epoch}    ${stats}
+		Sleep	4
+	END
+
+	# at the end of the hour the test will end and disconnect the SSH session (Suite Teardown) if your test scenario is
+	# configured to run for more than an hour the agent will automatically restart the test.
+
+*** Keywords ***
+
+
+Open Connection And Log In
+	[Arguments] 	${HOST}=${HOST} 	${USERNAME}=${USERNAME} 	${PASSWORD}=${PASSWORD}
+	# keywords from SHH Library
+	Open Connection     ${HOST}		width=160
+	Login               ${USERNAME}        ${PASSWORD}
+
+
+Collect Stats
+	${lscpu}=	lscpu
+	${vmstat}=	vmstat
+
+	${stats}=	Create Dictionary
+	FOR 	${k}	IN		@{lscpu.keys()}
+		log 	${k}
+		Set To Dictionary	${stats} 	lscpu: ${k}		${lscpu["${k}"]}
+	END
+	FOR 	${k}	IN		@{vmstat.keys()}
+		log 	${k}
+		log 	${vmstat["${k}"]}
+		Set To Dictionary	${stats} 	vmstat: ${k}		${vmstat["${k}"]}
+	END
+	[Return]	${stats}
+
+
+
+lscpu
+	Write              lscpu
+	${output}=         Read Until       $
+	# @{lines}=	Split To Lines	${output}
+	@{lines}=	Set Variable	${output.splitlines()}
+
+	${dict}=	Create Dictionary
+	FOR 	${line} 	IN		@{lines}
+		${key}	${val}=		Set Variable	${line.split(":")}
+		Set To Dictionary	${dict} 	${key.strip()}		${val.strip()}
+	END
+
+	[Return] 	${dict}
+
+
+
+vmstat
+	${output}=		Execute Command		vmstat -w 1 2
+	@{lines} =	Split To Lines	${output}
+	log 	${lines[-1]}
+	${line}=		Set Variable	${lines[-1]}
+	@{vals} =	Split String	${line}
+	log 	${vals}
+
+	${vmstat}=	Create Dictionary
+	# Proc
+	#
+	# r: The number of runnable processes. These are processes that have been launched and are either running or are waiting for their next time-sliced burst of CPU cycles.
+	# b: The number of processes in uninterruptible sleep. The process isn’t sleeping, it is performing a blocking system call, and it cannot be interrupted until it has completed its current action. Typically the process is a device driver waiting for some resource to come free. Any queued interrupts for that process are handled when the process resumes its usual activity.
+	Set To Dictionary	${vmstat} 	Processes: Runnable 		${vals[0]}
+	Set To Dictionary	${vmstat} 	Processes: Uninterruptible	${vals[1]}
+
+
+	# Memory
+	#
+	# swpd: the amount of virtual memory used. In other words, how much memory has been swapped out.,
+	# free: the amount of idle (currently unused) memory.
+	# buff: the amount of memory used as buffers.
+	# cache: the amount of memory used as cache.
+	Set To Dictionary	${vmstat} 	Memory: Swap 	${vals[2]}
+	Set To Dictionary	${vmstat} 	Memory: Free 	${vals[3]}
+	Set To Dictionary	${vmstat} 	Memory: Buffers	${vals[4]}
+	Set To Dictionary	${vmstat} 	Memory: Cache 	${vals[5]}
+
+
+	# Swap
+	#
+	# si: Amount of virtual memory swapped in from swap space.
+	# so: Amount of virtual memory swapped out to swap space.
+	Set To Dictionary	${vmstat} 	Swap: Swapped In 	${vals[6]}
+	Set To Dictionary	${vmstat} 	Swap: Swapped Out	${vals[7]}
+
+
+	# IO
+	#
+	# bi: Blocks received from a block device. The number of data blocks used to swap virtual memory back into RAM.
+	# bo: Blocks sent to a block device. The number of data blocks used to swap virtual memory out of RAM and into swap space.
+	Set To Dictionary	${vmstat} 	IO: Blocks received	${vals[8]}
+	Set To Dictionary	${vmstat} 	IO: Blocks sent 	${vals[9]}
+
+
+	# System
+	#
+	# in: The number of interrupts per second, including the clock.
+	# cs: The number of context switches per second. A context switch is when the kernel swaps from system mode processing into user mode processing.
+	Set To Dictionary	${vmstat} 	System: Interrupts/s		${vals[10]}
+	Set To Dictionary	${vmstat} 	System: Context Switches/s	${vals[11]}
+
+
+	# CPU
+	#
+	# These values are all percentages of the total CPU time.
+	#
+	# us: Time spent running non-kernel code. That is, how much time is spent in user time processing and in nice time processing.
+	# sy: Time spent running kernel code.
+	# id: Time spent idle.
+	# wa: Time spent waiting for input or output.
+	# st: Time stolen from a virtual machine. This is the time a virtual machine has to wait for the hypervisor to finish servicing other virtual machines before it can come back and attend to this virtual machine.
+	Set To Dictionary	${vmstat} 	CPU: User	${vals[12]}
+	Set To Dictionary	${vmstat} 	CPU: System	${vals[13]}
+	Set To Dictionary	${vmstat} 	CPU: Idle	${vals[14]}
+	Set To Dictionary	${vmstat} 	CPU: Wait	${vals[15]}
+	Set To Dictionary	${vmstat} 	CPU: Stolen	${vals[16]}
+
+
+	#
+	[Return] 	${vmstat}
+
+
+
+
+Post AUT Stats
+	[Documentation]		SSH: Post AUT Stats
+	[Arguments]		${AUT}	${AUTType}	${AUTTime}    ${Stats}
+
+	# keyword from Requests Library, reuse the session rather than creating a new one if possible
+	${exists}= 	Session Exists	rfs
+	Run Keyword Unless 	${exists} 	Create Session	rfs 	${RFS_SWARMMANAGER}
+
+	${data}=	Create Dictionary
+	Set To Dictionary	${data} 	PrimaryMetric		${AUT}
+	Set To Dictionary	${data} 	MetricType			${AUTType}
+	Set To Dictionary	${data} 	MetricTime			${AUTTime}
+	Set To Dictionary	${data} 	SecondaryMetrics	${Stats}
+
+	# keyword from json Library
+	${string_json}= 	json.Dumps	${data}
+
+	# keyword from Requests Library
+	${resp}=	Post Request	rfs 	/Metric 	${string_json}
+	Log	${resp}
+	Log	${resp.content}
+	Should Be Equal As Strings	${resp.status_code}	200
+
+
+```

Doc/About.md

@@ -14,4 +14,4 @@ rfswarm aims to solve this problem by allowing you to take an existing functiona
 
 rfswarm is written completely in python, so if you are already using Robot Framework, then you will already have most of what you need to use rfswarm and will be familiar with pip to get any extra components you need.
 
-[See who has sponsored this project](Doc/Sponsors.md)
+[See who has sponsored this project](Sponsors.md)

Doc/Agent_Communication.md

@@ -130,6 +130,8 @@ Response Body:
     "StartTime": 1572057404,
     "EndTime": 1572064628,
     "RunName": "Scenario_1572057404",
+	"Abort": false,
+	"UploadMode": "err",
     "Schedule": {
         "1_1": {
             "ScriptHash": "c4307dee904afe7df89fa33d193a7d30",

Doc/FAQ.md

@@ -3,6 +3,15 @@
 
 # rfswarm (Robot Framework Swarm) Frequently Asked Questions
 
+- [Can I run the Agent and the Manager on the same machine?](#can-i-run-the-agent-and-the-manager-on-the-same-machine)
+- [The Agent doesn't connect to the Manager?](#the-agent-doesnt-connect-to-the-manager)
+- [I have some experience in performance testing, can you translate the terminology between rfswarm and tool xzy?](#i-have-some-experience-in-performance-testing-can-you-translate-the-terminology-between-rfswarm-and-tool-xzy)
+- [Can you help me get started? Which test cases should I choose?](#can-you-help-me-get-started-which-test-cases-should-i-choose)
+- [Do we have any sample test cases?](#do-we-have-any-sample-test-cases)
+- [Is there a tutorial on how to use rfswarm?](#is-there-a-tutorial-on-how-to-use-rfswarm)
+- [does rfswarm support IPv6?](#does-rfswarm-support-ipv6)
+
+
 ## Can I run the Agent and the Manager on the same machine?
 
 Yes running the Agent and the Manager on the same machine is ok for small numbers of robots (users), but if you want to run any significant load then you will probably need some separate machines for the agent.
@@ -21,6 +30,7 @@ This should cover off the main components with the most common tools:
 |-------|----------|------|
 |Manager|Controller|JMeter client (JMeter GUI)|
 |Agents|Agent process (sometimes called Load Generators or Injectors)|JMeter servers (JMeterEngine)|
+|Reporter|Analysis||
 |Scenario|Scenario|Test Plan|
 |Test Case|Script|Thread Group|
 

Doc/HardwareRequirements.md

@@ -28,3 +28,13 @@ A [2013 11" Macbook Air 1.3GHz dual-core i5 4Gb ram](https://support.apple.com/k
 |---				|---	|---				|---					|
 |SeleniumLibrary	|30		| Memory (Ram) ~80%	|40 					|
 |RequestsLibrary	|230	| Memory (Ram) >70%, CPU > 60% at times|240	|
+
+## Reporter
+
+The Hardware Requirements for the Reporter are a bit higher, it will depend on how many sections and subsections you have in your report and how complicated the graphs are, but these minimums should allow you to plan a reasonable report. The most memory intensive report I have built so far used around 400MB of RAM after updating the preview and while exporting the report in html, word and excel at the same time.
+
+Recommended minimums:
+
+|CPU|Memory|Disk|
+|---|---|---|
+|Dual Core 1Ghz| 1GB above OS Requirements|1GB free|

Doc/Index.md

@@ -3,10 +3,12 @@
 ## Index
 1. [About](About.md)
 1. [Overview / Concepts](Overview.md)
-1. [rfswarm.py (Manager)](rfswarm_py.md)
-1. [rfswarm_agent.py (Agent)](rfswarm_agent_py.md)
+1. [rfswarm Manager](rfswarm_manager.md)
+1. [rfswarm Agent](rfswarm_agent.md)
+1. [rfswarm Reporter](rfswarm_reporter.md)
 1. [Hardware Requirements](HardwareRequirements.md)
 1. [Preparing a test case for performance](Preparing_for_perf.md)
+1. [AUT Monitoring](AUT_Monitoring.md)
 1. [Agent Communication](Agent_Communication.md)
 1. [Screen Shots](Screen_Shots.md)
 1. [FAQ](FAQ.md)

Doc/Overview.md

@@ -3,11 +3,12 @@
 
 ## Overview / Concepts
 
-rfswarm is made up of 2 components, the [Manager](rfswarm_py.md) where you plan and run your test scenario and the [Agents](rfswarm_agent_py.md) which runs the Robot Framework tests.
+rfswarm is made up of 3 components, the [Manager](rfswarm_manager.md) where you plan and run your test scenario, the [Agents](rfswarm_agent.md) which runs the Robot Framework tests, and the [Reporter](rfswarm_reporter.md)
 
 - [Manager](#manager)
 - [Agents](#Agents)
 - [Robot File handling (transfer from Manager to Agent)](#robot-file-handling-transfer-from-manager-to-agent)
+- [Reporter](#Reporter)
 
 
 <kbd>
@@ -70,3 +71,8 @@ The Agent then:
 	- The Agent update it's in memory file list using the hash as the key, the relative path and the local path (this is likely very different to the local path on the Manager) as properties of the hash.
 
 It is not recommended to use fixed paths in your robot file unless you are sure that the path will be the same on the Manager and all Agents.
+
+
+### Reporter
+
+This component is used to assist you in reporting the results of your performance tests and is a completly optional component. You only need one reporter machine, and if your test schedule is not too busy this can be the same machine as the manager.

Doc/Preparing_for_perf.md

@@ -176,7 +176,7 @@ Match Keyword
 ```
 
 #### No Operation
-Would not have a timing measured by default because this keyword belongs to the builtin which is one of the default [excluded libraries](./rfswarm_agent_py.md#exclude-libraries)
+Would not have a timing measured by default because this keyword belongs to the builtin which is one of the default [excluded libraries](./rfswarm_agent.md#exclude-libraries)
 
 #### Example Keyword
 Would have a timing measured by default, this would be reported in the Manager as "TC01 My Example Keyword" along with the time taken to perform the step No Operation

Doc/README.md

@@ -3,10 +3,12 @@
 ## Index
 1. [About](About.md)
 1. [Overview / Concepts](Overview.md)
-1. [rfswarm.py (Manager)](rfswarm_py.md)
-1. [rfswarm_agent.py (Agent)](rfswarm_agent_py.md)
+1. [rfswarm Manager](rfswarm_manager.md)
+1. [rfswarm Agent](rfswarm_agent.md)
+1. [rfswarm Reporter](rfswarm_reporter.md)
 1. [Hardware Requirements](HardwareRequirements.md)
 1. [Preparing a test case for performance](Preparing_for_perf.md)
+1. [AUT Monitoring](AUT_Monitoring.md)
 1. [Agent Communication](Agent_Communication.md)
 1. [Screen Shots](Screen_Shots.md)
 1. [FAQ](FAQ.md)