Q: Where are the relevant logs for perfSONAR services?¶
A: Please see Reading Log Files for more information.
+Q: How do I enable log compression with logrotate.conf?¶
+A: Sometimes, log files can grow bigger and consume disk space even before their rotation. Logrotate is a system utility installed by default and is configured to handle log rotation for all installed packages and applications. In order to enable system-wide log files compression change default settings in /etc/logrotate.conf and uncomment compress option. This implies that rotated files will be compressed with log files having a .gz file extension.
Q: Can I use a firewall?¶
A: Please see Firewalls and Security Software.
@@ -450,6 +454,7 @@Q: How does version numbering work for the perfSONAR project?Host and Network Administration Questions
- Q: Where are the relevant logs for perfSONAR services?
+- Q: How do I enable log compression with logrotate.conf?
- Q: Can I use a firewall?
- Q: How many NTP servers do I need, can I select them all?
- Q: When setting up a dual homed host, how can one get individual tests to use one interface or another?
diff --git a/release_candidates/5.1.0/_sources/FAQ.rst.txt b/release_candidates/5.1.0/_sources/FAQ.rst.txt
index da1721e7..d8411334 100644
--- a/release_candidates/5.1.0/_sources/FAQ.rst.txt
+++ b/release_candidates/5.1.0/_sources/FAQ.rst.txt
@@ -175,6 +175,11 @@ Q: Where are the relevant logs for perfSONAR services?
**A:** Please see :doc:`manage_logs` for more information.
+Q: How do I enable log compression with logrotate.conf?
+========================================================================================================================================================================================
+**A:** Sometimes, log files can grow bigger and consume disk space even before their rotation. ``Logrotate`` is a system utility installed by default and is configured to handle log rotation for all installed packages and applications. In order to enable system-wide log files compression change default settings in ``/etc/logrotate.conf`` and uncomment ``compress`` option. This implies that rotated files will be compressed with log files having a ``.gz`` file extension.
+
+
Q: Can I use a firewall?
========================================================================================================================================================================================
**A:** Please see :doc:`manage_security`.
diff --git a/release_candidates/5.1.0/_sources/cookbook_central_archive.rst.txt b/release_candidates/5.1.0/_sources/cookbook_central_archive.rst.txt
index 23bc7d5e..7135aed0 100644
--- a/release_candidates/5.1.0/_sources/cookbook_central_archive.rst.txt
+++ b/release_candidates/5.1.0/_sources/cookbook_central_archive.rst.txt
@@ -44,26 +44,25 @@ In this step we'll setup the *archive host* store measurements. Specifically we'
* **Debian/Ubuntu**::
cd /etc/apt/sources.list.d/
- curl -o perfsonar-release.list http://downloads.perfsonar.net/debian/perfsonar-release.list
- curl http://downloads.perfsonar.net/debian/perfsonar-official.gpg.key | apt-key add -
+ curl -o perfsonar-minor-snapshot.list downloads.perfsonar.net/debian/perfsonar-minor-snapshot.list
+ curl http://downloads.perfsonar.net/debian/perfsonar-snapshot.gpg.key | apt-key add -
apt-update
-4. Next we'll install the perfsonar-archive package. This will setup OpenSearch and Logstash. The command will again depend on the operating system::
+4. Next we'll install the perfsonar-archive package. This will setup OpenSearch and Logstash. The command will again depend on the operating system:
* **RedHat-based (e.g. Rocky, Alma)**::
-
+
dnf install perfsonar-archive
* **Debian/Ubuntu**::
apt install perfsonar-archive
-5. Let's quickly verify Opensearch and Logstash are running. You should see both listed as **active (running)** when you execute the following commands::
+5. Let's quickly verify the archive is running with the *psarchive troubleshoot* utility. It will check that components such as OpenSearch and Logstash are running as well as verify authentication credentials. It can also check if the archive has data, but since we have not yet configured our measurement hosts we will skip that check with the `--skip-opensearch-data` option. Run the command as follows and if everything is marked as *OK* then proceed, otherwise follow the instructions in the command output to debug::
- systemctl status opensearch
- systemctl status logstash
+ psarchive troubleshoot --skip-opensearch-data
-6. Next we'll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit **/etc/httpd/conf.d/apache-logstash.conf** to include the example IPs (see the diagram at the top of this guide). Note we'll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the *Require ip* lines are the relevant portions)::
+6. Next we'll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit **/etc/httpd/conf.d/apache-logstash.conf** (Rocky/Alma) or **/etc/apache2/conf-available/apache-logstash.conf** (Debian/Ubuntu) to include the example IPs (see the diagram at the top of this guide). Note we'll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the *Require ip* lines are the relevant portions)::
ProxyRequests Off
@@ -133,7 +132,7 @@ Now that the host can store measurements, let's prepare to visualize them. You s
.. image:: images/cbca_gf_verify.png
-3. Navigate to the **perfSONAR Host Metrics** page. There may be a link under "Recently Viewed Dashboards" (see previous screenshot) or type "perfSONARHost Metrics" in the search bar:
+3. Navigate to the **perfSONAR Host Metrics** page. There may be a link under "Recently Viewed Dashboards" (see previous screenshot) or type "perfSONAR Host Metrics" in the search bar:
.. image:: images/cbca_gf_search_hm.png
@@ -144,7 +143,7 @@ Now that the host can store measurements, let's prepare to visualize them. You s
We now have a working Grafana installation. For now there is not much to show beyond host statistics, but we will revisit this after we have the measurements running. In the next step, we'll define the measurements to run.
-Step 3: Build and Publish Measurement Configuration with pSConfig
+Step 3: Creating Your Measurement Configuration with pSConfig
==================================================================
In this step we'll create a file that defines the measurements we want all the testpoints to run. This is called a **pSConfig Template File**. If you want to learn more about pSConfig templates see :doc:`psconfig_intro`. In this example we'll generate the file by hand from a skeleton file then use some helpful utilities to fill-in and publish. Steps are as follows:
@@ -159,7 +158,7 @@ In this step we'll create a file that defines the measurements we want all the t
3. The first thing we'll add to the pSConfig template is instructions that tell testpoints how to send results to the archive. There is a helper script that helps us generate this definition. You will pass it a `-n` that tells it the public address of your archive. In our example it is *archive.local* but change that to the address of your archive host. The command for this example looks like the following::
- /usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local
+ /usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local -a ip
4. You output will look similar to the following (with archive.local replaced with your hostname)::
@@ -180,7 +179,7 @@ In this step we'll create a file that defines the measurements we want all the t
}
}
-4. We are going to copy and paste above into our `psconfig.json` file under the *archives* section and name it *example_http_archive*. This is what our psconfig.json looks like when we are done::
+5. We are going to copy and paste above into our `psconfig.json` file under the *archives* section and name it *example_http_archive*. This is what our psconfig.json looks like when we are done::
{
"_meta":{
@@ -209,7 +208,7 @@ In this step we'll create a file that defines the measurements we want all the t
"addresses": {
...
-5. Now well add the addresses of our testpoints and define a group that build a mesh of tests between all the testpoints::
+6. Now well add the addresses of our testpoints and define a group that build a mesh of tests between all the testpoints::
...
@@ -230,7 +229,7 @@ In this step we'll create a file that defines the measurements we want all the t
}
},
...
-6. The next section defines the test specifications, schedules and tasks. We'll just keep the defaults of the skeleton, but you can adjust these to meet your needs. Any test type or option supported by pscheduler is supported by this file. For reference, below is exactly what is in skeleton file::
+7. The next section defines the test specifications, schedules and tasks. We'll just keep the defaults of the skeleton, but you can adjust these to meet your needs. Any test type or option supported by pscheduler is supported by this file. For reference, below is exactly what is in skeleton file::
...
"tests": {
@@ -278,7 +277,8 @@ In this step we'll create a file that defines the measurements we want all the t
"schedule": "example_schedule_PT4H",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Throughput Tests"
+ "display-task-name": "Example Throughput Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_latencybg": {
@@ -286,7 +286,8 @@ In this step we'll create a file that defines the measurements we want all the t
"test": "example_test_latencybg",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Loss Tests"
+ "display-task-name": "Example Loss Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_trace": {
@@ -295,39 +296,45 @@ In this step we'll create a file that defines the measurements we want all the t
"schedule": "example_schedule_PT10M",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Traceroute Tests"
+ "display-task-name": "Example Traceroute Tests",
+ "display-task-group": [ "Example Tests" ]
}
}
}
-7. Save the file when you are done.
+8. Save the file when you are done.
+
+9. Run the following to validate your JSON file and correct any errors found::
+
+ psconfig validate psconfig.json
-8. We will now publish the file to a local web server so the testpoints can download it. We do this will the following command::
+Step 4: Publishing Your Measurement Configuration with pSConfig
+==================================================================
+1. We will now publish the file to a local web server so the testpoints can download it. We do this will the following command::
+
psconfig publish psconfig.json
-9. The output of the command looks like the following::
+2. The output of the command looks like the following::
Success! File saved to /usr/lib/perfsonar/web-psconfig/psconfig.json
Published file can be accessed at https://archive.local/psconfig/psconfig.json
- Execute the following on a host running an agent to use this file:
+ Execute the following on a host running an agent to use this file:
psconfig remote add "https://archive.local/psconfig/psconfig.json"
-10. That last command will be important when we setup the testpoints. We will also run it right now on our archive host (replace archive.local with your hostname)::
+3. That last command will be important when we setup the testpoints. We will also run it right now on our archive host (replace archive.local with your hostname)::
psconfig remote add "https://archive.local/psconfig/psconfig.json"
-11. The previous command tells a pSConfig agent running on the host to setup some dashboards. It also tells it to gather host statistics from each of the testpoints. We aren't getting any results yet, but lets take a quick look at the dashboards it created by visiting **https://archive.local/grafana/dashboards**.
+4. The previous command tells a pSConfig agent running on the host to setup some dashboards. It also tells it to gather host statistics from each of the testpoints. We aren't getting any results yet, but lets take a quick look at the dashboards it created by visiting **https://archive.local/grafana/dashboards**.
-12. Click the folder **perfSONAR pSConfig** folder on the page that load.
+5. Click the folder **perfSONAR pSConfig** folder on the page that load.
-13. Click on **All perfSONAR Measurements**. The dashboard doesn't have any data, so let's fix that by having our testpoints run some measurements.
+6. Click on **All perfSONAR Measurements**. The dashboard doesn't have any data, so let's fix that by having our testpoints run some measurements.
.. image:: images/cbca_gf_nodata.png
-
-
Step 4: Setup perfSONAR Testpoint hosts
========================================================
We will now logout of the archive host and login to a testpoint host. The steps in this section we will repeat for each of our testpoint hosts.
@@ -354,10 +361,10 @@ We will now logout of the archive host and login to a testpoint host. The steps
curl http://downloads.perfsonar.net/debian/perfsonar-official.gpg.key | apt-key add -
apt-update
-4. Next we'll install the perfsonar-testpoint package. The command will again depend on the operating system::
+4. Next we'll install the perfsonar-testpoint package. The command will again depend on the operating system:
* **RedHat-based (e.g. Rocky, Alma)**::
-
+
dnf install perfsonar-testpoint
* **Debian/Ubuntu**::
diff --git a/release_candidates/5.1.0/_sources/esmond_api_rest.rst.txt b/release_candidates/5.1.0/_sources/esmond_api_rest.rst.txt
index fb2dc390..3bc9259a 100644
--- a/release_candidates/5.1.0/_sources/esmond_api_rest.rst.txt
+++ b/release_candidates/5.1.0/_sources/esmond_api_rest.rst.txt
@@ -186,7 +186,7 @@ The **dns-match-rule** parameter controls what DNS lookups the server will perfo
A quick example that returns only results with a source that matches the AAAA record of host1.example.net is below:
::
- curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=v6-only"
+ curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=only-v6"
Searching by Event Type
diff --git a/release_candidates/5.1.0/_sources/install_debian.rst.txt b/release_candidates/5.1.0/_sources/install_debian.rst.txt
index 580247f4..b86b4551 100644
--- a/release_candidates/5.1.0/_sources/install_debian.rst.txt
+++ b/release_candidates/5.1.0/_sources/install_debian.rst.txt
@@ -89,6 +89,11 @@ Step 2: Install a Bundle
.. note:: The ``perfsonar-archive`` bundle is also supported on Debian 11 and Ubuntu 22.
+Setting the default user password in PostgreSQL
+++++++++++++++++++
+With a regular ``apt install``, the user is prompted to enter a password for the new PostgreSQL database created during installation. You can choose this password arbitrarily or leave it empty (the DB is only accessible locally).
+Alternatively, you can install packages with APT in non-interactive mode to avoid the password prompt by setting the following environment variable when installing: ``DEBIAN_FRONTEND=noninteractive apt install perfsonar-toolkit``
+
Optional Packages
++++++++++++++++++
In addition to any of the bundles above you may also **optionally** choose to install one or more of our add-on packages (these are automatically added on the perfsonar-toolkit bundle):
diff --git a/release_candidates/5.1.0/_sources/pscheduler_ref_tests_tools.rst.txt b/release_candidates/5.1.0/_sources/pscheduler_ref_tests_tools.rst.txt
index 2c56c3b9..76a250b4 100644
--- a/release_candidates/5.1.0/_sources/pscheduler_ref_tests_tools.rst.txt
+++ b/release_candidates/5.1.0/_sources/pscheduler_ref_tests_tools.rst.txt
@@ -47,7 +47,7 @@ dns
http
############
:Description: Measure HTTP response time
-:Tools: psurl
+:Tools: curl
latency
############
@@ -120,15 +120,23 @@ Throughput tests support the following arguments::
-t DURATION, --duration=DURATION
Total runtime of test
-i INTERVAL, --interval=INTERVAL
- How often to report results (internally, results still reported in aggregate at end)
+ How often to report results (internally, results still
+ reported in aggregate at end)
+ --link-rtt=LINK_RTT Approximate link round-trip time (ISO8601 or integer
+ ms)
-P PARALLEL, --parallel=PARALLEL
How many parallel streams to run during the test
-u, --udp Use UDP instead of TCP testing
-b BANDWIDTH, --bandwidth=BANDWIDTH
Bandwidth to rate limit the test to, supports SI
notation such as 1G
+ --bandwidth-strict Never go faster than --bandwidth, even to make up for
+ lost time.
+ --burst-size=BURST_SIZE
+ Limit bursts of packets to this number
-w WINDOW_SIZE, --window-size=WINDOW_SIZE
- TCP window size to use for the test, supports SI notation such as 64M
+ TCP window size to use for the test, supports
+ SI notation such as 64M
-m MSS, --mss=MSS TCP maximum segment size
-l BUFFER_LENGTH, --buffer-length=BUFFER_LENGTH
length of the buffer to read/write from
@@ -148,8 +156,14 @@ Throughput tests support the following arguments::
Set's the sending side's CPU affinity
--server-cpu-affinity=SERVER_CPU_AFFINITY
Set's the receiving's side's CPU affinity
+ --single-ended Run a test directly to a host without pscheduler.
+ --single-ended-port=SINGLE_ENDED_PORT
+ Run a test directly to a given port.
--reverse Reverses the direction of the test.
-
+ --reverse-connections
+ Make connections from destination to source where
+ possible.
+ --loopback Run both client and server side in a loopback test.
The currently supported throughput tools are *iperf2*, *iperf3*, and *nuttcp*. *iperf3* is the default.
Note that not every tool supports every option. The following table summarizes tool specific option.
@@ -210,6 +224,9 @@ Latency tests support the following arguments::
-f, --flip In multi-participant mode, have the dest start the
client and request a reverse test. Useful in some
firewall and NAT environments.
+ -r, --reverse Report results in the reverse direction (destination
+ to source) if possible.
+ --traverse-nat Take steps to traverse outbound NAT
-R, --output-raw Output individual packet statistics. This will
substantially increase the size of a successful
result.
@@ -298,9 +315,18 @@ HTTP tests support the following arguments::
-h, --help show this help message and exit
--url=URL URL to query
+ --header=HEADERS HTTP header for request, format 'HeaderName: Value'.
+ May be repeated.
--parse=PARSE String to parse for
--host=HOST Host to run the test
--host-node=HOST_NODE Host to run the test
+ --ip-version=IP_VERSION
+ Specificy which IP version to use, 4 or 6
+ --always-succeed Treat HTTP failures as successes, regardless of
+ response code
+ --keep-content=KEEP_CONTENT
+ Amount of content to keep in bytes; default none, 0
+ for no limit, SI units supported
--timeout=TIMEOUT Timeout for each query attempt
dns Tests
diff --git a/release_candidates/5.1.0/cookbook_central_archive.html b/release_candidates/5.1.0/cookbook_central_archive.html
index 28d7678b..f7d47f13 100644
--- a/release_candidates/5.1.0/cookbook_central_archive.html
+++ b/release_candidates/5.1.0/cookbook_central_archive.html
@@ -119,8 +119,8 @@ Step 1: Setup the perfSONAR Archive for Storing Measurementscd /etc/apt/sources.list.d/
-curl -o perfsonar-release.list http://downloads.perfsonar.net/debian/perfsonar-release.list
-curl http://downloads.perfsonar.net/debian/perfsonar-official.gpg.key | apt-key add -
+curl -o perfsonar-minor-snapshot.list downloads.perfsonar.net/debian/perfsonar-minor-snapshot.list
+curl http://downloads.perfsonar.net/debian/perfsonar-snapshot.gpg.key | apt-key add -
apt-update
@@ -145,13 +145,12 @@ Step 1: Setup the perfSONAR Archive for Storing Measurements
-Let’s quickly verify Opensearch and Logstash are running. You should see both listed as active (running) when you execute the following commands:
-systemctl status opensearch
-systemctl status logstash
+
Let’s quickly verify the archive is running with the psarchive troubleshoot utility. It will check that components such as OpenSearch and Logstash are running as well as verify authentication credentials. It can also check if the archive has data, but since we have not yet configured our measurement hosts we will skip that check with the –skip-opensearch-data option. Run the command as follows and if everything is marked as OK then proceed, otherwise follow the instructions in the command output to debug:
+psarchive troubleshoot --skip-opensearch-data
Next we’ll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit /etc/httpd/conf.d/apache-logstash.conf to include the example IPs (see the diagram at the top of this guide). Note we’ll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the Require ip lines are the relevant portions):
+Next we’ll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit /etc/httpd/conf.d/apache-logstash.conf (Rocky/Alma) or /etc/apache2/conf-available/apache-logstash.conf (Debian/Ubuntu) to include the example IPs (see the diagram at the top of this guide). Note we’ll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the Require ip lines are the relevant portions):
<IfModule proxy_module>
ProxyRequests Off
<Proxy *>
@@ -236,7 +235,7 @@ Step 2: Setup Visualization with Grafana
-Navigate to the perfSONAR Host Metrics page. There may be a link under “Recently Viewed Dashboards” (see previous screenshot) or type “perfSONARHost Metrics” in the search bar:
+Navigate to the perfSONAR Host Metrics page. There may be a link under “Recently Viewed Dashboards” (see previous screenshot) or type “perfSONAR Host Metrics” in the search bar:
@@ -249,8 +248,8 @@ Step 2: Setup Visualization with Grafana
-Step 3: Build and Publish Measurement Configuration with pSConfig¶
+
+Step 3: Creating Your Measurement Configuration with pSConfig¶
In this step we’ll create a file that defines the measurements we want all the testpoints to run. This is called a pSConfig Template File. If you want to learn more about pSConfig templates see What is pSConfig?. In this example we’ll generate the file by hand from a skeleton file then use some helpful utilities to fill-in and publish. Steps are as follows:
Create a directory where you will edit the templates:
@@ -265,7 +264,7 @@ Step 3: Build and Publish Measurement Configuration with pSConfig/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local
+/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local -a ip
@@ -289,8 +288,6 @@ Step 3: Build and Publish Measurement Configuration with pSConfig
We are going to copy and paste above into our psconfig.json file under the archives section and name it example_http_archive. This is what our psconfig.json looks like when we are done:
{
"_meta":{
@@ -391,7 +388,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT4H",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Throughput Tests"
+ "display-task-name": "Example Throughput Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_latencybg": {
@@ -399,7 +397,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"test": "example_test_latencybg",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Loss Tests"
+ "display-task-name": "Example Loss Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_trace": {
@@ -408,7 +407,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT10M",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Traceroute Tests"
+ "display-task-name": "Example Traceroute Tests",
+ "display-task-group": [ "Example Tests" ]
}
}
}
@@ -416,6 +416,16 @@ Step 3: Build and Publish Measurement Configuration with pSConfigpsconfig validate psconfig.json
+
+
+
+
+
+
+Step 4: Publishing Your Measurement Configuration with pSConfig¶
+
We will now publish the file to a local web server so the testpoints can download it. We do this will the following command:
psconfig publish psconfig.json
@@ -430,12 +440,16 @@ Step 3: Build and Publish Measurement Configuration with pSConfighttps://archive.local/psconfig/psconfig.json”
+psconfig remote add "https://archive.local/psconfig/psconfig.json"
+
+
The previous command tells a pSConfig agent running on the host to setup some dashboards. It also tells it to gather host statistics from each of the testpoints. We aren’t getting any results yet, but lets take a quick look at the dashboards it created by visiting https://archive.local/grafana/dashboards.
Click the folder perfSONAR pSConfig folder on the page that load.
Click on All perfSONAR Measurements. The dashboard doesn’t have any data, so let’s fix that by having our testpoints run some measurements.
-
+
+
+
Step 5: Viewing ResultsCentral Archive with Grafana Cookbook - NOT PRODUCTION. CURRENTLY USES NIGHTLY-MINOR REPO
- Step 1: Setup the perfSONAR Archive for Storing Measurements
- Step 2: Setup Visualization with Grafana
-- Step 3: Build and Publish Measurement Configuration with pSConfig
+- Step 3: Creating Your Measurement Configuration with pSConfig
+- Step 4: Publishing Your Measurement Configuration with pSConfig
- Step 4: Setup perfSONAR Testpoint hosts
- Step 5: Viewing Results
diff --git a/release_candidates/5.1.0/esmond_api_rest.html b/release_candidates/5.1.0/esmond_api_rest.html
index 6d1ae57b..50ec5af5 100644
--- a/release_candidates/5.1.0/esmond_api_rest.html
+++ b/release_candidates/5.1.0/esmond_api_rest.html
@@ -255,7 +255,7 @@ Listing all the measurementscurl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=v6-only"
+curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=only-v6"
diff --git a/release_candidates/5.1.0/install_debian.html b/release_candidates/5.1.0/install_debian.html
index bdf5a5cd..7da53e47 100644
--- a/release_candidates/5.1.0/install_debian.html
+++ b/release_candidates/5.1.0/install_debian.html
@@ -162,6 +162,11 @@ System RequirementsNote
The perfsonar-archive bundle is also supported on Debian 11 and Ubuntu 22.
+
+Setting the default user password in PostgreSQL¶
+With a regular apt install, the user is prompted to enter a password for the new PostgreSQL database created during installation. You can choose this password arbitrarily or leave it empty (the DB is only accessible locally).
+Alternatively, you can install packages with APT in non-interactive mode to avoid the password prompt by setting the following environment variable when installing: DEBIAN_FRONTEND=noninteractive apt install perfsonar-toolkit
+
Optional Packages¶
In addition to any of the bundles above you may also optionally choose to install one or more of our add-on packages (these are automatically added on the perfsonar-toolkit bundle):
@@ -333,6 +338,7 @@ Upgrade from Ubuntu 18 to Ubuntu 20Installation
- Step 1: Configure APT
- Step 2: Install a Bundle
diff --git a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
index 90a5349c..8f8ab5bf 100644
--- a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
+++ b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
@@ -125,7 +125,7 @@
http¶<
Measure HTTP response time
- Tools:
-psurl
+curl
@@ -240,15 +240,23 @@ throughput Tests-t DURATION, --duration=DURATION
Total runtime of test
-i INTERVAL, --interval=INTERVAL
- How often to report results (internally, results still reported in aggregate at end)
+ How often to report results (internally, results still
+ reported in aggregate at end)
+ --link-rtt=LINK_RTT Approximate link round-trip time (ISO8601 or integer
+ ms)
-P PARALLEL, --parallel=PARALLEL
How many parallel streams to run during the test
-u, --udp Use UDP instead of TCP testing
-b BANDWIDTH, --bandwidth=BANDWIDTH
Bandwidth to rate limit the test to, supports SI
notation such as 1G
+ --bandwidth-strict Never go faster than --bandwidth, even to make up for
+ lost time.
+ --burst-size=BURST_SIZE
+ Limit bursts of packets to this number
-w WINDOW_SIZE, --window-size=WINDOW_SIZE
- TCP window size to use for the test, supports SI notation such as 64M
+ TCP window size to use for the test, supports
+ SI notation such as 64M
-m MSS, --mss=MSS TCP maximum segment size
-l BUFFER_LENGTH, --buffer-length=BUFFER_LENGTH
length of the buffer to read/write from
@@ -268,7 +276,14 @@ throughput TestsSet's the sending side's CPU affinity
--server-cpu-affinity=SERVER_CPU_AFFINITY
Set's the receiving's side's CPU affinity
+ --single-ended Run a test directly to a host without pscheduler.
+ --single-ended-port=SINGLE_ENDED_PORT
+ Run a test directly to a given port.
--reverse Reverses the direction of the test.
+ --reverse-connections
+ Make connections from destination to source where
+ possible.
+ --loopback Run both client and server side in a loopback test.
The currently supported throughput tools are iperf2, iperf3, and nuttcp. iperf3 is the default.
@@ -343,6 +358,9 @@
latency Tests-f, --flip In multi-participant mode, have the dest start the
client and request a reverse test. Useful in some
firewall and NAT environments.
+ -r, --reverse Report results in the reverse direction (destination
+ to source) if possible.
+ --traverse-nat Take steps to traverse outbound NAT
-R, --output-raw Output individual packet statistics. This will
substantially increase the size of a successful
result.
@@ -429,9 +447,18 @@ http Tests-h, --help show this help message and exit
--url=URL URL to query
+--header=HEADERS HTTP header for request, format 'HeaderName: Value'.
+ May be repeated.
--parse=PARSE String to parse for
--host=HOST Host to run the test
--host-node=HOST_NODE Host to run the test
+--ip-version=IP_VERSION
+ Specificy which IP version to use, 4 or 6
+--always-succeed Treat HTTP failures as successes, regardless of
+ response code
+--keep-content=KEEP_CONTENT
+ Amount of content to keep in bytes; default none, 0
+ for no limit, SI units supported
--timeout=TIMEOUT Timeout for each query attempt
Step 1: Setup the perfSONAR Archive for Storing Measurementscd /etc/apt/sources.list.d/
-curl -o perfsonar-release.list http://downloads.perfsonar.net/debian/perfsonar-release.list
-curl http://downloads.perfsonar.net/debian/perfsonar-official.gpg.key | apt-key add -
+curl -o perfsonar-minor-snapshot.list downloads.perfsonar.net/debian/perfsonar-minor-snapshot.list
+curl http://downloads.perfsonar.net/debian/perfsonar-snapshot.gpg.key | apt-key add -
apt-update
@@ -145,13 +145,12 @@ Step 1: Setup the perfSONAR Archive for Storing Measurements
-Let’s quickly verify Opensearch and Logstash are running. You should see both listed as active (running) when you execute the following commands:
-systemctl status opensearch
-systemctl status logstash
+
Let’s quickly verify the archive is running with the psarchive troubleshoot utility. It will check that components such as OpenSearch and Logstash are running as well as verify authentication credentials. It can also check if the archive has data, but since we have not yet configured our measurement hosts we will skip that check with the –skip-opensearch-data option. Run the command as follows and if everything is marked as OK then proceed, otherwise follow the instructions in the command output to debug:
+psarchive troubleshoot --skip-opensearch-data
Next we’ll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit /etc/httpd/conf.d/apache-logstash.conf to include the example IPs (see the diagram at the top of this guide). Note we’ll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the Require ip lines are the relevant portions):
+Next we’ll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit /etc/httpd/conf.d/apache-logstash.conf (Rocky/Alma) or /etc/apache2/conf-available/apache-logstash.conf (Debian/Ubuntu) to include the example IPs (see the diagram at the top of this guide). Note we’ll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the Require ip lines are the relevant portions):
<IfModule proxy_module>
ProxyRequests Off
<Proxy *>
@@ -236,7 +235,7 @@ Step 2: Setup Visualization with Grafana
-Navigate to the perfSONAR Host Metrics page. There may be a link under “Recently Viewed Dashboards” (see previous screenshot) or type “perfSONARHost Metrics” in the search bar:
+Navigate to the perfSONAR Host Metrics page. There may be a link under “Recently Viewed Dashboards” (see previous screenshot) or type “perfSONAR Host Metrics” in the search bar:
@@ -249,8 +248,8 @@ Step 2: Setup Visualization with Grafana
-Step 3: Build and Publish Measurement Configuration with pSConfig¶
+
+Step 3: Creating Your Measurement Configuration with pSConfig¶
In this step we’ll create a file that defines the measurements we want all the testpoints to run. This is called a pSConfig Template File. If you want to learn more about pSConfig templates see What is pSConfig?. In this example we’ll generate the file by hand from a skeleton file then use some helpful utilities to fill-in and publish. Steps are as follows:
Create a directory where you will edit the templates:
@@ -265,7 +264,7 @@ Step 3: Build and Publish Measurement Configuration with pSConfig/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local
+/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local -a ip
@@ -289,8 +288,6 @@ Step 3: Build and Publish Measurement Configuration with pSConfig
We are going to copy and paste above into our psconfig.json file under the archives section and name it example_http_archive. This is what our psconfig.json looks like when we are done:
{
"_meta":{
@@ -391,7 +388,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT4H",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Throughput Tests"
+ "display-task-name": "Example Throughput Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_latencybg": {
@@ -399,7 +397,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"test": "example_test_latencybg",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Loss Tests"
+ "display-task-name": "Example Loss Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_trace": {
@@ -408,7 +407,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT10M",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Traceroute Tests"
+ "display-task-name": "Example Traceroute Tests",
+ "display-task-group": [ "Example Tests" ]
}
}
}
@@ -416,6 +416,16 @@ Step 3: Build and Publish Measurement Configuration with pSConfigpsconfig validate psconfig.json
+
+
+
+
+
+
+Step 4: Publishing Your Measurement Configuration with pSConfig¶
+
We will now publish the file to a local web server so the testpoints can download it. We do this will the following command:
psconfig publish psconfig.json
@@ -430,12 +440,16 @@ Step 3: Build and Publish Measurement Configuration with pSConfighttps://archive.local/psconfig/psconfig.json”
+psconfig remote add "https://archive.local/psconfig/psconfig.json"
+
+
The previous command tells a pSConfig agent running on the host to setup some dashboards. It also tells it to gather host statistics from each of the testpoints. We aren’t getting any results yet, but lets take a quick look at the dashboards it created by visiting https://archive.local/grafana/dashboards.
Click the folder perfSONAR pSConfig folder on the page that load.
Click on All perfSONAR Measurements. The dashboard doesn’t have any data, so let’s fix that by having our testpoints run some measurements.
-
+
+
+
Step 5: Viewing ResultsCentral Archive with Grafana Cookbook - NOT PRODUCTION. CURRENTLY USES NIGHTLY-MINOR REPO
- Step 1: Setup the perfSONAR Archive for Storing Measurements
- Step 2: Setup Visualization with Grafana
-- Step 3: Build and Publish Measurement Configuration with pSConfig
+- Step 3: Creating Your Measurement Configuration with pSConfig
+- Step 4: Publishing Your Measurement Configuration with pSConfig
- Step 4: Setup perfSONAR Testpoint hosts
- Step 5: Viewing Results
diff --git a/release_candidates/5.1.0/esmond_api_rest.html b/release_candidates/5.1.0/esmond_api_rest.html
index 6d1ae57b..50ec5af5 100644
--- a/release_candidates/5.1.0/esmond_api_rest.html
+++ b/release_candidates/5.1.0/esmond_api_rest.html
@@ -255,7 +255,7 @@ Listing all the measurementscurl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=v6-only"
+curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=only-v6"
diff --git a/release_candidates/5.1.0/install_debian.html b/release_candidates/5.1.0/install_debian.html
index bdf5a5cd..7da53e47 100644
--- a/release_candidates/5.1.0/install_debian.html
+++ b/release_candidates/5.1.0/install_debian.html
@@ -162,6 +162,11 @@ System RequirementsNote
The perfsonar-archive bundle is also supported on Debian 11 and Ubuntu 22.
+
+Setting the default user password in PostgreSQL¶
+With a regular apt install, the user is prompted to enter a password for the new PostgreSQL database created during installation. You can choose this password arbitrarily or leave it empty (the DB is only accessible locally).
+Alternatively, you can install packages with APT in non-interactive mode to avoid the password prompt by setting the following environment variable when installing: DEBIAN_FRONTEND=noninteractive apt install perfsonar-toolkit
+
Optional Packages¶
In addition to any of the bundles above you may also optionally choose to install one or more of our add-on packages (these are automatically added on the perfsonar-toolkit bundle):
@@ -333,6 +338,7 @@ Upgrade from Ubuntu 18 to Ubuntu 20Installation
- Step 1: Configure APT
- Step 2: Install a Bundle
diff --git a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
index 90a5349c..8f8ab5bf 100644
--- a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
+++ b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
@@ -125,7 +125,7 @@
http¶<
Measure HTTP response time
- Tools:
-psurl
+curl
@@ -240,15 +240,23 @@ throughput Tests-t DURATION, --duration=DURATION
Total runtime of test
-i INTERVAL, --interval=INTERVAL
- How often to report results (internally, results still reported in aggregate at end)
+ How often to report results (internally, results still
+ reported in aggregate at end)
+ --link-rtt=LINK_RTT Approximate link round-trip time (ISO8601 or integer
+ ms)
-P PARALLEL, --parallel=PARALLEL
How many parallel streams to run during the test
-u, --udp Use UDP instead of TCP testing
-b BANDWIDTH, --bandwidth=BANDWIDTH
Bandwidth to rate limit the test to, supports SI
notation such as 1G
+ --bandwidth-strict Never go faster than --bandwidth, even to make up for
+ lost time.
+ --burst-size=BURST_SIZE
+ Limit bursts of packets to this number
-w WINDOW_SIZE, --window-size=WINDOW_SIZE
- TCP window size to use for the test, supports SI notation such as 64M
+ TCP window size to use for the test, supports
+ SI notation such as 64M
-m MSS, --mss=MSS TCP maximum segment size
-l BUFFER_LENGTH, --buffer-length=BUFFER_LENGTH
length of the buffer to read/write from
@@ -268,7 +276,14 @@ throughput TestsSet's the sending side's CPU affinity
--server-cpu-affinity=SERVER_CPU_AFFINITY
Set's the receiving's side's CPU affinity
+ --single-ended Run a test directly to a host without pscheduler.
+ --single-ended-port=SINGLE_ENDED_PORT
+ Run a test directly to a given port.
--reverse Reverses the direction of the test.
+ --reverse-connections
+ Make connections from destination to source where
+ possible.
+ --loopback Run both client and server side in a loopback test.
The currently supported throughput tools are iperf2, iperf3, and nuttcp. iperf3 is the default.
@@ -343,6 +358,9 @@
latency Tests-f, --flip In multi-participant mode, have the dest start the
client and request a reverse test. Useful in some
firewall and NAT environments.
+ -r, --reverse Report results in the reverse direction (destination
+ to source) if possible.
+ --traverse-nat Take steps to traverse outbound NAT
-R, --output-raw Output individual packet statistics. This will
substantially increase the size of a successful
result.
@@ -429,9 +447,18 @@ http Tests-h, --help show this help message and exit
--url=URL URL to query
+--header=HEADERS HTTP header for request, format 'HeaderName: Value'.
+ May be repeated.
--parse=PARSE String to parse for
--host=HOST Host to run the test
--host-node=HOST_NODE Host to run the test
+--ip-version=IP_VERSION
+ Specificy which IP version to use, 4 or 6
+--always-succeed Treat HTTP failures as successes, regardless of
+ response code
+--keep-content=KEEP_CONTENT
+ Amount of content to keep in bytes; default none, 0
+ for no limit, SI units supported
--timeout=TIMEOUT Timeout for each query attempt
cd /etc/apt/sources.list.d/
-curl -o perfsonar-release.list http://downloads.perfsonar.net/debian/perfsonar-release.list
-curl http://downloads.perfsonar.net/debian/perfsonar-official.gpg.key | apt-key add -
+curl -o perfsonar-minor-snapshot.list downloads.perfsonar.net/debian/perfsonar-minor-snapshot.list
+curl http://downloads.perfsonar.net/debian/perfsonar-snapshot.gpg.key | apt-key add -
apt-update
Let’s quickly verify Opensearch and Logstash are running. You should see both listed as active (running) when you execute the following commands:
-systemctl status opensearch
-systemctl status logstash
+Let’s quickly verify the archive is running with the psarchive troubleshoot utility. It will check that components such as OpenSearch and Logstash are running as well as verify authentication credentials. It can also check if the archive has data, but since we have not yet configured our measurement hosts we will skip that check with the –skip-opensearch-data option. Run the command as follows and if everything is marked as OK then proceed, otherwise follow the instructions in the command output to debug:
+psarchive troubleshoot --skip-opensearch-data
Next we’ll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit /etc/httpd/conf.d/apache-logstash.conf to include the example IPs (see the diagram at the top of this guide). Note we’ll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the Require ip lines are the relevant portions):
+Next we’ll setup IP authentication so that the testpoint hosts will be able to send their results. This verification is handled by the Apache proxy in front of Logstash. We are going to edit /etc/httpd/conf.d/apache-logstash.conf (Rocky/Alma) or /etc/apache2/conf-available/apache-logstash.conf (Debian/Ubuntu) to include the example IPs (see the diagram at the top of this guide). Note we’ll add both the IPv4 and IPv6 addresses to make sure the host can authenticate via either protocol. The following is what we setup for the example (the Require ip lines are the relevant portions):
<IfModule proxy_module>
ProxyRequests Off
<Proxy *>
@@ -236,7 +235,7 @@ Step 2: Setup Visualization with Grafana
Navigate to the perfSONAR Host Metrics page. There may be a link under “Recently Viewed Dashboards” (see previous screenshot) or type “perfSONARHost Metrics” in the search bar:
+Navigate to the perfSONAR Host Metrics page. There may be a link under “Recently Viewed Dashboards” (see previous screenshot) or type “perfSONAR Host Metrics” in the search bar:
@@ -249,8 +248,8 @@
Step 2: Setup Visualization with Grafana
-Step 3: Build and Publish Measurement Configuration with pSConfig¶
+
+Step 3: Creating Your Measurement Configuration with pSConfig¶
In this step we’ll create a file that defines the measurements we want all the testpoints to run. This is called a pSConfig Template File. If you want to learn more about pSConfig templates see What is pSConfig?. In this example we’ll generate the file by hand from a skeleton file then use some helpful utilities to fill-in and publish. Steps are as follows:
Create a directory where you will edit the templates:
@@ -265,7 +264,7 @@ Step 3: Build and Publish Measurement Configuration with pSConfig/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local
+/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local -a ip
@@ -289,8 +288,6 @@ Step 3: Build and Publish Measurement Configuration with pSConfig
We are going to copy and paste above into our psconfig.json file under the archives section and name it example_http_archive. This is what our psconfig.json looks like when we are done:
{
"_meta":{
@@ -391,7 +388,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT4H",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Throughput Tests"
+ "display-task-name": "Example Throughput Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_latencybg": {
@@ -399,7 +397,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"test": "example_test_latencybg",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Loss Tests"
+ "display-task-name": "Example Loss Tests",
+ "display-task-group": [ "Example Tests" ]
}
},
"example_task_trace": {
@@ -408,7 +407,8 @@ Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT10M",
"archives": [ "example_http_archive" ],
"reference": {
- "display-task-name": "Example Traceroute Tests"
+ "display-task-name": "Example Traceroute Tests",
+ "display-task-group": [ "Example Tests" ]
}
}
}
@@ -416,6 +416,16 @@ Step 3: Build and Publish Measurement Configuration with pSConfigpsconfig validate psconfig.json
+
+
+
+
+
+
+Step 4: Publishing Your Measurement Configuration with pSConfig¶
+
We will now publish the file to a local web server so the testpoints can download it. We do this will the following command:
psconfig publish psconfig.json
@@ -430,12 +440,16 @@ Step 3: Build and Publish Measurement Configuration with pSConfighttps://archive.local/psconfig/psconfig.json”
+psconfig remote add "https://archive.local/psconfig/psconfig.json"
+
+
The previous command tells a pSConfig agent running on the host to setup some dashboards. It also tells it to gather host statistics from each of the testpoints. We aren’t getting any results yet, but lets take a quick look at the dashboards it created by visiting https://archive.local/grafana/dashboards.
Click the folder perfSONAR pSConfig folder on the page that load.
Click on All perfSONAR Measurements. The dashboard doesn’t have any data, so let’s fix that by having our testpoints run some measurements.
-
+
+
+
Step 5: Viewing ResultsCentral Archive with Grafana Cookbook - NOT PRODUCTION. CURRENTLY USES NIGHTLY-MINOR REPO
- Step 1: Setup the perfSONAR Archive for Storing Measurements
- Step 2: Setup Visualization with Grafana
-- Step 3: Build and Publish Measurement Configuration with pSConfig
+- Step 3: Creating Your Measurement Configuration with pSConfig
+- Step 4: Publishing Your Measurement Configuration with pSConfig
- Step 4: Setup perfSONAR Testpoint hosts
- Step 5: Viewing Results
diff --git a/release_candidates/5.1.0/esmond_api_rest.html b/release_candidates/5.1.0/esmond_api_rest.html
index 6d1ae57b..50ec5af5 100644
--- a/release_candidates/5.1.0/esmond_api_rest.html
+++ b/release_candidates/5.1.0/esmond_api_rest.html
@@ -255,7 +255,7 @@ Listing all the measurementscurl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=v6-only"
+curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=only-v6"
diff --git a/release_candidates/5.1.0/install_debian.html b/release_candidates/5.1.0/install_debian.html
index bdf5a5cd..7da53e47 100644
--- a/release_candidates/5.1.0/install_debian.html
+++ b/release_candidates/5.1.0/install_debian.html
@@ -162,6 +162,11 @@ System RequirementsNote
The perfsonar-archive bundle is also supported on Debian 11 and Ubuntu 22.
+
+Setting the default user password in PostgreSQL¶
+With a regular apt install, the user is prompted to enter a password for the new PostgreSQL database created during installation. You can choose this password arbitrarily or leave it empty (the DB is only accessible locally).
+Alternatively, you can install packages with APT in non-interactive mode to avoid the password prompt by setting the following environment variable when installing: DEBIAN_FRONTEND=noninteractive apt install perfsonar-toolkit
+
Optional Packages¶
In addition to any of the bundles above you may also optionally choose to install one or more of our add-on packages (these are automatically added on the perfsonar-toolkit bundle):
@@ -333,6 +338,7 @@ Upgrade from Ubuntu 18 to Ubuntu 20Installation
- Step 1: Configure APT
- Step 2: Install a Bundle
diff --git a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
index 90a5349c..8f8ab5bf 100644
--- a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
+++ b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
@@ -125,7 +125,7 @@
http¶<
Measure HTTP response time
- Tools:
-psurl
+curl
@@ -240,15 +240,23 @@ throughput Tests-t DURATION, --duration=DURATION
Total runtime of test
-i INTERVAL, --interval=INTERVAL
- How often to report results (internally, results still reported in aggregate at end)
+ How often to report results (internally, results still
+ reported in aggregate at end)
+ --link-rtt=LINK_RTT Approximate link round-trip time (ISO8601 or integer
+ ms)
-P PARALLEL, --parallel=PARALLEL
How many parallel streams to run during the test
-u, --udp Use UDP instead of TCP testing
-b BANDWIDTH, --bandwidth=BANDWIDTH
Bandwidth to rate limit the test to, supports SI
notation such as 1G
+ --bandwidth-strict Never go faster than --bandwidth, even to make up for
+ lost time.
+ --burst-size=BURST_SIZE
+ Limit bursts of packets to this number
-w WINDOW_SIZE, --window-size=WINDOW_SIZE
- TCP window size to use for the test, supports SI notation such as 64M
+ TCP window size to use for the test, supports
+ SI notation such as 64M
-m MSS, --mss=MSS TCP maximum segment size
-l BUFFER_LENGTH, --buffer-length=BUFFER_LENGTH
length of the buffer to read/write from
@@ -268,7 +276,14 @@ throughput TestsSet's the sending side's CPU affinity
--server-cpu-affinity=SERVER_CPU_AFFINITY
Set's the receiving's side's CPU affinity
+ --single-ended Run a test directly to a host without pscheduler.
+ --single-ended-port=SINGLE_ENDED_PORT
+ Run a test directly to a given port.
--reverse Reverses the direction of the test.
+ --reverse-connections
+ Make connections from destination to source where
+ possible.
+ --loopback Run both client and server side in a loopback test.
The currently supported throughput tools are iperf2, iperf3, and nuttcp. iperf3 is the default.
@@ -343,6 +358,9 @@
latency Tests-f, --flip In multi-participant mode, have the dest start the
client and request a reverse test. Useful in some
firewall and NAT environments.
+ -r, --reverse Report results in the reverse direction (destination
+ to source) if possible.
+ --traverse-nat Take steps to traverse outbound NAT
-R, --output-raw Output individual packet statistics. This will
substantially increase the size of a successful
result.
@@ -429,9 +447,18 @@ http Tests-h, --help show this help message and exit
--url=URL URL to query
+--header=HEADERS HTTP header for request, format 'HeaderName: Value'.
+ May be repeated.
--parse=PARSE String to parse for
--host=HOST Host to run the test
--host-node=HOST_NODE Host to run the test
+--ip-version=IP_VERSION
+ Specificy which IP version to use, 4 or 6
+--always-succeed Treat HTTP failures as successes, regardless of
+ response code
+--keep-content=KEEP_CONTENT
+ Amount of content to keep in bytes; default none, 0
+ for no limit, SI units supported
--timeout=TIMEOUT Timeout for each query attempt
Step 3: Creating Your Measurement Configuration with pSConfig¶
In this step we’ll create a file that defines the measurements we want all the testpoints to run. This is called a pSConfig Template File. If you want to learn more about pSConfig templates see What is pSConfig?. In this example we’ll generate the file by hand from a skeleton file then use some helpful utilities to fill-in and publish. Steps are as follows:
Create a directory where you will edit the templates:
@@ -265,7 +264,7 @@Step 3: Build and Publish Measurement Configuration with pSConfig
/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local +
@@ -289,8 +288,6 @@/usr/lib/perfsonar/archive/perfsonar-scripts/psconfig_archive.sh -n archive.local -a ip
Step 3: Build and Publish Measurement Configuration with pSConfig
We are going to copy and paste above into our psconfig.json file under the archives section and name it example_http_archive. This is what our psconfig.json looks like when we are done:
+ + + +{ "_meta":{ @@ -391,7 +388,8 @@
Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT4H", "archives": [ "example_http_archive" ], "reference": { - "display-task-name": "Example Throughput Tests" + "display-task-name": "Example Throughput Tests", + "display-task-group": [ "Example Tests" ] } }, "example_task_latencybg": { @@ -399,7 +397,8 @@
Step 3: Build and Publish Measurement Configuration with pSConfig"test": "example_test_latencybg", "archives": [ "example_http_archive" ], "reference": { - "display-task-name": "Example Loss Tests" + "display-task-name": "Example Loss Tests", + "display-task-group": [ "Example Tests" ] } }, "example_task_trace": { @@ -408,7 +407,8 @@
Step 3: Build and Publish Measurement Configuration with pSConfig"schedule": "example_schedule_PT10M", "archives": [ "example_http_archive" ], "reference": { - "display-task-name": "Example Traceroute Tests" + "display-task-name": "Example Traceroute Tests", + "display-task-group": [ "Example Tests" ] } } } @@ -416,6 +416,16 @@
Step 3: Build and Publish Measurement Configuration with pSConfig
+psconfig validate psconfig.json +
+ Step 4: Publishing Your Measurement Configuration with pSConfig¶
+We will now publish the file to a local web server so the testpoints can download it. We do this will the following command:
@@ -430,12 +440,16 @@psconfig publish psconfig.json
Step 3: Build and Publish Measurement Configuration with pSConfighttps://archive.local/psconfig/psconfig.json” +
+psconfig remote add "https://archive.local/psconfig/psconfig.json" +
The previous command tells a pSConfig agent running on the host to setup some dashboards. It also tells it to gather host statistics from each of the testpoints. We aren’t getting any results yet, but lets take a quick look at the dashboards it created by visiting https://archive.local/grafana/dashboards.
Click the folder perfSONAR pSConfig folder on the page that load.
Click on All perfSONAR Measurements. The dashboard doesn’t have any data, so let’s fix that by having our testpoints run some measurements.
-
++
+Step 5: Viewing ResultsCentral Archive with Grafana Cookbook - NOT PRODUCTION. CURRENTLY USES NIGHTLY-MINOR REPO
- Step 1: Setup the perfSONAR Archive for Storing Measurements
- Step 2: Setup Visualization with Grafana -
- Step 3: Build and Publish Measurement Configuration with pSConfig +
- Step 3: Creating Your Measurement Configuration with pSConfig +
- Step 4: Publishing Your Measurement Configuration with pSConfig
- Step 4: Setup perfSONAR Testpoint hosts
- Step 5: Viewing Results
Listing all the measurements
+curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=v6-only" +
diff --git a/release_candidates/5.1.0/install_debian.html b/release_candidates/5.1.0/install_debian.html index bdf5a5cd..7da53e47 100644 --- a/release_candidates/5.1.0/install_debian.html +++ b/release_candidates/5.1.0/install_debian.html @@ -162,6 +162,11 @@curl "http://archive.example.net/esmond/perfsonar/archive/?source=host1.example.net&dns-match-rule=only-v6"
System RequirementsNote
The
perfsonar-archivebundle is also supported on Debian 11 and Ubuntu 22.+ Setting the default user password in PostgreSQL¶
+With a regular
+apt install, the user is prompted to enter a password for the new PostgreSQL database created during installation. You can choose this password arbitrarily or leave it empty (the DB is only accessible locally). +Alternatively, you can install packages with APT in non-interactive mode to avoid the password prompt by setting the following environment variable when installing:DEBIAN_FRONTEND=noninteractive apt install perfsonar-toolkitOptional Packages¶
In addition to any of the bundles above you may also optionally choose to install one or more of our add-on packages (these are automatically added on the perfsonar-toolkit bundle):
@@ -333,6 +338,7 @@Upgrade from Ubuntu 18 to Ubuntu 20Installation
- Step 1: Configure APT
- Step 2: Install a Bundle
diff --git a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
index 90a5349c..8f8ab5bf 100644
--- a/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
+++ b/release_candidates/5.1.0/pscheduler_ref_tests_tools.html
@@ -125,7 +125,7 @@
http¶<
Measure HTTP response time
- Tools:
-psurl
+curl
throughput Tests-t DURATION, --duration=DURATION Total runtime of test -i INTERVAL, --interval=INTERVAL - How often to report results (internally, results still reported in aggregate at end) + How often to report results (internally, results still + reported in aggregate at end) + --link-rtt=LINK_RTT Approximate link round-trip time (ISO8601 or integer + ms) -P PARALLEL, --parallel=PARALLEL How many parallel streams to run during the test -u, --udp Use UDP instead of TCP testing -b BANDWIDTH, --bandwidth=BANDWIDTH Bandwidth to rate limit the test to, supports SI notation such as 1G + --bandwidth-strict Never go faster than --bandwidth, even to make up for + lost time. + --burst-size=BURST_SIZE + Limit bursts of packets to this number -w WINDOW_SIZE, --window-size=WINDOW_SIZE - TCP window size to use for the test, supports SI notation such as 64M + TCP window size to use for the test, supports + SI notation such as 64M -m MSS, --mss=MSS TCP maximum segment size -l BUFFER_LENGTH, --buffer-length=BUFFER_LENGTH length of the buffer to read/write from @@ -268,7 +276,14 @@
throughput TestsSet's the sending side's CPU affinity --server-cpu-affinity=SERVER_CPU_AFFINITY Set's the receiving's side's CPU affinity + --single-ended Run a test directly to a host without pscheduler. + --single-ended-port=SINGLE_ENDED_PORT + Run a test directly to a given port. --reverse Reverses the direction of the test. + --reverse-connections + Make connections from destination to source where + possible. + --loopback Run both client and server side in a loopback test.
The currently supported throughput tools are iperf2, iperf3, and nuttcp. iperf3 is the default. @@ -343,6 +358,9 @@
latency Tests-f, --flip In multi-participant mode, have the dest start the client and request a reverse test. Useful in some firewall and NAT environments. + -r, --reverse Report results in the reverse direction (destination + to source) if possible. + --traverse-nat Take steps to traverse outbound NAT -R, --output-raw Output individual packet statistics. This will substantially increase the size of a successful result. @@ -429,9 +447,18 @@
http Tests-h, --help show this help message and exit --url=URL URL to query +--header=HEADERS HTTP header for request, format 'HeaderName: Value'. + May be repeated. --parse=PARSE String to parse for --host=HOST Host to run the test --host-node=HOST_NODE Host to run the test +--ip-version=IP_VERSION + Specificy which IP version to use, 4 or 6 +--always-succeed Treat HTTP failures as successes, regardless of + response code +--keep-content=KEEP_CONTENT + Amount of content to keep in bytes; default none, 0 + for no limit, SI units supported --timeout=TIMEOUT Timeout for each query attempt