Merge branch 'devel' into refactor-workceptor

.github/workflows/coverage_reporting.yml

@@ -38,7 +38,7 @@ jobs:
           sudo cp ./receptor /usr/local/bin/receptor
 
       - name: Download kind binary
-        run: curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.11.1/kind-linux-amd64 && chmod +x ./kind
+        run: curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64 && chmod +x ./kind
 
       - name: Create k8s cluster
         run: ./kind create cluster

.github/workflows/pull_request.yml

@@ -15,7 +15,7 @@ jobs:
           fetch-depth: 0
 
       - name: golangci-lint
-        uses: golangci/golangci-lint-action@v3
+        uses: golangci/golangci-lint-action@v4
         with:
           version: v1.52
   lint-receptorctl:
@@ -70,7 +70,7 @@ jobs:
           sudo cp ./receptor /usr/local/bin/receptor
 
       - name: Download kind binary
-        run: curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.11.1/kind-linux-amd64 && chmod +x ./kind
+        run: curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64 && chmod +x ./kind
 
       - name: Create k8s cluster
         run: ./kind create cluster

docs/source/getting_started_guide/creating_a_basic_network.rst

@@ -0,0 +1,104 @@
+
+.. _creating_a_basic_network:
+
+###############################
+Creating a basic 3-node network
+###############################
+
+In this section, we will create a three-node network.
+The three nodes are: foo, bar, and mal.
+
+`foo -> bar <- mal`
+
+foo and mal are directly connected to bar with TCP connections.
+
+foo can reach mal by sending network packets through bar.
+
+***********************
+Receptor configurations
+***********************
+
+1. Create three configuration files, one for each node.
+
+ **foo.yml**
+
+.. code-block:: yaml
+
+  ---
+  - node:
+    id: foo
+
+    - control-service:
+      service: control
+      filename: /tmp/foo.sock
+
+    - tcp-peer:
+      address: localhost:2222
+      redial: true
+
+    - log-level: debug
+    ...
+
+**bar.yml**
+
+.. code-block:: yaml
+
+    ---
+    - node:
+      id: bar
+
+    - control-service:
+      service: control
+      filename: /tmp/bar.sock
+
+    - tcp-listener:
+      port: 2222
+
+    - log-level: debug
+    ...
+
+ **mal.yml**
+
+.. code-block:: yaml
+
+    ---
+    - node:
+      id: mal
+
+    - control-service:
+      service: control
+      filename: /tmp/mal.sock
+
+    - tcp-peer:
+      address: localhost:2222
+      redial: true
+
+    - log-level: debug
+
+    - work-command:
+      workType: echo
+      command: bash
+      params: "-c \"while read -r line; do echo $line; sleep 1; done\""
+      allowruntimeparams: true
+    ...
+
+2. Run the services in separate terminals.
+
+.. code-block:: bash
+
+    ./receptor --config foo.yml
+
+.. code-block:: bash
+
+    ./receptor --config bar.yml
+
+.. code-block:: bash
+
+    ./receptor --config mal.yml
+
+.. seealso::
+
+    :ref:`configuring_receptor_with_a_config_file`
+        Configuring Receptor with a configuration file
+    :ref:`connecting_nodes`
+        Detail on connecting receptor nodes

docs/source/getting_started_guide/index.rst

@@ -0,0 +1,28 @@
+#############################
+Getting started with Receptor
+#############################
+
+Receptor is an overlay network intended to ease the distribution of work across
+a large and dispersed collection of workers. Receptor nodes establish peer-to-
+peer connections with each other via existing networks. Once connected, the re-
+ceptor mesh provides datagram (UDP-like) and stream (TCP-like) capabilities to
+applications, as well as robust unit-of-work handling with resiliency against
+transient network failures.
+
+.. image:: mesh.png
+
+.. toctree::
+    :maxdepth: 1
+    :caption: Contents:
+
+    introduction
+    installing_receptor
+    creating_a_basic_network
+    trying_sample_commands
+
+.. seealso::
+
+    :ref:`interacting_with_nodes`
+        Further examples of working with nodes
+    :ref:`connecting_nodes`
+        Detail on connecting receptor nodes

docs/source/getting_started_guide/installing_receptor.rst

@@ -0,0 +1,21 @@
+
+.. _installing_receptor:
+
+###################
+Installing Receptor
+###################
+
+1. `Download receptor <https://github.com/ansible/receptor/releases>`_
+2. Install receptor (per installation guide below)
+3. Install receptorctl
+
+.. code-block:: bash
+
+    pip install receptorctl
+
+.. seealso::
+
+    :ref:`installing`
+        Detailed installation instructions
+    :ref:`using_receptor_containers`
+        Using receptor in containers

docs/source/getting_started_guide/introduction.rst

@@ -0,0 +1,16 @@
+########################
+Introduction to receptor
+########################
+
+Receptor is an overlay network.
+It eases the work distribution across a large and dispersed collection
+of workers
+
+Receptor nodes establish peer-to-peer connections with each other through
+existing networks
+
+Once connected, the receptor mesh provides:
+
+* Datagram (UDP-like) and stream (TCP-like) capabilities to applications
+* Robust unit-of-work handling
+* Resiliency against transient network failures

docs/source/getting_started_guide/trying_sample_commands.rst

@@ -0,0 +1,53 @@
+###################
+Try Sample Commands
+###################
+
+N.B. The prior steps of network setup and receptor installation
+need to be completed in order for these command to work
+
+1. Show network status
+
+.. code-block:: bash
+
+    receptorctl --socket /tmp/foo.sock status
+
+2. Ping node mal from node foo
+
+.. code-block:: bash
+
+    receptorctl --socket /tmp/foo.sock ping mal
+
+3. Submit work from foo to mal and stream results back to foo
+
+.. code-block:: bash
+
+    seq 10 | receptorctl --socket /tmp/foo.sock work submit --node mal echo --payload - -f
+
+4. List work units
+
+.. code-block:: bash
+
+    receptorctl --socket /tmp/foo.sock work list --node foo
+
+5. Get work unit id using jq
+
+.. code-block:: bash
+
+    receptorctl --socket /tmp/foo.sock work list --node foo | jq --raw-output '.|keys|first'
+
+6. Re-stream the work results from work unit
+
+.. code-block:: bash
+
+    receptorctl --socket /tmp/foo.sock work results work_unit_id
+
+Congratulations, you are now using Receptor!
+
+.. seealso::
+
+    :ref:`control_service_commands`
+        Control service commands
+    :ref:`creating_a_basic_network`
+        Creating a Basic Network
+    :ref:`installing_receptor`
+        Installing Receptor

docs/source/index.rst

@@ -13,6 +13,7 @@ Receptor is an overlay network intended to ease the distribution of work across
    :maxdepth: 2
 
    installation
+   getting_started_guide/index
    user_guide/index
    developer_guide
    contributing

docs/source/user_guide/basic_usage.rst

@@ -21,6 +21,7 @@ Supported log levels, in increasing verbosity, are Error, Warning, Info and Debu
 
 Note: stop the receptor process with ``ctrl-c``
 
+.. _configuring_receptor_with_a_config_file:
 Configuring Receptor with a config file
 ----------------------------------------
 
@@ -45,6 +46,7 @@ Start receptor using the config file
 
 Changing the configuration file does take effect until the receptor process is restarted.
 
+:: _using_receptor_containers:
 Use Receptor through a container image
 ---------------------------------------
 

docs/source/user_guide/connecting_nodes.rst

@@ -1,3 +1,4 @@
+.. _connecting_nodes:
 Connecting nodes
 ================
 

docs/source/user_guide/index.rst

@@ -8,27 +8,36 @@ This guide describes how to use receptor in multiple environments and uses the f
 
 .. glossary::
 
-   receptor
-      The receptor application taken as a whole, which typically runs as a daemon.
-
-   receptorctl
-      A user-facing command line used to interact with receptor, typically over a Unix domain socket.
-
-   node
-      A single running instance of receptor.
+   Backend
+      A type of connection that receptor nodes can pass traffic over. Current backends include TCP, UDP and websockets.
 
-   node ID
-      An arbitrary string identifying a single node, analogous to an IP address.
+   backend peers
+      A node connected to another through receptor backends.
 
-   backend
-      A type of connection that receptor nodes can pass traffic over. Current backends include TCP, UDP and websockets.
+   control node
+      A node running the receptor control service.
 
    control service
       A built-in service that usually runs under the name `control`.  Used to report status and to launch and monitor work.
 
    netceptor
       The component of receptor that handles all networking functionality.
 
+   netceptor peers
+      A receptor node directly connected to another receptor node.
+
+   node
+      A single running instance of receptor.
+
+   node ID
+      An arbitrary string identifying a single node, analogous to an IP address.
+
+   receptor
+      The receptor application taken as a whole, which typically runs as a daemon.
+
+   receptorctl
+      A user-facing command line used to interact with receptor, typically over a Unix domain socket.
+
    workceptor
       The component of receptor that handles work units.
 

docs/source/user_guide/interacting_with_nodes.rst

@@ -1,5 +1,6 @@
 
 
+.. _interacting_with_nodes:
 Interacting with nodes
 ======================
 
@@ -109,6 +110,7 @@ Once connected to a control service, one can issue commands like "status" or "wo
 
 Keep in mind that a "work submit" command will require a payload. Type out the payload contents and press ctrl-D to send the EOF signal. The socket will then close and work will begin. See :ref:`workceptor` for more on submitting work via receptor.
 
+.. _control_service_commands:
 Control service commands
 --------------------------
 

go.mod

@@ -20,11 +20,11 @@ require (
 	github.com/songgao/water v0.0.0-20200317203138-2b4b6d7c09d8
 	github.com/vishvananda/netlink v1.1.0
 	go.uber.org/mock v0.4.0
-	golang.org/x/net v0.20.0
+	golang.org/x/net v0.21.0
 	gopkg.in/yaml.v2 v2.4.0
-	k8s.io/api v0.29.1
-	k8s.io/apimachinery v0.29.1
-	k8s.io/client-go v0.29.1
+	k8s.io/api v0.29.2
+	k8s.io/apimachinery v0.29.2
+	k8s.io/client-go v0.29.2
 )
 
 require (
@@ -55,12 +55,12 @@ require (
 	github.com/quic-go/qtls-go1-20 v0.4.1 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/vishvananda/netns v0.0.4 // indirect
-	golang.org/x/crypto v0.18.0 // indirect
+	golang.org/x/crypto v0.19.0 // indirect
 	golang.org/x/exp v0.0.0-20240103183307-be819d1f06fc // indirect
 	golang.org/x/mod v0.14.0 // indirect
 	golang.org/x/oauth2 v0.10.0 // indirect
-	golang.org/x/sys v0.16.0 // indirect
-	golang.org/x/term v0.16.0 // indirect
+	golang.org/x/sys v0.17.0 // indirect
+	golang.org/x/term v0.17.0 // indirect
 	golang.org/x/text v0.14.0 // indirect
 	golang.org/x/time v0.3.0 // indirect
 	golang.org/x/tools v0.16.1 // indirect