Rebase to upstream v3.3.1

.github/workflows/install_deps.py

@@ -21,7 +21,7 @@ def install_requirements(what):
         from setup import EXTRAS_REQUIRE, read
     finally:
         sys.path = old_path
-    requirements = ['mock>=2.0.0', 'flake8', 'pytest', 'pytest-cov'] if what == 'all' else ['behave']
+    requirements = ['flake8', 'pytest', 'pytest-cov'] if what == 'all' else ['behave']
     requirements += ['coverage']
     # try to split tests between psycopg2 and psycopg3
     requirements += ['psycopg[binary]'] if sys.version_info >= (3, 8, 0) and\

.github/workflows/tests.yaml

@@ -186,7 +186,7 @@ jobs:
 
     - uses: jakebailey/pyright-action@v2
       with:
-        version: 1.1.356
+        version: 1.1.367
 
   docs:
     runs-on: ubuntu-latest

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2015 Compose, Zalando SE
+Copyright (c) 2024 Compose, Zalando SE, Patroni Contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.rst

@@ -14,9 +14,9 @@ We call Patroni a "template" because it is far from being a one-size-fits-all or
 
 Currently supported PostgreSQL versions: 9.3 to 16.
 
-**Note to Citus users**: Starting from 3.0 Patroni nicely integrates with the `Citus <https://github.com/citusdata/citus>`__ database extension to Postgres. Please check the `Citus support page <https://github.com/zalando/patroni/blob/master/docs/citus.rst>`__ in the Patroni documentation for more info about how to use Patroni high availability together with a Citus distributed cluster.
+**Note to Citus users**: Starting from 3.0 Patroni nicely integrates with the `Citus <https://github.com/citusdata/citus>`__ database extension to Postgres. Please check the `Citus support page <https://github.com/patroni/patroni/blob/master/docs/citus.rst>`__ in the Patroni documentation for more info about how to use Patroni high availability together with a Citus distributed cluster.
 
-**Note to Kubernetes users**: Patroni can run natively on top of Kubernetes. Take a look at the `Kubernetes <https://github.com/zalando/patroni/blob/master/docs/kubernetes.rst>`__ chapter of the Patroni documentation.
+**Note to Kubernetes users**: Patroni can run natively on top of Kubernetes. Take a look at the `Kubernetes <https://github.com/patroni/patroni/blob/master/docs/kubernetes.rst>`__ chapter of the Patroni documentation.
 
 .. contents::
     :local:
@@ -27,9 +27,7 @@ Currently supported PostgreSQL versions: 9.3 to 16.
 How Patroni Works
 =================
 
-Patroni originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.
-
-For an example of a Docker-based deployment with Patroni, see `Spilo <https://github.com/zalando/spilo>`__, currently in use at Zalando.
+Patroni (formerly known as Zalando's Patroni) originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.
 
 For additional background info, see:
 
@@ -41,15 +39,15 @@ For additional background info, see:
 Development Status
 ==================
 
-Patroni is in active development and accepts contributions. See our `Contributing <https://github.com/zalando/patroni/blob/master/docs/contributing_guidelines.rst>`__ section below for more details.
+Patroni is in active development and accepts contributions. See our `Contributing <https://github.com/patroni/patroni/blob/master/docs/contributing_guidelines.rst>`__ section below for more details.
 
-We report new releases information `here <https://github.com/zalando/patroni/releases>`__.
+We report new releases information `here <https://github.com/patroni/patroni/releases>`__.
 
 =========
 Community
 =========
 
-There are two places to connect with the Patroni community: `on github <https://github.com/zalando/patroni>`__, via Issues and PRs, and on channel `#patroni <https://postgresteam.slack.com/archives/C9XPYG92A>`__ in the `PostgreSQL Slack <https://pgtreats.info/slack-invite>`__.  If you're using Patroni, or just interested, please join us.
+There are two places to connect with the Patroni community: `on github <https://github.com/patroni/patroni>`__, via Issues and PRs, and on channel `#patroni <https://postgresteam.slack.com/archives/C9XPYG92A>`__ in the `PostgreSQL Slack <https://pgtreats.info/slack-invite>`__.  If you're using Patroni, or just interested, please join us.
 
 ===================================
 Technical Requirements/Installation
@@ -151,27 +149,27 @@ run:
 YAML Configuration
 ==================
 
-Go `here <https://github.com/zalando/patroni/blob/master/docs/dynamic_configuration.rst>`__ for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/zalando/patroni/blob/master/postgres0.yml>`__.
+Go `here <https://github.com/patroni/patroni/blob/master/docs/dynamic_configuration.rst>`__ for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/patroni/patroni/blob/master/postgres0.yml>`__.
 
 =========================
 Environment Configuration
 =========================
 
-Go `here <https://github.com/zalando/patroni/blob/master/docs/ENVIRONMENT.rst>`__ for comprehensive information about configuring(overriding) settings via environment variables.
+Go `here <https://github.com/patroni/patroni/blob/master/docs/ENVIRONMENT.rst>`__ for comprehensive information about configuring(overriding) settings via environment variables.
 
 ===================
 Replication Choices
 ===================
 
-Patroni uses Postgres' streaming replication, which is asynchronous by default. Patroni's asynchronous replication configuration allows for ``maximum_lag_on_failover`` settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the leader. This setting should be increased or decreased based on business requirements. It's also possible to use synchronous replication for better durability guarantees. See `replication modes documentation <https://github.com/zalando/patroni/blob/master/docs/replication_modes.rst>`__ for details.
+Patroni uses Postgres' streaming replication, which is asynchronous by default. Patroni's asynchronous replication configuration allows for ``maximum_lag_on_failover`` settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the leader. This setting should be increased or decreased based on business requirements. It's also possible to use synchronous replication for better durability guarantees. See `replication modes documentation <https://github.com/patroni/patroni/blob/master/docs/replication_modes.rst>`__ for details.
 
 ======================================
 Applications Should Not Use Superusers
 ======================================
 
 When connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from an application, you can potentially use the entire connection pool, including the connections reserved for superusers, with the ``superuser_reserved_connections`` setting. If Patroni cannot access the Primary because the connection pool is full, behavior will be undesirable.
 
-.. |Tests Status| image:: https://github.com/zalando/patroni/actions/workflows/tests.yaml/badge.svg
-   :target: https://github.com/zalando/patroni/actions/workflows/tests.yaml?query=branch%3Amaster
-.. |Coverage Status| image:: https://coveralls.io/repos/zalando/patroni/badge.svg?branch=master
-   :target: https://coveralls.io/github/zalando/patroni?branch=master
+.. |Tests Status| image:: https://github.com/patroni/patroni/actions/workflows/tests.yaml/badge.svg
+   :target: https://github.com/patroni/patroni/actions/workflows/tests.yaml?query=branch%3Amaster
+.. |Coverage Status| image:: https://coveralls.io/repos/patroni/patroni/badge.svg?branch=master
+   :target: https://coveralls.io/github/patroni/patroni?branch=master

docker-compose-citus.yml

@@ -8,7 +8,7 @@
 # The cluster could be started as:
 # $ docker-compose -f docker-compose-citus.yml up -d
 # You can read more about it in the:
-# https://github.com/zalando/patroni/blob/master/docker/README.md#citus-cluster
+# https://github.com/patroni/patroni/blob/master/docker/README.md#citus-cluster
 version: "2"
 
 networks:

docker-compose.yml

@@ -6,7 +6,7 @@
 # The cluster could be started as:
 # $ docker-compose up -d
 # You can read more about it in the:
-# https://github.com/zalando/patroni/blob/master/docker/README.md
+# https://github.com/patroni/patroni/blob/master/docker/README.md
 version: "2"
 
 networks:

docs/ENVIRONMENT.rst

@@ -80,7 +80,7 @@ Etcdv3
 Environment names for Etcdv3 are similar as for Etcd, you just need to use ``ETCD3`` instead of ``ETCD`` in the variable name. Example: ``PATRONI_ETCD3_HOST``, ``PATRONI_ETCD3_CACERT``, and so on.
 
 .. warning::
-    Keys created with protocol version 2 are not visible with protocol version 3 and the other way around, therefore it is not possible to switch from Etcd to Etcdv3 just by updating Patroni configuration.
+    Keys created with protocol version 2 are not visible with protocol version 3 and the other way around, therefore it is not possible to switch from Etcd to Etcdv3 just by updating Patroni configuration. In addition, Patroni uses Etcd's gRPC-gateway (proxy) to communicate with the V3 API, which means that TLS common name authentication is not possible.
 
 
 ZooKeeper
@@ -122,6 +122,7 @@ Kubernetes
 -  **PATRONI\_KUBERNETES\_PORTS**: (optional) if the Service object has the name for the port, the same name must appear in the Endpoint object, otherwise service won't work. For example, if your service is defined as ``{Kind: Service, spec: {ports: [{name: postgresql, port: 5432, targetPort: 5432}]}}``, then you have to set ``PATRONI_KUBERNETES_PORTS='[{"name": "postgresql", "port": 5432}]'`` and Patroni will use it for updating subsets of the leader Endpoint. This parameter is used only if `PATRONI_KUBERNETES_USE_ENDPOINTS` is set.
 -  **PATRONI\_KUBERNETES\_CACERT**: (optional) Specifies the file with the CA_BUNDLE file with certificates of trusted CAs to use while verifying Kubernetes API SSL certs. If not provided, patroni will use the value provided by the ServiceAccount secret.
 -  **PATRONI\_RETRIABLE\_HTTP\_CODES**: (optional) list of HTTP status codes from K8s API to retry on. By default Patroni is retrying on ``500``, ``503``, and ``504``, or if K8s API response has ``retry-after`` HTTP header.
+-  **PATRONI\_KUBERNETES\_XLOG\_CACHE\_TTL**: (optional) duration in seconds to retain the previous value of the member xlog position when updating the member pod metadata. Higher values reduce the frequency of pod metadata updates from Patroni, at the expense of having an outdated xlog position. The default value is ``0``, indicating that the system should always use the up-to-date position. Setting it to the value that is a multiple of ``loop_wait`` reduces the number of API server requests in the Kubernetes cluster.
 
 Raft (deprecated)
 -----------------

docs/README.rst

@@ -6,8 +6,6 @@ Introduction
 
 Patroni is a template for high availability (HA) PostgreSQL solutions using Python. Patroni originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.
 
-For an example of a Docker-based deployment with Patroni, see `Spilo <https://github.com/zalando/spilo>`__, currently in use at Zalando.
-
 For additional background info, see:
 
 * `PostgreSQL HA with Kubernetes and Patroni <https://www.youtube.com/watch?v=iruaCgeG7qs>`__, talk by Josh Berkus at KubeCon 2016 (video)
@@ -39,7 +37,7 @@ perfectly fine. You can add more standby nodes later.
 Running and Configuring
 -----------------------
 
-The following section assumes Patroni repository as being cloned from https://github.com/zalando/patroni. Namely, you
+The following section assumes Patroni repository as being cloned from https://github.com/patroni/patroni. Namely, you
 will need example configuration files `postgres0.yml` and `postgres1.yml`. If you installed Patroni with pip, you can
 obtain those files from the git repository and replace `./patroni.py` below with `patroni` command.
 
@@ -69,7 +67,7 @@ run:
 YAML Configuration
 ------------------
 
-Go :ref:`here <yaml_configuration>` for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/zalando/patroni/blob/master/postgres0.yml>`__.
+Go :ref:`here <yaml_configuration>` for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/patroni/patroni/blob/master/postgres0.yml>`__.
 
 
 Environment Configuration

docs/citus.rst

@@ -335,7 +335,7 @@ new Kubernetes objects ConfigMaps or Endpoints, it automatically puts the
 You can find a complete example of Patroni deployment on Kubernetes with Citus
 support in the `kubernetes`__ folder of the Patroni repository.
 
-__ https://github.com/zalando/patroni/tree/master/kubernetes
+__ https://github.com/patroni/patroni/tree/master/kubernetes
 
 There are two important files for you:
 

docs/conf.py

@@ -77,8 +77,8 @@
 
 # General information about the project.
 project = 'Patroni'
-copyright = '2015 Compose, Zalando SE'
-author = 'Zalando SE'
+copyright = '2024 Compose, Zalando SE, Patroni Contributors'
+author = 'Patroni Contributors'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -134,7 +134,7 @@
 # Replace "source" links with "edit on GitHub" when using rtd theme
 html_context = {
     'display_github': True,
-    'github_user': 'zalando',
+    'github_user': 'patroni',
     'github_repo': 'patroni',
     'github_version': 'master',
     'conf_py_path': '/docs/',
@@ -191,7 +191,7 @@
 #  author, documentclass [howto, manual, or own class]).
 latex_documents = [
     (master_doc, 'Patroni.tex', 'Patroni Documentation',
-     'Zalando SE', 'manual'),
+     'Patroni Contributors', 'manual'),
 ]
 
 

docs/contributing_guidelines.rst

@@ -16,7 +16,7 @@ Reporting bugs
 --------------
 
 Before reporting a bug please make sure to **reproduce it with the latest Patroni version**!
-Also please double check if the issue already exists in our `Issues Tracker <https://github.com/zalando/patroni/issues>`__.
+Also please double check if the issue already exists in our `Issues Tracker <https://github.com/patroni/patroni/issues>`__.
 
 Running tests
 -------------

docs/dynamic_configuration.rst

@@ -30,7 +30,7 @@ In order to change the dynamic configuration you can use either :ref:`patronictl
 -  **failsafe\_mode**: Enables :ref:`DCS Failsafe Mode <dcs_failsafe_mode>`. Defaults to `false`.
 -  **postgresql**:
 
-   -  **use\_pg\_rewind**: whether or not to use pg_rewind. Defaults to `false`.
+   -  **use\_pg\_rewind**: whether or not to use pg_rewind. Defaults to `false`. Note that either the cluster must be initialized with ``data page checksums`` (``--data-checksums`` option for ``initdb``) and/or ``wal_log_hints`` must be set to ``on``, or ``pg_rewind`` will not work.
    -  **use\_slots**: whether or not to use replication slots. Defaults to `true` on PostgreSQL 9.4+.
    -  **recovery\_conf**: additional configuration settings written to recovery.conf when configuring follower. There is no recovery.conf anymore in PostgreSQL 12, but you may continue using this section, because Patroni handles it transparently.
    -  **parameters**: list of configuration settings for Postgres.

docs/ha_multi_dc.rst

@@ -44,7 +44,7 @@ You should not use ``pg_ctl promote`` in this scenario, you need "manually promo
 
 In case you want to return to the "initial" state, there are only two ways of resolving it:
 
-- Add the standby_cluster section back and it will trigger pg_rewind, but there are chances that pg_rewind will fail.
+- Add the standby_cluster section back and it will trigger ``pg_rewind``; however, for ``pg_rewind`` to function properly, either the cluster must be initialized with ``data page checksums`` (``--data-checksums`` option for ``initdb``) and/or ``wal_log_hints`` must be set to ``on``, but there are still chances that ``pg_rewind`` might fail due to other factors.
 - Rebuild the standby cluster from scratch.
 
 Before promoting standby cluster one have to manually ensure that the source cluster is down (STONITH). When DC1 recovers, the cluster has to be converted to a standby cluster.

docs/kubernetes.rst

@@ -87,7 +87,7 @@ Note that if you migrate from default role labels to custom ones, you can reduce
 Examples
 --------
 
-- The `kubernetes <https://github.com/zalando/patroni/tree/master/kubernetes>`__ folder of the Patroni repository contains
+- The `kubernetes <https://github.com/patroni/patroni/tree/master/kubernetes>`__ folder of the Patroni repository contains
   examples of the Docker image, and the Kubernetes manifest to test Patroni Kubernetes setup.
   Note that in the current state it will not be able to use PersistentVolumes because of permission issues.
 
@@ -98,5 +98,5 @@ Examples
   to deploy the Spilo image configured with Patroni running using Kubernetes.
 
 - In order to run your database clusters at scale using Patroni and Spilo, take a look at the
-  `postgres-operator <https://github.com/zalando-incubator/postgres-operator>`_ project. It implements the operator pattern
+  `postgres-operator <https://github.com/zalando/postgres-operator>`_ project. It implements the operator pattern
   to manage Spilo clusters.

docs/patronictl.rst

@@ -1065,7 +1065,7 @@ Run a SQL command as ``postgres`` user, and take password from ``libpq`` environ
 
 .. code:: bash
 
-    $ PGPASSWORD=zalando patronictl -c postgres0.yml query batman -U postgres -c "SELECT now()"
+    $ PGPASSWORD=patroni patronictl -c postgres0.yml query batman -U postgres -c "SELECT now()"
     now
     2023-09-12 18:11:37.639500+00:00