…pache#18074)

…pache#18097)

…ad position for consumer (apache#18037)

### Motivation
Fixes apache#18036

### Modifications
- The backlog eviction policy should use `asyncMarkDelete` instead of `resetCursor` in order to move the mark delete position.

…t.PersistentSubscriptionMessageDispatchStreamingDispatcherThrottlingTest#setup (apache#18115)

…che#18072)

… entries. (apache#18106)

…pache#18094)

… tools (apache#15485)

* add a new section for websocket API

* Update site2/docs/security-tls-transport.md

Co-authored-by: Yunze Xu <[email protected]>

* Update site2/docs/security-tls-transport.md

Co-authored-by: Yunze Xu <[email protected]>

* Update site2/docs/security-tls-transport.md

Co-authored-by: Yunze Xu <[email protected]>

* updates

* updates

* updates

* update code snippet as per review comments

Co-authored-by: Yunze Xu <[email protected]>

…apache#17962)

* [feat][PIP-188] support blue-green cluster migration [part-1]

Add blue-green cluster migration

Fix dependency

* cleanup

…rom standalone.conf (apache#18126)

)

* [improve][io] JDBC sinks: implement JDBC Batch API

* more tests and transactions support

* remove .db files

* doc

* fix batch results and thread safey

* add next flush test - fix doc - improve code readability

* [fix][doc] Point Python and C++ docs to new GitHub repos

* Fix cpp examples link

* Add notice to redirect any users

…follow BookieServiceInfo updates (apache#18133)

* [fix] PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates

* remove debug

### Motivation
The current MAX_ACK_GROUP_SIZE is fixed at 1000, increase the configuration acknowledgementsGroupSize, support MAX_ACK_GROUP_SIZE configurable:
https://github.com/apache/pulsar/blob/afcdbf0e2b5fb905e1f82f0220436f8f9ec0c742/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java#L63-L64


### Modifications
1.Add configuration acknowledgementsGroupSize；

…ient memory limitation. (apache#17936)

…#18093)

* [feat][ci] check style for all source code

This closes apache#18131.

* fix violations

Signed-off-by: tison <[email protected]>

* no snapshot update

Signed-off-by: tison <[email protected]>

* remove mysterious dependency

Signed-off-by: tison <[email protected]>

Signed-off-by: tison <[email protected]>

… and check whether it is greater than the batch index of the previous ack (apache#18042)

Co-authored-by: leolinchen <[email protected]>

…tion (apache#17915)

…ache#18148)

Co-authored-by: leolinchen <[email protected]>

…ing) (apache#17398)

…x system topic (apache#16931)

Master Issue: apache#16913
### Motivation
Implement system topic client for snapshot segment topic and index topic to send segment snapshots or indexes.
The configuration `transactionBufferSegmentedSnapshotEnabled` is used in the Transaction Buffer to determine which `AbortedTxnProcessor` is adopted by this TB.
### Modification

In the new implementation of the Transaction Buffer Snapshot System topic, because the system topic that needs to be processed has changed from the original one to three with different schemes, we have added generics to the TransactionBufferSnapshotBaseSystemTopicClient class and the SystemTopicTxnBufferSnapshotService<T> class.
And Pulsar Service maintains a factory class TransactionBufferSnapshotServiceFactory used to obtain SystemTopicTxnBufferSnapshotService.
This way, we can obtain the required System topic client through pulsarService to read and send snapshots.
<img width="1336" alt="image" src="https://user-images.githubusercontent.com/55571188/197467173-9028e58a-79cc-4fe4-81e2-c299c568caee.png">

…n delete namespaces (apache#18153)

### Motivation

In the current implementation, don't pass the `force` parameter to `deleteNamespaceBundle` when deleting namespaces, we should pass the `force` parameter.

### Modifications

Passing `force` parameter to `deleteNamespaceBundle` when delete namespaces.

…ackAffinityMapping (apache#15640)

* [Autorecovery] Default reppDnsResolverClass to ZkBookieRackAffinityMapping

* Improve documentation

Fixes: apache#18012

### Motivation

The current Bookkeeper configuration defaults to using `org.apache.bookkeeper.net.ScriptBasedMapping` for the `DNSToSwitchMapping` implementation. However, this default configuration does not align with the Broker's default configuration, which is `org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping`. As such, the default configuration for a Pulsar cluster does not lead to ideal rack awareness when ledgers need to be recovered. The result is that a user can configure a cluster for rack awareness and the brokers will honor that configuration, but the autorecovery process will not because it does not have the correct bookkeeper cluster topology view.

I propose we configure bookkeeper to use the broker's `ZkBookieRackAffinityMapping` class. That way, autorecovery will honor the operator's configured rack awareness policies out of the box.

### Modifications

* Add default value for `reppDnsResolverClass` to the `conf/bookkeeper.conf` configuration. This change effectively switches the default from `org.apache.bookkeeper.net.ScriptBasedMapping` to `org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping`.

### Verifying this change

I manually verified that the `ZkBookieRackAffinityMapping` works by running some tests in a minikube cluster deployed with the DataStax helm chart. I set up 3 racks, 4 bookies, and a topic with a E=2, Qw=2, and Qa=2. I then verified that the autorecovery pod correctly discovered the racks and then identified when an ensemble was not following the rack placement policy after two bookies were removed. I documented my testing a bit more here: datastax/pulsar-helm-chart#214.

### Does this pull request potentially affect one of the following parts:

It changes a default value. The tradeoff is that a user relying on the `ScriptBasedMapping` default might accidentally get switched to using the `ZkBookieRackAffinityMapping` implementation. Given that `ScriptBasedMapping` doesn't work out of the box, and that the broker's default to `ZkBookieRackAffinityMapping`, I think this is an acceptable tradeoff.

- [x] `doc`

* [fix][doc] Remove archived and halted repositories

* Add back archived repositories

For anyone who wants to revive them.

Signed-off-by: tison <[email protected]>

Signed-off-by: tison <[email protected]>

…he#18177)

…tore (apache#18172)

…ically (apache#18181)

…#18184)

* do not serialize field bundleStats

* add unit test.

* use org.testng.Assert.assertFalse

* add apache license header

* fix doc

* fix doc and change the package level.

Co-authored-by: fengwenzhi <[email protected]>

* add go-client chunking doc

* Update site2/docs/client-libraries-go.md

Co-authored-by: Zixuan Liu <[email protected]>

* Update site2/docs/client-libraries-go.md

Co-authored-by: Anonymitaet <[email protected]>

* Update site2/docs/client-libraries-go.md

Co-authored-by: Anonymitaet <[email protected]>

* Update site2/docs/client-libraries-go.md

Co-authored-by: Anonymitaet <[email protected]>

Co-authored-by: Zixuan Liu <[email protected]>
Co-authored-by: Anonymitaet <[email protected]>

…multiple providers (apache#18035)

* Update security-athenz.md

* improve auth overview

* address review comments

* Improve OAuth2 authentication

* improve Kerberos authentication

* Update security-athenz.md

…he#18188)

* [improve][doc] cherry-pick cpp client docs installation section

Signed-off-by: tison <[email protected]>

* Apply suggestions from code review

Co-authored-by: momo-jun <[email protected]>

* Update client-libraries-cpp.md

Signed-off-by: tison <[email protected]>
Co-authored-by: momo-jun <[email protected]>

… daily (apache#18081)

apache#18192)

…pache#18211)

apache#18205)

…henTrimLedgers` (apache#18147)

…ache#18197)

…apache#18214)

apache#18219)

### Motivation
https://github.com/apache/pulsar/blob/b061c6ac5833c21e483368febebd0d30679a35e1/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java#L748-L774
The `pendingLookupRequestSemaphore` will leak when handleError. There are `LookUpRequestSemaphore` not released when removing it from `pendingRequests`

related PR: apache#17856 

### Modifications
We can't easily release the semaphore in `handleError`, because there are not only `LookUpRequest`. So release the semaphore when LookupException

### Verifying this change
Add unit test case to cover this change

### Documentation

<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->

- [ ] `doc-required` 
(Your PR needs to update docs and you will update later)

- [x] `doc-not-needed` 
bug fixs, no need doc

- [ ] `doc` 
(Your PR contains doc changes)

- [ ] `doc-complete`
(Docs have been already added)

…reconnecting (apache#18194)

…ache#18216)

Signed-off-by: Zixuan Liu <[email protected]>

…tup (apache#18179)

…he#18226)

…covery (apache#18228)

…cks in some cases when offload (apache#18221)

…multiple messages (apache#18238)

…with active consumer leaves topic metadata inconsistent. (apache#18193)