HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

KeeProMise · 2025-01-23T03:47:01Z

Description of PR

We need to add a description document to describe the function, configuration and other information of the asynchronous router.
seeAlso: https://issues.apache.org/jira/browse/HDFS-17715

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…by hfutatzhanghb. Reviewed-by: Jian Zhang <[email protected]>

…che#7188) Co-authored-by: Jian Zhang <[email protected]>

…nchrony. (apache#7139). Contributed by hfutatzhanghb. Co-authored-by: Jian Zhang <[email protected]> Signed-off-by: He Xiaoqiao <[email protected]>

…ributed by hfutatzhanghb. Reviewed-by: Jian Zhang <[email protected]> Signed-off-by: He Xiaoqiao <[email protected]>

hadoop-yetus · 2025-01-23T05:47:56Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ HDFS-17531 Compile Tests _
+1 💚	mvninstall	36m 57s		HDFS-17531 passed
+1 💚	mvnsite	0m 47s		HDFS-17531 passed
+1 💚	shadedclient	70m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	mvnsite	0m 37s		the patch passed
+1 💚	shadedclient	33m 13s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		119m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/1/artifact/out/Dockerfile
GITHUB PR	#7318
Optional Tests	dupname asflicense mvnsite codespell detsecrets markdownlint
uname	Linux ab467daa951a 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	HDFS-17531 / `4e4700c`
Max. process+thread count	700 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

KeeProMise · 2025-01-23T06:47:13Z

@Hexiaoqiao @ferhui
Hi, if you have time, please help to take a look at this PR, thanks!

hadoop-yetus · 2025-01-23T08:12:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 40s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ HDFS-17531 Compile Tests _
+1 💚	mvninstall	35m 40s		HDFS-17531 passed
+1 💚	mvnsite	0m 47s		HDFS-17531 passed
+1 💚	shadedclient	69m 47s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 36s		the patch passed
+1 💚	shadedclient	35m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 36s		The patch does not generate ASF License warnings.
		109m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/2/artifact/out/Dockerfile
GITHUB PR	#7318
Optional Tests	dupname asflicense mvnsite codespell detsecrets markdownlint
uname	Linux ebf5a0ed4347 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	HDFS-17531 / `f05c565`
Max. process+thread count	562 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-01-23T08:27:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 51s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ HDFS-17531 Compile Tests _
+1 💚	mvninstall	40m 31s		HDFS-17531 passed
+1 💚	mvnsite	0m 49s		HDFS-17531 passed
+1 💚	shadedclient	79m 23s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 36s		the patch passed
+1 💚	shadedclient	38m 28s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		121m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/3/artifact/out/Dockerfile
GITHUB PR	#7318
Optional Tests	dupname asflicense mvnsite codespell detsecrets markdownlint
uname	Linux 30525c0759bb 5.15.0-125-generic #135-Ubuntu SMP Fri Sep 27 13:53:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	HDFS-17531 / `f05c565`
Max. process+thread count	525 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/3/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-01-23T08:44:56Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ HDFS-17531 Compile Tests _
+1 💚	mvninstall	40m 0s		HDFS-17531 passed
+1 💚	mvnsite	0m 49s		HDFS-17531 passed
+1 💚	shadedclient	79m 23s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 36s		the patch passed
+1 💚	shadedclient	38m 36s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		121m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/4/artifact/out/Dockerfile
GITHUB PR	#7318
Optional Tests	dupname asflicense mvnsite codespell detsecrets markdownlint
uname	Linux 5268605eb913 5.15.0-125-generic #135-Ubuntu SMP Fri Sep 27 13:53:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	HDFS-17531 / `5393932`
Max. process+thread count	571 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/4/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-01-23T10:35:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ HDFS-17531 Compile Tests _
+1 💚	mvninstall	35m 38s		HDFS-17531 passed
+1 💚	mvnsite	0m 47s		HDFS-17531 passed
+1 💚	shadedclient	69m 31s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 36s		the patch passed
+1 💚	shadedclient	33m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		106m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/5/artifact/out/Dockerfile
GITHUB PR	#7318
Optional Tests	dupname asflicense mvnsite codespell detsecrets markdownlint
uname	Linux 5a1b24b31979 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	HDFS-17531 / `8743956`
Max. process+thread count	555 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/5/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-01-23T14:28:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ HDFS-17531 Compile Tests _
+1 💚	mvninstall	36m 0s		HDFS-17531 passed
+1 💚	mvnsite	0m 47s		HDFS-17531 passed
+1 💚	shadedclient	70m 9s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	70m 30s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 34s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	mvnsite	0m 37s		the patch passed
+1 💚	shadedclient	34m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		107m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/6/artifact/out/Dockerfile
GITHUB PR	#7318
Optional Tests	dupname asflicense mvnsite codespell detsecrets markdownlint
uname	Linux 730a28192c37 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	HDFS-17531 / `31852d5`
Max. process+thread count	709 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7318/6/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hfutatzhanghb and others added 5 commits January 20, 2025 14:38

HADOOP-19361. RPC DeferredMetrics bugfix. (apache#7220). Contributed …

1f1d7c1

…by hfutatzhanghb. Reviewed-by: Jian Zhang <[email protected]>

HDFS-17640.[ARR] RouterClientProtocol supports asynchronous rpc. (apa…

1e0dabc

…che#7188) Co-authored-by: Jian Zhang <[email protected]>

HDFS-17650. [ARR] The router server-side rpc protocol PB supports asy…

d83dae4

…nchrony. (apache#7139). Contributed by hfutatzhanghb. Co-authored-by: Jian Zhang <[email protected]> Signed-off-by: He Xiaoqiao <[email protected]>

HDFS-17651.[ARR] Async handler executor isolation (apache#7244). Cont…

616d707

…ributed by hfutatzhanghb. Reviewed-by: Jian Zhang <[email protected]> Signed-off-by: He Xiaoqiao <[email protected]>

[ARR] Add documentation for asynchronous router.

4e4700c

github-actions bot added HDFS trunk RBF labels Jan 23, 2025

KeeProMise added 3 commits January 23, 2025 14:21

fix blank

52e6276

fix blank

f05c565

fix blank

5393932

fix

8743956

KeeProMise force-pushed the HDFS-17531 branch from 616d707 to 15cc381 Compare January 23, 2025 12:34

KeeProMise added 2 commits January 23, 2025 20:36

Merge remote-tracking branch 'origin/HDFS-17531' into HDFS-17715

ea464b2

merge

31852d5

KeeProMise mentioned this pull request Jan 24, 2025

HDFS-17531. RBF: Aynchronous router RPC. #7308

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

KeeProMise commented Jan 23, 2025 •

edited

Loading

hadoop-yetus commented Jan 23, 2025

KeeProMise commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

Are you sure you want to change the base?

HDFS-17715. [ARR] Add documentation for asynchronous router. #7318

Conversation

KeeProMise commented Jan 23, 2025 • edited Loading

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Jan 23, 2025

KeeProMise commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

hadoop-yetus commented Jan 23, 2025

KeeProMise commented Jan 23, 2025 •

edited

Loading