feat: sync y-octo's logic #563

darkskygit · 2023-12-28T11:23:41Z

No description provided.

this PR partially solves y-crdt/y-octo##33 and greatly improves the performance of apply update test cases show a performance improvement of about 40-50%

…o#39)

vercel · 2023-12-28T11:23:46Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
octobase	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 31, 2024 7:41am

libs/jwst-rpc/src/utils/mod.rs

github-actions · 2023-12-28T12:05:51Z

Benchmark for Linux

Click to view benchmark

Test	Base	PR	%
codec/jwst decode var_int (32 bit)	199.1±3.35µs	196.4±4.35µs	-1.36%
codec/jwst decode var_uint (32 bit)	166.5±3.11µs	169.1±1.91µs	+1.56%
codec/jwst decode var_uint (64 bit)	166.3±3.03µs	168.7±2.55µs	+1.44%
codec/jwst encode var_int (32 bit)	299.9±6.41µs	297.6±6.08µs	-0.77%
codec/jwst encode var_uint (32 bit)	260.9±5.13µs	266.3±4.99µs	+2.07%
codec/jwst encode var_uint (64 bit)	264.7±19.23µs	266.6±3.19µs	+0.72%
codec/lib0 decode var_int (64 bit)	162.7±3.56µs	160.1±8.81µs	-1.60%
codec/lib0 decode var_uint (32 bit)	140.0±7.64µs	139.1±3.25µs	-0.64%
codec/lib0 decode var_uint (64 bit)	132.1±2.88µs	136.2±2.53µs	+3.10%
codec/lib0 encode var_int (64 bit)	377.1±6.82µs	383.9±1.31µs	+1.80%
codec/lib0 encode var_uint (32 bit)	363.2±6.86µs	370.1±5.30µs	+1.90%
codec/lib0 encode var_uint (64 bit)	367.2±7.00µs	367.1±6.05µs	-0.03%
ops/array/jwst/insert	107.9±2.11µs	107.1±1.16µs	-0.74%
ops/array/jwst/insert range	160.7±2.81µs	154.2±2.25µs	-4.04%
ops/array/jwst/remove	63.8±0.42µs	61.3±1.20µs	-3.92%
ops/array/yrs/insert	56.6±1.30µs	57.7±0.92µs	+1.94%
ops/array/yrs/insert range	72.3±1.96µs	73.9±1.45µs	+2.21%
ops/array/yrs/remove	64.3±1.02µs	64.8±0.83µs	+0.78%
ops/map/jwst/get	337.2±6.23ns	218.6±3.98ns	-35.17%
ops/map/jwst/insert	4.0±0.07µs	3.3±0.06µs	-17.50%
ops/map/jwst/remove	4.8±0.08µs	4.5±0.07µs	-6.25%
ops/map/yrs/get	248.9±4.41ns	247.1±3.95ns	-0.72%
ops/map/yrs/insert	3.8±0.05µs	3.6±0.08µs	-5.26%
ops/map/yrs/remove	4.9±0.07µs	4.7±0.08µs	-4.08%
ops/text/jwst/insert	66.4±1.82µs	64.6±1.55µs	-2.71%
ops/text/jwst/remove	2.0±0.03µs	1426.9±11.69ns	+71245.00%
ops/text/yrs/insert	26.9±0.46µs	27.3±0.32µs	+1.49%
ops/text/yrs/remove	1661.5±31.32ns	1665.4±26.92ns	+0.23%
update/parse with jwst/basic.bin	37.0±0.44µs	36.0±0.21µs	-2.70%
update/parse with jwst/database.bin	27.7±0.49µs	27.5±0.55µs	-0.72%
update/parse with jwst/large.bin	1757.2±33.74µs	1781.0±8.37µs	+1.35%
update/parse with jwst/with-subdoc.bin	4.6±0.09µs	4.9±0.06µs	+6.52%
update/parse with yrs/basic.bin	29.1±0.53µs	29.3±0.27µs	+0.69%
update/parse with yrs/database.bin	22.7±0.48µs	22.6±0.34µs	-0.44%
update/parse with yrs/large.bin	1317.1±24.44µs	1316.2±20.11µs	-0.07%
update/parse with yrs/with-subdoc.bin	3.8±0.06µs	4.6±0.07µs	+21.05%

Benchmark for Windows

Click to view benchmark

Test	Base	PR	%
codec/jwst decode var_int (32 bit)	214.5±2.54µs	213.3±2.98µs	-0.56%
codec/jwst decode var_uint (32 bit)	195.0±2.33µs	191.1±3.86µs	-2.00%
codec/jwst decode var_uint (64 bit)	191.8±3.02µs	195.6±2.32µs	+1.98%
codec/jwst encode var_int (32 bit)	303.9±2.95µs	333.8±3.75µs	+9.84%
codec/jwst encode var_uint (32 bit)	271.3±2.76µs	248.7±6.52µs	-8.33%
codec/jwst encode var_uint (64 bit)	261.8±2.88µs	246.4±6.10µs	-5.88%
codec/lib0 decode var_int (64 bit)	188.9±4.67µs	188.8±6.35µs	-0.05%
codec/lib0 decode var_uint (32 bit)	156.9±6.07µs	159.9±9.08µs	+1.91%
codec/lib0 decode var_uint (64 bit)	156.7±5.97µs	156.2±6.89µs	-0.32%
codec/lib0 encode var_int (64 bit)	386.7±3.01µs	385.7±1.80µs	-0.26%
codec/lib0 encode var_uint (32 bit)	373.5±0.93µs	373.4±1.26µs	-0.03%
codec/lib0 encode var_uint (64 bit)	374.0±1.34µs	373.2±1.12µs	-0.21%
ops/array/jwst/insert	149.8±0.92µs	147.7±1.70µs	-1.40%
ops/array/jwst/insert range	210.8±1.20µs	206.7±2.86µs	-1.94%
ops/array/jwst/remove	71.7±0.47µs	92.3±1.23µs	+28.73%
ops/array/yrs/insert	71.3±0.70µs	70.7±0.47µs	-0.84%
ops/array/yrs/insert range	96.3±0.51µs	96.4±0.43µs	+0.10%
ops/array/yrs/remove	100.4±0.43µs	103.1±0.63µs	+2.69%
ops/map/jwst/get	380.3±3.33ns	278.4±6.20ns	-26.79%
ops/map/jwst/insert	6.0±0.04µs	4.7±0.03µs	-21.67%
ops/map/jwst/remove	6.3±0.03µs	6.7±0.03µs	+6.35%
ops/map/yrs/get	277.5±6.06ns	274.8±4.94ns	-0.97%
ops/map/yrs/insert	5.7±0.03µs	5.7±0.03µs	-0.00%
ops/map/yrs/remove	7.0±0.03µs	7.1±0.03µs	+1.43%
ops/text/jwst/insert	88.2±1.38µs	82.5±0.94µs	-6.46%
ops/text/jwst/remove	2.4±0.01µs	2.2±0.02µs	-8.33%
ops/text/yrs/insert	32.2±0.33µs	32.5±0.16µs	+0.93%
ops/text/yrs/remove	2.5±0.01µs	2.6±0.02µs	+4.00%
update/parse with jwst/basic.bin	55.2±0.94µs	57.4±0.49µs	+3.99%
update/parse with jwst/database.bin	43.0±0.21µs	44.7±0.26µs	+3.95%
update/parse with jwst/large.bin	2.6±0.08ms	2.8±0.20ms	+7.69%
update/parse with jwst/with-subdoc.bin	8.1±0.04µs	8.1±0.04µs	-0.00%
update/parse with yrs/basic.bin	46.0±0.29µs	43.8±0.42µs	-4.78%
update/parse with yrs/database.bin	34.5±0.13µs	35.2±0.41µs	+2.03%
update/parse with yrs/large.bin	2.2±0.05ms	2.1±0.08ms	-4.55%
update/parse with yrs/with-subdoc.bin	7.2±0.06µs	7.3±0.04µs	+1.39%

codecov · 2023-12-28T14:04:16Z

Codecov Report

Attention: 82 lines in your changes are missing coverage. Please review.

Comparison is base (49a6b7a) 65.66% compared to head (dc2caeb) 65.97%.

Files	Patch %	Lines
libs/jwst-core/src/workspaces/sync.rs	0.00%	21 Missing ⚠️
libs/jwst-rpc/src/utils/mod.rs	0.00%	17 Missing ⚠️
libs/jwst-codec/src/doc/common/range.rs	89.56%	12 Missing ⚠️
libs/jwst-codec/src/doc/codec/io/codec_v1.rs	90.90%	7 Missing ⚠️
libs/jwst-codec-utils/bin/doc_merger.rs	0.00%	4 Missing ⚠️
libs/jwst-rpc/src/context.rs	0.00%	3 Missing ⚠️
libs/jwst-rpc/src/connector/memory.rs	0.00%	2 Missing ⚠️
libs/jwst-storage/src/storage/docs/database.rs	60.00%	2 Missing ⚠️
apps/keck/src/server/api/blocks/history.rs	0.00%	1 Missing ⚠️
libs/jwst-codec/src/doc/codec/io/writer.rs	0.00%	1 Missing ⚠️
... and 12 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #563      +/-   ##
==========================================
+ Coverage   65.66%   65.97%   +0.31%     
==========================================
  Files         118      119       +1     
  Lines       13138    13250     +112     
==========================================
+ Hits         8627     8742     +115     
+ Misses       4511     4508       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-01-30T09:19:44Z

Benchmark for Linux

Click to view benchmark

Test	Base	PR	%
codec/jwst decode var_int (32 bit)	201.6±1.24µs	202.6±0.53µs	+0.50%
codec/jwst decode var_uint (32 bit)	171.0±0.41µs	173.1±10.46µs	+1.23%
codec/jwst decode var_uint (64 bit)	170.9±0.52µs	170.8±0.32µs	-0.06%
codec/jwst encode var_int (32 bit)	273.6±17.74µs	303.1±5.26µs	+10.78%
codec/jwst encode var_uint (32 bit)	251.5±3.81µs	251.7±4.57µs	+0.08%
codec/jwst encode var_uint (64 bit)	252.0±3.88µs	251.7±3.78µs	-0.12%
codec/lib0 decode var_int (64 bit)	161.3±1.62µs	161.2±0.87µs	-0.06%
codec/lib0 decode var_uint (32 bit)	142.6±1.22µs	142.7±1.74µs	+0.07%
codec/lib0 decode var_uint (64 bit)	136.2±1.86µs	135.8±1.76µs	-0.29%
codec/lib0 encode var_int (64 bit)	386.0±2.66µs	384.0±0.91µs	-0.52%
codec/lib0 encode var_uint (32 bit)	374.1±0.96µs	373.8±0.45µs	-0.08%
codec/lib0 encode var_uint (64 bit)	374.2±1.25µs	376.0±12.36µs	+0.48%
ops/array/jwst/insert	110.9±1.59µs	101.8±0.42µs	-8.21%
ops/array/jwst/insert range	166.8±1.34µs	153.3±0.72µs	-8.09%
ops/array/jwst/remove	57.8±0.12µs	56.8±0.25µs	-1.73%
ops/array/yrs/insert	56.6±0.73µs	54.3±0.49µs	-4.06%
ops/array/yrs/insert range	72.7±0.57µs	69.6±0.24µs	-4.26%
ops/array/yrs/remove	65.3±0.54µs	64.5±1.39µs	-1.23%
ops/map/jwst/get	346.5±2.97ns	231.8±0.93ns	-33.10%
ops/map/jwst/insert	4.1±0.02µs	3.2±0.01µs	-21.95%
ops/map/jwst/remove	4.7±0.04µs	4.5±0.01µs	-4.26%
ops/map/yrs/get	240.7±1.43ns	249.9±1.06ns	+3.82%
ops/map/yrs/insert	3.8±0.11µs	3.7±0.09µs	-2.63%
ops/map/yrs/remove	4.8±0.02µs	4.8±0.02µs	-0.00%
ops/text/jwst/insert	69.9±1.97µs	65.4±1.45µs	-6.44%
ops/text/jwst/remove	2.1±0.01µs	1431.5±49.10ns	+68066.67%
ops/text/yrs/insert	27.5±0.16µs	28.0±0.12µs	+1.82%
ops/text/yrs/remove	1756.6±10.13ns	1686.3±13.39ns	-4.00%
update/parse with jwst/basic.bin	38.7±0.24µs	37.7±0.26µs	-2.58%
update/parse with jwst/database.bin	31.2±0.22µs	29.8±0.17µs	-4.49%
update/parse with jwst/large.bin	1900.5±8.87µs	1794.0±10.83µs	-5.60%
update/parse with jwst/with-subdoc.bin	4.5±0.10µs	4.9±0.08µs	+8.89%
update/parse with yrs/basic.bin	27.9±0.49µs	27.9±0.15µs	-0.00%
update/parse with yrs/database.bin	22.7±0.14µs	22.7±0.09µs	-0.00%
update/parse with yrs/large.bin	1281.2±6.04µs	1266.1±55.23µs	-1.18%
update/parse with yrs/with-subdoc.bin	3.3±0.04µs	3.7±0.02µs	+12.12%

Benchmark for Windows

Click to view benchmark

Test	Base	PR	%
codec/jwst decode var_int (32 bit)	244.0±4.91µs	242.8±0.90µs	-0.49%
codec/jwst decode var_uint (32 bit)	189.9±4.53µs	190.7±2.46µs	+0.42%
codec/jwst decode var_uint (64 bit)	190.9±2.76µs	196.2±4.94µs	+2.78%
codec/jwst encode var_int (32 bit)	335.4±7.79µs	302.6±1.87µs	-9.78%
codec/jwst encode var_uint (32 bit)	270.3±1.77µs	270.4±2.29µs	+0.04%
codec/jwst encode var_uint (64 bit)	281.4±2.30µs	270.8±2.91µs	-3.77%
codec/lib0 decode var_int (64 bit)	185.2±2.85µs	184.4±1.70µs	-0.43%
codec/lib0 decode var_uint (32 bit)	156.3±4.55µs	155.7±6.58µs	-0.38%
codec/lib0 decode var_uint (64 bit)	157.7±6.88µs	158.7±8.69µs	+0.63%
codec/lib0 encode var_int (64 bit)	385.0±3.88µs	385.4±7.06µs	+0.10%
codec/lib0 encode var_uint (32 bit)	373.3±0.96µs	374.1±5.15µs	+0.21%
codec/lib0 encode var_uint (64 bit)	374.3±1.26µs	387.3±4.32µs	+3.47%
ops/array/jwst/insert	138.0±1.22µs	135.4±1.27µs	-1.88%
ops/array/jwst/insert range	198.2±1.62µs	203.2±1.93µs	+2.52%
ops/array/jwst/remove	72.3±0.57µs	72.4±0.46µs	+0.14%
ops/array/yrs/insert	70.4±0.52µs	69.0±0.63µs	-1.99%
ops/array/yrs/insert range	95.8±0.45µs	94.3±0.34µs	-1.57%
ops/array/yrs/remove	98.3±0.30µs	103.1±0.95µs	+4.88%
ops/map/jwst/get	382.5±2.72ns	281.3±3.67ns	-26.46%
ops/map/jwst/insert	6.1±0.02µs	4.7±0.05µs	-22.95%
ops/map/jwst/remove	6.3±0.06µs	9.4±0.05µs	+49.21%
ops/map/yrs/get	276.9±3.75ns	276.8±4.34ns	-0.04%
ops/map/yrs/insert	5.8±0.04µs	5.8±0.03µs	-0.00%
ops/map/yrs/remove	7.0±0.04µs	7.0±0.05µs	-0.00%
ops/text/jwst/insert	91.4±0.86µs	87.1±0.99µs	-4.70%
ops/text/jwst/remove	2.6±0.01µs	2.2±0.01µs	-15.38%
ops/text/yrs/insert	32.6±0.25µs	32.4±0.90µs	-0.61%
ops/text/yrs/remove	2.6±0.02µs	2.6±0.02µs	-0.00%
update/parse with jwst/basic.bin	53.3±0.50µs	58.8±1.53µs	+10.32%
update/parse with jwst/database.bin	40.8±0.18µs	46.4±0.25µs	+13.73%
update/parse with jwst/large.bin	2.7±0.13ms	2.9±0.09ms	+7.41%
update/parse with jwst/with-subdoc.bin	8.3±0.04µs	8.4±0.04µs	+1.20%
update/parse with yrs/basic.bin	43.5±0.38µs	42.3±0.29µs	-2.76%
update/parse with yrs/database.bin	34.8±1.18µs	33.3±0.24µs	-4.31%
update/parse with yrs/large.bin	2.1±0.05ms	2.2±0.11ms	+4.76%
update/parse with yrs/with-subdoc.bin	7.2±0.04µs	7.0±0.03µs	-2.78%

darkskygit and others added 7 commits December 28, 2023 18:52

perf: improve apply update performance (y-crdt/y-octo#34)

d634f63

this PR partially solves y-crdt/y-octo##33 and greatly improves the performance of apply update test cases show a performance improvement of about 40-50%

refactor: decoder accept &[u8] as input (y-crdt/y-octo#35)

f15e0de

chore: apply clippy suggestions (y-crdt/y-octo#37)

5465a29

perf: avoid sort and squash on every push (y-crdt/y-octo#38)

20d7402

perf: don't extract changed crdt item when apply update (y-crdt/y-oct…

565e53f

…o#39)

feat: somlstr for parent key (y-crdt/y-octo#30)

f0cc5ac

chore: upgrade apis for jwst-codec

4737ffa

github-actions bot assigned darkskygit Dec 28, 2023

vercel bot deployed to Preview December 28, 2023 11:23 View deployment

github-advanced-security bot found potential problems Dec 28, 2023

View reviewed changes

libs/jwst-rpc/src/utils/mod.rs Fixed Show fixed Hide fixed

vercel bot deployed to Preview December 28, 2023 13:57 View deployment

styles: fix with clippy

be10f25

darkskygit force-pushed the darksky/yocto-sync branch from ee11c6f to be10f25 Compare December 28, 2023 13:59

vercel bot deployed to Preview December 28, 2023 13:59 View deployment

fix: clippy

78bd6ba

vercel bot deployed to Preview December 28, 2023 14:09 View deployment

chore: enable gc for cargo

d5d7287

vercel bot deployed to Preview December 28, 2023 14:14 View deployment

vercel bot deployed to Preview December 28, 2023 14:29 View deployment

chore: cleanup old codes

1a344f7

darkskygit force-pushed the darksky/yocto-sync branch from 6f66dc2 to 1a344f7 Compare December 28, 2023 14:32

vercel bot deployed to Preview December 28, 2023 14:33 View deployment

fix: conflict check when reference right is none (#40)

e68ef6f

vercel bot deployed to Preview January 30, 2024 08:13 View deployment

darkskygit force-pushed the darksky/yocto-sync branch from 1d355b3 to dce279c Compare January 30, 2024 08:15

vercel bot deployed to Preview January 30, 2024 08:15 View deployment

darkskygit force-pushed the darksky/yocto-sync branch from dce279c to 203e557 Compare January 30, 2024 08:20

vercel bot deployed to Preview January 30, 2024 08:21 View deployment

fix: deps

9b282ae

darkskygit force-pushed the darksky/yocto-sync branch from 203e557 to 9b282ae Compare January 30, 2024 08:35

vercel bot deployed to Preview January 30, 2024 08:35 View deployment

vercel bot deployed to Preview January 31, 2024 07:05 View deployment

darkskygit force-pushed the darksky/yocto-sync branch from 5d0e2c0 to 9fb6c2f Compare January 31, 2024 07:11

vercel bot deployed to Preview January 31, 2024 07:12 View deployment

vercel bot deployed to Preview January 31, 2024 07:20 View deployment

darkskygit force-pushed the darksky/yocto-sync branch from ca6feae to 0a11a74 Compare January 31, 2024 07:31

vercel bot deployed to Preview January 31, 2024 07:32 View deployment

fix: lint & fuzzing

dc2caeb

darkskygit force-pushed the darksky/yocto-sync branch from 0a11a74 to dc2caeb Compare January 31, 2024 07:41

vercel bot deployed to Preview January 31, 2024 07:41 View deployment

darkskygit merged commit e821e37 into master Jan 31, 2024
12 of 14 checks passed

darkskygit deleted the darksky/yocto-sync branch January 31, 2024 07:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: sync y-octo's logic #563

feat: sync y-octo's logic #563

darkskygit commented Dec 28, 2023

vercel bot commented Dec 28, 2023 •

edited

Loading

github-actions bot commented Dec 28, 2023

codecov bot commented Dec 28, 2023 •

edited

Loading

github-actions bot commented Jan 30, 2024

feat: sync y-octo's logic #563

feat: sync y-octo's logic #563

Conversation

darkskygit commented Dec 28, 2023

vercel bot commented Dec 28, 2023 • edited Loading

github-actions bot commented Dec 28, 2023

Benchmark for Linux

Benchmark for Windows

codecov bot commented Dec 28, 2023 • edited Loading

Codecov Report

github-actions bot commented Jan 30, 2024

Benchmark for Linux

Benchmark for Windows

vercel bot commented Dec 28, 2023 •

edited

Loading

codecov bot commented Dec 28, 2023 •

edited

Loading