Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLDR-17014 Change HashMap to ConcurrentHashMap in ExtraPaths #4321

Closed
wants to merge 1 commit into from

Conversation

btangmu
Copy link
Member

@btangmu btangmu commented Jan 29, 2025

-ConcurrentModificationException was thrown when ExtraPaths.getInstance called computeIfAbsent during cldr-generate-json.sh

-Unable to repro the exception before making this change

CLDR-17014

  • This PR completes the ticket.

ALLOW_MANY_COMMITS=true

-ConcurrentModificationException was thrown when ExtraPaths.getInstance called computeIfAbsent during cldr-generate-json.sh

-Unable to repro the exception before making this change; anyway, this should prevent it
@btangmu btangmu self-assigned this Jan 29, 2025
@btangmu
Copy link
Member Author

btangmu commented Jan 29, 2025

Compare older alternative #4317 which adds synchronized

@btangmu
Copy link
Member Author

btangmu commented Jan 29, 2025

This fails ConsoleCheckCLDR:

 Caused by: java.util.ConcurrentModificationException
	at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1511)
	at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
	at java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:351)
	at org.unicode.cldr.util.ExtraPaths.append(ExtraPaths.java:31)
	at org.unicode.cldr.util.CLDRFile.getRawExtraPathsPrivate(CLDRFile.java:3156)
	at org.unicode.cldr.util.CLDRFile.getRawExtraPaths(CLDRFile.java:3122)
	at org.unicode.cldr.util.CLDRFile.getExtraPaths(CLDRFile.java:3087)
	at org.unicode.cldr.util.CLDRFile$FullIterable.next(CLDRFile.java:1518)
	at org.unicode.cldr.util.CLDRFile$FullIterable.next(CLDRFile.java:1497)
	at org.unicode.cldr.util.With$ToIterator.next(With.java:383)
	at org.unicode.cldr.test.ConsoleCheckCLDR.lambda$main$0(ConsoleCheckCLDR.java:659)

@btangmu btangmu requested review from srl295 and macchiati January 29, 2025 17:59
@btangmu
Copy link
Member Author

btangmu commented Jan 29, 2025

@macchiati @srl295 tests passed locally but ConsoleCheckCLDR fails here, so maybe we need synchronized (#4317) after all or some other solution

@btangmu
Copy link
Member Author

btangmu commented Jan 29, 2025

I don't often run ConsoleCheckCLDR locally, and didn't run it before making this PR. Running it locally now, I do get ConcurrentModificationException:

$ java -DCLDR_DIR=$(pwd) -jar tools/cldr-code/target/cldr-code.jar check -S common,seed -e -z FINAL_TESTING
>> org.unicode.cldr.test.ConsoleCheckCLDR
[-S, common,seed, -e, -z, FINAL_TESTING]
#-f	file_filter	≝	.*
#-t	test_filter	≝	.*
#-o	organization	≝	.*
...
# fr_MU [French (Mauritius)]	Summary	modern	Items:	14586	Raw Missing:	14585	Raw Provisional:	0
# fr_MU Elapsed time: : 655.2 ms
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.unicode.cldr.tool.Main.invoke(Main.java:60)
	at org.unicode.cldr.tool.Main.main(Main.java:47)
Caused by: java.util.ConcurrentModificationException: java.util.ConcurrentModificationException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:600)
	at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661)
	at org.unicode.cldr.test.ConsoleCheckCLDR.main(ConsoleCheckCLDR.java:543)
	... 6 more
Caused by: java.util.ConcurrentModificationException
	at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1511)
	at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
	at java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:351)
	at org.unicode.cldr.util.ExtraPaths.append(ExtraPaths.java:31)
	at org.unicode.cldr.util.CLDRFile.getRawExtraPathsPrivate(CLDRFile.java:3157)
	at org.unicode.cldr.util.CLDRFile.getRawExtraPaths(CLDRFile.java:3122)
	at org.unicode.cldr.util.CLDRFile.getExtraPaths(CLDRFile.java:3087)
	at org.unicode.cldr.util.CLDRFile$FullIterable.next(CLDRFile.java:1518)
	at org.unicode.cldr.util.CLDRFile$FullIterable.next(CLDRFile.java:1497)
	at org.unicode.cldr.util.With$ToIterator.next(With.java:383)
	at org.unicode.cldr.test.ConsoleCheckCLDR.lambda$main$0(ConsoleCheckCLDR.java:659)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

Copy link
Member

@macchiati macchiati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see ticket

@btangmu btangmu closed this Jan 31, 2025
@btangmu btangmu deleted the t17014_g branch January 31, 2025 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants