Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

low priority: comments intermittently fail to highlight syntactically #46

Open
ghost opened this issue Aug 28, 2021 · 20 comments
Open

low priority: comments intermittently fail to highlight syntactically #46

ghost opened this issue Aug 28, 2021 · 20 comments
Labels
bug Something isn't working

Comments

@ghost
Copy link

ghost commented Aug 28, 2021

Hi, I apologize for a vague issue as I still haven't figured out how to reproduce, but if you start typing a lot of comments eventually the syntax highlighting breaks without recovery. This can also be triggered by pasting code into lalrpop files with syntax errors, seemingly randomly.

It may take me weeks till I even realize how to reproduce as it's honestly quite perplexing, but write enough comments // or paste enough code into a lalrpop file and this will happen to you.

I say this is low priority though since you can fix it simply by restarting intellij. So not a blocker, just an annoyance.

The following error logs are shown, though I'm not sure if or how they are related:

Caused by: java.lang.IndexOutOfBoundsException: Wrong offset: 2209. Should be in range: [0, 767]
| at com.intellij.openapi.editor.ex.util.SegmentArray.offsetOutOfRange(SegmentArray.java:89)
| at com.intellij.openapi.editor.ex.util.SegmentArray.findSegmentIndex(SegmentArray.java:103)
| at com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.incrementalUpdate(LexerEditorHighlighter.java:178)
| at com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.documentChanged(LexerEditorHighlighter.java:334)
| ... 74 more
| 2021-08-28 19:21:29,522 [ 257640] ERROR - enapi.editor.impl.DocumentImpl - IntelliJ IDEA 2021.2.1 Build #IC-212.5080.55
| 2021-08-28 19:21:29,523 [ 257641] ERROR - enapi.editor.impl.DocumentImpl - JDK: 11.0.11; VM: OpenJDK 64-Bit Server VM; Vendor: JetBrains s.r.o.
| 2021-08-28 19:21:29,523 [ 257641] ERROR - enapi.editor.impl.DocumentImpl - OS: Linux
| 2021-08-28 19:21:29,523 [ 257641] ERROR - enapi.editor.impl.DocumentImpl - Last Action: SelectNextOccurrence
| 2021-08-28 19:21:29,523 [ 257641] ERROR - enapi.editor.impl.DocumentImpl - Current Command: Typing
| 2021-08-28 19:21:29,525 [ 257643] ERROR - enapi.editor.impl.DocumentImpl - com.intellij.openapi.editor.ex.util.LexerEditorHighlighter(com.mdrobnak.lalrpop.LpLexerAdaptor): Error updating after DocumentEventImpl[myOffset=2279, myOldLength=0, myNewLength=1].
| com.intellij.openapi.editor.ex.util.LexerEditorHighlighter$InvalidStateException: com.intellij.openapi.editor.ex.util.LexerEditorHighlighter(com.mdrobnak.lalrpop.LpLexerAdaptor): Error updating after DocumentEventImpl[myOffset=2279, myOldLength=0, myNewLength=1].

@ghost
Copy link
Author

ghost commented Aug 28, 2021

this offset error seems to also disable diff views (comparing local history). I can do unified view but not side-by-side.

Not sure if I should open a separate issue as I'm guessing its same root cause. Here's the error:

2021-08-28 22:40:07,711 [4906397] ERROR - diff.impl.DiffRequestProcessor - Unexpected termination offset for lexer FlexAdapter for com.mdrobnak.lalrpop.lexer.LalrpopLexer
| java.lang.IllegalStateException: Unexpected termination offset for lexer FlexAdapter for com.mdrobnak.lalrpop.lexer.LalrpopLexer
| at com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.doSetText(LexerEditorHighlighter.java:461)
| at com.intellij.openapi.editor.ex.util.LexerEditorHighlighter.setText(LexerEditorHighlighter.java:420)
| at com.intellij.openapi.editor.impl.EditorImpl.setHighlighter(EditorImpl.java:1346)
| at com.intellij.diff.util.DiffUtil.setEditorHighlighter(DiffUtil.java:196)
| at com.intellij.diff.util.DiffUtil.configureEditor(DiffUtil.java:263)
| at com.intellij.diff.tools.holders.TextEditorHolder.create(TextEditorHolder.java:97)
| at com.intellij.diff.tools.holders.TextEditorHolder$TextEditorHolderFactory.create(TextEditorHolder.java:107)
| at com.intellij.diff.tools.holders.TextEditorHolder$TextEditorHolderFactory.create(TextEditorHolder.java:101)
| at com.intellij.diff.tools.util.side.TwosideDiffViewer.createEditorHolders(TwosideDiffViewer.java:106)
| at com.intellij.diff.tools.util.side.TwosideTextDiffViewer.createEditorHolders(TwosideTextDiffViewer.java:115)
| at com.intellij.diff.tools.util.side.TwosideDiffViewer.(TwosideDiffViewer.java:53)
| at com.intellij.diff.tools.util.side.TwosideTextDiffViewer.(TwosideTextDiffViewer.java:67)
| at com.intellij.diff.tools.simple.SimpleDiffViewer.(SimpleDiffViewer.java:63)
| at com.intellij.diff.tools.simple.SimpleDiffTool.createComponent(SimpleDiffTool.java:31)
| at com.intellij.diff.impl.DiffRequestProcessor.createState(DiffRequestProcessor.java:269)
| at com.intellij.diff.impl.DiffRequestProcessor.lambda$doApplyRequest$4(DiffRequestProcessor.java:344)
| at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:705)
| at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:647)
| at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:63)
| at com.intellij.openapi.progress.impl.CoreProgressManager.executeNonCancelableSection(CoreProgressManager.java:223)
| at com.intellij.diff.impl.DiffRequestProcessor.lambda$doApplyRequest$5(DiffRequestProcessor.java:338)
| at com.intellij.diff.util.DiffUtil.runPreservingFocus(DiffUtil.java:757)
| at com.intellij.diff.impl.DiffRequestProcessor.doApplyRequest(DiffRequestProcessor.java:328)
| at com.intellij.diff.impl.DiffRequestProcessor.lambda$applyRequest$3(DiffRequestProcessor.java:310)
| at com.intellij.openapi.wm.impl.FocusManagerImpl.lambda$doWhenFocusSettlesDown$3(FocusManagerImpl.java:184)
| at com.intellij.util.ui.EdtInvocationManager.invokeLaterIfNeeded(EdtInvocationManager.java:101)
| at com.intellij.ide.IdeEventQueue.ifFocusEventsInTheQueue(IdeEventQueue.java:186)
| at com.intellij.ide.IdeEventQueue.executeWhenAllFocusEventsLeftTheQueue(IdeEventQueue.java:140)
| at com.intellij.openapi.wm.impl.FocusManagerImpl.doWhenFocusSettlesDown(FocusManagerImpl.java:175)
| at com.intellij.openapi.wm.impl.FocusManagerImpl.doWhenFocusSettlesDown(FocusManagerImpl.java:181)
| at com.intellij.openapi.wm.impl.IdeFocusManagerImpl.doWhenFocusSettlesDown(IdeFocusManagerImpl.java:41)
| at com.intellij.diff.impl.DiffRequestProcessor.applyRequest(DiffRequestProcessor.java:318)
| at com.intellij.diff.impl.DiffRequestProcessor.applyRequest(DiffRequestProcessor.java:297)
| at com.intellij.diff.impl.DiffRequestPanelImpl$MyDiffRequestProcessor.updateRequest(DiffRequestPanelImpl.java:107)
| at com.intellij.diff.impl.DiffRequestProcessor.updateRequest(DiffRequestProcessor.java:180)
| at com.intellij.diff.impl.DiffRequestProcessor$DiffToolToggleAction.actionPerformed(DiffRequestProcessor.java:683)
| at com.intellij.openapi.actionSystem.ex.ActionUtil.lambda$performActionDumbAwareWithCallbacks$4(ActionUtil.java:240)
| at com.intellij.openapi.actionSystem.ex.ActionUtil.performDumbAwareWithCallbacks(ActionUtil.java:261)
| at com.intellij.openapi.actionSystem.ex.ActionUtil.performActionDumbAwareWithCallbacks(ActionUtil.java:240)
| at com.intellij.ui.popup.ActionPopupStep.performAction(ActionPopupStep.java:252)
| at com.intellij.ui.popup.ActionPopupStep.performAction(ActionPopupStep.java:242)
| at com.intellij.ui.popup.ActionPopupStep.lambda$onChosen$2(ActionPopupStep.java:228)
| at com.intellij.openapi.application.TransactionGuardImpl.performUserActivity(TransactionGuardImpl.java:94)
| at com.intellij.ui.popup.AbstractPopup.lambda$dispose$18(AbstractPopup.java:1503)
| at com.intellij.util.ui.EdtInvocationManager.invokeLaterIfNeeded(EdtInvocationManager.java:101)
| at com.intellij.ide.IdeEventQueue.ifFocusEventsInTheQueue(IdeEventQueue.java:186)
| at com.intellij.ide.IdeEventQueue.executeWhenAllFocusEventsLeftTheQueue(IdeEventQueue.java:140)
| at com.intellij.openapi.wm.impl.FocusManagerImpl.doWhenFocusSettlesDown(FocusManagerImpl.java:175)
| at com.intellij.openapi.wm.impl.IdeFocusManagerImpl.doWhenFocusSettlesDown(IdeFocusManagerImpl.java:36)
| at com.intellij.ui.popup.AbstractPopup.dispose(AbstractPopup.java:1500)
| at com.intellij.ui.popup.WizardPopup.dispose(WizardPopup.java:163)
| at com.intellij.ui.popup.list.ListPopupImpl.dispose(ListPopupImpl.java:337)
| at com.intellij.ui.popup.PopupFactoryImpl$ActionGroupPopup.dispose(PopupFactoryImpl.java:287)
| at com.intellij.openapi.util.ObjectTree.runWithTrace(ObjectTree.java:136)
| at com.intellij.openapi.util.ObjectTree.executeAll(ObjectTree.java:166)
| at com.intellij.openapi.util.Disposer.dispose(Disposer.java:155)
| at com.intellij.openapi.util.Disposer.dispose(Disposer.java:143)
| at com.intellij.ui.popup.WizardPopup.disposeAllParents(WizardPopup.java:266)
| at com.intellij.ui.popup.list.ListPopupImpl.handleNextStep(ListPopupImpl.java:444)
| at com.intellij.ui.popup.list.ListPopupImpl._handleSelect(ListPopupImpl.java:416)
| at com.intellij.ui.popup.list.ListPopupImpl.handleSelect(ListPopupImpl.java:372)
| at com.intellij.ui.popup.PopupFactoryImpl$ActionGroupPopup.handleSelect(PopupFactoryImpl.java:304)
| at com.intellij.ui.popup.list.ListPopupImpl$MyMouseListener.mouseReleased(ListPopupImpl.java:629)
| at java.desktop/java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:298)
| at java.desktop/java.awt.Component.processMouseEvent(Component.java:6652)
| at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3345)
| at com.intellij.ui.popup.list.ListPopupImpl$MyList.processMouseEvent(ListPopupImpl.java:706)
| at java.desktop/java.awt.Component.processEvent(Component.java:6417)
| at java.desktop/java.awt.Container.processEvent(Container.java:2263)
| at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5027)
| at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2321)
| at java.desktop/java.awt.Component.dispatchEvent(Component.java:4859)
| at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918)
| at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547)
| at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488)
| at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2307)
| at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2784)
| at java.desktop/java.awt.Component.dispatchEvent(Component.java:4859)
| at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:778)
| at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
| at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
| at java.base/java.security.AccessController.doPrivileged(Native Method)
| at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
| at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95)
| at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:751)
| at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:749)
| at java.base/java.security.AccessController.doPrivileged(Native Method)
| at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
| at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:748)
| at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:887)
| at com.intellij.ide.IdeEventQueue.dispatchMouseEvent(IdeEventQueue.java:816)
| at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:753)
| at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$7(IdeEventQueue.java:443)
| at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:825)
| at com.intellij.ide.IdeEventQueue.lambda$dispatchEvent$8(IdeEventQueue.java:442)
| at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:794)
| at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:494)
| at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
| at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
| at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
| at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
| at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
| at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
| 2021-08-28 22:40:07,712 [4906398] ERROR - diff.impl.DiffRequestProcessor - IntelliJ IDEA 2021.2.1 Build #IC-212.5080.55
| 2021-08-28 22:40:07,712 [4906398] ERROR - diff.impl.DiffRequestProcessor - JDK: 11.0.11; VM: OpenJDK 64-Bit Server VM; Vendor: JetBrains s.r.o.
| 2021-08-28 22:40:07,712 [4906398] ERROR - diff.impl.DiffRequestProcessor - OS: Linux

@AzureMarker
Copy link
Owner

What version of the plugin are you using?

@ghost
Copy link
Author

ghost commented Aug 30, 2021

this was with 0.2.4, I see you released again I'll try with that and get back to you in the next week or so, but it's unrelated to the replace feature I think. Thanks.

@AzureMarker
Copy link
Owner

I tried adding 500 lines containing the following and didn't see any issues:

// Test 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Unfortunately the stack traces don't give much info about what might be going wrong in my code. This issue gives some possible reasons for the error:
antlr/jetbrains-plugin-sample#9

Did you have any unclosed strings or comments when the error occurred?

@ghost
Copy link
Author

ghost commented Aug 31, 2021

I just got the same issue again after upgrading lalrpop plugin. No unclosed strings or comments and I still don't know what triggers it. Play around with the code for a bit though and eventually the comments break. Nothing is a long line. I'll just be typing, eventually cause a syntax error and they break. Other times I cause syntax errors and it won't happen.

Sorry, this one is very hard to catch. It may take weeks before I notice a pattern. It is seemingly random! The wrong offset error still prints though - that is the only consistency.

@AzureMarker
Copy link
Owner

AzureMarker commented Sep 1, 2021

Do you see the problem when you're typing embedded Rust code or LALRPOP code? Or maybe when you go between them?

@ghost
Copy link
Author

ghost commented Sep 1, 2021

"Going between them" is probably a more accurate statement. Certainly just typing comments endlessly doesn't seem to trigger it. It's when I'm coding intensely - a lot of changes occurring rapidly, syntax errors, copying and pasting, etc.

That's why it's so difficult to catch. What I may need to do is record my desktop and convert it to a gif while I trigger the behavior, if you're not able to reproduce. Would that be helpful?

@AzureMarker
Copy link
Owner

AzureMarker commented Sep 2, 2021

That would be helpful, and at very least help document the behavior.

@ghost
Copy link
Author

ghost commented Sep 8, 2021

@AzureMarker it may be a while till I screen record, but I did just activate a .lalrpop file that was previously open and all comments highlighting was lost as soon as I typed:

// TODO: use &str instead of String

This time I got logged:

2021-09-08 17:57:50,257 [14208845] ERROR - enapi.editor.impl.DocumentImpl - com.intellij.openapi.editor.ex.util.LexerEditorHighlighter(com.mdrobnak.lalrpop.LpLexerAdaptor): Error updating after DocumentEventImpl[myOffset=279, myOldLength=0, myNewLength=59].

and then:

2021-09-08 17:57:55,051 [14213639] ERROR - chcopy.TextWithMarkupProcessor - IdeaLoggingEvent[message=Error generating text with markup, throwable=java.lang.AssertionError: Unexpected range returned by highlighter: 43:44, prevEnd: 0, scanned range: 242:338, resulting range: 242:44, highlighter: com.intellij.openapi.editor.ex.util.LexerEditorHighlighter(com.mdrobnak.lalrpop.LpLexerAdaptor): 'use std::str::FromStr;

I'll try to do a recording, but that was the simplest scenario I've witnessed so thought to share. The file had been opened for longer and I can't remember all changes I made prior to typing that TODO. When I do record I'll try to start the recording showing freshly loading intellij.

@AzureMarker
Copy link
Owner

Thanks, looks like there is an issue in the highlighter. I'll take a closer look soon.

@AzureMarker
Copy link
Owner

Unexpected range returned by highlighter: 43:44, prevEnd: 0

This makes me think that the Rust code sections might be causing the issue. They are handled specially by the lexer (basically as one big token). I can't find any issue just looking at the code though. Additionally, the highlighting lexer is implemented by JetBrains, so there's not much code on my side to mess up. I'll continue looking into this though, thanks for continuing to share details.

Did this issue start happening recently, or has it always been there?

@ghost
Copy link
Author

ghost commented Oct 10, 2021

I am now noticing on 0.2.6 IntelliJ shows a baloon message for "IDE FATAL ERRORS" and an UI appears that would normally allow me to send the error report to the author of the plugin, but the "Report and Clear All" button is grayed out (disabled). Anyway you can enable that feature and then I can send you the error report next time?

@AzureMarker
Copy link
Owner

That's a good idea. I should be able to look into that this week.

@AzureMarker
Copy link
Owner

AzureMarker commented Oct 16, 2021

I've started looking at this (error reporting). Actually sending the error report is up to the plugin developer, so I'll need to set up something like Sentry or redirect to GitHub issues. Working on that now.

@AzureMarker
Copy link
Owner

Added the error reporter, will do a release soon: 7e6c6d1

@AzureMarker
Copy link
Owner

Released 0.2.7 with the error reporter.

@AzureMarker
Copy link
Owner

FYI I think at least one of these exceptions got submitted to Sentry:
image

I fixed a few other errors I saw in Sentry, and will get to this one hopefully soon.

@AzureMarker
Copy link
Owner

I think I found a way to reproduce this error. Start with this:

grammar;

Test: () = "test";

Then quickly add an empty code block like so:

grammar;

Test: () = "test" => {};

The error details will show that it got the exception when analyzing the file right before the second curly brace.

@AzureMarker AzureMarker added the bug Something isn't working label Feb 3, 2022
@evbo
Copy link

evbo commented Feb 26, 2022

I noticed this behavior a while ago, but seems more stable now in the latest release. Maybe okay to close?

@AzureMarker
Copy link
Owner

AzureMarker commented Mar 6, 2022

I just tried the reproduction steps I posted, and I still see an error. I investigated it a bunch a while ago, but it's really hard to debug since it is almost all in the internals of IntelliJ's lexer code.

I haven't seen the error pop up on Sentry recently, so maybe it's not seen as much in practice anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants