Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improper parsing]: Hackerone #129

Open
abhinavsecond opened this issue Oct 12, 2023 · 6 comments
Open

[Improper parsing]: Hackerone #129

abhinavsecond opened this issue Oct 12, 2023 · 6 comments
Assignees
Labels
Bug Something isn't working Effort: 1 week The implementation should require about one week Performance Things are unreasonably slow

Comments

@abhinavsecond
Copy link

InQL version

BApp Store

GraphQL API

https://hackerone.com/graphql

GraphQL specification version

No response

What isn't working?

I have waited more that 30+ min to show the results of HackerOne Graphql schemas but it is not showing any result
Note:- You need to be authenticated to query graphql on hackerone
image
It is stuck a this point

@abhinavsecond
Copy link
Author

Well after 10 more mins the result finally appeared but blank
image

@execveat
Copy link
Contributor

Thank you for the report! The Activity query alone takes 2 minutes to get parsed on my machine. I filed the issue with GQLSpection, will take a look at optimizations we could add here: doyensec/GQLSpection#32

The next InQL version contains syntax highlighting which causes some additional drop in performance, so we definitely need to address it.

@execveat execveat added Bug Something isn't working Effort: 1 week The implementation should require about one week labels Oct 12, 2023
@abhinavsecond
Copy link
Author

Thanks for looking into it🙂
I also noticed that while using this, it's using a lot of resources in Burp Suite, up to 7.2 GB of RAM.
image
I was running it on my main computer, which has plenty of RAM, so it didn't crash Burp. If it was on a virtual machine (VM), I think it might have caused the VM to crash.
I believe you should investigate this further because even though it's a great tool, if it crashes a user's Burp Suite or VM, they won't be able to use it effectively.

Thanks
Abhinav

@execveat execveat added the Performance Things are unreasonably slow label Oct 20, 2023
@execveat
Copy link
Contributor

So, this turned out to be way more complicated than I initially thought. I'm planning to address the performance issue by reducing recursion to an iterative loop and generating queries on demand (so, only when you click one, not ahead of time). Initially this was planned for a future release, but it looks like a release breaking issue now.

In case of H1 specifically, some queries right now are more than 1GB in size which explains the observed RAM usage. Obviously, this size makes them completely impractical to work with. So we could just add a sanity check that detects these huge queries and refuses to work with them, but obviously these problematic queries might very well be the most interesting ones, so in my view you should be able to work with them using Burp & InQL (on a beefy machine).

So, it's a complicated problem that will likely need to be addressed both from performance tuning perspective (everything's single core right now...) as well as from the UI/UX to intelligently disable heavyweight features such as syntax highlighting and inline comments for larger queries. Stay tuned.

@execveat
Copy link
Contributor

I replaced the recursion with a loop and the parsing time for the whole h1 schema (on my machine) now is under 2 minutes. I will try to reduce this loading time further by generating queries on demand (upon the click) instead of ahead-of-time, as well as introducing parallelization.

You can try the new version, but it's not release ready yet. Clicking larger queries (like activity) completely freezes Burp, presumably due to expensive highlighting routine. I'll try to introduce auto-disabling highlighting for larger queries as a stop-gap measure, followed by performance optimization.

@marz-hunter
Copy link

I also experienced this problem. I load from introspection (json file) takes -+ 45 minutes. but when clicking to view it can't be seen (I'm using the version from the baap store 10 oct 2023)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Effort: 1 week The implementation should require about one week Performance Things are unreasonably slow
Projects
None yet
Development

No branches or pull requests

3 participants