Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(node): Keep created node_modules directories until after resolution #9781

Merged
merged 1 commit into from
Jan 24, 2025

Conversation

sschuberth
Copy link
Member

@sschuberth sschuberth commented Jan 20, 2025

Please have a look at the individual commit messages for the details.

Copy link

codecov bot commented Jan 20, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.11%. Comparing base (e77fadf) to head (8179dfb).
Report is 6 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9781      +/-   ##
============================================
+ Coverage     68.08%   68.11%   +0.02%     
- Complexity     1286     1287       +1     
============================================
  Files           249      249              
  Lines          8829     8837       +8     
  Branches        918      918              
============================================
+ Hits           6011     6019       +8     
  Misses         2432     2432              
  Partials        386      386              
Flag Coverage Δ
funTest-docker 65.15% <100.00%> (+0.14%) ⬆️
funTest-non-docker 33.35% <ø> (ø)
test-ubuntu-24.04 35.78% <0.00%> (-0.07%) ⬇️
test-windows-2022 35.76% <0.00%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sschuberth sschuberth force-pushed the pm-before-after-semantics branch 2 times, most recently from e1a026a to dbad6a6 Compare January 20, 2025 15:26
@sschuberth sschuberth force-pushed the pm-before-after-semantics branch 3 times, most recently from d96ca20 to 5882fb9 Compare January 20, 2025 17:31
@sschuberth sschuberth marked this pull request as ready for review January 20, 2025 17:51
@sschuberth sschuberth requested a review from a team as a code owner January 20, 2025 17:51
@sschuberth sschuberth changed the title refactor(analyzer): Change when {before,after}Resolution() are called Change when {before,after}Resolution() are called to make these hooks more useful Jan 20, 2025
analyzer/src/funTest/kotlin/AnalyzerFunTest.kt Outdated Show resolved Hide resolved
analyzer/src/funTest/kotlin/AnalyzerFunTest.kt Outdated Show resolved Hide resolved
analyzer/src/funTest/kotlin/AnalyzerFunTest.kt Outdated Show resolved Hide resolved
analyzer/src/main/kotlin/Analyzer.kt Outdated Show resolved Hide resolved
analyzer/src/funTest/kotlin/AnalyzerFunTest.kt Outdated Show resolved Hide resolved
plugins/package-managers/gradle/src/main/kotlin/Gradle.kt Outdated Show resolved Hide resolved
plugins/package-managers/maven/src/main/kotlin/Maven.kt Outdated Show resolved Hide resolved
@sschuberth sschuberth force-pushed the pm-before-after-semantics branch 2 times, most recently from fba4e24 to a2afa30 Compare January 21, 2025 13:37
@sschuberth sschuberth force-pushed the pm-before-after-semantics branch from a2afa30 to 33ab11e Compare January 21, 2025 14:30
@sschuberth sschuberth requested a review from fviernau January 21, 2025 14:31
@sschuberth sschuberth force-pushed the pm-before-after-semantics branch from 33ab11e to 8c82740 Compare January 21, 2025 14:50
plugins/package-managers/maven/src/main/kotlin/Maven.kt Outdated Show resolved Hide resolved
@@ -38,10 +38,10 @@ import org.ossreviewtoolkit.plugins.packagemanagers.node.NodePackageManagerType
import org.ossreviewtoolkit.plugins.packagemanagers.node.PackageJson
import org.ossreviewtoolkit.plugins.packagemanagers.node.parsePackageJson
import org.ossreviewtoolkit.utils.common.CommandLineTool
import org.ossreviewtoolkit.utils.common.DirectoryStash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still have to think about this commit a bit, haven't finished the review.

The previous commits LGTM now. Do you want to split out this commit so you can merge the rest ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to split out this commit so you can merge the rest ?

I'm not sure... since this is (to me) the most important commit that concludes the series and allows for the React-Native NPM / Gradle use-case to be analyzed, there is not much value in merging the other commits without this.

I still have to think about this commit a bit

Maybe it helps if you look at the current implementation of stashDirectories(): It's basically just a shortcut for creating a DirectoryStash instance, that is then closed via use {}. All that the last commit intends to do is to split the creation of the DirectoryStash instance and the closing across beforeResolution() / afterResolution() by calling close() explicitly instead of leveraging use {}.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it helps if you look at the current implementation of stashDirectories()

I got that bit, but I need to first better understand some details of the analyzer, e.g. how this fits with mapDefinitionFiles, but also in general.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also address issues, if any, as a follow-up? I'd like to include this feature into tomorrow's ORT release.

Regarding the mapping of definition files: That happens in findManagedFiles() before analyze{InParallel}() is called. So {before,after}Resolution() see the mapped files, just like originally, if that was one of the questions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit short in time before the release, and I'm not 100% confident yet about the approach of the top-most commit (I'd need do research). In general, I believe it'd be nice to also have @mnonnenmacher's view.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I believe it'd be nice to also have @mnonnenmacher's view.

Or also @MarcelBochtler. Whatever it takes to move this forward.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous commits LGTM now. Do you want to split out this commit so you can merge the rest ?

Despite my original plan, I've done that now to move things forward. Please approve @fviernau: #9824

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I believe it'd be nice to also have @mnonnenmacher's view.

Looks good to me, I don't see any issues with the approach.

@sschuberth sschuberth force-pushed the pm-before-after-semantics branch from 8c82740 to fa5c65b Compare January 22, 2025 09:22
@sschuberth sschuberth requested review from fviernau and a team January 22, 2025 09:27
…ution

This fixes the case where a Gradle analysis (that is configured to run
after e.g. a Yarn analysis) needs to access Gradle projects inside the
`node_modules` directory created by a React-Native build.

Signed-off-by: Sebastian Schuberth <[email protected]>
@sschuberth sschuberth force-pushed the pm-before-after-semantics branch from fa5c65b to 8179dfb Compare January 24, 2025 10:07
@sschuberth sschuberth changed the title Change when {before,after}Resolution() are called to make these hooks more useful feat(node): Keep created node_modules directories until after resolution Jan 24, 2025
@@ -38,10 +38,10 @@ import org.ossreviewtoolkit.plugins.packagemanagers.node.NodePackageManagerType
import org.ossreviewtoolkit.plugins.packagemanagers.node.PackageJson
import org.ossreviewtoolkit.plugins.packagemanagers.node.parsePackageJson
import org.ossreviewtoolkit.utils.common.CommandLineTool
import org.ossreviewtoolkit.utils.common.DirectoryStash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I believe it'd be nice to also have @mnonnenmacher's view.

Looks good to me, I don't see any issues with the approach.

@sschuberth sschuberth dismissed fviernau’s stale review January 24, 2025 13:17

Actually, no actionable changes were requested. The review was just blocked out of a lack of time to think about it.

@sschuberth sschuberth merged commit 78ac845 into main Jan 24, 2025
26 checks passed
@sschuberth sschuberth deleted the pm-before-after-semantics branch January 24, 2025 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants