Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add static analysis to build job #1324

Merged
merged 3 commits into from
Sep 11, 2023
Merged

Add static analysis to build job #1324

merged 3 commits into from
Sep 11, 2023

Conversation

iBicha
Copy link
Contributor

@iBicha iBicha commented Jul 3, 2023

Changes

Add static analysis to build job. This downloads https://devtools.web.roku.com/#static-channel-analysis-tool and analyze the build.
I just added the tool to Playlet, I thought it might save me from unpleasant surprises when I try to publish.
Things to consider:

  • I only added this to the build-prod job, you might be interested in other jobs (dev, at PR level, etc)
  • I'm analyzing the staging folder, not the zipped artifact
  • We're downloading from https://devtools.web.roku.com/static-channel-analysis/sca-cmd.zip without a pinned version, let's hope that Roku folks are good at not breaking things

That being said, feel free to modify, merge, repurpose, or close this PR

Issues

N/A

@iBicha
Copy link
Contributor Author

iBicha commented Jul 3, 2023

Output should look like this on the Github action

Run ./sca-cmd/bin/sca-cmd ./release/playlet.zip --exit error
Channel title: Playlet
Channel version: 0.10.00001
Channel locales: 

************************************************************************************************************************
*                                               Channel analysis results                                               *
************************************************************************************************************************
------------------------------------------------------------------------------------------------------------------------
-                                                    Uncategorized                                                     -
------------------------------------------------------------------------------------------------------------------------
Warning:  If your channel has streamed more than an average of 5 million hours per month over the last three months or you have a new channel projected to reach the specified streaming hour threshold shortly after launch, you must participate in Roku’s Continue Watching program. See Continue Watching for more detail. TVOD, live linear, and made-for-kids channels are excluded from this requirement. [Cert Requirements: 4.[13](https://github.com/iBicha/playlet/actions/runs/5448944849/jobs/9912702764#step:9:14)]. Documentation: https://developer.roku.com/docs/developer-program/certification/certification.md#4-channel-operation.
------------------------------------------------------------------------------------------------------------------------
-                                                     Performance                                                      -
------------------------------------------------------------------------------------------------------------------------
Warning:  For your channel to pass certification, your application must fire the "AppDialogInitiate" and "AppDialogComplete" beacons if the channel UI displays a login, user selection, EULA, or any other dialog before the home page. [Cert Requirements: 3.2]. Documentation: https://developer.roku.com/docs/developer-program/certification/certification.md#3-performance.
************************************************************************************************************************

@cewert
Copy link
Member

cewert commented Jul 4, 2023

Thanks for the PR!

I made a PR on my fork so I could see the output and it seems to work well. One thing I noticed is for some reason it's not picking up all of the locales. Roku supports 9, we currently support 7, but only 5 are showing up in the static analysis output:

static-anal

As far as this PR, I think we should add the static analysis to both the production build and the dev build workflows so that we know if a PR breaks something. Testing the staging folder makes sense and should match the artifact zip but we may want to test the zip file instead just to verify the zip is what it should be. @neilsb @jimdogx @1hitsong what do you guys think?

@iBicha
Copy link
Contributor Author

iBicha commented Jul 4, 2023

For the locales not detected I think I have the answer.

I was digging into this tool and got curious on how it worked. It's an obfuscated java library, that downloads config from http://devtools.web.roku.com/static-channel-analysis/config.json.

What's even weirder is that it's an encrypted json file that gets decrypted client side (weird, as that offers no security whatsoever, so I don't know why they did that)

Which got me even more curious, so I decrypted the config file, you can find it at https://gist.github.com/iBicha/8b9d542138c0319e1b21038d91cadd13

It's a big json payload with lots of fields related to the checks ran during the analysis. But if you scroll all the way down, you can see there is a channelLocales field with 5 items in it. My guess is that Roku added support for additional languages, but forgot to update this tool. That should explain why you're only seeing 5.

@jellyfin-bot
Copy link
Contributor

This pull request has been inactive for 21 days and will be automatically closed in 7 days if there is no further activity.

@jellyfin-bot jellyfin-bot added the stale This issue/PR has gone stale. label Jul 26, 2023
@jellyfin-bot
Copy link
Contributor

This pull request has been closed because it has been inactive for 28 days. You may submit a new pull request if desired.

@cewert cewert reopened this Aug 17, 2023
@cewert cewert removed the stale This issue/PR has gone stale. label Aug 17, 2023
@cewert
Copy link
Member

cewert commented Aug 17, 2023

Definitely want to add this just haven't had time. The only thing to figure out is whether this should be added to the main build jobs or make a new job for it.

@neilsb
Copy link
Member

neilsb commented Aug 20, 2023

The only thing to figure out is whether this should be added to the main build jobs or make a new job for it.

My first thought is probably a separate job, since there is still some benefit in knowing that the app builds but fails static analysis. But that said, we can't (shouldn't) submit it to the store until the static analysis passes so that may not be a reason for a separate job.

@cewert
Copy link
Member

cewert commented Aug 20, 2023

Yea I think I could argue it either way but I'm leaning on a separate job. For the reason you mentioned and also the added flexibility because we don't have to make the new job a required check on github. That way if roku changes the URL or breaks the tool somehow, our main build jobs won't fail and we'll still be able to merge PRs.

@cewert
Copy link
Member

cewert commented Aug 23, 2023

Changes

  • Create new workflow called roku-analysis.yml
    • Workflow is triggered by push and pull_request
    • Create new job called static
    • Move new code from this PR to the static job
    • Add logic to run the appropriate build command based on which branch triggered the workflow (still need to test this works right on master)

@neilsb how's this look?

@cewert cewert requested a review from neilsb August 23, 2023 02:51
Copy link
Member

@neilsb neilsb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All makes sense, and looks good.

@cewert cewert merged commit 0bf7057 into jellyfin:unstable Sep 11, 2023
9 checks passed
@cewert cewert added the dev-improvement This improves the dev experience in some way. label Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev-improvement This improves the dev experience in some way.
Projects
Development

Successfully merging this pull request may close these issues.

4 participants