[question] Scope of measurements?

[Malvoz]

Hello, awesome folks!

I've spent a lot of my time as a hobby developer understanding and (mainly privately) documenting best practices related to accessibility, compatibility, performance, privacy/security etc.

I'm a frequent contributor and a follower of a wide spread of organizations on github, and seeing that there are quite a few organizations doing overlapping work related to web app configuration/measurement, I sometimes file the same issue in more than 1 repository. But for web.dev I'm uncertain as to how profound audit checks you're inclined to implement.

For example, authors may get their HTTP header syntax wrong, or they'll respond with HTTP/2 incompatible headers over such a connection.

Maybe they'll set a strict security header such as CSP, but not for all applicable resources?

Are these things out of scope for measurement?

Happy weekend! :)

[robdodson]

Good questions. I'm going to transfer this over to the Lighthouse repo because under the hood we're using Lighthouse as a service, so they'd be the ones to decide on new audits.

[connorjclark]

I actually proposed audits for both headers and CSP here. Needs further discussion though.

I'll mark this as dupe, but please jump in on the conversation in the linked issue if you like :)

[brendankenny]

But for web.dev I'm uncertain as to how profound audit checks you're inclined to implement....
Are these things out of scope for measurement?

Agreed on checking out #8659 for header ideas in particular, but to address the larger topic:

Those kinds of things are absolutely in scope for Lighthouse. The main issue is balancing what's gained from a new audit with what it costs. We try to take a look at that in terms of the effect on Lighthouse performance, correctness (false positives and false negatives and how those trade on user trust in the report), and the long-term maintenance of the audit's code.

We've tried to capture all that in our "new audit checklist", which we typically ask folks to fill out in an issue to kick off discussion of a new audit idea.

We also recently introduced Lighthouse plugins, which are one way to try out ideas for audits, or to write audits that might only be useful for very specific types of users, like users of a single framework or server setup.

Let us know if you have more questions!