diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..725c776
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1 @@
+*.bs linguist-language=HTML
diff --git a/.github/workflows/auto-publish.yml b/.github/workflows/auto-publish.yml
new file mode 100644
index 0000000..4ca6ca9
--- /dev/null
+++ b/.github/workflows/auto-publish.yml
@@ -0,0 +1,36 @@
+# Workflow based on the main w3c/spec-prod action example:
+# https://github.com/w3c/spec-prod/#basic-usage
+
+name: Build, Validate, Deploy and Publish
+
+on:
+ # Worflow runs on pull requests where it makes sure that the spec can still be
+ # generated, that markup is valid and that there are no broken links, as
+ # well as on pushes to the default branch where it also deploys the generated
+ # spec to the gh-pages branch and publishes the result to /TR.
+ # The "workflow_dispatch" hook allows admins to also trigger the workflow
+ # manually from GitHub's UI.
+ pull_request: {}
+ push:
+ branches: [main]
+ workflow_dispatch:
+
+jobs:
+ main:
+ runs-on: ubuntu-20.04
+ steps:
+ # See doc at https://github.com/actions/checkout#checkout-v2
+ - name: Checkout repository
+ uses: actions/checkout@v2
+
+ # See doc at https://github.com/w3c/spec-prod/#spec-prod
+ # The action only deploys the generated spec to the gh-pages branch when
+ # the workflow was triggered by a push to the default branch.
+ - name: Build and validate index.html, push to gh-pages branch if needed
+ uses: w3c/spec-prod@v2
+ with:
+ GH_PAGES_BRANCH: gh-pages
+ W3C_ECHIDNA_TOKEN: ${{ secrets.ECHIDNA_TOKEN }}
+ W3C_WG_DECISION_URL: https://github.com/w3c/media-wg/issues/27
+ W3C_BUILD_OVERRIDE: |
+ status: WD
diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index 81f8185..0000000
--- a/.travis.yml
+++ /dev/null
@@ -1,12 +0,0 @@
-language: generic
-
-script: bash ./deploy.sh
-
-branches:
- only:
- - master
-
-env:
- global:
- - ENCRYPTION_LABEL: 4064b46999b2
- - COMMIT_AUTHOR_EMAIL: travis-ci@w3.org
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index a9818c2..b09737a 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -2,7 +2,7 @@
Contributions to this repository are intended to become part of Recommendation-track documents
governed by the [W3C Patent Policy](http://www.w3.org/Consortium/Patent-Policy-20040205/) and
-[Document License](http://www.w3.org/Consortium/Legal/copyright-documents). To contribute, you must
+[Software and Document License](http://www.w3.org/Consortium/Legal/copyright-software). To contribute, you must
either participate in the relevant W3C Working Group or make a non-member patent licensing
commitment.
diff --git a/LICENSE.md b/LICENSE.md
index 6b9ee26..e432d6f 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -1,2 +1,2 @@
-All documents in this Repository are licensed by contributors under the [W3C Document
-License](http://www.w3.org/Consortium/Legal/copyright-documents).
+All documents in this Repository are licensed by contributors under the [W3C
+Software and Document License](http://www.w3.org/Consortium/Legal/copyright-software).
diff --git a/deploy.sh b/deploy.sh
deleted file mode 100644
index 938f993..0000000
--- a/deploy.sh
+++ /dev/null
@@ -1,59 +0,0 @@
-#!/bin/bash
-set -e # Exit with nonzero exit code if anything fails
-
-SOURCE_BRANCH="master"
-TARGET_BRANCH="gh-pages"
-
-# Pull requests and commits to other branches shouldn't try to deploy, just build to verify
-if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_BRANCH" != "$SOURCE_BRANCH" ]; then
- echo "Skipping deploy; just doing a build."
- make
- exit 0
-fi
-
-# Save some useful information
-REPO=`git config remote.origin.url`
-SSH_REPO=${REPO/https:\/\/github.com\//git@github.com:}
-SHA=`git rev-parse --verify HEAD`
-
-# Clone the existing gh-pages for this repo into out/
-# Create a new empty branch if gh-pages doesn't exist yet (should only happen on first deply)
-git clone $REPO out
-cd out
-git checkout $TARGET_BRANCH || git checkout --orphan $TARGET_BRANCH
-cd ..
-
-# Clean out existing contents
-rm -rf out/* || exit 0
-
-# Build the html output
-make && mv index.html out/index.html
-
-# Now let's go have some fun with the cloned repo
-cd out
-git config user.name "Travis CI"
-git config user.email "$COMMIT_AUTHOR_EMAIL"
-
-# If there are no changes to the compiled out (e.g. this is a README update) then just bail.
-if [[ -z $(git status -s) ]]; then
- echo "No changes to the output on this push; exiting."
- exit 0
-fi
-
-# Commit the "changes", i.e. the new version.
-# The delta will show diffs between new and old versions.
-git add .
-git commit -m "Deploy to GitHub Pages: ${SHA}"
-
-# Get the deploy key by using Travis's stored variables to decrypt deploy_key.enc
-ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key"
-ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv"
-ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR}
-ENCRYPTED_IV=${!ENCRYPTED_IV_VAR}
-openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in ../deploy_key.enc -out deploy_key -d
-chmod 600 deploy_key
-eval `ssh-agent -s`
-ssh-add deploy_key
-
-# Now that we're all set up, we can push.
-git push $SSH_REPO $TARGET_BRANCH
diff --git a/deploy_key.enc b/deploy_key.enc
deleted file mode 100644
index e1e1a03..0000000
Binary files a/deploy_key.enc and /dev/null differ
diff --git a/explainer.md b/explainer.md
index 610d817..01fee05 100644
--- a/explainer.md
+++ b/explainer.md
@@ -2,7 +2,7 @@
This is the explainer for the Media Capabilities API. The document explains the goals and non-goals of the API and in general helps understand the thought process behind the API. The API shape in the document is mostly for information and might not be final.
-This document is a bit more dense that some readers might want. A quick one-pager can be found in the [README.md](https://github.com/WICG/media-capabilities/blob/master/README.md) file.
+This document is a bit more dense that some readers might want. A quick one-pager can be found in the [README.md](https://github.com/w3c/media-capabilities/blob/main/README.md) file.
# Objective
@@ -99,12 +99,11 @@ This aim of this API is to help websites provide an optimal initial experience.
## Encryption
-Playbacks using [Encrypted Media Extensions](
-s://w3c.github.io/encrypted-media/) (aka EME) employ specialized decoding and rendering code paths. This means different codec support and performance compared to clear playbacks. Hence, callers should describe a key system configuration as part of the `MediaDecodingConfiguration` dictionary.
+Playbacks using [Encrypted Media Extensions](https://w3c.github.io/encrypted-media/) (aka EME) employ specialized decoding and rendering code paths. This means different codec support and performance compared to clear playbacks. Hence, callers should describe a key system configuration as part of the `MediaDecodingConfiguration` dictionary.
```Javascript
partial dictionary MediaDecodingConfiguration {
- MediaCapabilitiesKeySystemConfiguration keySystemConfig;
+ MediaCapabilitiesKeySystemConfiguration keySystemConfiguration;
};
```
@@ -126,7 +125,7 @@ This replicates the inputs provided to EME's [requestMediaKeySystemAccess](https
Specifically, `rMKSA` takes a sequence of `MediaKeySystemConfigurations`, ordered by preference. Each entry may contain a sequence of initDataTypes and sequences of audio and video contentTypes with robustness. In the dictionary above, all of these sequences are reduced to single values.
-This is a fundamental difference between the APIs. MediaCapabilities aims to describe the quality (smoothness and power efficiency) of support for a single pair of audio and video streams without making a decision for the caller. Callers should still order media configurations as they do with `rMKSA`, only now they walk the list themselves, calling MediaCapabiliites once for each option. These calls will return immediately with the promises resolving asynchronously.
+This is a fundamental difference between the APIs. MediaCapabilities aims to describe the quality (smoothness and power efficiency) of support for a single pair of audio and video streams without making a decision for the caller. Callers should still order media configurations as they do with `rMKSA`, only now they walk the list themselves, calling MediaCapabilities once for each option. These calls will return immediately with the promises resolving asynchronously.
When a key system configuration is included in the `MediaDecodingConfiguration`, `mediaCapabilities.decodingInfo()` will return a promise containing the usual three booleans (`supported`, `smooth`, and `powerEfficient`) plus a `MediaKeySystemAccess` object whenever `supported = true`. The caller may use the `MediaKeySystemAccess` as they would in traditional EME to request media keys and setup encrypted media playback. This removes the need to call `rMKSA`.
@@ -167,7 +166,7 @@ const capabilitiesPromises = orderedMediaConfigs
### Permission prompts
-EME specifies that a handful of steps in `rMKSA` may request consent from the user. This consent is critical to knowing what encrypted media capabilities are available. Hence, MediaCapabilies will prompt in the same way as `rMKSA`.
+EME specifies that a handful of steps in `rMKSA` may request consent from the user. This consent is critical to knowing what encrypted media capabilities are available. Hence, MediaCapabilities will prompt in the same way as `rMKSA`.
The spec will make clear that calling the API with a key system configuration may result in permission prompts. In practice, such prompts are rare. Currently only Chrome and Mozilla show EME prompts, and Mozilla limits theirs to once per browser profile.
@@ -181,70 +180,71 @@ Media Capabilities should offer a means of surfacing when different MediaDecodin
## HDR
-HDR support in browsers is nonexistent. The API is intended to enable high end media playback on the Web as soon as it becomes more mainstream so the platform does not lag behind the curve. This is also a great example of including more formats into the web and keeping the API extensible.
+The API is intended to enable high end media playback on the Web as soon as it becomes more mainstream so the platform does not lag behind the curve. This is also a great example of including more formats into the web and keeping the API extensible.
-### Screen capabilities
-
-Even if a device is able to decode HDR content, if the screen isn’t able to show this content appropriately, it might not be worth using HDR content for the website because of the higher bandwidth consumptions but also the rendering might be worse than a SDR optimised video.
-
-The following data can be used to define whether a screen is HDR-worthy:
-
-* Colour gamut: HDR content requires a wider colour gamut than SDR content. Most screens use the sRGB colour gamut but p3 or BT.2020 are colour gamut that would usually be expected for HDR content.
-* Colour/pixel depth: even if the screen has a wide colour gamut, the pixels need to be encoded in 30 bits (10 bits per colour component) instead of the usual 24 bits (8 bits per colour component). Otherwise, even if wider colour can be represented, precision would be lost.
-* Brightness: because of darker blacks and brighter whites, a screen needs to have a large contrast ratio in order to be used for HDR content.
-
-#### Colour gamut
+For HDR support detection, there are three main components whose capabilities need to be surfaced -- the decoder, renderer, and screen. The decoder takes in an encoded stream and produces a decoded stream understood by the renderer, which in turn maps the stream's signals to those the screen can properly output. Most of the time, the decoder and renderer are part of the UA while the screen represents the physical output monitor, whether this be a computer monitor or TV. To match this natural modularity between the UA and the screen, this API is compartmentalized into two parts:
-The colour gamut of the screen could be exposed on the *Screen* interface. It is already part of the work in progress [CSS Media Queries 4](https://drafts.csswg.org/mediaqueries-4/#color-gamut) but because various information will have to be read from the *Screen* object for HDR content, it would make sense to have all of them grouped together.
+* *MediaCapabilities.decodingInfo()*: handles the UA pieces, namely decoding and rendering.
+* TODO: various aspects of the screen are being discussed.
-#### Colour/Pixel Depth
-
-It is already exposed on the *Screen* object but only for compatibility reasons. The [CSSOM View Module](https://www.w3.org/TR/cssom-view-1/#dom-screen-pixeldepth) should be updated or amended to make this information available.
-
-#### Brightness
-
-The minimum and maximum brightness should be exposed on the *Screen* object. In order to know the effective brightness, a website would need to know the brightness of the room which can be achieved with the [Ambient Light Sensor](https://w3c.github.io/ambient-light/).
-
-#### Example
+### Screen capabilities
-```JavaScript
-function canDisplayMyHDRStreams() {
- // The conditions below are entirely made up :)
- return window.screen.colorGamut == "rec2020" &&
- window.screen.pixelDepth == "30" &&
- window.screen.brightness.max > 500 &&
- window.screen.brightness.min < 0.1;
-}
-```
+When the UA reports ability to decode HDR content, but the screen does not report ability to render HDR, it is not recommended to transmit HDR content because it wastes bandwidth and may result in actual rendering that is worse than SDR optimized video on some devices.
-### Screen change
+The shape of this API is actively being discussed, specifically how the two-plane problem in TVs should be handled. Please refer to issue [#135](https://github.com/w3c/media-capabilities/issues/135).
-The Web Platform only exposes the current screen associated to the website window. That means that a window changing screen will get its `window.screen` updated. A page can poll to find out about this but adding a *change* event to the *Screen* interface might be a more efficient way to expose this information.
+***Work in progress***
### Decode capabilities
-HDR videos have some information that need to be understood by the user agent in order to be rendered correctly. A website might want to check that the user agent will be able to interpret its HDR content before providing it.
+HDR content has 3 properties that need to be understood by the decoder and renderer: color gamut, transfer function, and frame metadata if applicable. They can be used to determine whether a UA supports a particular HDR format.
-HDR content has 4 properties that need to be understood by the decoder: primaries, yuv-to-rgb conversion matrix, transfer function and range. In addition, certain HDR content might also contain frame metadata. Metadata informs user agents of the required brightness for a given content or the transformation to apply for different values. Sometimes, all of these are combined in buckets like [HDR10](https://en.wikipedia.org/wiki/HDR10), [Dolby Vision](https://en.wikipedia.org/wiki/Dolby_Vision) and [HLG](https://en.wikipedia.org/wiki/Hybrid_Log-Gamma).
+* Color gamut: HDR content requires a wider color gamut than SDR content. Most UAs support the sRGB color gamut but p3 or Rec. 2020 are color gamut that would usually be expected for HDR content.
+* Transfer function: To map the wider color gamut of HDR content to the screen's signals, the UA needs to understand transfer functions like PQ and HLG.
+* Frame metadata: Certain HDR content might also contain frame metadata. Metadata informs user agents of the required brightness for a given content or the transformation to apply for different values.
-**Work in progress**
+Sometimes, all of these are combined in buckets like [HDR10](https://en.wikipedia.org/wiki/HDR10), [HDR10+](https://en.wikipedia.org/wiki/High-dynamic-range_video#HDR10+) [Dolby Vision](https://en.wikipedia.org/wiki/Dolby_Vision) and [HLG](https://en.wikipedia.org/wiki/Hybrid_Log-Gamma). Below are the minimum requirements for frame metadata, color gamut, and transfer respectively for each of the buckets:
-At this point, it is unclear what should be exposed in this API: HDR frame metadata formats are not yet standardised, and it remains unclear if other properties should be exposed granularly or in buckets. The first iteration of this specification will not include HDR decoding capabilities until it receives implementer feedback. This is currently slated for $todo.
+* HDR10: SMPTE-ST-2086 static metadata, Rec. 2020 color space, and PQ transfer function.
+* HDR10+: SMPTE-ST-2094-40 dynamic metadata, Rec. 2020 color space, and PQ transfer function.
+* Dolby Vision: SMPTE-ST-2094-10 dynamic metadata, Rec. 2020 color space, and PQ transfer function.
+* HLG: No metadata, Rec. 2020 color space, and HLG transfer function.
-At the moment, no operating system besides Android exposes HDR capabilities. Android exposes HDR capabilities using the buckets mentioned above. See [HdrCapabilities](https://developer.android.com/reference/android/view/Display.HdrCapabilities.html) interface and the [HDR Types](https://developer.android.com/reference/android/view/Display.HdrCapabilities.html#getSupportedHdrTypes()).
+Color gamut, transfer function, and frame metadata -- as they they have to do with decoding and rendering -- are exposed individually on the *MediaCapabilities* interface as part of *VideoConfiguration*, which is queried with *MediaCapabilities.decodingInfo()*.
-Regardless of what is exposed, the HDR information will be part of an *hdr* sub-dictionary as part of the *video* information.
+#### Example
```JavaScript
navigator.mediaCapabilities.decodingInfo({
- type: 'file',
- video: { contentType: "video/webm; codecs=vp09.00.10.08", width: 1280, height: 720,
- framerate: 24, bitrate: 123456,
- hdr: { ... } },
- audio: { contentType: "audio/webm; codecs=opus" },
+ video: {
+ // Determine UA support for decoding and rendering HDR10.
+ hdrMetadataType: "smpteSt2086",
+ colorGamut: "rec2020",
+ transferFunction: "pq",
+ ...
+ }
+}).then(result => {
+ // Do things based on results.
+ // Note: While some clients are able to map HDR content to SDR screens, check
+ // Screen capabilities to ensure high-fidelity playback.
+ console.log(result.supported);
+ console.log(result.smooth);
+ console.log(result.powerEfficient);
+ ...
});
```
+### Fingerprinting
+
+While exposing HDR capabilities could add many bits of entropy for certain platforms, this API was designed with fingerprinting in mind and does its best to adhere to the Privacy Interest Group's suggested best practices:
+
+1. Avoid unnecessary or severe increases to fingerprinting surface, especially for passive fingerprinting.
+* This API returns only a single boolean per set of input.
+2. Narrow the scope and availability of a feature with fingerprinting surface to what is functionally necessary.
+* Various mitigations are suggested in the normative specification.
+3. Mark features that contribute to fingerprintability.
+* The normative specification highlights fingerprinting concerns.
+
## Transitioning between stream configurations
The MediaCapabilities `transition()` API will surface when a media element is capable of transitioning between different stream configurations. The primary motivations are:
@@ -357,12 +357,101 @@ Unlike encrypted content, clear playbacks do not have an analog to the MediaKeyS
## HDCP support
-Now covered in a [separate repository](https://github.com/WICG/hdcp-detection/blob/master/explainer.md).
+Now covered in a [separate repository](https://github.com/WICG/hdcp-detection/blob/main/explainer.md).
## Audio channels/speakers configuration
This is already exposed by the Web Audio API [somehow](https://webaudio.github.io/web-audio-api/#ChannelLayouts). If the Web Audio API is not sufficient, the Media Capabilities API might expose this information too. However, the Web Audio API exposes this information on the destination node which is better than what the Media Capabilities API would be able to do.
+## Spatial audio
+
+This API aims to enable spatial audio on the Web as increasingly more online content providers serve high-end media playback experiences; examples include [Dolby Atmos](https://en.wikipedia.org/wiki/Dolby_Atmos) and [DST:X](https://en.wikipedia.org/wiki/DTS_(sound_system)#DTS:X). Like [HDR](https://github.com/w3c/media-capabilities/blob/main/explainer.md#hdr), this is an example of web's growth and this API's extensibility.
+
+### Spatial rendering
+
+Spatial rendering describes the UA's ability to to render spatial audio to a given output device; it can be used in conjunction with the stream's mime type to determine support for a specific spatial audio format.
+
+A Web API exposing spatial rendering is necessary for the following reasons:
+
+* Because spatial audio is not a codec per se, a client's ability to decode a statial-compatible mime type does not necessitate support for rendering spatial audio.
+* WebAudio's maxChannelCount API cannot be used to discern support for spatial audio, because formats like Dolby Atmos supports two-channel headphones in addition to N-channel speaker systems.
+* Serving content with spatial audio to clients that can decode but not render it results in wasted bandwidth and potentially lower quality user experience.
+
+Spatial rendering is exposed as a boolean included in AudioConfiugration, which can be used to query *MediaCapabilities.decodingInfo()*.
+
+### Example
+
+```JavaScript
+navigator.mediaCapabilities.decodingInfo({
+ audio: {
+ // Determine support for Dolby Atmos by checking Dolby Digital Plus and spatial rendering.
+ contentType: "audio/mp4; codecs=ec-3",
+ spatialRendering: true,
+ ...
+ }
+}).then(result => {
+ // Do things based on results.
+ console.log(result.supported);
+ console.log(result.smooth);
+ console.log(result.powerEfficient);
+ ...
+});
+```
+
+## WebRTC
+
+The API also supports the WebRTC usec case and makes it possible to determine both send and receive capabilities by calling the methods `encodingInfo` and `decodingInfo`. This gives complementary information to what is otherwise received from the methods `RTCRtpSender.getCapabilities` and `RTCRtpReceiver.getCapabilities`. There are a couple of differences to the input to the API when the type `webrtc` is used:
+
+* The contentType should now be a valid media type according to what's defined for RTP. See the examples below and the specification for more details on this.
+* An optional field `scalabilityMode` can be used in the video configuration when calling `encodingInfo` to query if a specific scalability mode is supported. See [Scalable Video Coding (SVC) Extension for WebRTC](https://www.w3.org/TR/webrtc-svc/).
+* An optional field `spatialScalability` can be used in the video configuration when calling `decodingInfo` to query if the decoder can handle spatial scalability. A bit simplified this can be interpreted as any stream that is encoded with dependent spatial layers according to [Scalable Video Coding (SVC) Extension for WebRTC](https://www.w3.org/TR/webrtc-svc/).
+
+### Examples
+
+#### Decoding info
+```JavaScript
+navigator.mediaCapabilities.decodingInfo({
+ type: 'webrtc',
+ video: {
+ contentType: 'video/VP9; profile-id="2"',
+ spatialScalability: false,
+ height: 1080,
+ width: 1920,
+ framerate: 24,
+ bitrate: 2826848,
+ },
+ audio: {
+ contentType: 'audio/opus',
+ }
+}).then(result => {
+ console.log(result.supported);
+ console.log(result.smooth);
+ console.log(result.powerEfficient);
+});
+```
+
+#### Encoding info
+```JavaScript
+navigator.mediaCapabilities.encodingInfo({
+ type: 'webrtc',
+ video: {
+ contentType: 'video/VP9',
+ scalabilityMode: 'L3T3_KEY',
+ height: 720,
+ width: 1280,
+ framerate: 24,
+ bitrate: 1216848,
+ },
+ audio: {
+ contentType: 'audio/opus',
+ }
+}).then(result => {
+ console.log(result.supported);
+ console.log(result.smooth);
+ console.log(result.powerEfficient);
+});
+```
+
# Privacy Considerations
The Media Capabilities API will provide a lot of information to enable websites to optimise the user experience. None of this is sensitive information about the user. However, some of the data being exposed could be used to fingerprint users or track them across websites.
diff --git a/hdr_explainer.md b/hdr_explainer.md
new file mode 100644
index 0000000..2f5079e
--- /dev/null
+++ b/hdr_explainer.md
@@ -0,0 +1,64 @@
+# HDR Capability Detection
+HDR capabilities are detected with a combination of CSS and MediaCapabilities APIs. Both APIs should be called to determine if HDR can be correctly presented.
+
+Please note that both of these APIs are fairly new and are not yet implemented by all user agents. See Implementations section below.
+
+# CSS: is the display HDR capable?
+
+Media Queries Level 5 defines a `dynamic-range` feature, with values `"standard"` and `"high"`.
+
+https://www.w3.org/TR/mediaqueries-5/#descdef-media-dynamic-range
+
+```
+let isScreenHDR = window.matchMedia('(dynamic-range: high)').matches;
+```
+
+This indicates whether the current display is HDR capable.
+
+# CSS: video-* prefixed MediaQueries
+User agents on TVs tend to offer video capabilities at the native resolution and dynamic range (ex: 4k HDR) while offering much lower capabilities for the rest of the web page (maybe 1080p SDR). To surface these distinct capabilities, Media Queries Level 5 defines features prefixed by "video-".
+
+https://www.w3.org/TR/mediaqueries-5/#video-prefixed-features
+
+This includes `video-dynamic-range`. On televisions, this value may be `"high"` while the non-prefixed `dynamic-range` remains `"standard"`.
+
+On desktop and mobile devices, values for video-* features will generally match their non-prefixed counterparts.
+
+The sizing video-\* prefixed features are still under discussion, including `video-width`, `video-height`, and `video-resolution`. See https://github.com/w3c/csswg-drafts/issues/5044
+
+
+# MediaCapabilities: does the UA support decoding/rendering my brand of HDR video?
+Use MediaCapabilities to ask about HDR video decoding/rendering capabilities, **independent of the display capabilities**.
+* Does it support the desired transfer function? https://www.w3.org/TR/media-capabilities/#transferfunction
+* Does it support rendering in the desired color gamut? https://www.w3.org/TR/media-capabilities/#colorgamut
+* Does it support the desired HDR metadata? https://www.w3.org/TR/media-capabilities/#hdrmetadatatype
+
+Here's a code sample (note the last 3 fields in `hdrConfig`).
+```
+let hdrConfig = {
+ type: 'media-source',
+ video: {
+ contentType: 'video/webm; codecs="vp09.00.10.08"',
+ width: 1920,
+ height: 1080,
+ bitrate: 2646242,
+ framerate: '25',
+ transferFunction: 'pq',
+ colorGamut: 'p3',
+ hdrMetadataType: 'smpteSt2086',
+ }
+};
+
+navigator.mediaCapabilities.decodingInfo(hdrConfig).then(function(info) {
+ if (info.supported) {
+ // Playback is supported! See also, info.smooth and info.powerEfficient.
+ }
+});
+```
+
+# Implementations
+
+As of 2020/11, here's the unofficial status of user agent implementations for the above:
+* CSS dynamic-range is implemented by Safari. Not yet started by Chrome (ETA early 2021), nor Firefox
+* CSS video-* features are not yet implemented anywhere.
+* MediaCapabilities HDR inputs are implemented by Safari. Chrome's implementation is work-in-progress. Not yet started in Firefox.
diff --git a/index.bs b/index.bs
index 4af3f7b..dc2a4ce 100644
--- a/index.bs
+++ b/index.bs
@@ -1,12 +1,18 @@
Title: Media Capabilities
-Repository: wicg/media-capabilities
-Status: CG-DRAFT
-ED: https://wicg.github.io/media-capabilities/
+Repository: w3c/media-capabilities
+Status: ED
+ED: https://w3c.github.io/media-capabilities/
+TR: https://www.w3.org/TR/media-capabilities/
Shortname: media-capabilities
-Level: 1
-Group: wicg
-Editor: Mounir Lamouri, w3cid 45389, Google Inc. https://google.com/
+Level: None
+Group: mediawg
+Editor: Jean-Yves Avenard, w3cid 115886, Apple Inc. https://www.apple.com/
+
+Former Editor: Will Cassella, w3cid 139598, Google Inc. https://www.google.com/
+Former Editor: Mounir Lamouri, w3cid 45389, Google Inc. https://www.google.com/
+Former Editor: Chris Cunningham, w3cid 114832, Google Inc. https://www.google.com/
+Former Editor: Vi Nguyen, w3cid 116349, Microsoft Corporation https://www.microsoft.com/
Abstract: This specification intends to provide APIs to allow websites to make
Abstract: an optimal decision when picking media content for the user. The APIs
@@ -14,58 +20,80 @@ Abstract: will expose information about the decoding and encoding capabilities
Abstract: for a given format but also output capabilities to find the best match
Abstract: based on the device's display.
-!Participate: Git Repository.
-!Participate: File an issue.
-!Version History: https://github.com/wicg/media-capabilities/commits
+!Participate: Git Repository.
+!Participate: File an issue.
+!Version History: https://github.com/w3c/media-capabilities/commits
-spec: media-source; urlPrefix: https://w3c.github.io/media-source/
+spec: media-source; urlPrefix: https://www.w3.org/TR/media-source/
type: interface
- for: MediaSource; text: MediaSource; url: #media-source
+ for: MediaSource; text: MediaSource; url: #mediasource
type: method
for: MediaSource; text: isTypeSupported(); url: #dom-mediasource-istypesupported
-spec: html; urlPrefix: https://html.spec.whatwg.org/multipage/;
- type: method
- urlPrefx: embedded-content.html/
- for: HTMLMediaElement; text: canPlayType(); url: #dom-navigator-canplaytype
- type: dfn
- text: rules for parsing floating-point number values
-
-spec: ECMAScript; urlPrefix: https://tc39.github.io/ecma262/#
- type: interface
- text: TypeError; url: sec-native-error-types-used-in-this-standard-typeerror
-
-spec: cssom-view; urlPrefix: https://drafts.csswg.org/cssom-view/#
- type: interface
- text: Screen; url: screen
-
-spec: mediaqueries-4; urlPrefix: https://drafts.csswg.org/mediaqueries-4/#
- type: interface
- text: color-gamut
-
-spec: mediacapture-record; urlPrefix: https://www.w3.org/TR/mediastream-recording/#
+spec: mediastream-recording; urlPrefix: https://www.w3.org/TR/mediastream-recording/#
type:interface
text: MediaRecorder; url: mediarecorder
-spec: webrtc-pc; urlPrefix: https://www.w3.org/TR/webrtc/#
- type: interface
- text: RTCPeerConnection; url: interface-definition
-
spec: mimesniff; urlPrefix: https://mimesniff.spec.whatwg.org/#
type: dfn; text: valid mime type; url: valid-mime-type
-spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
- type: dfn; text: present
+spec: encrypted-media; for: EME; urlPrefix: https://www.w3.org/TR/encrypted-media/#
+ type: attribute
+ text: keySystem; url: dom-mediakeysystemaccess-keysystem
+ text: initDataTypes; url: dom-mediakeysystemconfiguration-initdatatypes
+ text: robustness; url: dom-mediakeysystemmediacapability-robustness
+ text: distinctiveIdentifier; url: dom-mediakeysystemconfiguration-distinctiveidentifier
+ text: persistentState; url: dom-mediakeysystemconfiguration-persistentstate
+ text: sessionTypes; url: dom-mediakeysystemconfiguration-sessiontypes
+ type: dfn
+ text: encrypted media
+ text: Key System; url: key-system
+ text: Get Supported Configuration; url: get-supported-configuration
+ type: interface
+ text: MediaKeySystemAccess; url: mediakeysystemaccess-interface
+ text: MediaKeys; url: mediakeys-interface
+ text: MediaKeySystemConfiguration; url: mediakeysystemconfiguration-dictionary
+ text: requestMediaKeySystemAccess(); url: navigator-extension:-requestmediakeysystemaccess()
+ text: MediaKeySystemMediaCapability; url: mediakeysystemmediacapability-dictionary
+ text: MediaKeysRequirement; url: dom-mediakeysrequirement
+ text: audioCapabilities; url: dom-mediakeysystemconfiguration-audiocapabilities
+ text: contentType; url: dom-mediakeysystemmediacapability-contenttype
+
+spec: encrypted-media-draft; for: EME; urlPrefix: https://w3c.github.io/encrypted-media/#
+ type: attribute
+ text: encryptionScheme; url: dom-mediakeysystemmediacapability-encryptionscheme
+
{
- "media-playback-quality": {
- "href": "https://wicg.github.io/media-playback-quality/",
- "title": "Media Playback Quality Specification",
- "status": "CG-DRAFT",
- "publisher": "WICG"
+ "SMPTE-ST-2084": {
+ "href": "https://ieeexplore.ieee.org/document/7291452",
+ "title": "High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays",
+ "publisher": "SMPTE",
+ "date": "2014",
+ "id": "SMPTE-ST-2084"
+ },
+ "SMPTE-ST-2086": {
+ "href": "https://ieeexplore.ieee.org/document/7291707",
+ "title": "Mastering Display Color Volume Metadata Supporting High Luminance and Wide Color Gamut Images",
+ "publisher": "SMPTE",
+ "date": "2014",
+ "id": "SMPTE-ST-2086"
+ },
+ "SMPTE-ST-2094": {
+ "href": "https://ieeexplore.ieee.org/document/7513361",
+ "title": "Dynamic Metadata for Color Volume Transform Core Components",
+ "publisher": "SMPTE",
+ "date": "2016",
+ "id": "SMPTE-ST-2094"
+ },
+ "ENCRYPTED-MEDIA-DRAFT": {
+ "href": "https://w3c.github.io/encrypted-media",
+ "title": "Encrypted Media Extensions",
+ "publisher": "W3C",
+ "date": "13 December 2019"
}
}
@@ -124,24 +152,25 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
-
+ <![CDATA[
dictionary MediaConfiguration {
VideoConfiguration video;
AudioConfiguration audio;
};
- </pre>
+ ]]>
-
+ <![CDATA[
dictionary MediaDecodingConfiguration : MediaConfiguration {
required MediaDecodingType type;
+ MediaCapabilitiesKeySystemConfiguration keySystemConfiguration;
};
- </pre>
+ ]]>
-
+ <![CDATA[
dictionary MediaEncodingConfiguration : MediaConfiguration {
required MediaEncodingType type;
};
- </pre>
+ ]]>
The input to the decoding capabilities is represented by a
@@ -150,23 +179,66 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
For a {{MediaConfiguration}} to be a valid
- MediaConfiguration, audio or video MUST
- be present.
+ MediaConfiguration, all of the following conditions MUST be true:
+
+ -
+
audio and/or video MUST [=map/exist=].
+
+ -
+
audio MUST be a valid audio configuration if
+ it [=map/exists=].
+
+ -
+
video MUST be a valid video configuration if
+ it [=map/exists=].
+
+
+
+
+ For a {{MediaDecodingConfiguration}} to be a valid
+ MediaDecodingConfiguration, all of the following conditions MUST
+ be true:
+
+ -
+ It MUST be a valid MediaConfiguration.
+
+ -
+ If
keySystemConfiguration [=map/exists=]:
+
+ -
+ The
type MUST be {{media-source}} or {{file}}.
+
+ -
+ If
keySystemConfiguration.audio [=map/exists=],
+ audio MUST also [=map/exist=].
+
+ -
+ If
keySystemConfiguration.video [=map/exists=],
+ video MUST also [=map/exist=].
+
+
+
+
+
+
+ For a {{MediaDecodingConfiguration}} to describe [[!ENCRYPTED-MEDIA]], a
+ {{keySystemConfiguration}} MUST [=map/exist=].
-
+ <![CDATA[
enum MediaDecodingType {
"file",
"media-source",
+ "webrtc"
};
- </pre>
+ ]]>
- A {{MediaDecodingConfiguration}} has two types:
+ A {{MediaDecodingConfiguration}} has three types:
- file is used to
represent a configuration that is meant to be used for a plain file
@@ -175,6 +247,9 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
to represent a configuration that is meant to be used for playback of
a {{MediaSource/MediaSource}} as defined in the [[media-source]]
specification.
+ - webrtc is used to
+ represent a configuration that is meant to be received using
+ {{RTCPeerConnection}} as defined in [[webrtc]]).
@@ -182,22 +257,23 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
-
+ <![CDATA[
enum MediaEncodingType {
"record",
- "transmission"
+ "webrtc"
};
- </pre>
+ ]]>
A {{MediaEncodingConfiguration}} can have one of two types:
- record is used to
- represent a configuration for recording of media, e.g. using
- {{MediaRecorder}} as defined in [[mediastream-recording]].
- - transmission is used
- to represent a configuration meant to be transmitted over electronic
- means (e.g. using {{RTCPeerConnection}}).
+ represent a configuration for recording of media,
+ e.g. using {{MediaRecorder}} as defined in
+ [[mediastream-recording]].
+ - webrtc is used to
+ represent a configuration that is meant to be transmitted using
+ {{RTCPeerConnection}} as defined in [[webrtc]]).
@@ -251,30 +327,66 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
- A valid audio MIME type is a string that is valid media
- MIME type and for which the type per [[RFC7231]] is
+ Please note that the definition of MIME subtypes and parameters is
+ context dependent. For {{file}}, {{media-source}}, and {{record}}, the
+ MIME types are specified as defined for [[#http|HTTP]], whereas for
+ {{MediaDecodingType/webrtc}} the MIME types are specified as
+ defined for [[#rtp|RTP]].
+
+
+
+ A valid audio MIME type is a string that is a valid media
+ MIME type and for which the type per [[RFC9110]] is
either audio or application.
A valid video MIME type is a string that is a valid media
- MIME type and for which the type per [[RFC7231]] is
+ MIME type and for which the type per [[RFC9110]] is
either video or application.
+
+
+ HTTP
+
+
+ If the MIME type does not imply a codec, the string MUST also have one
+ and only one parameter that is named codecs with a value
+ describing a single media codec. Otherwise, it MUST contain no
+ parameters.
+
+
+
+
+ RTP
+
+
+ The MIME types used with RTP are defined in the specifications of the
+ corresponding RTP payload formats [[RFC4855]] [[RFC6838]]. The codec
+ name is typically specified as subtype and zero or more parameters
+ may be present depending on the codec.
+
+
VideoConfiguration
-
+ <![CDATA[
dictionary VideoConfiguration {
required DOMString contentType;
required unsigned long width;
required unsigned long height;
required unsigned long long bitrate;
- required DOMString framerate;
+ required double framerate;
+ boolean hasAlphaChannel;
+ HdrMetadataType hdrMetadataType;
+ ColorGamut colorGamut;
+ TransferFunction transferFunction;
+ DOMString scalabilityMode;
+ boolean spatialScalability;
};
- </pre>
+ ]]>
The contentType member
@@ -291,24 +403,14 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
abort these steps.
- If none of the following is true, return false and
- abort these steps:
-
+ If {{VideoConfiguration/framerate}} is not finite or is not greater
+ than 0, return false and abort these steps.
+
+ If an optional member is specified for a {{MediaDecodingType}} or
+ {{MediaEncodingType}} to which it's not applicable, return
+ false and abort these steps. See applicability rules
+ in the member definitions below.
Return true.
@@ -336,22 +438,199 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
The framerate member
represents the framerate of the video track. The framerate is the number
- of frames used in one second (frames per second). It is represented
- either as a double or as a fraction.
+ of frames used in one second (frames per second). It is represented as a
+ double.
+
+
+
+ The hasAlphaChannel member
+ represents whether the video track contains alpha channel information. If
+ true, the encoded video stream can produce per-pixel alpha channel information
+ when decoded. If false, the video stream cannot produce per-pixel alpha channel
+ information when decoded. If undefined, the UA should determine whether the
+ video stream encodes alpha channel information based on the indicated
+ {{VideoConfiguration/contentType}}, if possible. Otherwise, the UA should
+ presume that the video stream cannot produce alpha channel information.
+
+
+
+ If present, the hdrMetadataType
+ member represents that the video track includes the specified HDR
+ metadata type, which the UA needs to be capable of interpreting for tone
+ mapping the HDR content to a color volume and luminance of the output
+ device. Valid inputs are defined by {{HdrMetadataType}}. hdrMetadataType is
+ only applicable to {{MediaDecodingConfiguration}} for types {{media-source}}
+ and {{file}}.
+
+
+
+ If present, the colorGamut
+ member represents that the video track is delivered in the specified
+ color gamut, which describes a set of colors in which the content is
+ intended to be displayed. If the attached output device also supports
+ the specified color, the UA needs to be able to cause the output device
+ to render the appropriate color, or something close enough. If the
+ attached output device does not support the specified color, the UA
+ needs to be capable of mapping the specified color to a color supported
+ by the output device. Valid inputs are defined by {{ColorGamut}}. colorGamut
+ is only applicable to {{MediaDecodingConfiguration}} for types
+ {{media-source}} and {{file}}.
+
+
+
+ If present, the transferFunction
+ member represents that the video track requires the specified transfer
+ function to be understood by the UA. Transfer function describes the
+ electro-optical algorithm supported by the rendering capabilities of a
+ user agent, independent of the display, to map the source colors in the
+ decoded media into the colors to be displayed. Valid inputs are defined
+ by {{TransferFunction}}. transferFunction is only applicable to
+ {{MediaDecodingConfiguration}} for types {{media-source}} and {{file}}.
+
+
+
+ If present, the scalabilityMode
+ member represents the scalability mode as defined in [[webrtc-svc]]. If
+ absent, the implementer defined default mode for this
+ {{VideoConfiguration/contentType}} is assumed (i.e., the mode you get if
+ you don't specify one via {{RTCRtpSender/setParameters()}}).
+ scalabilityMode is only applicable to {{MediaEncodingConfiguration}} for
+ type {{MediaEncodingType/webrtc}}.
+
+
+
+ If present, the spatialScalability
+ member represents the ability to do spatial prediction, that is,
+ using frames of a resolution different than the current resolution as
+ dependencies. If absent, spatialScalability will default to
+ false. spatialScalability is closely coupled to
+ {{VideoConfiguration/scalabilityMode}} in the sense that streams encoded
+ with modes using spatial scalability (e.g. "L2T1") can only be decoded
+ if spatialScalability is supported. spatialScalability is only
+ applicable to {{MediaDecodingConfiguration}} for types {{media-source}},
+ {{file}}, and {{MediaDecodingType/webrtc}}.
+
+
+
+
+
+
+
+
<![CDATA[
+ enum HdrMetadataType {
+ "smpteSt2086",
+ "smpteSt2094-10",
+ "smpteSt2094-40"
+ };
+ ]]>
+
+
+ If present, {{HdrMetadataType}} describes the capability to interpret HDR metadata
+ of the specified type.
+
+
+
+ The {{VideoConfiguration}} may contain one of the following types:
+
+ -
+ smpteSt2086,
+ representing the static metadata type defined by
+ [[!SMPTE-ST-2086]].
+
+ -
+ smpteSt2094-10,
+ representing the dynamic metadata type defined by
+ [[!SMPTE-ST-2094]].
+
+ -
+ smpteSt2094-40,
+ representing the dynamic metadata type defined by
+ [[!SMPTE-ST-2094]].
+
+
+
+
+
+
+
+ ColorGamut
+
+
+
<![CDATA[
+ enum ColorGamut {
+ "srgb",
+ "p3",
+ "rec2020"
+ };
+ ]]>
+
+
+ The {{VideoConfiguration}} may contain one of the following types:
+
+ -
+ srgb, representing the
+ [[!sRGB]] color gamut.
+
+ -
+ p3, representing the DCI
+ P3 Color Space color gamut. This color gamut includes the
+ {{ColorGamut/srgb}} gamut.
+
+ -
+ rec2020, representing
+ the ITU-R Recommendation BT.2020 color gamut. This color gamut
+ includes the {{ColorGamut/p3}} gamut.
+
+
+
+
+
+
+
+ TransferFunction
+
+
+
<![CDATA[
+ enum TransferFunction {
+ "srgb",
+ "pq",
+ "hlg"
+ };
+ ]]>
+
+
+ The {{VideoConfiguration}} may contain one of the following types:
+
+ -
+ srgb, representing
+ the transfer function defined by [[!sRGB]].
+
+ -
+ pq, representing the
+ "Perceptual Quantizer" transfer function defined by
+ [[!SMPTE-ST-2084]].
+
+ -
+ hlg, representing the
+ "Hybrid Log Gamma" transfer function defined by BT.2100.
+
+
+
AudioConfiguration
-
+ <![CDATA[
dictionary AudioConfiguration {
required DOMString contentType;
DOMString channels;
unsigned long long bitrate;
unsigned long samplerate;
+ boolean spatialRendering;
};
- </pre>
+ ]]>
The contentType member
@@ -375,7 +654,10 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
The channels member
- represents the audio channels used by the audio track.
+ represents the audio channels used by the audio track. channels is only
+ applicable to the decoding types {{media-source}}, {{file}}, and
+ {{MediaDecodingType/webrtc}} and the encoding type
+ {{MediaEncodingType/webrtc}}.
@@ -394,7 +676,10 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
The samplerate
represents the samplerate of the audio track in. The samplerate is the
- number of samples of audio carried per second.
+ number of samples of audio carried per second. samplerate is only
+ applicable to the decoding types {{media-source}}, {{file}}, and
+ {{MediaDecodingType/webrtc}} and the encoding type
+ {{MediaEncodingType/webrtc}}.
@@ -404,340 +689,526 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
thousands of samples of audio per second.
44100 Hz is equivalent to 44.1 kHz.
+
+
+ The spatialRendering
+ member indicates that the audio SHOULD be renderered spatially. The
+ details of spatial rendering SHOULD be inferred from the
+ {{AudioConfiguration/contentType}}. If it does not [=map/exist=], the UA
+ MUST presume spatialRendering is not required. When true,
+ the user agent SHOULD only report this configuration as
+ {{MediaCapabilitiesInfo/supported}} if it can support spatial
+ rendering *for the current audio output device* without failing back to a
+ non-spatial mix of the stream. spatialRendering is only applicable to
+ {{MediaDecodingConfiguration}} for types {{media-source}} and {{file}}.
+
+
+
+
+ <![CDATA[
+ dictionary MediaCapabilitiesKeySystemConfiguration {
+ required DOMString keySystem;
+ DOMString initDataType = "";
+ MediaKeysRequirement distinctiveIdentifier = "optional";
+ MediaKeysRequirement persistentState = "optional";
+ sequence<DOMString> sessionTypes;
+ KeySystemTrackConfiguration audio;
+ KeySystemTrackConfiguration video;
+ };
+ ]]>
+
+
+ This dictionary refers to a number of types defined by
+ [[ENCRYPTED-MEDIA]] (EME). Sequences of EME types are
+ flattened to a single value whenever the intent of the sequence was to
+ have {{EME/requestMediaKeySystemAccess()}} choose a subset it supports.
+ With MediaCapabilities, callers provide the sequence across multiple
+ calls, ultimately letting the caller choose which configuration to use.
+
+
+
+ The keySystem
+ member represents a {{EME/keySystem}} name as described in
+ [[!ENCRYPTED-MEDIA]].
+
+
+ The initDataType
+ member represents a single value from the {{EME/initDataTypes}} sequence
+ described in [[!ENCRYPTED-MEDIA]].
+
+
+ The distinctiveIdentifier
+ member represents a {{EME/distinctiveIdentifier}} requirement as
+ described in [[!ENCRYPTED-MEDIA]].
+
+
+ The persistentState
+ member represents a {{EME/persistentState}} requirement as described in
+ [[!ENCRYPTED-MEDIA]].
+
+
+ The sessionTypes
+ member represents a sequence of required {{EME/sessionTypes}} as
+ described in [[!ENCRYPTED-MEDIA]].
+
+
+ The audio member
+ represents a {{KeySystemTrackConfiguration}} associated with the {{AudioConfiguration}}.
+
+
+ The video member
+ represents a {{KeySystemTrackConfiguration}} associated with the {{VideoConfiguration}}.
+
+
+
+
+
+ KeySystemTrackConfiguration
+
+
+ <![CDATA[
+ dictionary KeySystemTrackConfiguration {
+ DOMString robustness = "";
+ DOMString? encryptionScheme = null;
+ };
+ ]]>
+
+
+ The robustness
+ member represents a {{EME/robustness}} level as described in [[!ENCRYPTED-MEDIA]].
+
+
+
+ The encryptionScheme
+ member represents an {{EME/encryptionScheme}} as described in [[!ENCRYPTED-MEDIA-DRAFT]].
+
+
+
-
+ <![CDATA[
dictionary MediaCapabilitiesInfo {
required boolean supported;
required boolean smooth;
required boolean powerEfficient;
};
- </pre>
+ ]]>
-
- The {{MediaCapabilitiesInfo}} has an associated configuration which is a
- {{MediaDecodingConfiguration}} or {{MediaEncodingConfiguration}}.
-
+ <![CDATA[
+ dictionary MediaCapabilitiesDecodingInfo : MediaCapabilitiesInfo {
+ required MediaKeySystemAccess keySystemAccess;
+ MediaDecodingConfiguration configuration;
+ };
+ ]]>
+
+ <![CDATA[
+ dictionary MediaCapabilitiesEncodingInfo : MediaCapabilitiesInfo {
+ MediaEncodingConfiguration configuration;
+ };
+ ]]>
- A {{MediaCapabilitiesInfo}} has associated supported, smooth, supported, smooth, powerEfficient fields which are
booleans.
+
+ Authors can use {{MediaCapabilitiesInfo/powerEfficient}} in concordance
+ with the Battery Status API [[battery-status]] in order to determine
+ whether the media they would like to play is appropriate for the user
+ configuration. It is worth noting that even when a device is not power
+ constrained, high power usage has side effects such as increasing the
+ temperature or the fans noise.
+
+
- When the create a MediaCapabilitiesInfo algorithm with a
- configuration is invoked, the user agent MUST run the following
- steps:
-
- -
- Let info be a new {{MediaCapabilitiesInfo}} instance.
- Unless stated otherwise, reading and writing apply to info
- for the next steps.
-
- -
- Set configuration to
- configuration.
-
- -
- If configuration is of type
- {{MediaDecodingConfiguration}}, run the following substeps:
+ A {{MediaCapabilitiesDecodingInfo}} has associated
+ keySystemAccess
+ which is a {{EME/MediaKeySystemAccess}} or
null as
+ appropriate.
+
+
+
+ If the encrypted decoding configuration is supported, the
+ resulting {{MediaCapabilitiesInfo}} will include a
+ {{EME/MediaKeySystemAccess}}. Authors may use this to create
+ {{EME/MediaKeys}} and setup encrypted playback.
+
+
+
+ A {{MediaCapabilitiesDecodingInfo}} has an associated configuration which
+ is the decoding configuration properties used to generate the
+ {{MediaCapabilitiesDecodingInfo}}.
+
+
+
+ A {{MediaCapabilitiesEncodingInfo}} has an associated configuration which
+ is the encoding configuration properties used to generate the
+ {{MediaCapabilitiesEncodingInfo}}.
+
+
+
+ Algorithms
+
+
+
+
+ Given a {{MediaEncodingConfiguration}} configuration, this
+ algorithm returns a {{MediaCapabilitiesEncodingInfo}}. The following
+ steps are run:
-
- If the user agent is able to decode the media represented by
- configuration, set
- supported to
true.
- Otherwise set it to false.
+ Let info be a new {{MediaCapabilitiesEncodingInfo}}
+ instance. Unless stated otherwise, reading and writing apply to
+ info for the next steps.
-
- If the user agent is able to decode the media represented by
- configuration at a pace that
- allows a smooth playback, set smooth to
true.
- Otherwise set it to false.
+ Set {{MediaCapabilitiesEncodingInfo/configuration}} to be a
+ new {{MediaEncodingConfiguration}}. For every property in
+ configuration create a new property with the same name and
+ value in {{MediaCapabilitiesEncodingInfo/configuration}}.
- -
- If the user agent is able to decode the media represented by
- configuration in a power efficient
- manner, set powerEfficient to
-
true. Otherwise set it to false. The
- user agent SHOULD NOT take into consideration the current power
- source in order to determine the decoding power efficiency unless
- the device's power source has side effects such as enabling
- different decoding modules.
-
-
-
- -
- If configuration is of type
- {{MediaEncodingConfiguration}}, run the following substeps:
-
-
If the user agent is able to encode the media represented by
- configuration, set
- supported to
true.
- Otherwise set it to false.
+ configuration, set {{MediaCapabilitiesInfo/supported}}
+ to true. Otherwise set it to false.
-
If the user agent is able to encode the media represented by
- configuration at a pace that
- allows encoding frames at the same pace as they are sent to the
- encoder, set smooth to
-
true. Otherwise set it to false.
+ configuration at the indicated framerate, set
+ {{MediaCapabilitiesInfo/smooth}} to true. Otherwise
+ set it to false.
-
If the user agent is able to encode the media represented by
- configuration in a power
- efficient manner, set powerEfficient to
true.
+ configuration in a power efficient manner, set
+ {{MediaCapabilitiesInfo/powerEfficient}} to true.
Otherwise set it to false. The user agent SHOULD NOT
take into consideration the current power source in order to
determine the encoding power efficiency unless the device's power
source has side effects such as enabling different encoding
modules.
+ -
+ Return info.
+
- -
- Return info.
-
-
-
-
-
-
- The supported attribute
- MUST return supported.
-
+
+
-
- The smooth attribute MUST
- return smooth.
-
+
+
+
+ Given a {{MediaDecodingConfiguration}} configuration, this
+ algorithm returns a {{MediaCapabilitiesDecodingInfo}}. The following
+ steps are run:
+
+ -
+ Let info be a new {{MediaCapabilitiesDecodingInfo}}
+ instance. Unless stated otherwise, reading and writing apply to
+ info for the next steps.
+
+ -
+ Set {{MediaCapabilitiesDecodingInfo/configuration}} to be a new
+ {{MediaDecodingConfiguration}}. For every property in
+ configuration create a new property with the same name and
+ value in {{MediaCapabilitiesDecodingInfo/configuration}}.
+
+ -
+ If
configuration.keySystemConfiguration [=map/exists=]:
+
+ -
+ Set {{MediaCapabilitiesDecodingInfo/keySystemAccess}}
+ to the result of running the Check Encrypted Decoding
+ Support algorithm with configuration.
+
+ -
+ If {{MediaCapabilitiesDecodingInfo/keySystemAccess}}
+ is not
null set
+ {{MediaCapabilitiesInfo/supported}} to
+ true. Otherwise set it to false.
+
+
+
+ -
+ Otherwise, run the following steps:
+
+ -
+ Set {{MediaCapabilitiesDecodingInfo/keySystemAccess}}
+ to
null.
+
+ -
+ If the user agent is able to decode the media represented
+ by configuration, set
+ {{MediaCapabilitiesInfo/supported}} to
true.
+
+ - Otherwise, set it to
false.
+
+
+ -
+ If the user agent is able to decode the media represented by
+ configuration at the indicated framerate
+ without dropping frames, set {{MediaCapabilitiesInfo/smooth}}
+ to
true. Otherwise set it to false.
+
+ -
+ If the user agent is able to decode the media represented by
+ configuration in a power efficient
+ manner, set {{MediaCapabilitiesInfo/powerEfficient}} to
+
true. Otherwise set it to false. The
+ user agent SHOULD NOT take into consideration the current
+ power source in order to determine the decoding power
+ efficiency unless the device's power source has side effects
+ such as enabling different decoding modules.
+
+ -
+ Return info.
+
+
+
+
-
- The powerEfficient
- attribute MUST return powerEfficient.
-
+
+
+ Check Encrypted Decoding Support
+
+
+ Given a {{MediaDecodingConfiguration}} config where
+ {{keySystemConfiguration}} [=map/exists=], this algorithm returns a
+ {{EME/MediaKeySystemAccess}} or null as appropriate. The
+ following steps are run:
+
+ -
+ If the {{keySystem}} member of
+
config.keySystemConfiguration is not one of the
+ Key Systems supported by the user agent, return
+ null. String comparison is case-sensitive.
+
+ -
+ Let origin be the [=/origin=] of the calling context's
+ Document.
+
+ -
+ Let implementation be the implementation of
config.keySystemConfiguration.keySystem
+
-
- Authors can use {{MediaCapabilitiesInfo/powerEfficient}} in concordance
- with the Battery Status API [[battery-status]] in order to determine
- whether the media they would like to play is appropriate for the user
- configuration. It is worth noting that even when a device is not power
- constrained, high power usage has side effects such as increasing the
- temperature or the fans noise.
-
+ -
+ Let emeConfiguration be a new
+ {{EME/MediaKeySystemConfiguration}}, and initialize it as follows:
+
+
+ -
+ Set the {{EME/initDataTypes}} attribute to a sequence containing
+
config.keySystemConfiguration.initDataType.
+
+ -
+ Set the {{EME/distinctiveIdentifier}} attribute to
+
config.keySystemConfiguration.distinctiveIdentifier.
+
+ -
+ Set the {{EME/persistentState}} attribute to
+
config.keySystemConfiguration.peristentState.
+
+ -
+ Set the {{EME/sessionTypes}} attribute to
+
config.keySystemConfiguration.sessionTypes.
+
+ -
+ If {{MediaConfiguration/audio}} [=map/exists=] in config, set the
+ {{EME/audioCapabilities}} attribute to a sequence containing a
+ single {{EME/MediaKeySystemMediaCapability}}, initialized as
+ follows:
+
+ -
+ Set the {{EME/contentType}} attribute to
+
config.audio.contentType.
+
+ -
+ If
config.keySystemConfiguration.audio
+ [=map/exists=]:
+
+ -
+ Set the {{EME/robustness}} attribute to
+ config.keySystemConfiguration.audio.robustness
+
+ -
+ Set the {{EME/encryptionScheme}} attribute to
+ config.keySystemConfiguration.audio.encryptionScheme
+
+
+
+
+
+ -
+ If {{MediaConfiguration/video}} [=map/exists=] in config, set the
+ videoCapabilities attribute to a sequence containing a single
+ {{EME/MediaKeySystemMediaCapability}}, initialized as follows:
+
+ -
+ Set the {{EME/contentType}} attribute to
+
config.video.contentType.
+
+ -
+ If
config.keySystemConfiguration.video [=map/exists=]:
+
+ -
+ Set the {{EME/robustness}} attribute to
+ config.keySystemConfiguration.video.robustness.
+
+ -
+ Set the {{EME/encryptionScheme}} attribute to
+ config.keySystemConfiguration.video.encryptionScheme
+
+
+
+
+
+
+ -
+ Let supported configuration be the result of
+ executing the Get Supported Configuration
+ algorithm on implementation,
+ emeConfiguration, and origin.
+
+ -
+ If supported configuration is
+
NotSupported, return null and abort
+ these steps.
+
+ -
+ Let access be a new {{EME/MediaKeySystemAccess}}
+ object, and initialize it as follows:
+
+ -
+ Set the {{EME/keySystem}} attribute to
+
emeConfiguration.keySystem.
+
+ -
+ Let the configuration value be
+ supported configuration.
+
+ -
+ Let the cdm implementation value be
+ implementation.
+
+
+
+ - Return access
+
+
+
Navigator and WorkerNavigator extension
-
+ <![CDATA[
[Exposed=Window]
partial interface Navigator {
[SameObject] readonly attribute MediaCapabilities mediaCapabilities;
};
- </pre>
- <pre class='idl'>
+ ]]>
+
+ <![CDATA[
[Exposed=Worker]
partial interface WorkerNavigator {
[SameObject] readonly attribute MediaCapabilities mediaCapabilities;
};
- </pre>
+ ]]>
-
+ <![CDATA[
[Exposed=(Window, Worker)]
interface MediaCapabilities {
- [NewObject] Promise<MediaCapabilitiesInfo> decodingInfo(MediaDecodingConfiguration configuration);
- [NewObject] Promise<MediaCapabilitiesInfo> encodingInfo(MediaEncodingConfiguration configuration);
+ [NewObject] Promise<MediaCapabilitiesDecodingInfo> decodingInfo(MediaDecodingConfiguration configuration);
+ [NewObject] Promise<MediaCapabilitiesEncodingInfo> encodingInfo(MediaEncodingConfiguration configuration);
};
- </pre>
+ ]]>
- The decodingInfo() method and
- the encodingInfo() method MUST
- run the following steps:
+ The {{decodingInfo()}} method method MUST run the following steps:
-
- If configuration is not a valid MediaConfiguration,
- return a Promise rejected with a
TypeError.
-
- -
- If
configuration.video is present and is not a
- valid video configuration, return a Promise rejected with a
- TypeError.
+ If configuration is not a valid
+ MediaDecodingConfiguration, return a Promise rejected with a
+ newly created {{TypeError}}.
-
- If
configuration.audio is present and is not a
- valid audio configuration, return a Promise rejected with a
- TypeError.
+ If configuration.keySystemConfiguration [=map/exists=],
+ run the following substeps:
+
+ -
+ If the [=/global object=] is of type {{WorkerGlobalScope}},
+ return a Promise rejected with a newly created {{DOMException}}
+ whose name is {{InvalidStateError}}.
+
+ -
+ If the [=/global object's=] relevant settings object is a
+ [=non-secure context=], return a Promise rejected with a newly
+ created {{DOMException}} whose name is {{SecurityError}}.
+
+
-
Let p be a new promise.
-
- In parallel, run the create a MediaCapabilitiesInfo
- algorithm with configuration and resolve p
- with its result.
+ In parallel, run the Create a
+ MediaCapabilitiesDecodingInfo algorithm with
+ configuration and resolve p with its result.
-
Return p.
-
-
-
-
- Display Capabilities
-
-
- This section is still Work In Progress and has no shipping implementation.
- Please look into it in details before implementing it.
-
-
-
- Screen Luminance
-
-
- interface ScreenLuminance {
- readonly attribute double min;
- readonly attribute double max;
- readonly attribute double maxAverage;
- };
-
-
-
- The {{ScreenLuminance}} object represents the known luminance
- characteristics of the screen.
-
-
-
- The min attribute MUST return
- the minimal screen luminance that a pixel of the screen can emit in
- candela per square metre. The minimal screen luminance is the luminance
- used when showing the darkest color a pixel on the screen can display.
-
-
-
- The max attribute MUST return
- the maximal screen luminance that a pixel of the screen can emit in
- candela per square metre. The maximal screen luminance is the luminance
- used when showing the whitest color a pixel on the screen can display.
-
-
-
- The maxAverage attribute MUST
- return the maximal average screen luminance that the screen can emit in
- candela per square metre. The maximal average screen luminance is the
- maximal luminance value such as all the pixels of the screen emit the same
- luminance. The value returned by {{ScreenLuminance/maxAverage}} is
- expected to be different from {{ScreenLuminance/max}} as screens usually
- can't apply the maximal screen luminance to the entire panel.
-
-
-
-
- Screen Color Gamut
-
- enum ScreenColorGamut {
- "srgb",
- "p3",
- "rec2020",
- };
-
-
-
- The {{ScreenColorGamut}} represents the color gamut supported by a
- {{Screen}}, that means the range of color that the screen can display.
+
+ Note, calling {{decodingInfo()}} with a {{keySystemConfiguration}} present
+ may have user-visible effects, including requests for user consent. Such
+ calls should only be made when the author intends to create and use a
+ {{EME/MediaKeys}} object with the provided configuration.
- The {{ScreenColorGamut}} values are:
-
+ The {{encodingInfo()}} method MUST run the following steps:
+
-
- srgb, it represents the
- [[sRGB]] color gamut.
+ If configuration is not a valid MediaConfiguration,
+ return a Promise rejected with a newly created {{TypeError}}.
-
- p3, it represents the DCI
- P3 Color Space color gamut. This color gamut includes the
- {{ScreenColorGamut/srgb}} gamut.
+ Let p be a new promise.
-
- rec2020, it represents
- the ITU-R Recommendation BT.2020 color gamut. This color gamut
- includes the {{ScreenColorGamut/p3}} gamut.
+ In parallel, run the Create a MediaCapabilitiesEncodingInfo
+ algorithm with configuration and resolve p
+ with its result.
-
-
-
-
- Screen extension
-
-
- Part of this section is 🐵 patching of the CSSOM View Module. Issue #4
- is tracking merging the changes. This partial interface requires the
- {{Screen}} interface to become an {{EventTarget}}.
-
-
-
- partial interface Screen {
- readonly attribute ScreenColorGamut colorGamut;
- readonly attribute ScreenLuminance? luminance;
-
- attribute EventHandler onchange;
- };
-
-
-
- The colorGamut attribute SHOULD return
- the {{ScreenColorGamut}} approximately supported by the screen. In other
- words, the screen does not need to fully support the given color gamut but
- needs to be close enough. If the user agent does not know the color gamut
- supported by the screen, if the supported color gamut is lower than
- {{ScreenColorGamut/srgb}}, or if the user agent does not want to expose
- this information for privacy consideration, it SHOULD return
- {{ScreenColorGamut/srgb}} as a default value. The value returned by
- {{Screen/colorGamut}} MUST match the value returned by the {{color-gamut}}
- CSS media query.
-
-
-
- The luminance attribute SHOULD return
- a {{ScreenLuminance}} object that will expose the luminance
- characteristics of the screen. If the user agent has no access to the
- luminance characteristics of the screen, it MUST return null.
- The user agent MAY also return null if it does not want to
- expose the luminance information for privacy reasons.
-
-
-
- The onchange attribute is an event
- handler whose corresponding event handler event type is
- change.
+
+ Return p.
+
+
-
- Whenever the user agent is aware that the state of the {{Screen}}
- object has changed, that is if one the value exposed on the {{Screen}}
- object or in an object exposed on the {{Screen}} object, it MUST
- queue a task to fire an event named change on
- {{Screen}}.
-
@@ -772,22 +1243,24 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
- If an implementation wishes to implement a fingerprint-proof version of
- this specification, it would be recommended to fake a given set of
- capabilities (ie. decode up to 1080p VP9, etc.) instead of returning
- always yes or always no as the latter approach could considerably degrade
- the user's experience.
+ HDR detection is more nuanced. Adding colorGamut, transferFunction, and
+ hdrMetadataType has the potential to add significant entropy. However,
+ for UAs whose decoders are implemented in software and therefore whose
+ capabilities are fixed across devices, this feature adds no effective
+ entropy. Additionally, for many cases, devices tend to fall into large
+ categories, within which capabilities are similar thus minimizing
+ effective entropy.
-
-
-
- Display and Fingerprinting
- The information exposed by the display capabilities can already be
- accessed via CSS for the most part. The specification also provides
- default values when the user agent does not which to expose the feature
- for privacy reasons.
+ If an implementation wishes to implement a fingerprint-proof version of
+ this specification, it would be recommended to fake a given set of
+ capabilities (ie. decode up to 1080p VP9, etc.) instead of returning
+ always yes or always no as the latter approach could considerably
+ degrade the user's experience. Another mitigation could be to limit
+ these Web APIs to top-level browsing contexts. Yet another is to use a
+ privacy budget that throttles and/or blocks calls to the API above a
+ threshold.
@@ -797,7 +1270,149 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
Examples
- Query recording capabilities with {{encodingInfo()}}
+ Query playback capabilities with {{decodingInfo()}}
+
+ The following example shows how to use {{decodingInfo()}} to query
+ media playback capabilities when using Media Source Extensions
+ [[media-source]].
+
+
+
+
+ <script>
+ const contentType = 'video/mp4;codecs=avc1.640028';
+
+ const configuration = {
+ type: 'media-source',
+ video: {
+ contentType: contentType,
+ width: 640,
+ height: 360,
+ bitrate: 2000,
+ framerate: 29.97
+ }
+ };
+
+ navigator.mediaCapabilities.decodingInfo(configuration)
+ .then((result) => {
+ console.log('Decoding of ' + contentType + ' is'
+ + (result.supported ? '' : ' NOT') + ' supported,'
+ + (result.smooth ? '' : ' NOT') + ' smooth and'
+ + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
+ })
+ .catch((err) => {
+ console.error(err, ' caused decodingInfo to reject');
+ });
+ </script>
+
+
+ The following examples show how to use {{decodingInfo()}} to query
+ WebRTC receive capabilities [[webrtc]].
+
+
+
+
+ <script>
+ const contentType = 'video/VP8';
+
+ const configuration = {
+ type: 'webrtc',
+ video: {
+ contentType: contentType,
+ width: 640,
+ height: 360,
+ bitrate: 2000,
+ framerate: 25
+ }
+ };
+
+ navigator.mediaCapabilities.decodingInfo(configuration)
+ .then((result) => {
+ console.log('Decoding of ' + contentType + ' is'
+ + (result.supported ? '' : ' NOT') + ' supported,'
+ + (result.smooth ? '' : ' NOT') + ' smooth and'
+ + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
+ })
+ .catch((err) => {
+ console.error(err, ' caused decodingInfo to reject');
+ });
+ </script>
+
+
+
+ <script>
+ const contentType = 'video/H264;level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f';
+
+ const configuration = {
+ type: 'webrtc',
+ video: {
+ contentType: contentType,
+ width: 640,
+ height: 360,
+ bitrate: 2000,
+ framerate: 25
+ }
+ };
+
+ navigator.mediaCapabilities.decodingInfo(configuration)
+ .then((result) => {
+ console.log('Decoding of ' + contentType + ' is'
+ + (result.supported ? '' : ' NOT') + ' supported,'
+ + (result.smooth ? '' : ' NOT') + ' smooth and'
+ + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
+ })
+ .catch((err) => {
+ console.error(err, ' caused decodingInfo to reject');
+ });
+ </script>
+
+
Query recording capabilities with {{encodingInfo()}}
+
+
+ The following example shows how to use {{encodingInfo()}} to query
+ WebRTC send capabilities [[webrtc]] including the optional field
+ {{VideoConfiguration/scalabilityMode}}.
+
+
+
+
+ <script>
+ const contentType = 'video/VP9';
+
+ const configuration = {
+ type: 'webrtc',
+ video: {
+ contentType: contentType,
+ width: 640,
+ height: 480,
+ bitrate: 10000,
+ framerate: 29.97,
+ scalabilityMode: "L3T3_KEY"
+ }
+ };
+
+ navigator.mediaCapabilities.encodingInfo(configuration)
+ .then((result) => {
+ console.log(contentType + ' is:'
+ + (result.supported ? '' : ' NOT') + ' supported,'
+ + (result.smooth ? '' : ' NOT') + ' smooth and'
+ + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
+ })
+ .catch((err) => {
+ console.error(err, ' caused encodingInfo to reject');
+ });
+ </script>
+
+
The following example can also be found in e.g.
@@ -808,26 +1423,29 @@ spec: webidl; urlPrefix: https://heycam.github.io/webidl/#
<script>
+ const contentType = 'video/webm;codecs=vp8';
+
const configuration = {
- type : 'record',
- video : {
- contentType : 'video/webm;codecs=vp8',
- width : 640,
- height : 480,
- bitrate : 10000,
- framerate : '30'
+ type: 'record',
+ video: {
+ contentType: contentType,
+ width: 640,
+ height: 480,
+ bitrate: 10000,
+ framerate: 29.97
}
};
+
navigator.mediaCapabilities.encodingInfo(configuration)
- .then((result) => {
- console.log(result.contentType + ' is:'
- + (result.supported ? '' : ' NOT') + ' supported,'
- + (result.smooth ? '' : ' NOT') + ' smooth and'
- + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
- })
- .catch((err) => {
- console.error(err, ' caused encodingInfo to throw');
- });
+ .then((result) => {
+ console.log(contentType + ' is:'
+ + (result.supported ? '' : ' NOT') + ' supported,'
+ + (result.smooth ? '' : ' NOT') + ' smooth and'
+ + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
+ })
+ .catch((err) => {
+ console.error(err, ' caused encodingInfo to reject');
+ });
</script>
diff --git a/security-privacy-questionnaire.md b/security-privacy-questionnaire.md
index 7f04056..ab06b2e 100644
--- a/security-privacy-questionnaire.md
+++ b/security-privacy-questionnaire.md
@@ -1,91 +1,133 @@
# Media Capabilities - Security and Privacy Questionnaire
This document answers the [W3C Security and Privacy
-Questionnaire](https://www.w3.org/TR/security-privacy-questionnaire/)for the
+Questionnaire](https://w3ctag.github.io/security-questionnaire/) for the
Media Capabilities specification.
-Last Update: 2017-11-06
+Last Update: 2020-01-30
-**Does this specification deal with personally-identifiable information?**
+**What information might this feature expose to Web sites or other parties, and
+for what purposes is that exposure necessary?**
-No.
+The API exposes information about the support, performance (smoothness), and
+power efficiency of various media configurations.
-**Does this specification deal with high-value data?**
+Example query: Can you decode 4k AV1 as part of a media-source playback?
+Example answer: Yes, but it will not play smoothly and it will not be power
+efficient.
-No.
+This information is used by sites to select optimal media configurations. Sites
+will often have multiple configurations to choose from for a given stream and
+will weigh trade-offs like bandwidth savings offered by the latest codecs
+against their performance and power efficiency.
-**Does this specification introduce new state for an origin that persists across
-browsing sessions?**
+**Do features in your specification expose the minimum amount of information
+necessary to enable their intended uses?**
-No.
+Yes. The API aims to answer the important questions without providing any
+extraneous detail (e.g. we make a simple performance prediction without offering
+a confidence interval).
-**Does this specification expose persistent, cross-origin state to the web?**
+**How do the features in your specification deal with personal information,
+personally-identifiable information (PII), or information derived from them?**
-The Media Capabilities of a device are somewhat persistent user information that
-will be easier to access. Indeed, websites can guess this information by playng
-media and checking the dropped frames. Exposed information can also heavily
-correlate with other information such as device model.
+This specification does not deal with PII.
-**Does this specification expose any other data to an origin that it doesn’t
-currently have access to?**
+**How do the features in your specification deal with sensitive information?**
-No.
+This specification does not deal with sensitive information.
-**Does this specification enable new script execution/loading mechanisms?**
+**Do the features in your specification introduce new state for an origin that
+persists across browsing sessions?**
No.
-**Does this specification allow an origin access to a user’s location?**
+**Do the features in your specification expose information about the underlying
+platform to origins?**
-No.
+Yes. Information about codec support, performance and power efficiency is
+implicitly describing the underlying platform. The information is not
+origin-specific.
-**Does this specification allow an origin access to sensors on a user’s
-device?**
+The specification describes the predictions about performance and
+powerEfficiency, but does not prescribe how the UA makes those predictions, nor
+how often the UA should update the predicted values. Acceptable solutions
+include updating the data continuously, at every browser reboot, or even never
+(the UA may hard code it's answers).
+
+
+**Do features in this specification allow an origin access to sensors on a
+user’s device?**
No.
-**Does this specification allow an origin access to aspects of a user’s local
-computing environment?**
+**What data do the features in this specification expose to an origin? Please
+also document what data is identical to data exposed by other features, in the
+same or different contexts.**
-Not directly. The exposed persistent information could be used to guess the
-local computing environment simalarly to what web pages can do today by running
-local benchmark.
+3 booleans indicating codec support, performance (smoothness), and
+power efficiency.
-**Does this specification allow an origin access to other devices?**
+The question of support can also be answered by older APIs. Examples include
+HTMLMediaElement.canPlayType(), MediaSource.isTypeSupported(),
+MediaRecorder.isTypeSupported(), and RTCRtpReceiver.getCapabilities().
-No.
+Performance (smoothness) predictions are unique to this API. Though it is
+possible to observe real-time performance statistics via other APIs. Examples
+include HTMLMediaElement.getVideoPlaybackQuality() and
+RTCPeerConnection.getStats().
+
+Power efficiency predictions are mostly unique to this API. A configuration is
+considered "power efficient" when it is hardware accelerated or when its
+complexity (e.g. resolution) is sufficiently low such that a software codec
+would use a similar amount of power to a hardware accelerated alternative.
+For WebRTC, real-time hardware acceleration information is available via the
+RTCPeerConnection.getStats() API (decoderImplementation == "ExternalDecoder") on
+some UAs (at least Chromium).
-**Does this specification allow an origin some measure of control over a user
-agent’s native UI?**
+
+**Do features in this specification enable new script execution/loading
+mechanisms?**
No.
-**Does this specification expose temporary identifiers to the web?**
+
+**Do features in this specification allow an origin to access other devices?**
No.
-**Does this specification distinguish between behavior in first-party and
-third-party contexts?**
+
+**Do features in this specification allow an origin some measure of control over
+ a user agent’s native UI?**
No.
-**How should this specification work in the context of a user agent’s
-"incognito" mode?**
-The feature should not behave differently in incognito mode.
+**What temporary identifiers do the features in this specification create or
+expose to the web?**
+
+None.
+
+
+**How does this specification distinguish between behavior in first-party and
+third-party contexts?**
+
+It does not distinguish.
-**Does this specification persist data to a user’s local device?**
+**How do the features in this specification work in the context of a browser’s
+Private Browsing or Incognito mode?**
-One implementation strategy would be to store data about historical decoding
-information in order to provide better values to the querying API. When such an
-implementation strategy is used, the data should be cleared when users clear
-their browser information.
+The specification does not prescribe a specific behavior for these modes. The
+specified features do not inherently allow for detection such modes, nor do they
+leak information between modes.
-**Does this specification have a "Security Considerations" and
-"Privacy Considerations" section?**
+**Does this specification have both "Security Considerations" and "Privacy
+Considerations" sections?**
-[Yes](https://wicg.github.io/media-capabilities/#security-privacy-considerations).
+It has a [combined section on Security and Privacy](https://w3c.github.io/media-capabilities/#security-privacy-considerations),
+mostly focused on Privacy. There are no known security impacts of the features in this specification.
-**Does this specification allow downgrading default security characteristics?**
+**Do features in your specification enable downgrading default security
+characteristics?**
No.
diff --git a/w3c.json b/w3c.json
new file mode 100644
index 0000000..15ef466
--- /dev/null
+++ b/w3c.json
@@ -0,0 +1,9 @@
+{
+ "group": 115198,
+ "contacts": [
+ "tidoust",
+ "jernoble",
+ "mounirlamouri"
+ ],
+ "repo-type": "rec-track"
+}
\ No newline at end of file