Intent to Ship: Document picture-in-picture

In the future, we may be able to do something on Android that doesn't support input without any change to Android APIs. If we wanted input in the PiP window, it would require changes to Android itself. That said, we haven't seen any interest from web developers for an inputless document PiP on Android, and there isn't really anything you can do with an inputless document PiP that you can't do with a canvas-back video PiP, so we haven't pursued anything on Android.

On Tue, Jun 13, 2023 at 7:06 AM Philip Jägenstedt <[email protected]> wrote:

Thanks Domenic for mentoring and vouching :) I took a very cursory look at the API shape and sent a small fix, thanks for reviewing that.

The concerns on https://github.com/WebKit/standards-positions/issues/41 are device independence and portability. For the Android support, can you say anything about future expectations? Is this feature simply not important on mobile and it might be a desktop-only API for the foreseeable future, or would Android support make sense for some use cases? If so, does it seem tractable given the requisite changes to Android APIs?

On Tue, Jun 13, 2023 at 4:16 AM Domenic Denicola <[email protected]> wrote:

I was the spec mentor for Tommy's work on this feature, and am chiming in to provide the requested spec maturity summary.

I'm very happy with the spec work for this feature. Tommy worked hard to incorporate all my feedback about how to ensure everything was rigorously defined, and integrated well with existing concepts like opening new windows (navigables). The monkey patches are small, focused, and easy to understand.

There is some TAG feedback that seems to wish this was part of window.open() or some other more-general API, but I think that advice is not correct, and the current API design is good, due to the singleton-per-top-level-traversable nature of a document PiP window. In my opinion this makes the window.documentPictureInPicture entrypoint, with its requestWindow(), window, and onenter properties, a good API for the use case.

On Tue, Jun 13, 2023 at 1:51 AM 'Tommy Steimel' via blink-dev <[email protected]> wrote:

Contact emails

[email protected], [email protected], [email protected]

Explainer

https://github.com/WICG/document-picture-in-picture/blob/main/README.md

Specification

https://wicg.github.io/document-picture-in-picture

Summary

Document PiP adds a new API to open an always-on-top window that can be populated with arbitrary HTMLElements. This is an expansion upon the existing HTMLVideoElement API that only allows for an HTMLVideoElement to be put into a PiP window. This allows web developers to provide a better PiP experience to users.

Blink component

Blink>Media>PictureInPicture

TAG review

https://github.com/w3ctag/design-reviews/issues/798

TAG review status

Issues open

Risks

Interoperability and Compatibility

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/670)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/41) no official position, but have some concerns

Web developers: Positive (https://discourse.wicg.io/t/proposal-document-picture-in-picture/5736/2?u=steimel) In addition to the linked comment, we have some signals from partners that this would be valuable, and are checking to see if they are OK making that public

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

N/A since we are not enabling this API on Android

Debuggability

The new methods and properties proposed in this spec will show up in autocomplete functionality (e.g. window.documentPictureInPicture). The enter event will support event listener breakpoints in the "Picture-in-Picture" category.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

At least initially, we're not supporting Android since the Android APIs don't easily support always-on-top windows with arbitrary interaction

On Tue, Jun 13, 2023 at 7:25 PM 'Tommy Steimel' via blink-dev <[email protected]> wrote:

In the future, we may be able to do something on Android that doesn't support input without any change to Android APIs. If we wanted input in the PiP window, it would require changes to Android itself. That said, we haven't seen any interest from web developers for an inputless document PiP on Android, and there isn't really anything you can do with an inputless document PiP that you can't do with a canvas-back video PiP, so we haven't pursued anything on Android.

On Tue, Jun 13, 2023 at 7:06 AM Philip Jägenstedt <[email protected]> wrote:

Thanks Domenic for mentoring and vouching :) I took a very cursory look at the API shape and sent a small fix, thanks for reviewing that.

The concerns on https://github.com/WebKit/standards-positions/issues/41 are device independence and portability. For the Android support, can you say anything about future expectations? Is this feature simply not important on mobile and it might be a desktop-only API for the foreseeable future, or would Android support make sense for some use cases? If so, does it seem tractable given the requisite changes to Android APIs?

On Tue, Jun 13, 2023 at 4:16 AM Domenic Denicola <[email protected]> wrote:

I was the spec mentor for Tommy's work on this feature, and am chiming in to provide the requested spec maturity summary.

I'm very happy with the spec work for this feature. Tommy worked hard to incorporate all my feedback about how to ensure everything was rigorously defined, and integrated well with existing concepts like opening new windows (navigables). The monkey patches are small, focused, and easy to understand.

There is some TAG feedback that seems to wish this was part of window.open() or some other more-general API, but I think that advice is not correct, and the current API design is good, due to the singleton-per-top-level-traversable nature of a document PiP window. In my opinion this makes the window.documentPictureInPicture entrypoint, with its requestWindow(), window, and onenter properties, a good API for the use case.

On Tue, Jun 13, 2023 at 1:51 AM 'Tommy Steimel' via blink-dev <[email protected]> wrote:

Contact emails

[email protected], [email protected], [email protected]

Explainer

https://github.com/WICG/document-picture-in-picture/blob/main/README.md

The examples use the unload event for the PiP document window where, where elsewhere we're working on getting rid of it..

Is there an alternative API we can recommend instead?