Suggestions: ticking during sleep, comparison across contexts, time origin + now semantics, and skew definition

[jonathanmayer]

Hi all,

My research group at Princeton has been collaborating with Mozilla on a project that involves precise monotonic cross-context timing. We've encountered a recurring browser- and platform-specific implementation issue, where a context's High Resolution Time monotonic clock does not tick during sleep: mdn/content#4713. Contributors to the standard previously discussed a version of this issue in Chrome and reached consensus that the clock should tick during sleep: #65.

I would like to suggest several related revisions to the High Resolution Time standard for the Web Performance Working Group's consideration.

First, the language that closed #65 (PR #69) does not expressly address what should happen during sleep.

    <p class="note">In certain scenarios (e.g. when a tab is backgrounded), the
    user agent may choose to throttle timers and periodic callbacks run in that
    context or even freeze them entirely. Any such throttling should not affect
    the resolution or accuracy of the time returned by the monotonic clock.</p>

Addressing the scenario would be valuable for implementers of the API, since operating systems typically offer separate monotonic clocks APIs that do and don't include sleep. Addressing the scenario would also be valuable for users of the API, who could better understand the expected behavior and document inconsistent behavior. Here's an example revision.

    <p class="note">In certain scenarios (e.g., when a tab is backgrounded, 
    the thread or process for a tab is sleeping, or the system has entered a 
    suspended state), the user agent may choose to throttle timers and 
    periodic callbacks run in a context or even freeze them entirely. Any 
    such throttling or freezing should not affect the resolution or accuracy 
    of the time returned by the monotonic clock.</p>

Second, I would suggest upgrading the informative note on this issue to a normative requirement. If the monotonic clock in a context depends on past sleep for that context, then the following use case cannot be reliably supported in real-world conditions.

      <p>This specification defines a few different capabilities: it provides
        timestamps based on a stable, monotonic clock, comparable across
        contexts, with potential sub-millisecond resolution.</p>

      ...

      <p>Comparing timestamps between contexts is essential e.g. when
        synchronizing work between a {{Worker}} and the main thread or when
        instrumenting such work in order to create a unified view of the event
        timeline.</p>

If a High Resolution Time timestamp can vary by a context's sleep history, then timestamps are not comparable across contexts in real-world scenarios (where thread, process, and system sleep are frequent and unpredictable). Similarly, the example provided in Section 1.2 (Example 2) is not reliable in real-world scenarios. A stronger version of this normative requirement would be to migrate the per-context monotonic clocks entirely to the shared monotonic clock (i.e., the same clock ticking in all contexts, just with a per-context time origin). That might not have been realistic with OS capabilities ~6 years ago, but it might be realistic now. Some relevant context that I found helpful: #22 and #29.

Third, I would suggest adding semantics for performance.timeOrigin + performance.now() and how that compares to Date.now() in the explanation of monotonic clocks (Section 6). There are normative requirements that DOMHighResTimeStamp count in milliseconds, that the reference point for the shared monotonic clock is ordinary ECMAScript time, and that a context's time origin be set with the shared monotonic clock. The only way I see to satisfy these normative requirements is if performance.timeOrigin + performance.now() and Date.now() both represent the current time, and should only differ because of either 1) user or automated adjustments to the system clock after the shared monotonic clock starts ticking, or 2) differences in underlying clock implementation (e.g., there might be different hardware clocks with slightly different frequencies). There are definitely use cases for those performance.timeOrigin + performance.now() semantics, where the goal is to timestamp events in a way that approximates real-world time, is monotonic, ideally is comparable across contexts (the prior item), and is not subject to user or operating system clock adjustments, assuming that the system clock was approximately correct when the shared monotonic timer started ticking. I recognize there have been prior discussions about how High Resolution Time relates to Date.now(), with a resolution that the two APIs shouldn't have their values compared (e.g., #27). To be clear, I'm not suggesting a quantitative guarantee that the values provided by the two APIs will be similar, but rather, a semantic definition of how the two APIs relate and the specific conditions that can cause divergence over time.

Fourth, I would suggest defining what the term "skew" means in the document, and perhaps using a different term instead. The notion, if I understand correctly from discussion in prior issues, is that NTP and similar automatic adjustments for clock errors should not change (skew) the monotonic clocks. Unfortunately, skew also typically means a difference between two clocks (which NTP tries to mitigate). So... depending on the meaning of "skew," the normative requirement that monotonic clocks not have "system clock skew" either means they must not change when the system clock changes, or they must change when the system clock changes. I would also suggest clarifying that monotonic adjustments to a monotonic clock are permissible (e.g., monotonic NTP corrections for oscillator frequency or adjtime corrections, see CLOCK_MONOTONIC vs. CLOCK_MONOTONIC_RAW in clock_gettime); the current text could be interpreted to prohibit any adjustments.

[npm1]

Hi, thanks for the feedback!

First, the language that closed #65 (PR #69) does not expressly address what should happen during sleep.

Your suggestion is just to add more examples of 'scenarios' to explicitly call out sleep, is that right? That seems good to me. I also noticed that the note uses 'should', which is not allowed @yoavweiss. Which is related to the second point below...

Second, I would suggest upgrading the informative note on this issue to a normative requirement. If the monotonic clock in a context depends on past sleep for that context, then the following use case cannot be reliably supported in real-world conditions.

That seems fine, although AFAIK implementations haven't changed to behave properly during sleep. I guess this is a good reminder to do so :) I thought we had a bug for this but I couldn't find it so I filed https://bugs.chromium.org/p/chromium/issues/detail?id=1206450 instead, just so we don't keep forgetting. I don't think this will be a trivial change though.

Third, I would suggest adding semantics for performance.timeOrigin + performance.now() and how that compares to Date.now() in the explanation of monotonic clocks (Section 6).

Date.now() would jump, potentially backwards, if the user changes the system clock, right? That is one example where they would vastly differ.

Fourth, I would suggest defining what the term "skew" means in the document, and perhaps using a different term instead.

So if I understand correctly, you're saying clock skew is a well defined term and our usage in the spec does not really use it properly? My guess this is indeed trying to say that there should be no adjustments, and perhaps it should say that even monotonic clocks are adjusted but with certain conditions met (like monotonicity is preserved and adjustment sizes are very small).

[jonathanmayer]

Hi, thanks for the feedback!

Thank you for the valuable spec!

First, the language that closed #65 (PR #69) does not expressly address what should happen during sleep.

Your suggestion is just to add more examples of 'scenarios' to explicitly call out sleep, is that right? That seems good to me.

Yep.

I also noticed that the note uses 'should', which is not allowed @yoavweiss. Which is related to the second point below...

Good point. In the prior discussions I've found about High Resolution Time monotonic clocks and sleep, there's been some ambiguity about whether ticking during sleep is normative or informative. A should in an informative note probably contributes to that ambiguity.

Second, I would suggest upgrading the informative note on this issue to a normative requirement. If the monotonic clock in a context depends on past sleep for that context, then the following use case cannot be reliably supported in real-world conditions.

That seems fine, although AFAIK implementations haven't changed to behave properly during sleep.

I did some digging into the Chrome, Firefox, and Safari codebases to help MDN document implementations: mdn/content#4713. Here's a summary table of what I think current browser behaviors are, where ✅ means High Resolution Time clocks tick during sleep and 🚫 means they don't.

	Chrome/Chromium	Firefox	Safari/WebKit
Windows	✅	✅	N/A
macOS	🚫	🚫	🚫
Linux	🚫	🚫	🚫 (except maybe WebKitGTK?)
Android	🚫	🚫	N/A
iOS	N/A (WebKit wrapper)	N/A (WebKit wrapper)	🚫

I guess this is a good reminder to do so :) I thought we had a bug for this but I couldn't find it so I filed https://bugs.chromium.org/p/chromium/issues/detail?id=1206450 instead, just so we don't keep forgetting. I don't think this will be a trivial change though.

I found a couple relevant Chrome bugs, though neither is quite framed as a spec conformance issue.

• https://bugs.chromium.org/p/chromium/issues/detail?id=166153
• https://bugs.chromium.org/p/chromium/issues/detail?id=948384

A proposal associated with the earlier bug was especially helpful in understanding Chrome's behavior and what would be required for changes: https://docs.google.com/document/d/1Ll-8Nvo6JlhzKEttst8GHWCc7_A8Hluy2fX99cy4Sfg/edit

I do think there's a small bug in the proposal, though... confusingly, clock_gettime with CLOCK_MONOTONIC on macOS ticks during sleep, but the exact same call on Linux does not tick during sleep.

Third, I would suggest adding semantics for performance.timeOrigin + performance.now() and how that compares to Date.now() in the explanation of monotonic clocks (Section 6).

Date.now() would jump, potentially backwards, if the user changes the system clock, right? That is one example where they would vastly differ.

Definitely. Let me take another try at formalizing normative semantics for performance.timeOrigin + performance.now() and Date.now().

• At the moment that the shared monotonic clock starts ticking, performance.timeOrigin + performance.now() and Date.now() are equal. This is already a normative requirement in the spec.
• After the shared monotonic clock starts ticking, Date.now() can change to any value at any time (e.g., the user changes the clock or an NTP time sync occurs). This is already in the ECMAScript definition of Date.now().
• The shared monotonic clock and per-context monotonic clocks may only change if there is a small or gradual monotonic clock adjustment (e.g., NTP oscillator adjustments or slewing). This is just a clarification of the normative requirements for monotonic clocks that are already in the spec. The current text does not clearly address these types of adjustments, and it could be interpreted to preclude current major implementations of the spec (all of which include these types of adjustments).
• The shared monotonic clock and per-context monotonic clocks must not freeze during sleep. This is a clarification of the current informative note on clock freezing, plus an upgrade from informative to normative.
• The shared monotonic clock and per-context monotonic clocks must tick at the same frequency. The spec introduction, which is informative rather than normative, states a goal of allowing comparison and synchronization of timestamps across contexts. That goal is (implicitly) equivalent to saying the monotonic clocks must tick at the same frequency. This change would make explicit what's already implicit in the spec, plus upgrade the provision from informative to normative.

There are several important properties that follow from these semantics.

• At any given moment, performance.timeOrigin + performance.now() has the same value in all contexts (before fuzzing for security and privacy reasons). That's a very useful normative property for API users, and a normative realization of what are currently informative goals and an informative example in the spec.
• If Date.now() is approximately correct at the moment that the shared monotonic clock starts ticking, then performance.timeOrigin + performance.now() produces timestamps that approximate real-world time, are monotonic, are comparable across contexts, and are not subject to large or non-monotonic clock adjustments (including user clock adjustments). That's also a useful normative property for certain use cases.
• After the shared monotonic clock starts ticking, differences between performance.timeOrigin + performance.now() and Date.now() should only be attributable to 1) adjustments to the system clock that are large or non-monotonic, or 2) differences in underlying clock implementations (e.g., different hardware clocks for non-monotonic and monotonic time). That property isn't helpful for API users—it means all bets are off when comparing performance.timeOrigin + performance.now() and Date.now(). That property might, however, be valuable for API implementers—who could use it to choose among platform-specific clock APIs with subtle differences in how they treat monotonicity, sleep, and various adjustments.

Fourth, I would suggest defining what the term "skew" means in the document, and perhaps using a different term instead.

So if I understand correctly, you're saying clock skew is a well defined term and our usage in the spec does not really use it properly?

The current usage of the term "clock skew" is not consistent with the primary ways that term is used in computer science or electrical/computer engineering. Unfortunately, there's also inconsistency in those fields... sometimes clock skew means the difference in time values between two clocks (that's probably the most common meaning), sometimes clock skew means the difference in frequencies between two clocks (that's how NTP version 3 / RFC 1305 defined the term), and sometimes clock skew means the difference between when clock signals arrive (in circuit design). Looking over the spec again, I would suggest just replacing the term.

My guess this is indeed trying to say that there should be no adjustments, and perhaps it should say that even monotonic clocks are adjusted but with certain conditions met (like monotonicity is preserved and adjustment sizes are very small).

That sounds good. I don't believe Linux currently offers a monotonic clock that both ticks during sleep and is not subject to small or gradual monotonic adjustments, so it would be helpful to clarify that those types of adjustments are permitted.

[jonathanmayer]

Since the High Resolution Time API is essentially an abstraction over operating system and hardware clocks, I looked further into what's currently possible on widely deployed platforms. Short version: there is now near-universal support for implementing the semantics above, which wasn't the case when the API was initially designed. That might be a reason to make the semantics above—which are already implicit in the spec—explicit and normative.

Long version: here's what I've learned, with the caveats that this isn't an area of expertise and this is a deep rabbit hole.

Hardware

• On x86, the invariant Time Stamp Counter (TSC) provides a clock with the following properties:
  • monotonic (except reset to 0 on CPU reset and certain types of sleep),
  • runs at a fixed frequency (i.e., is unaffected by changes in CPU core frequency),
  • synchronized across CPU cores (on Intel CPUs by broadcasting a per-CPU clock to per-core clocks, AMD CPUs might do that or might do dynamic resynchronization between a per-CPU clock and per-core clock when a core reads the TSC),
  • synchronized across CPUs on a multi-socket board (by resetting the TSC on all CPUs at the same time, on CPU reset),
  • continues ticking during a range of power modes (including certain low-power/sleep modes), and
  • resets during other power modes (other low-power/sleep modes) in a way where synchronization is preserved, the operating system can apply an adjustment to preserve logical monotonicity, and the operating system can apply an adjustment to account for sleep time.

Intel has implemented invariant TSC as a standard feature since the Nehalem architecture (~2008), and AMD has supported invariant TSC as a standard feature since the K10 architecture (also ~2008).

• On ARM, the architecture is conceptually very similar, but with different terminology. There is a central System Counter that is monotonic, runs at a fixed frequency, continues ticking during all power modes, and broadcasts a synchronized count to a set of per-core timers. The overall design, dubbed Generic Timers (or sometimes Architected Timers), has been an extension for ARMv7 and required in ARMv8+ since ~2011.

Operating Systems

• Windows provides an abstraction layer over the TSC, QueryPerformanceCounter (QPC), that has been available since Windows 2000. On a system with an invariant TSC, QPC provides the attributes needed for the High Resolution Time API semantics above: a clock that is monotonic, that is synchronized across contexts (even if contexts are running on different threads, processes, cores, or CPUs), and that accounts for ticks during sleep.
• macOS on x86 provides an abstraction layer over the TSC with the attributes needed, callable as an old Apple-specific API (mach_continuous_time) or on 10.12+ as a recommended POSIX API (clock_gettime or clock_gettime_nsec_np with CLOCK_MONOTONIC or CLOCK_MONOTONIC_RAW). macOS on ARM (and other Darwin-based OSes like iOS) provide an abstraction layer over ARM Generic Timers, callable with the same APIs.
• Linux provides an abstraction layer over the TSC (on x86) and Generic Timers (on ARM) with the attributes needed, callable as clock_gettime with CLOCK_BOOTTIME. Linux also supports other hardware clock sources, but TSC and Generic Timers appear to be the defaults on popular distributions because they're so efficient, and certain of the other clock sources have the attributes needed anyway (i.e., some are single system clocks, so there's no synchronization issue).
• Android provides the same API as Linux, plus wrappers for Java and Kotlin (elapsedRealtime and elapsedRealtimeNanos). As best I can tell, the underlying implementation is just Linux.

References

• Hardware
  • x86
    • Intel
      • https://software.intel.com/content/www/us/en/develop/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4.html (latest)
      • https://patents.google.com/patent/US20110154090
    • AMD
      • https://www.amd.com/system/files/TechDocs/55898_pub.zip (Zen 3, latest)
      • https://www.amd.com/system/files/TechDocs/31116.pdf (K10)
      • https://patents.google.com/patent/US20090222683A1/en
    • https://stackoverflow.com/questions/10921210/cpu-tsc-fetch-operation-especially-in-multicore-multi-processor-environment/11060619#11060619
    • http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/
    • http://btorpey.github.io/blog/2014/02/18/clock-sources-in-linux/
  • ARM
    • https://developer.arm.com/documentation/102379/0000
    • https://developer.arm.com/documentation/ddi0406/latest/
    • https://developer.arm.com/documentation/ddi0487/latest/
    • https://wiki.osdev.org/ARMv7_Generic_Timers
• Operating Systems
  • Windows
    • https://docs.microsoft.com/en-us/windows/win32/sysinfo/acquiring-high-resolution-time-stamps
    • https://docs.microsoft.com/en-us/windows/win32/dxtecharts/game-timing-and-multicore-processors
    • https://docs.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancecounter
    • https://docs.microsoft.com/en-us/windows/win32/api/profileapi/nf-profileapi-queryperformancefrequency
  • Darwin (macOS, iOS, etc.)
    • https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time
    • https://opensource.apple.com/source/Libc/Libc-1439.40.11/gen/clock_gettime.3.auto.html
    • https://opensource.apple.com/source/Libc/Libc-1439.40.11/gen/clock_gettime.c.auto.html
  • Linux
    • https://man7.org/linux/man-pages/man2/clock_gettime.2.html
    • https://github.com/torvalds/linux/blob/master/Documentation/core-api/timekeeping.rst
    • https://github.com/torvalds/linux/blob/master/kernel/time/posix-stubs.c
    • https://github.com/torvalds/linux/blob/master/kernel/time/posix-timers.c
    • https://github.com/torvalds/linux/blob/master/include/linux/timekeeping.h
    • https://github.com/torvalds/linux/blob/master/kernel/time/timekeeping.c
    • https://github.com/torvalds/linux/blob/master/kernel/time/timekeeping_internal.h
    • https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/tsc.h
    • https://github.com/torvalds/linux/blob/master/arch/x86/kernel/tsc.c
    • https://github.com/torvalds/linux/blob/master/arch/x86/kernel/tsc_sync.c
    • https://github.com/torvalds/linux/blob/master/include/clocksource/arm_arch_timer.h
    • https://github.com/torvalds/linux/blob/master/drivers/clocksource/arm_arch_timer.c
    • https://github.com/torvalds/linux/blob/master/arch/arm/kernel/arch_timer.c
    • https://github.com/torvalds/linux/blob/master/arch/arm/kernel/arch_timer.c
    • https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/arch_timer.h
  • Android
    • https://cs.android.com/android/platform/superproject/+/master:system/core/libutils/include/utils/SystemClock.h
    • https://cs.android.com/android/platform/superproject/+/master:system/core/libutils/SystemClock.cpp
    • https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/android/os/SystemClock.java
    • https://cs.android.com/android/platform/superproject/+/master:system/core/libutils/include/utils/Timers.h
    • https://cs.android.com/android/platform/superproject/+/master:system/core/libutils/Timers.cpp

[jonathanmayer]

Thanks all for hearing me out on today's call. Here are the Chrome and Firefox performance.timeOrigin bugs that I mentioned at the end.

• https://bugs.chromium.org/p/chromium/issues/detail?id=1207386
• https://bugzilla.mozilla.org/show_bug.cgi?id=1710762

In short, both browsers depart from the spec in how they set performance.timeOrigin. The spec normatively requires using a shared monotonic clock that's synced once to the system clock (e.g., on browser startup). The browsers are instead syncing to the system clock per-global (Chrome) or per-process (Firefox). That behavior means comparing performance.timeOrigin across globals has clock change risks. Addressing these bugs is probably much easier than addressing the system sleep bugs.

There's another relevant Chrome bug that I didn't touch on during the call.

• https://bugs.chromium.org/p/chromium/issues/detail?id=948384

Chrome uses a macOS monotonic clock API that doesn't allow monotonic small/gradual adjustments (e.g., NTP oscillator adjustments or slewing). The choice of clock API appears to be causing a drift between the monotonic clock and the system clock. That behavior is probably unexpected and unwanted for API users, and it probably increases the browser fingerprinting surface. Some additional reasons that normative text about clock adjustments would be valuable.

[clelland]

@yoavweiss, is this actionable at all on the spec level?

[yoavweiss]

I think this is mostly an implementation issue. @jonathanmayer - is that correct?

[jonathanmayer]

There are specific actionable shortcomings in the specification. These problems contribute to the inconsistencies in implementations. Here's an abbreviated summary.

1. Ticking during system sleep. The specification does not clearly state the consensus about performance.now() ticking during system sleep.

Recommended Revision: State in Section 7 normative text that performance.now() should (or better, must) tick during system sleep.

2. Ticking when a context is suspended. The specification states a use case of comparing high-resolution timestamps across contexts. Achieving that property necessitates that performance.now() continue ticking regardless of whether a context is suspended (i.e., is frozen or throttled). The specification does not, however, require that property. The related note in Section 7 also violates W3C style guidance, because it uses a normative RFC 2119 term in an informative paragraph.

Recommended Revision: State in Section 7 normative text that performance.now() must tick when a context is suspended. Remove the related note.

3. Reference point for the shared monotonic clock. The specification contradicts itself. Section 7 requires setting the shared monotonic clock with the ECMA-262 system clock. Section 8, however, says that is only a recommendation.

Recommended Revision: Remove the inconsistent clause about ECMA-262 time in Section 8.

4. Clock skew. The term "clock skew" is undefined, ambiguous, inconsistent with what the term usually means (depending on the interpretation), and not testable (because it does not map to particular clock APIs offered by operating systems).

Recommended Revision: Remove the term clock skew. Define the term "clock adjustment" to mean 1) manual changes to the system clock or 2) automated changes to the system clock that are either non-monotonic or large (e.g., NTP time jumps). This definition would not include automated changes to the system clock that are monotonic and small (e.g., NTP oscillator adjustments or clock slewing).

5. Shared monotonic clock. The specification is ambiguous about when the clock is "shared" and when it isn't, with a mix of normative and informative description in Section 7. Implementations are inconsistent, in part because of this lack of clarity. The related note in Section 7 also violates W3C style guidance, for the same reason as the other note.

Recommended Revision: State in Section 7 normative text that the shared monotonic clock must be shared across all contexts, regardless of implementation (e.g., different processes or threads), and the only circumstances in which the shared monotonic clock may reset are a browser restart, private browsing mode, or different browsing profile. Revise the related note to conform to W3C style guidance.

6. Comparison between high-performance timestamps and ordinary timestamps. Many implementers are confused about how performance.timeOrigin + performance.now() relates to Date.now().

Recommended Revision: State in informative text that performance.timeOrigin + performance.now() will be similar to Date.now(), unless there is a clock adjustment.