NicJ.net

Beaconing in Practice: An Update on Reliability and the Pending Beacon API

Nic

Table of Contents

  1. Introduction
  2. Pending Beacon API
  3. Why Pending Beacons?
  4. Pending Beacon Experiments
    1. Methodology
    2. Reliability of XMLHttpRequest vs. sendBeacon() vs. Pending Beacon in Event Handlers
      1. onload
      2. pagehide or visibilitychange
      3. onload or pagehide or visibilitychange
      4. Conclusion
    3. Reliability of Pending Beacon "now" vs "backgroundTimeout"
    4. Reliability of Pending Beacon "backgroundTimeout" once vs. sendBeacon() in Event Handlers
  5. Misc Findings
  6. Follow-Ups
  7. TL;DR

Introduction

A few years ago, I wrote an article titled Beaconing In Practice that covered all of the aspects of sending telemetry from your web app to a back-end server for analysis (aka "beaconing").

While the contents of that article are still relatively fresh and accurate, there are two new aspects of beaconing that I would like to cover in this post:

Pending Beacon API

The Pending Beacon API is an exciting new proposal from Google Chrome engineers.

The goal is to provide an API for developers where they can "queue" data to be sent when a page is being unloaded (by the browser, automatically), rather than requiring developers to explicitly send beacons themselves in events like pagehide or visibilitychange (which don’t always fire reliably).

It is meant to be similar to the navigator.sendBeacon() API, with a simple calling style.

Here’s an example of using the Pending Beacon API to send a beacon when the page is being hidden/unloaded (or a maximum of 60 seconds after "now"):

// queue a beacon for the unloading or +60s
var beacon = new window.PendingGetBeacon(
    beaconUrl,
    {
        timeout: 60000, 
        backgroundTimeout: 0 
    });

(note the above API shape is outdated and the Pending Beacon API will utilize fetch() in future versions)

The API is still being discussed, and is actively evolving based on community and browser vendor feedback.

If you want to experiment with the Pending Beacon in Chrome today, you can register for an Origin Trial for Chrome 107-115. Though, again, note that the current API shape (with the window.PendingGetBeacon and window.PendingPostBeacon interfaces) is evolving towards being an option of fetch() instead.

Why Pending Beacons?

One of the challenges highlighted in the Beaconing In Practice article is how to reliably send data once it’s been gathered in a web app.

Developers frequently use events such as beforeunload/unload or pagehide/visibilitychange as a trigger for beaconing their data, but these events are not reliably fired on all platforms. If the events don’t fire, the beacons don’t get sent.

For example, if you want to gather all of your data and only send it once as the page is unloading, registering for all 4 of those events will only give you ~82.9% reliability in ensuring the data arrives at your server, even when using the sendBeacon() API.

So, wouldn’t it be lovely if developers had a more reliable way of "queuing" data to be sent, and have the browser automagically send it once the page starts to unload? That’s where the Pending Beacon API comes in.

The Pending Beacon API gives developers a way to build a "pending" beacon. That pending beacon can then be mutated over time, or later discarded. The browser will then handle sending it (in its latest state) when the page is being hidden or unloading, so developers no longer need to listen to the beforeunload/unload/pagehide/visibilitychange events.

Ideally, Pending Beacon will be a mechanism that can replace usage of sendBeacon() in browsers that support it, giving more reliable delivery of beacon data and better developer ergonomics (by not having to listen for, and send data during, unload-ish events).

Pending Beacon Experiments

Given those goals, I was curious to see how reliable Pending Beacon would be compared to existing APIs like XMLHttpRequest (XHRs) or the sendBeacon() API. I performed three experiments comparing how reliably data arrived after using one of those APIs in different scenarios.

Let’s explore three questions:

  1. Can we swap PendingBeacon in for usage of XHR and/or sendBeacon() in unload event handlers?
  2. How reliable is asking PendingBeacon to send data "now" vs with a backgroundTimeout?
  3. How reliable is queuing PendingBeacon data to be sent at page unload vs. listening to event handlers and using sendBeacon() in them?

Methodology

Over the course of a month, on a site that I control (with approx 2M page views), I ran an experiment gathering data from browsers using the following three APIs:

All of these APIs sent a small GET request back to the same domain / origin.

For all of the data below, I am only looking at Chrome and Chrome Mobile v107-115 (per the User-Agent string) with support for window.PendingGetBeacon, to ensure a level playing field. The data in Beaconing In Practice looks at reliability across all User-Agents, but the experiments below will focus solely on browsers supporting the Pending Beacon API.

Note that all of these tests were done with the PendingGetBeacon interface, before the current proposal to have this be a fetch() option. I’m unsure how the most recent proposal will affect these results, but I will re-do the test once that fetch() update is available.

Reliability of XMLHttpRequest vs. sendBeacon() vs. Pending Beacon in Event Handlers

The first question I wanted to know was: Can Pending Beacon be easily swapped into existing analytics libraries (like boomerang.js) to replace sendBeacon() and XMLHttpRequest (XHR) usage, and retain the same (or better) reliability (beacon received rate)?

In boomerang for example, we listen to beforeunload and pagehide to send our final "unload" beacon. Can we just use Pending Beacon instead?

For this experiment, I segmented visitors into 3 equally-distributed A/B/C groups (given Pending Beacon API support):

Each group then attempted to send 6 beacons per page load:

  1. Immediately in the <head> of the HTML
  2. In the page onload event
  3. In the page beforeunload event
  4. In the page unload event
  5. In the page pagehide event
  6. In the page visibilitychange event (for hidden)

By seeing how often each of those beacons arrived, we can consider the reliability of each API, during different page lifecycle events. I’m only showing data for page loads where the first step (sending data immediately in the <head>) occurred.

Let’s break the experimental data down by event first:

onload

The onload event is probably the most common event for an analytics library to fire a beacon. Marketing and performance analytics tools will often send their main payload at that point in time.

Based on our experimentation, when firing a beacon just at the onload event, sendBeacon() seems slightly more reliable than XHR, which is slightly more reliable than PendingGetBeacon.

sendBeacon() being more reliable than XHR is expected — the whole point of sendBeacon() is to allow the browser to send data asynchronously of the page, in case it unloads after the beacon is queued up.

However, I’m surprised that PendingGetBeacon appears to be the least reliable (by about 1% less than XHR), at least from my experiments.

Broken down by Desktop and Mobile:

Desktop is able to deliver beacons more reliably across all 3 APIs than mobile. On mobile, PendingGetBeacon is about 2.8% less reliable than sendBeacon().

Note: that the above results are for only measuring a beacon sent immediately during the page’s onload event, without accounting for any abandons that happen prior to onload. That is why these numbers are so low — if a user abandoned the page prior to the onload event, they would not be counted in the above chart. See the additional breakdowns below for how these numbers change if you use the suggested abandonment strategy of listening to onload, pagehide and visibilitychange.

I was hoping the Pending Beacon API would be at-least-or-better reliable than sendBeacon(), so I think there’s something to investigate here.

pagehide or visibilitychange

If the intent is to measure events that occur in the page beyond the onload event, i.e. additional performance or reliability metrics (such as Core Web Vitals or JavaScript errors), tools can send a beacon during one of the page’s unload events, such as beforeunload, unload, pagehide or visibilitychange.

Our recommended strategy is to listen to just pagehide and visibilitychange (for hidden), and not listen to the beforeunload or unload events (which are less reliable and can break BFCache navigations).

So let’s look at the result of sending a beacon immediately during a pagehide or visibilitychange event (if a beacon was received for either event):

This is showing that sendBeacon() is still reigning supreme for reliability (95.8%), with PendingGetBeacon slightly behind (89.1%) and XHR trailing that (84.9%).

However, when we break it down by Desktop:

PendingGetBeacon is nearly as reliable as sendBeacon(), with XHR trailing behind, while on Mobile:

There appears to be a huge drop-off in reliability for PendingGetBeacon on Mobile vs. Desktop.

Possibly a bug with Pending Beacon in Chrome’s initial implementation here? This data would give me pause in swapping to Pending Beacon right now.

onload or pagehide or visibilitychange

Finally, let’s combine the above three events per the suggested abandonment strategy, and see how reliable each API is if we’re listening for all 3 events (and sending data once in any of them).

Of course, this increases the reliability of receiving beacons to the maximum possible, with sendBeacon() able to get a beacon to the server 98% of the time:

Broken down by Desktop vs. Mobile, we see that Desktop is has an extremely high rate of receiving beacons:

While Mobile continues to show a possible issue with PendingGetBeacon vs. sendBeacon() (a 7.7% drop-off)!

Conclusion

From this experiment at least, it appears sendBeacon() continues to be the most reliable way of sending beacon data.

If sending data during onload, sendBeacon() is slightly more reliable than PendingGetBeacon.

However, there appears to be a bug with PendingGetBeacon during a page-unloading scenario like pagehide or visibilitychange, particularly on Mobile. If the Chrome engineers can figure out a way to increase the reliability there, I would expect the Pending Beacon API to be equivalent to using sendBeacon() (which is our preferred mechanism today).

NOTE: I measured the reliability of sending beacons during beforeunload and unload as well, but since those events are deprecated / not-recommended / unreliable / break BFCache events, I’ll skip those results in this post.

Reliability of Pending Beacon "now" vs "backgroundTimeout"

The next experiment I ran was to determine if the backgroundTimeout functionality of the Pending Beacon API was reliable to use.

Here’s the description of the parameter (which has changed slightly with the fetch()-based proposal, but I would guess would operate similarly):

In other words, ask the browser to send a beacon after backgroundTimeout milliseconds of being hidden.

This can be very useful as an alternative to listening to the pagehide / visibilitychange events for beaconing your "last" bits of data. If you regularly update your Pending Beacon, you may not need to listen to those event at all.

But can we trust the browser to still send our Pending Beacon, after we’ve queued it up?

For this experiment, I segmented visitors into 2 equally-distributed A/B groups (given Pending Beacon API support):

We are considering doing something similar to group B for boomerang.js, i.e. send all beacons within 60 seconds of the page load (so the data is still "real-time fresh" in dashboards), and asking the browser to send the data immediately if the user navigates away or closes the browser before then.

Let’s look at the results of using PendingGetBeacon to send a beacon "now" vs. "when the page is hidden/unloads":

Given a baseline of 100% meaning we received a "now" PendingGetBeacon, we’re seeing the 60s timeout + @hidden beacon about 98.5% of the time across Desktop and Mobile.

Desktop is slightly more reliable (99.7%) vs. Mobile (96.4%).

I think this is a great result, confirming the value-add of PendingGetBeacon. Instead of having to add event listeners for pagehide visibilitychange and a 60s setTimeout(), the browser delivered the beacon very reliably on its own!

Remember, listening to all 4 unload-ish events (beforeunload, unload, pagehide, visibilitychange) and sending a beacon in those events only resulted in ~82.9% reliability!

And in the meantime, the pending beacon could be manipulated to add/remove additional data up until the page unloads.

Reliability of Pending Beacon "backgroundTimeout" once vs. sendBeacon() in Event Handlers

Given that the last experiment showed that Pending Beacon with backgroundTimeout was very reliable in sending beacons at page unload, what is the difference between using PendingGetBeacon with backgroundTimeout: 0 vs. listening for pagehide and visibilitychange and sending a beacon with sendBeacon()?

Great news! Not only is the PendingGetBeacon more ergonomic (not having to listen for pagehide and visibilitychange events), it’s more reliably sending data when the page is unloading.

One interesting result I see here, is that the PendingGetBeacon reliability with backgroundTimeout: 0 was more reliable than listening to pagehide and visibilitychange and using PendingGetBeacon (now) in those events directly. This is likely due to the fact that pagehide and visibilitychange aren’t 100% reliable in the first place, but I would hope for it to be as-close-to sendBeacon() reliable as possible.

Misc Findings

Follow-Ups

First, I want to say that my experiments and conclusions probably have some flaws. I’ve reviewed and re-reviewed my methodology and queries several times, but I am a human (I think!) and make mistakes. I’m hoping others can review this data.

Given that, some follow-ups I plan on doing based on the above findings:

TL;DR

Share this: