Let’s start with an assertion: Web performance is important. You agree with that and that’s why you’re here. Both real user monitoring and synthetic web performance tooling offer ways to shed light on how your website is performing for your users. Which to use? Easy, they offer different views and are both important. A simple way to differentiate is tools measure and evaluate, and monitoring tracks trends and deviations. If I may insert a pedantic opinion here, the former is best suited to build time and the latter for your production website.

What is real user monitoring

Real user monitoring (RUM) is a mechanism for monitoring

  • aspects of your site’s functionality
  • aspects of user behaviour on your site

The mechanism is usually implemented by adding a script to your website that read certain data and sends them to servers where you can sees summaries and aggregations of the data either live or near real time.

A simple example would be something like the following.

  <script>
    window.addEventListener('load', async(event) => {
      const time = window.performance.timing
      const pageloadtime = time.loadEventStart - time.navigationStart;
      fetch('/my-rum-endpoint', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json'
        },
        body: JSON.stringify({pageloadtime})
      });
    
    });
  </script>

Caveats about browser support aside, this example uses Browser navigation timing to calculate pageloadtime and send it off somewhere to be stored for later viewing. Something like this plug screenshot for our monitoring solution :)

Page load over time image

As you can see, real data from real users.

What are synthetic web performance tools

Synthetic web performance tools are tools that we run (or have run for us by a service, which as we’ll see later begins to bridge the gap towards RUM) against a url which measure things like, for example, page load time, and evaluate things like, for example, whether your html is following accessibility best practices.

A great and simple example is Chrome’s own Lighthouse which can be accessed via devtools Chrome Lighthouse image

The important differentiator from real user monitoring is that you (or a proxy of you) as the engineer/company have instigated the run of test. This generally means the environment is in some way controlled or at least understood. Hence the earlier opinion about build time vs production.

When, where, how?!

There are of course some stark differences. For example, tooling cannot detect a user bouncing, and RUM cannot detect a 2MB new JPEG on the page before deploying to production.

Those differences sort of make sense. It is the muddy middle ground that can be a little confusing. Let’s clear that up. At first glance both solutions have an overlap in detected metrics. Both are capable of detecting among other things various page load timings, errors, and ajax request information.

A natural tendency is to do the simplest (and cheapest) thing: run a tool from your laptop. This is the correct thing to do. It is a fast feedback loop. It is cheap. It is repeatable. Run this once a week from your laptop? You’ve got the right idea. Add this to your build and fail when something is wrong? Now you’re winning.

Another step is synthetic tooling as a service, something like the incredible WebPageTest. You could set this up, or pay someone, to run against (likely) your production website. These services have the same benefits of feedback and repeatability, and above that they allow such tests to be run from various real geographic locations, devices, and connections. Services also allow us to start tracking trends. Now we have even more data and more insight into the behaviour of our website. It starts to become clear how this is a middle ground between local tolling and RUM.

The above 2 scenarios are individually and collectively extremely powerful to have in your arsenal. To be fair if you have one or both of those and no RUM solution in place, you’re doing very well already.

So why put a Real User Monitoring solution in place? So far we know our website works in a controlled manner. We’ve decided which pages to test, where to test them from, and which devices to test them with.

We’re not yet able to honestly and positively answer the very simple question “Is our website working for your all your users right now?”.

Server metrics can get us a good part of the way there - a surge in 500’s on an api endpoint is a very strong call to action! The truth however is that user experience is greater than the sum of the constituent parts that make up your website.

To answer the holistic question, a RUM solution gives us:

  • Coverage - This is by far the largest consideration. Our users represent a much larger combinatorial space of location/pages/devices/behaviour than your tests
  • Immediacy - RUM data (or beacons) are sent from user browsers as they access the site. Synthetics test run on a timetable, meaning there are gaps in visibility.
  • Visibility of external forces the internet is a moving target - 3rd party resources and dependencies can change without notice.

I hope this has helped clarify how each of local tooling, tooling as a service, and RUM gives us different pieces of a beautiful and often complex puzzle.

If you enjoyed this, perhaps you’ll enjoy Part 1 of our series Diagnosing Website Issues

Tags: #frontend #tools