April 15, 2021

How does real user monitoring work?

4 minutes read

Before answering the question “How does real user monitoring work?” let’s first make sure we have a baseline understanding.

Real User Monitoring and Synthetic Tooling

Broadly speaking, when monitoring or measuring websites there are two types of approaches to take:

  • Synthetic Tools
  • Real User Monitoring

Synthetic tools will programatically measure aspects of your site. These tools can be set up to run ad-hoc or ongoing depending on your requirements. Real User Monitoring is a mechanism to track the behaviour of, well, real users on your website.

For a deeper look into the differences between the two and the benefits of both approaches, take a look at Real user monitoring vs synthetic web performance tools

Beacons

In order to gain insight into real user behaviour we need some way of recording the steps they have taken on our website. This might be

  • pages they have visited
  • time spent on a task (e.g reading or scrolling through a blog)
  • interactions on a page (e.g using filters)
  • conversions (e.g a purchase)
  • bounce/leaving (implicitly recorded when no new information is recorded)

They are often sent with contextual information about the user’s visit. For example:

  • Browser type
  • Location / IP address
  • Language preference
  • Page load times

To do this recording we send “beacons” from the users' browser to a server that stores the information. In web terms, beacons are small pieces of information sent back as network requests to servers.

History of beacons

Now, you may hear about “beacons” or “tags” when discussing monitoring or tracking users and these two terms are often used interchangeably. That’s ok. Possibly not strictly correct these days, but fine to use. Historically these beacons we’re actually implemented in the form of HTML image <img> tags, and instead of returning an actual image, the server would return a tiny invisible 1x1 pixel image. This way a user’s visit to a page could be recorded (when the server received the img request) and the user experience would be only marginally affected (1 extra network request). Contextual information was either sent implicitly via request headers or explicitly via query params. So a request might look something like

https://myserver.somewhere/beaconImage.gif?firstVisit=true

with headers

...

Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: SESSION_ID=fooBar&someValuableInformation=livesHere
User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Mobile Safari/537.36
...

While modern techniques build and send the beacon using javascript, the underlying network request is the same. Using javascript just allows us to collect richer information to send back to our servers.

##The power of a picture From the example above, we can see that cookie is used to store a SESSION_ID. Essentially this means, when storing the information as it is received on the server, we are able to group the user’s actions and build a complete picture of their activity on the website. Powerful stuff. When this information is aggregated it means we can start to see how users tend to interact with our sites

  • What gets used a lot
  • What is not so popular
  • Where they leave
  • Blockers / Things that make them leave

Installation

As already mentioned, these beacons are generally sent using javascript. We use the term “installing the beacon” and what is really meant is adding a small piece of javascript to your website. The complexity of this javascript depends on the richness of information you want to collect as well as the accuracy with which you would like to collect it. Often specialist 3rd party solutions are used (yes BytesMatter is one such solution) because of their focused knowledge in the sector. Google analytics is probably the most common and widely used example.

In general the steps are (if you’re not implementing your own solution):

  • Do your research to figure out what information you’re wanting
  • sign up with 3rd party provider
  • copy and paste a piece of javascript they supply into all your HTML pages
  • automatically start collecting some super valuable information.

Thanks for reading. In terms of our BytesMatter solution, one interesting aspect we focus on is the thresholds of various page load timings at which users start to leave/bounce purely for performance reasons. Give us a try - 3 weeks zero obligation free trial! We think you’ll like what you see!

Tags: #beacons #frontend #tags