Replay Server

TechDocs

System Design

Overall, the purpose of this system is to receive GET requests being fetched from the browser for replay playback and go load those assets. Applying the above rules where appropriate in the system. One thing we need to account for is taking relative URL's, and ensuring they're transformed into fully qualified URL's. Additionally, we need "nested resources" to also reshape their requests for additional resources back through this replay resource loader. This of this resource-requesting sequence like a recursive function. If a CSS file has an @import statement in it, we want those css resources and any resource (like background-image declarations) to also be pointed to load through the RRL. To accomplish this, all URL's need 3 minium query-params:

path - This is the resource to load. It can be a fully qualified URL, or a relative path which may have come from an attribute like src="/static/things.css"
base - This is the base URL for the location the resource is being requested from. Could be the URL of a page, or the URL of a stylesheet if handling an @import statement
root - This is the URL location of this system. For recursive calls we need to load that nested resource from this system. Say you're handling an @imported style sheet. Where do you have it's CSS contents go load their fonts/images/svg's/etc? You need to point back to this system's /resource url, with the new base & path. The root param is unlikely to change given the lifecycle of recursively handling resource loading as it's always the same RRL system in play.

Replay Resource Loader (RRL)

Replays have HTML documents that need to load external resources. For example:

CSS
Fonts
Images
SVG's

We want replay to be able to fully control the loading aspect of these resources for some of the following reasons:

Cache control
URL Mapping (point a resource to an alternative destination such as a QM sponsored proxy/repository/etc)
Blocking (we don't want certain content types to be loaded within replay)
CSS needs specific handling, we'll cover more in detail below as it's has the most complex use cases
Load resources from specific IP's (proxies) for networking reasons

Replay Instrumentation Loader (RIL)

Replays are described by a set of "Hits" (pages in the user's journey) and "Events" that occur on each page. Each of these "Hits" and "Events" load up resources for the resources to reference.

Instrumentation is the combination of these "Page Hits" and their child "Events". Running through them is really all the Replay does.

Instrumentation accomplishes the following tasks

Combine the "Session Details" with the "Session Hits" and all their "Events" and deliver them in one payload to the UI
Return only the requested amount of data based on the "sessionAfter" parameter passed

Session Data manager

As time goes on and we want to leverage the data within the session, we'll want to be able to validate a Session and get some basic "summary" data out of the session for future use. We'll want to accomplish all this in the server as GoLang is much faster at this than NodeJS

Serve local json files

Download session json file either from the bucket or via replay-cli
Copy session json to /downloadedSessions/[sessionid].json
- You can control which directory this writes to with the field: downloadedSessionsDirectory in .env file if you have that setup

Export to CSV

If you download the replay first, you can then export the file into a csv file

replay-cli download --session #SESSION# --sub #SUB# -o test.json
- Make sure your output file here matches the input file in the next line.
replay-cli export-events --input-file test.json --export-mode summary -o test.csv
- If you don't add a format, it defaults to csv
- If you don't add an extension to the filename, we append .csv to it

Cool tricks

Want to download AND format a replay?
1. (Ensure you have jq installed)
2. Run replay-cli download --session #SESSION# --sub #SUB# | jq '.' > formatted-file.json
3. (jq is hella fast, and OOTB will format the JSON, piped then to a file called formatted-file.json)
Another example:
1. replay-cli download --replay-url "https://subname.quantummetric.com/\#/replay/4686292\?ts\=1654067479-1654153879" | jq '.' > some-file.json (basically in the replay-url field, just paste the raw URL from the UI)