feat: Add in-memory session persistence option

[andrewatwood]

Motivation

A not-insignificant cohort (>5%) of users for the application I work on have their cookies and local storage disabled. Their sessions are short (~15s happy path) and self-contained, and the relevant context for any given session is nearly entirely limited to that individual single-page session, so long term persistence or multi-tab/window support is wholly unnecessary. However, tracking these sessions and any issues that my arise is still a requirement.

In-memory session storage for the RUM SDK allows us to bypass the requirement for enabled cookies or local storage, and have full access to the sessions originating from these users.

This has been tested extensively, and deployed successfully in production, with over 4M sessions using this persistence strategy in the last two weeks alone.

Changes

• Adds a new session persistence strategy (SessionPersistence.IN_MEMORY) that stores session data in memory
• Updates configuration options to expose this strategy to the end user
• Refactors and rewords existing configuration unit tests related to the allowFallbackToLocalStorage flag for clarity

Test instructions

• Disable all cookies and site storage in browser settings (On-device site data in Chrome)
• Launch a RUM session configured to use sessionPersistence: 'in-memory'
• Verify that within a single tab, the session is recorded and uploaded successfully

I do likely need some help fleshing out any further e2e test coverage deemed necessary, as some of the test scripts seem to require more setup information than I am privy to.

Checklist

• Tested locally
• Tested on staging
• Added unit tests for this change.
• Added e2e/integration tests for this change.
• Updated documentation and/or relevant AGENTS.md file

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[andrewatwood]

I have read the CLA Document and I hereby sign the CLA

[bcaudan]

Hi @andrewatwood, thanks for your contribution!

Having in-memory session storage would be a great addition, two concerns though:

1. Fallback strategy:

From my understanding of your issue, you'd want to track users that have both cookies and local storage disabled and you'd want to use in memory session in that case.
During your tests, did you then set the storage in memory for all users or did you detect that cookies and local storage were disabled to configure the sdk accordingly?
We'd prefer to provide and document this session storage strategy with the possibility to configure a fallback to it, in order to avoid any misconfiguration that could have an impact on billing.
As we have deprecated the allowFallbackToLocalStorage configuration option, our plan to continue to support the fallback behavior is to update the sessionPersistence configuration option to take an array, declaring the allowed storages and fallback order.
Something like sessionPersistence: ['cookie', 'local-storage', 'memory'] would try cookie first, then local storage and finally memory.
We plan to work on this update before the next major release which should be next quarter.

2. Session sharing between rum and logs:

To correlate RUM and Logs data, the two SDKs share the same storage entry and we want to have the same behavior with session in memory.
We could probably achieve that by sharing the session state in a global variable, something like DD_SESSION.

We don't anticipate the changes needed to address those concerns to be too tricky, so if you want to have a go at it, we could provide additional support.
Otherwise, we'd adjust your PR with those changes when we'll work on the sessionPersistence update.

[andrewatwood]

Thanks for taking a look!

1. Yes, exactly! Originally this was patched on our end as an extra, separate fallback flag and handled internally by the SDK, but since the deprecation of the main localStorage fallback flag, I'd switched it to match the pattern of explicit option that is configured externally to the SDK. But using an array to dictate explicit priority rather than relying on separate fallback patterns sounds like a great change.

2. Great point. Didn't solve for this as we don't presently use the Logs feature.

I'll take a timeboxed go at these changes just to see if it can get to resolution a little quicker. But it's really keeping the patch updated between major versions that may have changes that's the biggest effort for us. So, if those changes (including a new in-memory option 🤞) end up being rolled into changes for the next major version anyway, it's still solved on effectively the same timeline from our perspective.

Appreciate your time, thanks again @bcaudan!

[andrewatwood]

Updated! Left the persistence array as optional to keep the change non-breaking.

@bcaudan let me know if this is what you had in mind when you get some time, thanks!

[andrewatwood]

@bcaudan thanks for getting this to the finish line! I'm now seeing the requirement for signed commits blocking merge lol. I'll let you rebase and sign if need be?

I didn't have signing set up on this account on my work machine, and I don't have vigilant mode enabled so I think it should be fine for you to replay my commits and sign them all as co-author yeah? Appreciate the quick turnaround on this!

[thomas-lebeau]

💯 Thanks a ton for your contribution @andrewatwood !

[bcaudan]

@andrewatwood it is available with v6.27.0!