Giter Site home page Giter Site logo

newrelic-browser-agent's Introduction

New Relic Open Source community plus project banner.

New Relic Browser Agent

The New Relic browser agent instruments your web application or site and provides observability into performance, errors, and other behaviors.

  • The instructions on this page pertain to installing the browser agent as an NPM package.

  • The browser agent is also generally available as a copy-paste JavaScript snippet and via auto-injection by backend apps. See Install the browser agent for info on these alternatives. Releases are deployed to customers gradually, so the version available via these other methods often lags the current release of this package.

  • For questions and feedback on this package, please visit the Explorer's Hub, New Relic's community support forum.

  • Looking to contribute to the browser agent code base? See DEVELOPING.md for instructions on building and testing the browser agent library, and CONTRIBUTING.md for general guidance.

Adding the agent package to your project

To make the agent available to your application, install via NPM or Yarn.

$ npm install @newrelic/browser-agent --save
$ yarn add @newrelic/browser-agent

Creating an app in New Relic

Before instrumenting your app using the NPM package, a Browser App should be configured in New Relic. This may be done with or without a corresponding APM agent. Once the app has been created, the Copy/Paste JavaScript code on the app's Application settings page will contain the configuration values needed to define options when instantiating the agent via the NPM package.

  1. If a browser app does not already exist, create one:
    • From the New Relic navigation panel, click Add Data.
    • Select the Browser monitoring data source.
    • Choose the APM or Copy/Paste method.
    • Select or name your app and click Enable.
  2. From the navigation panel, select Browser to view browser apps.
  3. Select the desired app and navigate to the Application settings page.
  4. From the Copy/Paste JavaScript box, copy the configuration values assigned to the NREUM object (init, info, and loader_config). You will use these configuration values when instantiating the agent using the NPM package.

Instantiating the agent

For best results, import and instantiate the BrowserAgent class as close to the top of the head element of your app's HTML output as possible. The specific location and method will vary based on your application's architecture or framework. See Library Support for more information.

Populate the options parameter using configuration values found in the the Copy/Paste JavaScript box in your browser app's Application settings page in New Relic.

import { BrowserAgent } from '@newrelic/browser-agent/loaders/browser-agent'

// Populate using values in copy-paste JavaScript snippet.
const options = {
  init: { ... }, // NREUM.init
  info: { ... }, // NREUM.info
  loader_config: { ...} // NREUM.loader_config
}

// The agent loader code executes immediately on instantiation.
new BrowserAgent(options)

Turning off features at runtime

Features may be turned off at runtime, which may have a meaningful peformance impact in some cases.

In the example below, the page_view_timing and session_trace features are disabled.

const options = {
  info: { ... },
  loader_config: { ... },
  init: {
    page_view_timing: { enabled: false },
    session_trace: { enabled: false },
    ...
  }
}

The following features may be disabled by adding init entries as shown above. Note that the page_view_event feature may not be disabled.

  • ajax
  • jserrors
  • metrics
  • page_action
  • page_view_timing
  • session_replay
  • session_trace
  • spa

See the New Relic documentation site for information on the above features.

Composing a custom agent with selected feature modules

The examples above use the BrowserAgent class, which is the best option for most use cases. The class makes all browser agent features available but provides the ability to turn off individual features selectively.

Using the base Agent class, it is also possible to compose a custom agent by passing an array called features in the options object, containing only the desired feature modules. Depending on which features are included, this may yield a smaller loader script and improved performance.

The example below includes three feature modules: Metrics, PageViewEvent, and PageViewTiming.

import { Agent } from '@newrelic/browser-agent/loaders/agent'
import { Metrics } from '@newrelic/browser-agent/features/metrics'
import { PageViewEvent } from '@newrelic/browser-agent/features/page_view_event'
import { PageViewTiming } from '@newrelic/browser-agent/features/page_view_timing'

const options = {
  info: { ... },
  loader_config: { ... },
  init: { ... }
}

new Agent({
  ...options,
  features: [
    Metrics,
    PageViewEvent,
    PageViewTiming
  ]
})

The following feature modules are available for inclusion in the features array:

import { Ajax } from '@newrelic/browser-agent/features/ajax';
import { JSErrors } from '@newrelic/browser-agent/features/jserrors';
import { Metrics } from '@newrelic/browser-agent/features/metrics';
import { PageAction } from '@newrelic/browser-agent/features/page_action';
import { PageViewEvent } from '@newrelic/browser-agent/features/page_view_event';
import { PageViewTiming } from '@newrelic/browser-agent/features/page_view_timing';
import { SessionTrace } from '@newrelic/browser-agent/features/session_trace';
import { Spa } from '@newrelic/browser-agent/features/spa';

Deploying one or more "micro" agents per page

The examples above use the Agent class at their core, which is ideal for most cases as it will automatically detect page-level events across your web application.

Using the MicroAgent class, it is possible to skip the "auto" instrumentation phases of the other loader types, and provide a very small agent designed for capturing data in a controlled manner via the API interfaces. The MicroAgent captures a distinct PageView event when instantiated, and additional PageAction and JavaScriptError events may be captured by calling the noticeError and addPageAction methods.

Because it does not wrap the page-level globals in the same way as the base Agent class, the MicroAgent is not only smaller but can easily be instantiated multiple times on a single page with low overhead, with each instance configured to report to a different Browser App entity in New Relic if desired. This accommodates specialized use cases, such as segmented UI designs (e.g., the micro front-end pattern) or applications requiring subsets of manually-handled data to be reported to different application entities.

The example below illustrates how to instantiate and interact with two separate MicroAgent instances on one page.

import { MicroAgent } from '@newrelic/browser-agent/loaders/micro-agent'

const options_1 = {
  info: { ... }, // configuration for application 1
  loader_config: { ... },
  init: { ... }
}

const microAgent1 = new MicroAgent(options_1)

const options_2 = {
  info: { ... }, // configuration for application 2
  loader_config: { ... },
  init: { ... }
}

const microAgent2 = new MicroAgent(options_2)

// manually handle a JavaScript Error with microAgent1
try{
  ...
} catch(err) {
  microAgent1.noticeError(err)
}

// manually capture a Page Action with microAgent2
microAgent2.addPageAction('myData', {hello: 'world'})

Browser Agent APIs

All Browser Agent APIs are exposed for use in two ways:

  • Via the newrelic window-level global object
  • At the top-level of the Agent instance

Please see our official documentation for more information about the Browser Agent APIs.

  newrelic.noticeError(...)
  // or
  const NrAgent = new BrowserAgent(...)
  NrAgent.noticeError(...)

Session Replay

The Session Replay feature is now available for limited free use by all customers. The data collected by this feature will become billable starting May 15th, 2024. Please see the Session Replay documentation to get started using this new feature.

Supported browsers

Our supported browser list can be accessed here.

If you desire more control over how the agent is built, our package also includes the source code. The source code is written to target the latest ECMAScript standards (ES2022). Please note that if you import the browser agent source code instead of the transpiled distribution code, you may need to alter your project's build system to apply transformations to the browser agent source code. Without these transforms, the code produced by your project's build system may not be compatible with your desired target browsers. Below is an example of how to import the source code.

// Note the /src/ in the import paths below
import { Agent } from '@newrelic/browser-agent/src/loaders/agent'
import { Metrics } from '@newrelic/browser-agent/src/features/metrics'
import { PageViewEvent } from '@newrelic/browser-agent/src/features/page_view_event'
import { PageViewTiming } from '@newrelic/browser-agent/src/features/page_view_timing'

const options = {
  info: { ... },
  loader_config: { ... },
  init: { ... }
}

new Agent({
  ...options,
  features: [
    Metrics,
    PageViewEvent,
    PageViewTiming
  ]
})

Neither the browser agent development team nor the New Relic support teams can support the numerous build systems that exist in the JavaScript ecosystem. If you run into issues, you may be asked to revert to importing from the transpiled distribution code.

DON'T mix imports between the browser agent source code and transpiled distribution code. The below will break.

// THIS IS BAD - do not do this
import { Agent } from '@newrelic/browser-agent/loaders/agent'
import { Metrics } from '@newrelic/browser-agent/src/features/metrics'

Library Support

The browser agent is written to be agnostic to any JavaScript library or framework. The agent exposes a number of API methods that can be incorporated into libraries and frameworks. For example, export or make available the initialized agent and create a new error boundary in your react application that calls browserAgent.noticeError().

Server-Side Rendering

A lot of new frameworks support the concept of server-side rendering the pages of an application to improve SEO and the performance of the user experience. The browser agent must be imported and instantiated within a browser context and will not work if executed on a server or in the server context of a server-side rendered application. These frameworks typically provide support for defining code that should only be run on the client. Check your frameworks documentation for more information.

Disclaimers

The session replay library shipping as part of the browser agent is not turned on by default. For information on the use of this feature, see Session Replay

Support

New Relic hosts and moderates an online forum where customers can interact with New Relic employees as well as other customers to get help and share best practices. Like all official New Relic open source projects, there's a related Community topic in the New Relic Explorers Hub. You can find this project's topic/threads here:

https://discuss.newrelic.com/c/full-stack-observability/browser

Contribute

We encourage your contributions to improve the Browser agent! Keep in mind that when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project.

If you have any questions, or to execute our corporate CLA (which is required if your contribution is on behalf of a company), drop us an email at [email protected].

For more details on how best to contribute, see CONTRIBUTING.md

A note about vulnerabilities

As noted in our security policy, New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.

If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through our bug bounty program.

If you would like to contribute to this project, review these guidelines.

To all contributors, we thank you! Without your contribution, this project would not be what it is today. We also host a community project page dedicated to the Browser agent.

License

The Browser agent is licensed under the Apache 2.0 License.

The Browser agent also uses source code from third-party libraries. Full details on which libraries are used and the terms under which they are licensed can be found in the third-party notices document.

newrelic-browser-agent's People

Contributors

andrewpomeroy avatar aubreymasten avatar bjfield avatar cwli24 avatar dami avatar elaguerta-nr avatar jeffalder avatar joe-degler avatar luisbravoa avatar melissaklein24 avatar metal-messiah avatar mlong-nr avatar newrelic-browser-agent-team avatar paddyo avatar patrickhousley avatar prathamesh-sonpatki avatar rhetoric101 avatar wrhector avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

newrelic-browser-agent's Issues

Documentation for CSP

Summary

There have been questions on CSP this summer and support shared the self hosting option/workaround to several customers which has been useful.

Desired Behavior

Let's put this solution into our github repo for reference.

Possible Solution

csp-v2-host-install-browser-agent (1).pdf

Agent does not report 0 for CLS

Description

0 is the perfect CLS score and needs to be included in our calculations (e.g. 75th percentile). Currently the agent only records a CLS value when the browser provides one (when content actually shifts). When there is no shift, we do not report anything, and the cumulativeLayoutShift attribute is not added to the PageViewTiming events.

We cannot simply update our queries to default to 0 because that would include events from browsers that do not support CLS. We could filter our queries to those browsers, but that makes for complicated queries, and we also want to make it easy for our customers to query CLS values.

Therefore, the agent should detect if the browser supports CLS. And if it does, it should report 0 by default.

Provide Subresource Integrity check for the agent script (SRI)

https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity

This pertains to the agent (aggregator) script that the loader is adding to the page dynamically. To provide SRI, we should generate an integrity hash, and include it in an "integrity" attribute of the script tag.

This work would include:

minor change to the agent (adding the integrity attribute)

update the build process to generate the integrity hash and include it in the agent

From Marino Alvarez: I have a customer reach out to me with CSP concerns. Created an internal ticket with the details and also a Feature Request. Our docs mention to reach out to support to discuss other options but there aren't any other options that support can rely to our customers (should we change that doc?).

Support for web workers

Summary

Provide support for monitoring Web Workers.

Desired Behavior

Collect at least the following:

  • Error reporting
  • External calls
  • Custom tracers

Agent can cause memory issues when it runs after the window load event

Description

When the initial agent script (loader) executes after the load event, it will never load the second part of the agent. It will also continue buffering data indefinitely.

There is a failsafe mechanism to stop collecting data when the second part of the agent does not load within 30 seconds. But this mechanism is setup when the window load event fires. Therefore, if the event never fires, data will continue to be collected in a buffer causing the web page to eventually run out of memory.

Expected Behavior

The agent should detect if the window load event has already occurred, and either initiate the second script download or abort.

Meet New Standards for CLS

Background

CLS captures how much the page shifts and how often. There are use cases, especially for single page applications, where the page can be open for a very long time which results in a score that is too high. Google therefore has capped that to account for the maximum layout shift window aka capturing the most frustrating burst of layout shift on the page.

What will we do

This effort is to match the new standard for our CLS reporting.

Customer & business value

This is an iterating of work for capture more data/better signal. It is also work we need to do to be on top of latest key metrics for RUM monitoring.


Explore OpenTelemetry Partnership with AWS

AWS would like to partner with our browser agent team to help define RUM OTel

The scope of this work in May will be limited to conversations with AWS to work toward a scope of this project.

Related Koan Key Result: https://newrelic.koan.co/team/f5bc77b8-f32c-40c3-a912-32f1a4ae473d/goals/f/goals/e29c7226-fa04-4d54-bc59-83fa99c1d526?inviteModal=false&view=nested&modal=&status=active&columns=leads%2CdueDate%2Cparent%2Cmetrics%2Cconfidence%2CcloseOutDetails&rosterModal=false

Related Row in Commits Spreadsheet: None.


Capture All AJAX Requests

Problem

Customers would like all Ajax Requests available to query in Insights. The confusion lies when just a subset of ajax requests are captured as event data, the rest are not-so-easy-to-query aggregate metric data.

Ajax calls are currently captured as events (AjaxRequest) _ only _when the call happens as part of a browser interaction that is captured. Since out of the box we capture only interactions that are classified as route changes, many ajax calls do not get recorded as events.

Ajax requests that are stored as event data are the starting point for Distributed Tracing, so by not representing all requests in the system, we lose some visibility.

What we need to do

We need to capture all AJAX requests in a way that customers can facet and filter by AJAX and not missing AJAX requests when using DT

The goal of this effort is to consolidate all AJAX requests into one data collection type, that is queryable via Insights and a seamless experience for our customers.

Proposed Solutions

All ajax requests as events (which come with storage concerns)

Ajax requests as dimensional metrics, which likely requires some rework for DT? This is likely the better candidate.


BROWAGENT-50


Related Operational Links

Running agent tests does not work on Windows

Running npm run test on Windows errors with message:

'tools' is not recognized as an internal or external command.

Even when using node at the beginning of the command does not seem to run tests.

  • small effort

JSONP instrumentation is holding up interactions from finishing when its callback is reused

The following snippet can be used as a repro. The interaction is captured the first time it is run, but not on subsequent times.

document.addEventListener('click', run, false)

function run () {
var url = '/jsonp?cb=foo'
var el = document.createElement('script')
el.src = url
window.document.body.appendChild(el)
}

function foo (data) {
window.location.hash = Math.random()
}

The agent calls clearResourceTimings, which affects other scripts

This was reported by a user
https://twitter.com/simonhearne/status/977200179236483072
https://discuss.newrelic.com/t/do-not-clear-the-resource-timing-buffer/54771

This affects session traces. The solution proposed by the user is to either stop collecting resources when the buffer becomes full, or increase the buffer.

One possible solution is to switch to using the observer API. It is not supported by older browsers, but probably vast majority at this point, so it might be sufficient.

Another possibility is to make this configurable, so that the customer using the NR agent can decide how long sessions traces they want to collect.

Reduce Performance Impact of Running the Agent

Summary

When monitoring some of our pages using Calibre (which essentially runs Lighthouse in a lab) we can see nr-loader-spa sometimes blocking > 5s of main thread execution.

We are including the browser agent on our page with the following HTML:

<script type="text/javascript">
(function () {
window.NREUM\|\|(NREUM={});
NREUM.init={distributed_tracing:{enabled: true,cors_use_newrelic_header: true,cors_use_tracecontext_headers:true,allowed_origins:[]},privacy:{cookies_enabled:true}};
NREUM.loader_config={accountID:"OUR_ACCOUNT_ID",trustKey:"OUR_ACCOUNT_ID",agentID:"OUR_APPLICATION_ID"};
NREUM.info={licenseKey:"OUR_LICENSE_KEY",applicationID:"OUR_APPLICATION_ID",agent:"https://OUR_CDN_DOMAIN/nr-spa.1210.js"};
}());
</script>
<script src="https://OUR_CDN_DOMAIN/nr-loader-spa.1210.js"></script>

To be fair this is running on a particularly slow page, elsewhere we only see a blocking time of 145ms.

Desired Behavior

Ideally we want as little blocking time as possible, or perhaps some insight into why this might be taking so long in this case.

Possible Solution

We've started using the style above as documented in this project's README because our apps are single page apps and we can't include the snippet via server-side monitoring and we found that the snippets provided by the old REST API seemed out of date.

Potentially we could change our script that generates the above HTML to instead inline nr-spa.1210.js, though in our trace it doesn't seem like loading the script is the issue.

I'm not sure if it would be workable to delay instantiation of the agent and simply collect data until there doesn't seem to be a lot of other activity.

Additional context

The Calibre trace is emulating desktop Chrome running over a cable connection; we have other tests using a similar profile that don't experience these long blocking times.

Unable to track sessions when browsers do not accept 3rd-party cookies

When the agent connects to our backend, the backend sets a cookie with a session ID. This is used to tie together page views (session attribute on PageView events). When the cookie cannot be created, the session value is different for every page view.

This keeps coming up in support tickets, and it basically prevents us from correctly recording sessions. If customers are using our session attribute to count unique sessions, this number is often incorrect.

This has become a higher severity with Safari 12, which by default blocks 3rd-party cookies.

This issue also affects recent Firefox users where the default behavior is to block these cookies: https://blog.mozilla.org/blog/2019/09/03/todays-firefox-blocks-third-party-tracking-cookies-and-cryptomining-by-default/

involve browser wizards on this one - Aidan Cuffe made an augmentation. See which customers have tried it.

Increase default limit of PageAction events per harvest

Summary

Currently the agent limits the number of PageAction events collected to 60 per harvest (every 30 seconds). This is often not enough. We would like to increase the number to meet our customers' needs.

Desired Behavior

Customers are able to collect all their PageAction events.

Possible Solution

Start by doubling the current limit, measure the impact and decide on next steps.

[Repolinter] Open Source Policy Issues

Repolinter Report

🤖This issue was automatically generated by repolinter-action, developed by the Open Source and Developer Advocacy team at New Relic. This issue will be automatically updated or closed when changes are pushed. If you have any problems with this tool, please feel free to open a GitHub issue or give us a ping in #help-opensource.

This Repolinter run generated the following results:

❗ Error ❌ Fail ⚠️ Warn ✅ Pass Ignored Total
0 0 0 7 0 7

Passed #

Click to see rules

license-file-exists #

Found file (LICENSE). New Relic requires that all open source projects have an associated license contained within the project. This license must be permissive (e.g. non-viral or copyleft), and we recommend Apache 2.0 for most use cases. For more information please visit https://docs.google.com/document/d/1vML4aY_czsY0URu2yiP3xLAKYufNrKsc7o4kjuegpDw/edit.

readme-file-exists #

Found file (README.md). New Relic requires a README file in all projects. This README should give a general overview of the project, and should point to additional resources (security, contributing, etc.) where developers and users can learn further. For more information please visit https://github.com/newrelic/open-by-default.

readme-starts-with-community-plus-header #

The first 5 lines contain all of the requested patterns. (README.md). The README of a community plus project should have a community plus header at the start of the README. If you already have a community plus header and this rule is failing, your header may be out of date, and you should update your header with the suggested one below. For more information please visit https://opensource.newrelic.com/oss-category/.

readme-contains-link-to-security-policy #

Contains a link to the security policy for this repository (README.md). New Relic recommends putting a link to the open source security policy for your project (https://github.com/newrelic/<repo-name>/security/policy or ../../security/policy) in the README. For an example of this, please see the "a note about vulnerabilities" section of the Open By Default repository. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

readme-contains-discuss-topic #

Contains a link to the appropriate discuss.newrelic.com topic (README.md). New Relic recommends directly linking the your appropriate discuss.newrelic.com topic in the README, allowing developer an alternate method of getting support. For more information please visit https://nerdlife.datanerd.us/new-relic/security-guidelines-for-publishing-source-code.

code-of-conduct-should-not-exist-here #

New Relic has moved the CODE_OF_CONDUCT file to a centralized location where it is referenced automatically by every repository in the New Relic organization. Because of this change, any other CODE_OF_CONDUCT file in a repository is now redundant and should be removed. Note that you will need to adjust any links to the local CODE_OF_CONDUCT file in your documentation to point to the central file (README and CONTRIBUTING will probably have links that need updating). For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view. Did not find a file matching the specified patterns. All files passed this test.

third-party-notices-file-exists #

Found file (THIRD_PARTY_NOTICES.md). A THIRD_PARTY_NOTICES.md file can be present in your repository to grant attribution to all dependencies being used by this project. This document is necessary if you are using third-party source code in your project, with the exception of code referenced outside the project's compiled/bundled binary (ex. some Java projects require modules to be pre-installed in the classpath, outside the project binary and therefore outside the scope of the THIRD_PARTY_NOTICES). Please review your project's dependencies and create a THIRD_PARTY_NOTICES.md file if necessary. For JavaScript projects, you can generate this file using the oss-cli. For more information please visit https://docs.google.com/document/d/1y644Pwi82kasNP5VPVjDV8rsmkBKclQVHFkz8pwRUtE/view.

Distributed Tracing Doesn't Seem to Work With Built Files

Description

I've tried using the built files to host the SPA agent locally with a configuration like the following:

<script type="text/javascript">
  window.NREUM||(NREUM={});
  NREUM.init = {
    distributed_tracing: {
      enabled: true,
      cors_use_newrelic_header: true,
      cors_use_tracecontext_headers: true,
      allowed_origins: [/* array of strings of origins, e.g. 'https://example.com', 'https://mydomain' */]
    },
    privacy: {
      cookies_enabled: true
    }
  };
  NREUM.info = {
    licenseKey: "/* my license key */",
    applicationID: /* my application id */,
    agent: "https://MY_CDN/nr-spa.js"
  };
</script>
<script src="https://MY_CDN/nr-loader-spa.js"></script>

It appears that most of the standard browser monitoring works, but distributed tracing does not.

Steps to Reproduce

  1. clone this project
  2. run npm run build
  3. upload the generated nr-spa.js and nr-spa-loader.js to a CDN, web server, etc.
  4. initialise the agent to use distributed tracing with a set of allowed domains that are also configured to support distributed tracing headers
  5. in a browser app using the generated scripts make XHR requests which should be included in distributed traces

Expected Behavior

  • AJAX requests are made with tracestate, traceparent and newrelic headers
  • requests appear in Distributed Tracing

Relevant Logs / Console output

Your Environment

  • Chome 91.0.4472.101
  • OS X 11.4

Additional context

Remove references to Opera from our testing matrix

Opera is no longer a supported browser in SauceLabs (source). While it was supported, we experienced issues with it that lead us to explicitly exclude it from certain tests in the agent (search for .exclude('opera@<=12') in the agent repo for examples), these are unnecessary now, those and any other references to opera should be cleaned up and it should be documented somewhere than Opera may not be an officially supported browser.

Investigate Cross-origin Isolation (impact on browser agent)

This feature will be available in Chrome 91https://developer.chrome.com/blog/enabling-shared-array-buffer/#cross-origin-isolation

This is related to CORPhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Cross-Origin_Resource_Policy_(CORP)

We are planning to test when Chrome 91 is available. Chrome 91 should come out at the end of May 2021, beta should be available at the end of April.

Context in this thread: https://newrelic.slack.com/archives/G019T80B1EJ/p1614288779011800

CLS - Report pageHide event when page is unloading

Summary

Currently we record pageHide PageViewTiming event for the first time a page is hidden. The original intent was to record an event when the user hides the page without leaving, e.g. switching to a different tab or window.

We do not record this event when the first time the page becomes hidden is when it is unloading. We could argue that this is covered by the windowUnload event. However, we use the pageHide event to query CLS values, and since that query can use only one type of the PVT event (to not include multiple events per page view), some pages are completely excluded from our aggregate calculations.

Desired Behavior

Therefore, this is a proposal for recording pageHide event when the page page is unloading - if and only if the pageHide event has not been already recorded. We can make argument that this makes sense since the visibilitychange event fires on windowUnload.

Additional context

The secondary implications of this change is that the pageHide event will change meaning a little. Users will no longer be able to ask how often users switch tabs, or compare windowUnload to pageHide. However, we do not know if customers query for this information, and it is not the primary use case for these events. If it turns out that customers want this, then we could decorate the pageHide PVT events with secondary attribute when they happen during unload.

Remove label github action stopped working

Description

The Remove Label github action is now failing with the error below. This is new behavior.

Run andymckay/labeler@master
TypeError: Cannot read property 'pull_request' of undefined
    at getIssueNumber (/home/runner/work/_actions/andymckay/labeler/master/label.js:32:59)
    at label (/home/runner/work/_actions/andymckay/labeler/master/label.js:50:21)
    at Object.<anonymous> (/home/runner/work/_actions/andymckay/labeler/master/label.js:99:1)
    at Module._compile (internal/modules/cjs/loader.js:959:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.Module._load (internal/modules/cjs/loader.js:727:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

Steps to Reproduce

Add the safe to test label to a PR. Push a new commit to the PR, which will trigger this workflow. Note that the safe to test label has not been removed.

Expected Behavior

The safe to test label should be removed when a new commit is pushed to a PR.

Beacon requests will fail when attempting to send more than 64k of data

When sending beacon (unload) requests using the agent, there is no limit in place for the amount of data added to the body of that request.

According to this post, most browsers (at least at the time) implemented a 64k limit on the maximum size of data allowed to be sent in a given beacon (Edge & Firefox) or navigation context (Chrome): https://github.com/w3c/beacon/issues/38This issue came up in a GTSE ticket with a similar behavior in Safari for iOS. An unload session trace harvest exceeded 64k in length and failed.

Answer question: Is this still an issue (github issue from 2016) How often this happens? How close to this limit? Size of messages consumer sees from the agent.

check how large our request is and how much space is available.

Effort: Medium, open questions

Replace or mask sensitive segments of a URL path

Summary

New Relic Browser collect lots of information about URLs (page urls, ajax-request urls, etc.), and sometimes these can contain information deemed sensitive (either by the authorities or internal company security). Currently there is no way actively mask or replace the segments of these URLs.

Desired Behavior

Ideally, the browser agent API would be extended with functions to replace all URLs with specific patterns, eg. nreum.replaceURL('regex-pattern', replacement). These pattern-replacement pairs would have to be checked for all the URLs being reported to New Relic (including URLs for page-views, browser-interactions, ajax-requests and session-traces).

Possible Solution (field solution)

We created a field solution for this, where we extended the compiled agent and manually added the patterns to the loader.js. The agent.js would then call a specific function before storing the url, and the function would check for patterns and replace where matched are found. I have attached both loader and agent (unminified).
In the loader.js you can find the code at the bottom of the file (line 2392). There are 2 cleaner-functions (used in the agent.js) + an array of regex-patterns and -replace entries.
agent+loader.zip

Additional comments

Although the field solution works, it isn't ideal. Upgrades would have to be extended, source-files changed, etc. A productised solution would be useful for any customer needing to mask or even group their URLs in a custom manner. The agent API would solve the inconvenience of having to re-implement after each agent upgrade.

CLS - Update Documentation

MMF

#89

Summary

Update documentation for CLS capture methods

Desired Behavior

Documentation should include:

  • Explain how and why values are different from Google
    • Different Sample Set
    • Chrome/Lighthouse have different lifecycles
  • Update public docs in the current Web Vitals section

Sustaining Work

  • Open sourcing optional work - development processes better
  • Small customer tickets - not clearing resource cache, etc.
  • Small bugs
  • Look over GTSEs and see if it could have been deflected if we had troubleshooting docs, available

Koan Link: https://newrelic.koan.co/team/f5bc77b8-f32c-40c3-a912-32f1a4ae473d/goals/f/goals/fe1fa213-8bf7-45c4-9f6a-1d58d777fd24?view=nested&modal=&status=active&columns=leads%2CdueDate%2Cparent%2Cmetrics%2Cconfidence%2CcloseOutDetails&rosterModal=false

Related Row in Commits Spreadsheet: None.

Capture URL of images for LCP

We are currently capturing the ID of the element. Not all elements will have IDs, and for images it would make sense to capture the URL instead. This would be useful for customers to identify exactly which element (image) resulted in the LCP value to be captured.

https://wicg.github.io/largest-contentful-paint/#sec-largest-contentful-paint-interface

In addition to the URL, we should also consider capturing the type of element (e.g. image, text, video) and class name (for elements that can be identified by class rather than URL or ID).

Feedback from Joel Newman

"If adding a new attribute isn’t prohibitive due to attribute limits or storage costs, I would encourage making the ID and URL data separate things. They’re in very different formats, and would be used in different ways, and using one instead of the other will cause confusion and unintended difficulties in querying downstream.

I think it would be worth exploring more verbose messages than we typically use for situations in which data for one or the other isn’t available. For instance, if the LCP element is not an external resource, we won’t be able to provide a URL for it, but rather than leaving the URL attribute blank, we could instead give it a value of “Inline” or something of that sort. Similarly, when there’s no HTML ID, we could revert to another value (such as class if it’s available) or give it a value of “not set”. This would help circumvent a lot of confusion on the customer’s side, trying to figure out why the agent is not capturing data, and probably help us have fewer support tickets with customers thinking the agent is broken.

This sort of context is especially important for something like LCP because the element it points to won’t be consistent across every page view, which makes it difficult to reproduce the exact behavior from a specific page view for troubleshooting purposes. "

Add mechanism to collect supportability metrics

Summary

In order to make better decisions about which features to implement and bugs to fix, it would be useful to be able to collect internal metrics from the agent. This would be useful to find out how the agent works internally in the wild, what data it sees etc.

Desired Behavior

The agent provides an API to collect a metric. This data would be collected by the agent.

Additional context

Examples of when this would be useful

  • #71 (Is this piece of code in the agent ever run?)
  • #75 (Does the agent every send data that is larger than a given limit?)
  • #78 (How often do we see stack traces modified by a 3rd-party that we cannot normalize?)

Exclude paint timing when page is not visible

When web page is hidden (e.g. not active tab), the paint timing events will not fire until the page becomes visible. This makes sense from technical perspective (no paint actually happens when page is hidden), but it affects performance metrics as paint timing values can be very high.

The easiest way to solve this would be to exclude values that are collected on a page that was hidden when it was loading. There is an open discussion around this on W3C -

Discussion around this happened here: w3c/paint-timing#40

and summarized here: https://docs.google.com/document/d/1A0u26wVip78yxlNrJdNKBQj3qdUrbWljY_NQgaIKab4/edit

Page Visibility APIhttps://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API

Full Fetch Support

The Problem

Customers are not seeing all of their Fetch calls.

Why Now

We are focusing on efforts on health of the agent and robust instrumentation. There have been increased support tickets and FRs around missing Fetch calls.

Appetite

2 engineer x 2.5 weeks = 5 engineering weeks

What will we do

SPA captures XHR and Fetch as events but only if they are part of route changes. We will now support all Fetch calls, sending them as metrics

Related Operational Links

Fix URLs Rolling Up into Domain Only

The Problem

Customers are not seeing their full URLs when on Chrome 88 and 89 and on Safari. URLs instead are rolling up into the route domain or ‘/’. This happens when using the value from the referrer header. Note: this impacts PageView events and not browser interaction events.

Why Now

We have a workaround behind a feature flag that send the value instead via a query string. 66% percent of traffic is coming from Chrome. 90% of that traffic is using Chrome 88 or Chrome 99.

Appetite

1 engineer x 2 weeks = 2 engineering weeks

What We will do

We’ll either need to make the feature-flagged behavior default, or pass through new configuration to auto-injected agent.

Success Metric

Boolean - customers using Chrome 88 and Chrome 89 can see their full urls


Related Koan Key Result: https://newrelic.koan.co/team/f5bc77b8-f32c-40c3-a912-32f1a4ae473d/goals/f/goals/df725e16-56cf-4e28-a543-0b82d3fa630f?inviteModal=false&view=nested&modal=&status=active&columns=leads%2CdueDate%2Cparent%2Cmetrics%2Cconfidence%2CcloseOutDetails&rosterModal=false

Related Row in Commits Spreadsheet: None.

Remove support for error stack traces in old opera versions

Opera is no

If time ever comes up to work on JS Errors or removal of legacy code, there are some comments not suitable for public GitHub that identify a portion of the stack trace parsing code built exclusively to support old Opera versions. The work would involve not parsing stack traces for versions of Opera older than 9 and allow us to delete old code.

See comments in this commit: https://source.datanerd.us/browser/browser_agent/blob/903edda05371cd7b253668fd73a680a009cc6acc/feature/err/aggregate/compute-stack-trace.js#L264-L266

https://source.datanerd.us/browser/browser_agent/blob/903edda05371cd7b253668fd73a680a009cc6acc/feature/err/aggregate/compute-stack-trace.js#L322-L327

NullpointerException in newrelic browserscript

We are getting many javascript nullpointer exceptions logged in our newrelic dashboard. The Stacktrace points to a specific line in the minified newrelic browserscript:

e.params.status=t.status

This line of the minified version comes from here https://github.com/newrelic/newrelic-browser-agent/blob/main/feature/xhr/instrument/index.js#L278

It seems that params can be undefined at that point. This error is raised only for a small amount of users but for them it happens very frequently.

It happened on Chrome 90 and Edge 90 so far with version nr-spa-1208

Stacktrace

Support zone.js style stacktrace

Summary

Since zone.js rewrites the stack trace, New Relic agent doesn't recognize the stack trace in some errors when zone.js is using.
Can we support zone.js style stack trace to show all stack traces?

Desired Behavior

[NOTE]: Show all stack traces

note: angular/angular#31588

Possible Solution

Additional context

This issue was previously discussed in GTSE-9182 and the customer and New Relic engineering decided to submit this feature request.

CLS - Investigate Usage of WebVitals Library

MMF

#89

Summary

It is useful to know if switching to use the webvitals library has any benefits, and the level of effort it would take to get there.

Desired Outcome

Document outlining the research and the recommendation of action

Things to identify

  • Differences between webvitals lib and our current implementation
  • Advantages and disadvantages
  • Strategy to stay up to date with library updates
  • Compatibilities with using the lib (typescript?)

Investigate: Add CORP headers

Chrome 92 will force certain interactions to have a Cross-Origin-Resource-Policy: cross-origin header present. Currently our CDN and Ingest do not return this header. We need to investigate the impact this will have and whether we need to take action. Chrome 92 is set for July.

FR for Demplatform team

Publish browser agent as npm package

Summary

The current mechanism of using the NewRelic browser agent is to embed the code snippet - shown in the "Configure the agent" section in the README.md file - directly to the website.

An additional option might be to allow website developers to integrate the browser agent by using npm packages. This mechanism has several advantages, e.g. developers can decide when to update and test a new version of the browser agent. Furthermore, type-safety (when using TypeScript) could be enabled.

For the NewRelic node client, this feature is already implemented. See https://github.com/newrelic/node-newrelic/releases and https://www.npmjs.com/package/newrelic

Desired Behavior

I am able to install the NewRelic browser agent with npm.

Possible Solution

Standard npm publish / GitHub release mechanism. See https://github.com/newrelic/node-newrelic/releases and https://www.npmjs.com/package/newrelic

Additional context

none

Track CSP Violations out of the box

Summary

NR Browser Agent could (and should) track CSP errors out of the box. This would likely be a relatively simple implementation and could provide insight on malicious frontend attacks / vulnerabilities.

Desired Behavior

Implementing the NR Browser Agent should capture and forward CSP violation errors to .noticeError() automatically.

Possible Solution

Subscribe to the DOM Event securitypolicyviolation and have the callback issue a newrelic.noticeError(...) with relevant info.

Additional context

Customers could gain insight on malicious activity and vulnerabilities.

##use cases/questions
Shows up as an error in console and not in NR1
How does it show up in the UI given it happens outside of a javascript error
Would you want to track it in a RUM environment
How often CSP blocked something: example: tried to run agent extension and it gets blocked by CSP

Decouple DT from SPA

  • metrics for everything
  • spans for some things - SPA? others?
  • sampling? - what are the most valuable spans for browser? configuration?

Use Client Hints API instead of `navigator.userAgent` when possible

Summary

The agent is currently calling navigator.userAgent in a few places. This causes a warning in Chrome DevTools (Issues tab) starting with version 92. Chrome is gradually reducing the information available in the legacy User Agent API and moving to client hints.

The Client Hints API to use is navigator.userAgentData.

Desired Behavior

Agent does not cause Chrome to display a warning message about the User Agent API in DevTools Issues.

Possible Solution

Use navigator.userAgentData (when available) instead of navigator.userAgent.

Browser Agent Errors for Passing Strings Instead of Objects in NREUM

This customer had a deploy that changes their NREUM.loader_config and NREUM.info into strings, which caused the Browser agent to not load or report data properly. We expect these values to be passed as objects, but it took weeks to figure out that they had been wrapped with quotes.



If the Browser agent could throw an error when these values are not passed as an object, troubleshooting this issue would be a lot quicker.

CLS - Update Agent Logic

MMF

#89

Summary

Update the agent to implement the approach outlined in Issue 104

Desired Behavior

  • Agent should capture CLS according to new specs as decided on in MMF Doc and Issue 104
  • Agent should have new unit and functional tests to validate new capture method
  • Engineer should validate previous/existing tests still work

Add passive flag for touchstart EventListener

Summary

Currently, the touchstart EventListener is not using the passive flag therefore the Lighthouse passive event listener audit fails. This was already brought up in the NewRelic Forum in November 2020 but the issues still seem to persist.

Possible Solution

Add the {passive: true} flag to the touchstart EventListener

Additional context

Google Lighthouse flags event listeners that may delay page scrolling: web.dev documentation

Effort: Small

Run tests on PRs in headless browsers

Currently, all browser-based tests in the agent are run using SauceLabs. This requires a private authentication key and means PRs from external contributors require a manual review by a member of the team to ensure the code is safe to run in an environment where these SauceLabs authentication keys could be accessed.

Instead of running cross-browser tests using SauceLabs, we could instead host a Selenium WebDriver in the same way to run these tests against headless versions of the browsers within the GitHub workflow itself.

This may require some change to the protocol we use to communicate between the test and the WebDriver. Some options for hosting the selenium server are below.

Use WebDriver IO: https://developers.google.com/web/updates/2017/04/headless-chrome#drivers

Host Selenium Hub as a service in the GitHub Action (example here)

Value: currently we have to add a label to run tests, this would allow tests to run automatically on the PR. This would have an additional advantage of contributor PRs auto-running tests

Medium Effort

Chrome in iOS: Diagnose which AjaxRequest (and some BrowserInteraction) events not shown for iPhone and Chrome

Dependency: Before determining the size of work necessary to resolve this issue, we need to determine which versions of Chrome for iOS are incompatible with the XHR feature and which should be feature-gated.

Rebecca Note: We need to get estimations on total amount of traffic impacted.

There are still some context details on this ticket: https://docs.google.com/document/d/15Z3v_SGCFCqyW5Cib2ODdYHIaPyQLv-ZmU4uzr8e-RA/edit?usp=sharing

Update 7/7/2020: Because of the behavior of older versions of Chrome for iOS, the browser agent has not enabled XHR or Session Trace instrumentation in that browser. This means AjaxRequest and Span events will not be harvested for traffic from Chrome for iOS. It also means BrowserInteraction duration measurements will not include any network (XMLHttpRequest and Fetch) related duration. BrowserInteractions from Chrome for iOS will currently show NavigationTiming API related values, but no ajaxCount and little to no value for duration.

The work this ticket now represents is removing that override, instrumenting the latest version Chrome for iOS, confirming whether the data produced is valid and ticketing the necessary work to release a version of the agent which instruments SPA on Chrome for iOS for versions older than the tested working version. See comments below on how to locally inject and test the agent in Chrome on iOS.

We cannot see any BrowserInteraction events from iPhone X using Chrome 80. I have tried using emulation, and that reports using Safari. They have implemented the suggestion from the other ticket, https://newrelic.zendesk.com/agent/#/tickets/392985

Please see Zendesk Support tab for further comments and attachments.

other comments from Jira card:
Aubrey: Attaching another related behavior, currently creating custom interactions using the SPA API in Chrome for iOS causes 1000s of interactions with no timing data to be created, this should hopefully be resolved if/when SPA was properly enabled for this user agent.

Lindsy: Updated this ticket to include AJAX request events per previous ticket JS-5807: Chrome in iOS and iPad is not reporting XHRREJECTED

Notes form comments in that ticket:Ticket that triggered this inquiry:
https://newrelic.zendesk.com/agent/tickets/344795

Impact is significant since it's causing them to miss AJAX data on a retail site. They are open to implementing a workaround if one is available. I will ask Dublin support team to do a temperature check with this information.

Closing 5087 to work from this ticket for both issues as they are the same culprit.

Aubrey:
For whoever picks up this ticket to inject the agent and determine whether we can enable XHR and SPA for Chrome, this is the process I used to inject the agent on customer or local sites on iOS:

Proxy traffic between iOS and Charles to inspect network requests made by Chrome. Use. this tutorial to enable and trust certificate: https://benoitpasquier.com/charles-ssl-proxy-ios/

Use Charles' external proxy settings to forward traffic to agent in the middle. Replace the port in the screenshot with the port agent-in-the-middle exposes

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.