Skip to main content

Website Scan

DebugBear can scan your website sitemap and make it very easy to set up monitoring for many of your website's pages all at once.

You can also use the website scan to identify slow pages based on Google CrUX data.

How to scan your website​

  1. Open Lab Tests in the sidebar
  2. Select Website Scan

Find the Website Scan feature

  1. Enter a domain to scan, and click Start Scan. DebugBear will take a few seconds to scan your site and then present the results.

Scan result

Identify slow pages based on CrUX data​

After getting a list of page URLs from your website, DebugBear can check each page URL to see if Google CrUX data is available for it.

tip

Google only publishes URL-level CrUX data for pages that receive a lot of traffic.

Click Load CrUX Metrics to trigger data collection. The scan will usually take 10 to 20 minutes, and you'll receive an email when it's complete.

CrUX data scan

When the data collection is complete you can open the scan result again and it will contain Core Web Vitals data for URLs where this data is available.

CrUX Core Web Vitals data in the scan result

Adding pages to monitoring​

You can simply select individual URLs and click Add Pages to set up monitoring for them. You can search your results or filter them to only show pages with CrUX data or pages that fail Google's Core Web Vitals assessment.

If your website has many pages, the Select All toggle will provide additional options to auto-select a subset of 100 or 1000 pages. The auto-select picks a diverse set of pages based on the website URL.

Auto-select

How does the website scan work?​

By default, we look for a sitemap file on your website. If one doesn't exist or we aren't able to access it we fall back to a website crawl starting on your website's homepage.

Instead of scanning your domain, you can also specify a direct URL to a specific XML sitemap file as the scan target.

Troubleshooting​

If the website scan fails this is usually because your server is blocking access to your sitemap.xml file, for example with a 403 Forbidden error.

The sitemap is fetched with a user agent containing dbb-scan. Allow this user agent in your server configuration to ensure DebugBear can access your sitemap.

Last updated on