Track Google indexation automatically for up to 140,000 URLs per site

It’s finally here — the ultimate indexation tracking tool you’ve always wanted

Indexation is a crucial part of SEO — after all, if a URL hasn’t been indexed there’s no way it can appear in search results. Yet for so long we’ve worked with very limited tools in monitoring indexation.

Starting today, those days are finally behind us. Dragon Metrics has levered Google’s newly-released Page Inspection API, to bring you the ultimate indexation tracking tool. We couldn’t be more excited to reveal this new feature today (and even to have it for our own purposes!)

With Dragon Metrics new Indexation Tracking, you can monitor indexation status automatically, at an individual URL level, for up to 140,000 URLs per site to help answer questions for each page like:

• Has Google crawled this URL?
• If so, when was it crawled most recently?
• If not, why didn’t Google crawl it?
• If it’s been crawled, has it been indexed?
• If not, why is it not indexed?
• Which device (desktop/mobile) was used when it was crawled?
• Did Google encounter any issues when trying to crawl it?

Both site-level and URL-level details are available to easily monitor thousands of pages while still being able to dive into the details of a single page.

140,000 URLs! What kind of sorcery is this?

We know this is the question on everyone’s mind. Those who are not familiar with Google’s Page Inspection API daily limit of 2,000 URLs per site may be thinking Why only 140,000? That’s not enough! While others that are aware of this limit may be thinking How can you get 140,000! That’s so far past Google’s limits!

Well not to fear, we’re here to reveal the secret to how we’re able to do it.

Keeping in mind Google’s limits are 2,000 URLs per day and per site:

• If you wish to track your data daily in Dragon Metrics, you certainly can! But if you set the frequency to weekly updates, you can now get 7 days * 2,000 URLs = 14,000 URLs per site.
• Google defines a site as a Google Search Console site — not a root domain. By breaking your domain into multiple Google Search Console sites (e.g. one site per subfolder), you now get 2,000 URLs for each subdivision of your site.
• Dragon Metrics lets you associate up to 10 Google Search Console properties with a single campaign.
• Combined together, you can track:
  • 2,000 URLs * 10 sites = 20,000 URLs monitored per day or
  • 2,000 URLs * 7 days * 10 sites = 140,000 URLs monitored per week.

While 140,000 URLs is way more than enough for small-medium sites, it’s can be just a drop in the ocean for the biggest sites on the web. So while you may not be able to monitor all pages on the site, it can still be incredibly useful for monitoring the launch of new URLs, or keeping an eye on the most important pages on the site.

Indexation Tracking in action

Once indexation tracking has been enabled for your campaign by adding your Google Search Console site and which URLs you want to track, we’ll fetch the data directly from Google and track it over time in the platform.

At the top of the page, a chart showing the primay indexation status of URLs in the set:

• Indexed: The URL has been indexed, can appear in Google Search results, and no problems were found with any enhancements found in the page
• Indexed with issues: The URL has been indexed and can appear in Google Search results, but there are some problems that might prevent it from appearing with the enhancements that you applied to the page.
• Not indexed: This URL won't appear in Google Search results, but we think that was your intention. Common reasons include that the page is password-protected or robots.txt protected, or blocked by a noindex directive.
• Not indexed (error): There was at least one critical error that prevented the URL from being indexed, and it cannot appear in Google Search until those issues are fixed.

You’ll also see this same data trended over time:

Next, we’ll show how many URLs were found with each type of crawling/indexation issue:

A coverage summary will be shown next, which provide a bit more details on the indexation status of pages on your site. Clicking on any row will filter the table below to reveal all the URLs with this status.

To help measure the freshness and frequency of Google’s crawls, a distribution showing how many days its been since Google has crawled each of the pages will be shown in a chart below.

Finally, at the bottom of the page, we can see in-depth details for each tracked URL.

If a crawl issue is found, hover on the icon to reveal additional information.

All fields in the table can be sorted and filtered.

For even more details about a specific URL, click on the page in the table.

This will link to the Indexation Status of this page in URL X-Ray, where you can see everything Google knows about this page.

Even with indexation tracking disabled for a campaign, you can still use URL X-Ray to search for any URL on your site and see Google’s indexation status in the same report.

Get indexed!

Indexation tracking is available today for all plans and packages, so you can get started right away. We’re so excited for you to try it out and let us know what you think.