How to set up the Google Indexing API to speed up indexing

If you are a business owner, marketer, or head of a digital department, you’ve likely encountered this situation: a new section of the site, a promotion, or a product goes live online, but it doesn’t appear in Google search for weeks. In our observations, in 80% of cases an indexing delay is not a technical accident, but the result of outdated approaches to SEO and a lack of understanding of modern tools.

Traditional methods — sending a Sitemap, internal linking, manually requesting indexing through Google Search Console — no longer keep up with the dynamics of the modern internet. This is felt particularly sharply by projects with a large number of pages, frequent updates, or high competition for traffic.

In this article I will explain how the BUSINESS SITE team solves the problem of speeding up indexing for clients across different industries: from pharmaceuticals to tourism and banking. We’ll cover what indexing and crawling are, why traditional methods often fail, and how the Google Indexing API can greatly accelerate the inclusion of new pages in the index. You will receive a step-by-step guide to setting up the API, an analysis of common mistakes, and tools for monitoring the results. The article is not just theory, but a concentrate of practical experience that we have accumulated over years of working with projects of various scales.

If you want your site to appear in search quickly and predictably, and for investments in content and advertising to deliver maximum ROI, read on. I promise: after reading you’ll be able not only to understand how modern indexing works but also to implement this knowledge in your business today.

Website and page indexing: basics for SEO

Only after this can pages appear in search results and attract SEO traffic. If a page is not indexed: for Google users it simply does not exist.

In my experience, many entrepreneurs and marketers underestimate the importance of the speed and completeness of indexing. For example, for an online store with daily product updates or for a site with promotions and special offers, an indexing delay of 1–2 weeks means direct loss of sales. In projects where we implemented indexing automation, organic traffic growth on new pages began within 1–3 days instead of the usual 7–14.

Website indexing and its impact on SEO traffic

Indexing is the first and mandatory step to getting SEO traffic. But it’s important to understand: inclusion in the index does not guarantee high rankings. Google analyzes hundreds of ranking factors, including content quality, page load speed, mobile optimization, behavioral metrics (time on page, bounce rate) and even the author’s expertise. At BUSINESS SITE we always recommend a comprehensive approach: technical optimization + quality content + accelerated indexing.

Lemmatization of key queries is another important aspect. Google increasingly understands the meaning of queries and content, so it’s important not just to insert keywords, but to create materials that fully answer users’ questions. In one of the case studies for a pharmaceutical company we achieved a 40% traffic increase by optimizing not only the technical part but also the content structure for lemmatized queries.

When to block pages from indexing?

Not all site pages should be included in the index. For example, administrative sections, duplicates, pages under development, or pages with confidential information are better blocked from indexing using robots.txt directives or the noindex meta tag. This helps preserve crawl budget and avoid problems with duplicate content. At BUSINESS SITE we regularly conduct indexing audits for clients and find “extra” pages that hinder promotion of the main sections.

Crawling and crawl budget: problems and impact

Without crawling, indexing is impossible, so any issues with access, structure or the technical state of the site directly affect the speed and completeness of being included in the index.

Crawling issues and their impact on indexing

The most common errors the BUSINESS SITE team encounters when auditing sites:

• Missing or incorrect robots.txt file – the bot cannot understand which pages to crawl and which not.
• No Sitemap.xml or it contains broken links — Google understands the site structure worse and may miss important sections.
• Redirects and broken links – if a page returns a 4xx or 5xx code, the bot will not be able to crawl it.
• Slow page loads — if the site takes longer than 2–3 seconds to load, Googlebot may stop crawling and leave.
• Poor internal linking, if pages are not linked to each other, the bot may simply not find them.

In a project for a construction company we encountered a situation where, due to an incorrect robots.txt and missing Sitemap, only 30% of pages were indexed. After fixing these errors, indexing rose to 95% within a month.

Crawl budget: what it is and how to optimize it

If the site is large and the budget is small, some pages may remain unscanned. You can optimize the budget by:

• Eliminating duplicates and broken links so the bot doesn’t waste time on unnecessary pages.
• Optimizing load speed — the faster the site loads, the more pages the bot can crawl.
• Proper internal linking setup so important pages are as accessible to the bot as possible.

Features of Google’s Mobile-First Index

Since 2021 Google has used the mobile version of a site as the basis for indexing and ranking. If you don’t have a responsive version or the mobile site is stripped of functionality, this will directly affect indexing and rankings. At BUSINESS SITE we always check that the site displays and works correctly on mobile devices before launching any SEO campaigns.

Site indexing problems in Google Search Console: common errors and diagnostics

Even if the site is technically fine, indexing can be slowed by problems with content, structure, or sanctions from Google. We’ll go through the most common scenarios.

Common errors and Google Search Console statuses

Google Search Console is the main tool for diagnosing indexing problems. Here you can see which pages are indexed and which are not, and why. Most often we encounter the following statuses:

• Page not indexed – it may be blocked in robots.txt, have a noindex tag, or simply not be found by the crawler.
• Duplicate content: if the site has pages with identical or very similar content, Google may index only one of them.
• 4xx and 5xx errors: page not found (404), server not responding (500), etc.
• Manual Google sanctions – if the site violates rules (for example, uses hidden text or links), Google may manually lower its ranking or remove it from the index entirely.

In one case for a travel agency we found that because of duplicate content only 50% of the tour pages were indexed. After implementing canonical tags and optimizing the site structure, indexing rose to 90%.

Risks of manual Google sanctions

Manual sanctions are direct interventions by a Google employee in a site’s ranking. They are applied for serious violations: spam, cloaking, hidden text, paid links, etc. If your site is hit by sanctions, indexing of new pages may stop completely. At BUSINESS SITE we always recommend clients avoid “gray” promotion methods and regularly check the site’s status in Google Search Console.

Google shadow ban: what it is and how to avoid it?

A shadow ban is an unofficial term for a situation where a site remains in the index but its pages are hardly shown in search results. Causes can vary: low content quality, mass use of noindex, technical errors. To avoid such problems, it’s important to regularly monitor indexing and promptly respond to changes.

Accelerating page indexing – methods and tools

Traditional methods of speeding up indexing still work, but their effectiveness is limited, especially for large and frequently updated sites. Let’s review the main approaches.

Traditional methods for speeding up a site

• Internal linking – the more links point to a new page from other sections of the site, the faster a crawler will find it.
• Sitemap.xml, a regularly updated sitemap file helps Google discover new and changed pages faster.
• Optimizing load speed – a fast site is crawled more efficiently.
• Eliminating broken links and unnecessary redirects – so the crawler doesn’t waste time on errors.

In a project for an online store we achieved a 30% speed-up in indexing new products by optimizing internal linking and the sitemap.

Ping services and Google Search Console — what are they?

After publishing a new page you can submit its URL to Google via the URL Inspection tool in Google Search Console. This speeds up discovery but does not guarantee fast indexing, especially if the site is large or frequently updated. Ping services (for example, pingomatic.com) can also help, but their effectiveness is limited.

For more effective and faster indexing it’s worth using the Google Indexing API — a tool we will describe below.

Google Indexing API: what it is and how it works

For projects where indexing speed is critical (news portals, job aggregators, sites with frequent promotions), traditional methods are no longer enough. The Google Indexing API is a tool that allows you to submit URLs for indexing directly, bypassing standard crawling. At BUSINESS SITE we implement this API for clients who need new pages to appear in search within hours, not days.

Google Indexing API: capabilities and application

Google Indexing API is a programmatic interface that allows developers and SEO specialists to submit URLs for indexing to Google automatically. Let’s look at its key features.

Which pages does the API support?

It’s important to understand: Google Indexing API does not support all page types. As of 2025, the API is primarily intended for:

• Job Posting: pages with job postings marked up with Schema.org microdata (JobPosting).
• BroadcastEvent: pages with information about events, also with the corresponding microdata.

Regular website pages (for example, articles, product pages, categories) cannot be submitted via this API: they are crawled via standard crawling. This limitation is related to Google’s policy and the API’s purpose: to speed up indexing of exactly those pages that are frequently updated and critical for users (job postings, events).

In one project for an HR agency, we managed to get new job postings into Google’s index within 15–30 minutes after publication, using the Google Indexing API and correct microdata.

Now let’s look at the limits and quotas that affect the use of the Google Indexing API.

API limits and quotas: what you need to know

By default the Google Indexing API has strict limits: no more than 200 URLs per day per project. For most businesses this is sufficient, but large aggregators or news portals may need to increase the quota. A quota increase request is submitted through Google Cloud Platform, but approval is not guaranteed: Google reviews each case individually.

What does access security depend on?

Working with the API requires a service account in Google Cloud Platform and a JSON key for authentication. The service account must be granted owner or editor rights in Google Search Console for the relevant site. This ensures security and access control: only authorized users can submit URLs for indexing.

Setting up the Google Indexing API for bulk page indexing

Now let’s move on to the practical part: how to set up the Google Indexing API to speed up page indexing. The instructions are based on our experience implementing this solution for BUSINESS SITE clients.

Creating a service account and a JSON key in Google Cloud

1. Go to Google Cloud Platform (GCP) and create a new project (or select an existing one).
2. Enable the Indexing API in the “API Library” section.
3. Create a service account in the “IAM & Admin” → “Service Accounts” section.
4. Generate a JSON key for this account and save the file on your computer.

Step 2: project preparation and adding the key

1. Create a folder for the project on the server or locally.
2. Place the JSON key in this folder (for example, service-account-key.json).
3. Install the required dependencies (for example, google-api-nodejs-client for Node.js).

Step 3: Service account permissions in Google Search Console

1. Go to Google Search Console and select the site you need.
2. Add the service account as an owner or editor.
3. Check access – the account should see the site in the list of properties.

Step 4: Enabling the Web Search Indexing API in GCP

1. Return to GCP and make sure the Indexing API is enabled for your project.
2. Check the quotas – by default 200 URLs per day.

Installing and configuring the Node.js library for the API

1. Install Node.js on the server or local machine.
2. Install the googleapis library via npm.
3. Write a script to send URLs for indexing. Example code:

const { google } = require('googleapis');
const indexing = google.indexing('v3');
const key = require('./service-account-key.json');

const jwtClient = new google.auth.JWT(
  key.client_email,
  null,
  key.private_key,
  ['https://www.googleapis.com/auth/indexing'],
  null
);

jwtClient.authorize(function(err, tokens) {
  if (err) {
    console.log(err);
    return;
  }

  const options = {
    url: 'https://example.com/job-posting-1',
    type: 'URL_UPDATED'
  };

  indexing.urlNotifications.publish({
    auth: jwtClient,
    requestBody: options
  }, (err, res) => {
    if (err) {
      console.log('Error:', err);
    } else {
      console.log('Response:', res.data);
    }
  });
});

Submitting URLs for indexing via a Node.js script

1. Prepare a list of URLs to send (only supported page types).
2. Run the script – each URL will be submitted for indexing.
3. Monitor the status through Google Search Console.

At BUSINESS SITE, we automated this process for clients in the HR and event industries, which reduced indexing time from several days to a few minutes.

Google Indexing API errors and limits: how to fix them?

Like any API, the Google Indexing API has its limitations and common errors. Let’s go over the most frequent scenarios and ways to resolve them.

Code 429 – Too Many Requests: what does it mean?

If you have exceeded the daily quota (200 URLs), the API will return error 429. The solution is to distribute URL submissions evenly throughout the day or request a quota increase through the Google Cloud Platform. At BUSINESS SITE we recommend that clients with a large number of pages plan their load in advance and, if necessary, submit a request to increase the limit.

Code 403 Forbidden: what does it mean?

This error means that the service account does not have permission to submit URLs for this site. Check that the account is added as an owner or editor in Google Search Console and has access to the required property.

Error «The system cannot find the path»

This error occurs if the URL is incorrect or the page does not exist. Always check that the URL is working and corresponds to a supported type (Job Posting, BroadcastEvent).

Request for an API quota increase

If your business needs more than the standard quota, you can submit a request for an increase through the Google Cloud Platform. Prepare a justification: describe your business model, the volume of pages, and the update frequency. Google reviews each request individually and there are no guarantees, but for large projects this is the only way to scale indexing.

Best practices for preventing errors

• Control the number of requests: do not exceed the limit.
• Check access rights: the service account must have the required level of permissions.
• Validate URLs: submit only working and supported pages.
• Monitor API responses, log errors and respond promptly.

Checking URL indexing: how to check a page?

After submitting a URL through the Google Indexing API, it’s important to make sure the pages have actually been indexed. Let’s look at the main tools and methods for checking.

Checking a URL in Google Search Console

The simplest method is to enter the URL into the URL inspection tool in Google Search Console. There you’ll see the page status, the date of the last crawl, and any possible errors. This method is suitable for single checks.

Operator site:: checking page indexing

In Google search, enter the operator site:your-domain.com/page-url. If the page appears in the results, it is indexed. This method is convenient for a quick check but is not suitable for bulk analysis.

Bulk indexing check in Serpstat

For large sites, it’s convenient to use specialized services like Serpstat. They allow you to upload a list of URLs and get a report on the indexing of each page. At BUSINESS SITE we use such tools for regular indexing audits for our clients.

Indexing issues: how to identify and fix them

If a page is not indexed, check:

• Technical errors (4xx, 5xx, redirects).
• Presence of structured data (for Job Posting, BroadcastEvent).
• Service account access rights.
• API limits: whether the quota has been exceeded.

In case of recurring problems, we recommend conducting a comprehensive site audit and addressing the technical issues.

Google Indexing API and speeding up indexing: answers to common questions

In this section I will answer the most frequent questions our clients and BUSINESS SITE partners ask about speeding up indexing and working with the Google Indexing API.

• How does the Google Indexing API help speed up the appearance of new pages in the index and increase SEO traffic?
  The API allows you to submit URLs directly to Google, bypassing standard crawling. This is especially important for pages that are frequently updated and critical to the business (job openings, events). As a result, new pages appear in search within minutes or hours instead of days, which directly affects SEO traffic and conversions.
• What limits and quotas exist when using the Google Indexing API and how can they be increased?
  By default – 200 URLs per day per project. To increase the quota you need to submit a request in Google Cloud Platform with justification. There are no guarantees, but for large projects this is the only way to scale the process.
• What types of pages can be submitted for indexing via the Google Indexing API, and why do regular pages not always qualify?
  The API supports only pages with Job Posting and BroadcastEvent microdata. Regular pages (articles, products, categories) cannot be submitted; they rely on standard crawling. This limitation is related to Google’s policy and the purpose of the API.
• How to correctly configure a service account and access rights in Google Cloud and Search Console to work with the API?
  You need to create a service account in GCP, generate a JSON key, add the account as an owner or editor in Search Console, and enable the Indexing API in GCP.
• What are the main errors (for example, 429, 403) that occur when working with the Google Indexing API and how to avoid them?
  429: quota exceeded, 403: no access rights. Solution: control the number of requests, check the service account permissions, validate URLs before submitting.
• How to integrate the Google Indexing API with automated processes (for example, with Node.js) to scale indexing?
  Use the googleapis library for Node.js, write a script for bulk URL submission and integrate it into your CI/CD or CMS.
• What metrics and KPIs should be used to evaluate the effectiveness of accelerated indexing with the API?
  Time from publication to appearance in search, growth of organic traffic on new pages, conversions from those pages. At BUSINESS SITE we always link technical metrics to business results.
• How do crawl budget and site structure affect indexing speed, and how to optimize them?
  The better the structure and internal linking, the faster the crawler will find new pages. Budget optimization includes eliminating duplicates, broken links, and speeding up load times.
• What risks and long-term consequences can arise from improper use of the Google Indexing API?
  Temporary blocks for exceeding quotas, loss of domain trust when mass-submitting unsupported URLs. It is recommended to strictly follow Google’s rules.
• How to check that pages are actually indexed after submitting via the API, and which tools to use for this?
  Google Search Console, the site: operator, bulk services like Serpstat.
• How cost-effective is implementing the Google Indexing API for a business in terms of ROI and marketing goals?
  For projects where indexing speed is critical (HR, events, news), the ROI is very high. For ordinary sites, traditional methods are often sufficient.
• How to ensure security and access control when working with service accounts and API keys?
  Store JSON keys in a secure location, restrict access to the service account, and regularly update permissions.
• What are the best practices for working with Job Posting and BroadcastEvent microdata for successful indexing?
  Use Schema.org validators, check the markup in the Google Rich Results Test, and update data when information on the page changes.
• How to avoid manual Google penalties when mass-indexing through the API?
  Do not abuse the API, submit only supported page types, respect limits, and do not attempt to game the system.
• What alternative methods exist to speed up indexing and how do they compare to using the Google Indexing API?
  Traditional methods (sitemaps, internal linking, ping services) work but are slower. The API is the fastest way for supported page types.

The strategic importance of indexing and the Google Indexing API for business

Indexing is the foundation of SEO and online sales. If your site isn’t indexed in time, you lose traffic, customers, and money. Traditional methods for speeding up indexing are still important, but for projects with highly dynamic content they are no longer sufficient.

At BUSINESS SITE we implement this API for clients for whom the speed of new content appearing in search is critical. The result: increased organic traffic, higher conversions, and transparent ROI from investments in content and promotion.

But it’s important to remember: the API is not a panacea. Effective indexing requires a comprehensive approach: technical site optimization, quality content, correct micromarkup, error control, and regular monitoring. Only this way will you achieve maximum results and avoid risks.

If you want your site to always be visible in search and for new pages to appear where your business needs them, start with an indexing audit, implement best practices and consider using the Google Indexing API to automate the process. At BUSINESS SITE we are ready to help with analysis, setup, and support – contact us for expert advice and real solutions that deliver results.