To get Google to crawl your website quickly, submit your sitemap through Google Search Console and request indexing for your main pages. You can also improve your site structure, fix broken links, and ensure your robots.txt file does not block search engine bots.
Submit your sitemap to Google Search Console
The fastest and most reliable way to tell Google that your website exists is through Google Search Console. This free tool allows you to submit your XML sitemap directly to the search engine, which provides a complete list of all the important pages on your domain. Once submitted, Google bots can systematically discover and read your content much faster than waiting for natural discovery through external links.
If you use a content management system like WordPress, your platform likely generates this sitemap automatically, often located at your domain name followed by sitemap.xml. Simply copy that address and paste it into the sitemap submission section of your Search Console dashboard to alert Google of your pages.
- Create a free Google Search Console account for your domain
- Verify your website ownership using DNS records or HTML tags
- Locate your XML sitemap URL within your website platform
- Paste the sitemap URL into the Search Console sitemap tool and click submit
Request manual indexing for individual pages
When you publish a brand new page or make significant updates to an existing one, you do not have to wait for Google to find it organically. Google Search Console includes a URL inspection tool that lets you paste any specific page link and request an immediate review. This tells the system to prioritize crawling that particular URL within a short timeframe.
Keep in mind that requesting indexing does not guarantee instant appearance in search results, but it does prompt the crawler to visit and evaluate the page content sooner. Use this feature sparingly for your most important homepage, service pages, and fresh blog posts rather than trying to force every minor update all at once.
- Copy the exact URL of the page you want Google to crawl
- Paste the link into the search bar at the top of Google Search Console
- Wait for the tool to analyze the current index status of the page
- Click the Request Indexing button to queue the URL for a fresh crawl
Fix technical barriers that block crawlers
Sometimes websites fail to get crawled because technical errors accidentally block search engine bots. Review your robots.txt file to ensure you are not unintentionally telling search engines to stay away from your important directories. Additionally, clean up broken internal links and redirect loops that waste crawler time and prevent bots from reaching deep into your site architecture.
Make sure your hosting server responds quickly and reliably when requests come in. If a website takes too long to load or experiences frequent downtime, search engine crawlers may abandon their visit before reading your content, assuming the site is broken or abandoned.
- Check your robots.txt file for accidental disallow rules
- Ensure your navigation menus use standard crawlable links
- Fix broken links and eliminate unnecessary redirect chains
- Verify your web hosting uptime and server response speed
Crawling, indexing and ranking are three different steps
- Crawling is when Googlebot visits a page and downloads it.
- Indexing is when Google decides the page is worth storing in its search database.
- Ranking is where the page shows up when someone searches.
Getting crawled does not guarantee the other two. Google crawls far more pages than it indexes, and it skips pages it sees as thin, duplicated or low value. So the goal is to make your pages easy to find, and also worth keeping.
How to tell if Google has crawled your page
- URL Inspection in Search Console. Paste the page address. It tells you whether the URL is on Google, when it was last crawled, which sitemap or page Google found it through, and whether anything blocked it.
- The Page indexing report. Under Indexing > Pages, Search Console lists which of your pages are indexed and gives a reason for every page that is not.
- The Crawl stats report. Under Settings, it shows how many requests Googlebot made to your site, how fast your server answered, and any server errors.
- A quick site: search. Typing site:yourdomain.com into Google gives a rough look at what is indexed. It is not a complete count, so use Search Console for real answers.
New to Search Console? See what Google Search Console is.
What the most common Search Console statuses mean
| Status | What it means | What to do |
|---|---|---|
| Discovered – currently not indexed | Google knows the URL exists but has not crawled it yet. | Add internal links to it from important pages, make sure it is in your sitemap, and check that your server is not slow. |
| Crawled – currently not indexed | Google read the page and chose not to index it for now. | Improve the content. Make it more complete and clearly different from your other pages. |
| Excluded by noindex tag | The page tells search engines not to index it. | Remove the noindex setting if you want the page in Google. |
| Blocked by robots.txt | Your robots.txt file stops Googlebot from crawling the page. | Edit robots.txt to allow it. |
| Page with redirect | The URL sends visitors to a different address. | Normal for old URLs. Make sure your sitemap and menus link to the final address. |
| Duplicate without user-selected canonical | Google sees the page as a copy of another one. | Merge similar pages or set a canonical URL that points to the main version. |
Check for an accidental “do not index” setting
A surprising number of new WordPress sites launch with search engines still blocked. In the WordPress dashboard, go to Settings > Reading and make sure “Discourage search engines from indexing this site” is not checked. Developers often turn it on while building the site and forget to switch it off at launch.
Also look at the SEO plugin settings for each page. Yoast, Rank Math and similar plugins can set single pages, categories or tags to noindex. That is fine for thin archive pages, but not for your service pages.
Build links that lead Google to your pages
Googlebot finds most pages by following links. A page that nothing links to, called an orphan page, may be crawled rarely or not at all, even if it is in your sitemap.
- Link every service page from your main menu or from a clear services page.
- Link new blog posts and articles from related pages, not just from the blog list.
- Use plain text links with words that describe the page, not only buttons or images.
- Keep important pages within a few clicks of the home page.
- Earn links from other sites, such as your Google Business Profile, local directories, chambers of commerce and partners. See what backlinks are and local citations.
Keep your sitemap clean
Your sitemap should only list the pages you want in Google. Leave out redirected URLs, deleted pages that return errors, pages set to noindex, thank-you pages and login pages. A sitemap full of dead or blocked URLs makes it harder for Google to trust it. In Search Console, open Indexing > Sitemaps to confirm the status says Success and to see how many pages Google found.
Do small sites need to worry about crawl budget?
Usually not. Crawl budget is the amount of crawling Google is willing to do on a site, and Google says it mainly matters for very large sites with many thousands of pages or pages that change constantly. A typical small business site with a few dozen pages can be crawled easily. If your pages are not being indexed, the cause is far more likely to be weak content, missing internal links, a noindex tag or a slow, unreliable server.
How long does it take?
Google says crawling can take anywhere from a few days to a few weeks. Requesting indexing in URL Inspection can speed up a single page, but there is a daily limit, and asking again for the same URL does not make it go any faster. Request it once, then give it time. For sites with many changed pages, an updated sitemap is the better option.
Do not forget Bing, which also feeds results to other search tools and AI assistants. Add your site to Bing Webmaster Tools and submit the same sitemap. Bing also supports IndexNow, a way for your site to notify it instantly when a page changes. Some WordPress SEO plugins can send these notices automatically. Google does not use IndexNow.
If your pages still are not showing up, contact us and we can take a look at your Tacoma business site.
Key takeaways
- Submit your XML sitemap directly through Google Search Console to speed up discovery.
- Use the URL inspection tool to request manual indexing for new or updated pages.
- Ensure your robots.txt file and server settings do not block search engine crawlers.
- Maintain a clear site structure with working internal links for easy bot navigation.
Related questions
How long does it take for Google to crawl a new website?
It can take anywhere from a few days to a few weeks for Google to discover and crawl a brand new website. Submitting your sitemap through Google Search Console can help speed up this initial discovery process significantly.
Why is Google crawling my site but not showing my pages in search results?
Crawling means Google has read your page, but indexing and ranking depend on content quality, relevance, and competition. If your content is thin, duplicate, or lacks clear value, Google may choose not to include it in the main search results.
Do I need to pay Google to crawl my website?
No, Google crawling and indexing are entirely free processes. You never need to pay any service or fee to get search engine bots to visit your website or display your pages in natural search results.
Should I click Request Indexing every day until my page shows up?
No. Google says repeated requests for the same URL do not get it crawled faster, and there is a daily quota. Request once, then work on internal links and content quality while you wait.
What does “Discovered – currently not indexed” mean?
Google knows the page exists but has not crawled it yet. Adding links to the page from your important pages and making sure it is in your sitemap usually helps. If many pages show this status, check your server speed too.
Does a Google Business Profile help Google find my website?
Adding your website link to your Google Business Profile gives Google another path to your site and connects your business listing to it. It is not a replacement for submitting a sitemap in Search Console.
