Your website might look great and load fast, but if search engines can’t find and crawl your pages efficiently, you’re invisible to potential customers. That’s where XML sitemap best practices come in. An XML sitemap acts as a roadmap for search engines like Google and Bing, telling them exactly which pages exist on your site, when they were last updated, and how important each page is relative to the others.
At Avatar Website Design, we build websites for small businesses that are designed to get found, and a properly configured XML sitemap is a critical piece of that puzzle. We’ve seen firsthand how a missing or poorly structured sitemap can hold back an otherwise solid site from ranking where it should in local search results. It’s one of those behind-the-scenes technical details that makes a real difference in whether your business shows up when customers are searching.
This guide covers everything you need to know: the technical rules your sitemap must follow, optimization tips to help search engines crawl your site more effectively, and step-by-step instructions for submitting your sitemap to major search engines. Whether you’re building your first website or cleaning up an existing one, these practices will help your pages get indexed faster and more reliably.
Why XML sitemaps matter for crawling and indexing
Search engines use automated programs called crawlers to navigate the web and discover pages. These crawlers follow links from one page to another, but they don’t always find every page on your site that way. An XML sitemap gives crawlers a direct, structured list of your URLs, so they don’t have to rely solely on following links. Following xml sitemap best practices ensures that crawlers spend their time on the pages that actually matter to your business, rather than wasting time on duplicates or dead ends.
How search engines discover your pages
When Googlebot visits your site, it starts by reading your sitemap if one is available. It then uses the information in that file to prioritize which pages to crawl and how frequently to revisit them. Without a sitemap, the crawler depends entirely on your internal link structure to find pages. If you have a page that isn’t linked from anywhere else on your site, it becomes what’s called an "orphan page," and crawlers may never find it at all.
A well-structured sitemap is your most direct line of communication with search engines about what exists on your site.
This matters most when you publish new content or update existing pages. A sitemap signals to search engines that something has changed and gives them the URL to check. That reduces the gap between when you publish a page and when it actually appears in search results, which is especially important for small business owners trying to attract local customers quickly.
The crawl budget problem small sites often face
Google allocates a crawl budget to every site, which is essentially the number of pages Googlebot will crawl within a given timeframe. For large sites with thousands of pages, this is a well-known technical concern. But small business websites face the same issue in a different way: if your site has thin pages, duplicate content, or broken links, crawlers waste their budget on those instead of your core service pages.
Your sitemap directly addresses this. By listing only the pages you want indexed, you guide crawlers toward your most valuable content. You’re not forcing search engines to waste resources crawling a terms-and-conditions page when you want them focused on your services or product pages. Google’s own documentation on sitemaps confirms that sitemaps are especially useful for large sites, new sites with few external links, and sites with a lot of rich media content.
What indexing actually depends on
Crawling and indexing are two separate steps, and your sitemap influences both. Crawling means a search engine visits the page. Indexing means it stores and considers that page for search results. A page can be crawled but not indexed if it has thin content, a noindex tag, or other issues. Your sitemap doesn’t guarantee indexing, but it removes the first barrier by making sure crawlers can find the page in the first place.
For small business websites where every page represents a service, a location, or a product, making sure those pages get crawled and indexed isn’t optional. It’s the foundation of any working SEO strategy.
XML sitemap rules you must follow
Before you focus on optimization, your sitemap needs to meet specific technical requirements set by search engines. These aren’t suggestions. If your sitemap violates these rules, Google may ignore the file entirely or fail to process it correctly, which defeats the whole purpose of having one. Understanding these requirements is the starting point for any effective xml sitemap best practices strategy.
Technical format requirements
Your sitemap must be a valid XML file that follows the sitemap protocol established at sitemaps.org, which Google, Bing, and other major search engines recognize. Every sitemap file needs a proper XML declaration at the top, and all URLs must be wrapped in the correct tags. The root element must be <urlset>, and each individual URL must be enclosed in a <url> tag with a required <loc> child element that contains the full, absolute URL including the protocol (https:// or http://).
If you use relative URLs like /services instead of https://yoursite.com/services, search engines will reject those entries.
Your sitemap file must also use UTF-8 encoding, and any special characters in URLs, such as ampersands or spaces, must be properly escaped. An ampersand in a URL, for example, must be written as & in the XML. Skipping this causes parsing errors that break the entire file.
Size and URL limits
Google enforces strict limits on sitemap file size and URL count. Each sitemap file can contain a maximum of 50,000 URLs and cannot exceed 50MB when uncompressed. These limits apply per file, not per site. If your site exceeds either limit, you need to split your URLs across multiple sitemap files and reference them using a sitemap index file.
For most small business websites, these limits are not a concern. But if you run an e-commerce site with hundreds of product pages, or a site with a large blog archive, you should monitor your URL count as your site grows. Keeping each sitemap file under these thresholds ensures search engines can fully process your sitemap without errors or incomplete reads.
How to build an XML sitemap that search engines trust
Building a sitemap that follows proper xml sitemap best practices doesn’t require you to write XML by hand. Most modern website platforms and content management systems give you built-in tools or plugins that handle the technical formatting automatically. What matters is that you understand what a trustworthy sitemap looks like so you can verify the output rather than just assume it’s correct.
Generate your sitemap with your CMS
If you use WordPress, plugins like Yoast SEO or Rank Math generate your sitemap automatically and keep it updated as you add or remove pages. Shopify and Squarespace also create sitemaps automatically, and most hosted platforms place yours at a standard location like yoursite.com/sitemap.xml. Before you do anything else, check that URL in your browser. If you see a structured list of URLs rendered in XML format, your sitemap exists. If you see a blank page or an error, you need to create or fix the file before moving forward.
Check your sitemap URL directly in your browser before submitting anything to search engines. A broken file wastes your time and delays indexing.
Validate your sitemap file
Once you have a sitemap file, run it through a validation check. Google Search Console flags errors after submission, but catching problems before you submit saves time and avoids delays in indexing. Manually review the XML structure to confirm that every URL uses the full absolute path, the file opens without browser errors, and no restricted or noindex pages appear in the list.
Pay attention to consistency as well. Every URL in your sitemap should use the same version of your domain, either the www version or the non-www version, but never both. The same applies to HTTP versus HTTPS. Mixed URL formats create duplicate content signals that confuse search engines and undermine the reliability of your file.
Update your sitemap regularly
Your sitemap should reflect the current state of your site at all times. When you add a new service page, publish a blog post, or remove outdated content, your sitemap needs to reflect those changes. Automated tools handle most of this for you, but checking your sitemap after major site updates confirms the file stayed accurate and complete.
What to include and exclude in your XML sitemap
Knowing the technical rules is only half the work. The other half is making deliberate choices about which pages belong in your sitemap and which ones should stay out. A bloated sitemap filled with low-value URLs sends search engines in the wrong direction, while a lean, accurate sitemap helps them focus on the pages that actually drive traffic and conversions for your business.
Pages that belong in your sitemap
Your sitemap should contain every page you want Google to crawl and index. That means your core service pages, location pages, product pages, category pages, and any blog posts or articles that provide genuine value to readers. If a page is live, canonical, and has real content worth ranking, it belongs in your sitemap. Following xml sitemap best practices means treating your sitemap as a curated list, not a complete dump of every URL your site generates.
Include only the pages you actively want to appear in search results. Every unnecessary URL in your sitemap dilutes the signal you’re sending to search engines.
Each URL you list should also be accessible without redirects and return a 200 status code. If a URL currently redirects to another page, list the final destination URL instead. Submitting redirect URLs forces search engines to follow an extra step, which wastes crawl budget and introduces unnecessary confusion.
Pages to leave out
Certain pages hurt more than they help when included in a sitemap. Paginated archive pages, tag pages, search result pages, and filtered product listing URLs typically generate duplicate or thin content that search engines should not index. The same applies to admin pages, login screens, thank-you pages, and any page protected by a password.
You should also exclude any URL that carries a noindex tag. Including a noindex page in your sitemap creates a direct contradiction: you’re telling search engines to visit the page while simultaneously telling them not to store or rank it. Search engines may follow one instruction and ignore the other, which produces unpredictable results. Remove those pages from your sitemap and let the noindex tag handle them on its own.
When to use sitemap indexes and special sitemaps
A single XML sitemap file works well for most small business websites, but certain situations require a different approach. When your site grows in size or hosts specific types of media, a standard sitemap either hits its limits or fails to give search engines the structured data they need to properly crawl and rank your content. Knowing when to step up your setup is part of applying solid xml sitemap best practices to a real-world site.
Sitemap index files for large sites
Once your site exceeds 50,000 URLs or the 50MB file size limit, you can no longer fit everything into a single sitemap file. A sitemap index file solves this by acting as a container that references multiple individual sitemap files. Instead of listing URLs directly, the index file lists the locations of your other sitemaps using <sitemap> and <loc> tags. Search engines read the index first and then follow each referenced file.
A sitemap index is also useful for organizing large sites by content type, such as keeping product pages, blog posts, and location pages in separate files for cleaner crawl management.
You submit the sitemap index file to Google Search Console the same way you would a regular sitemap. Google then processes each referenced file individually. This approach keeps your sitemap structure clean and makes it easier to spot indexing issues by isolating specific sections of your site when errors appear.
Image, video, and news sitemaps
Standard XML sitemaps tell search engines about your web pages, but they don’t communicate the details of rich media content like images and videos. Google supports extension tags that you can add directly to your existing sitemap or place in a dedicated media sitemap. Image sitemaps help Google discover images it might otherwise miss, especially images loaded through JavaScript or CSS, and they support additional metadata like captions and licenses.
Video sitemaps follow a similar structure and allow you to provide details like video duration, description, and thumbnail URLs. If your business publishes video content, a video sitemap helps Google surface that content in video search results, which is a separate and valuable source of traffic.
How to submit and monitor your sitemap in GSC
Submitting your sitemap to Google Search Console is the most direct way to tell Google your sitemap exists and to get accurate data on how many of your pages are being discovered and indexed. This step is free, takes only a few minutes, and gives you an ongoing window into how search engines see your site. Following xml sitemap best practices means you don’t just create a sitemap and forget about it. You actively use GSC to confirm it works.
Submitting your sitemap through Search Console
Log into Google Search Console and select your property. In the left sidebar, navigate to the Sitemaps section under Indexing. You’ll see a field where you can enter the URL path of your sitemap, typically something like sitemap.xml or sitemap_index.xml. Enter the path and click Submit. Google will immediately attempt to fetch the file and display a status result.
If Google returns an error on first submission, check that your sitemap URL is publicly accessible and that the file is valid XML before resubmitting.
After submission, GSC shows you the date it was last read, the number of URLs discovered, and whether any errors occurred. A successful submission doesn’t guarantee indexing, but it confirms Google can access and read the file without problems.
Reading GSC sitemap reports to catch issues
Once your sitemap is submitted, check the Sitemaps report at least once a month to catch problems before they affect your rankings. The report shows you how many URLs were submitted versus how many were actually indexed. A large gap between those two numbers signals that Google is finding pages but choosing not to index them, which points to content quality issues, duplicate content, or conflicting directives like noindex tags on pages you want ranked.
GSC also flags specific errors such as unreachable URLs, incorrectly formatted entries, or sitemap files that return a non-200 status code. Fixing those errors promptly keeps your sitemap working correctly and ensures search engines continue to trust the data you’re providing. Treat the Sitemaps report as a routine health check for your site’s indexing performance.
Quick recap and next steps
You now have everything you need to put xml sitemap best practices into action. A valid, well-structured sitemap removes the first barrier between your pages and search engine indexes. The core rules are straightforward: use proper XML formatting, stay within the 50,000 URL and 50MB limits, include only canonical and indexable pages, and exclude anything with a noindex tag or redirect chain. Build your sitemap with your CMS tools, validate it before submission, and submit it through Google Search Console so you can track discovery and indexing over time. Check the Sitemaps report monthly to catch errors early and close gaps between submitted and indexed URLs.
Getting these technical details right matters most when your website needs to generate real business results. If you want a site built with proper SEO foundations from day one, Avatar Website Design helps small businesses get online with professionally designed, search-ready websites that work as hard as you do.