XML Sitemap SEO: What to Submit and Leave Out

A sitemap is the smallest piece of technical SEO with the largest amount of misunderstanding attached to it.

In 13 years of fixing technical problems on Australian websites, the most common version I find is a file generated by a plugin the day the site launched and never looked at again, pointing at redirected URLs, noindexed tag archives and filter parameters. The site owner has been told a green tick in an audit tool means the sitemap is fine.

What follows is what actually belongs in the file, what should never be in it, and how to use it so it helps rather than wastes attention.

What an XML sitemap does and does not do

A discovery aid, not a ranking lever

A sitemap tells search engines which URLs exist and when they were last changed. That is it. It is a list, not a recommendation, and Google is direct about the limits. Google Search Central's guidance on sitemaps states that "A sitemap helps search engines discover URLs on your site, but it doesn't guarantee that all the items in your sitemap will be crawled and indexed."

Nothing in that sentence should be a surprise, yet almost every conversation I have about a page that will not index starts with someone wanting to add it to the sitemap. Discovery and indexing are different problems. The sitemap solves the first one.

When you genuinely do not need one

Small sites that are linked together properly are usually found without help. Google says as much: "If your site's pages are properly linked, Google can usually discover most of your site." A brochure site of a few dozen pages, with a working menu and internal links, is discoverable without a sitemap.

Keep one anyway, because it costs almost nothing to maintain and it earns its place the moment the site grows, the moment you add images or video worth surfacing, or the moment a section becomes orphaned by a navigation change. The mistake is treating it as a fix for something it cannot fix.

What belongs in your sitemap

Canonical, indexable URLs only

The rule is simple to state. Include the canonical, indexable version of each URL you want found, and nothing else. If a page is noindexed, redirected, blocked in robots.txt or a duplicate created by filters and parameters, it does not belong in the list.

Every URL in the file should answer yes to three questions: is this the address a search engine should use, is this page allowed to be indexed, and is this a page a human would want to land on from search. If any answer is no, leave it out.

The pages you want found, not every page you have

A sitemap is a curated statement of intent. It says these are the pages that matter. Plugins do not know that, which is why an untouched WordPress sitemap often includes author archives, date archives and tag pages that add nothing to the site.

Decide what matters by looking at the site the way a customer would. Product or service pages, category pages, location pages, useful articles, and the key static pages such as contact and about. Everything else has to earn its place.

What to leave out

Noindexed, redirected and parameter URLs

A URL that redirects should never be in a sitemap. You are telling a search engine to fetch an address and then immediately sending it somewhere else, which is a wasted fetch and a contradictory signal. A noindexed URL is worse, because the sitemap says include and the page says exclude, and the page wins.

Parameter URLs are the quiet destroyer of crawl efficiency. A filter combination that creates thousands of near-identical addresses will flood the file if the plugin generates them. If those pages are not meant to be indexed, they should not be listed.

Drafts, staging copies and internal search results

Drafts, staging domains and internal site search result pages should never appear. Staging copies are the most dangerous of the three, because a staging URL in a sitemap can be indexed and can outrank the real page for a while.

Internal search results are a lesser sin with a similar effect. They are generated pages with no unique value, and they can produce an unlimited number of URLs from a finite site.

The limits that force a split

The 50,000 URL and 50MB ceilings

Every format has the same ceiling, and it is documented. Google's guide to building and submitting a sitemap states that "All formats limit a single sitemap to 50MB (uncompressed) or 50,000 URLs. If you have a larger file or more URLs, you must break your sitemap into multiple sitemaps."

If you are anywhere near either number, split well before you hit it. A sitemap that times out mid-fetch is worse than three smaller files, because a partial read gives an incomplete picture of the site.

Sitemap index files and how they nest

A sitemap index is a file that lists other sitemaps. It has its own limits and its own placement rules. Google's documentation on managing sitemaps with a sitemap index file explains that "You can submit up to 500 sitemap index files for each site in your Search Console account." The same page notes that "Sitemaps that are referenced in the sitemap index file must be in the same directory as the sitemap index file, or lower in the site hierarchy."

That second rule catches people out during a rebuild. If the index sits at the domain root, every file it references has to live at the root or in a folder below it. A sitemap parked in a sibling directory will be ignored.

lastmod: the one field Google reads

What counts as a significant change

Of the three optional fields, only one is used, and the company's own documentation is precise about the conditions attached to it. Google's guide to building and submitting a sitemap states that "Google uses the <lastmod> value if it's consistently and verifiably (for example by comparing to the last modification of the page) accurate." The two conditions matter equally: consistently and verifiably.

The company's own guidance on the ping endpoint makes the intent explicit. As Google's announcement on the sitemaps ping endpoint puts it, "And when we say "last modification", we actually mean "last significant modification"." A price change, a rewritten section, new structured data or changed internal links count. A timestamp updated by a plugin on every save does not.

Why inflated dates eventually stop working

A lastmod value that claims every page changed yesterday, every day, is not a useful signal. It is a signal that gets discounted, and once it is discounted the pages that genuinely did change lose their advantage too.

The practical approach is to let the field reflect reality, or to omit it. An absent lastmod is honest. A lastmod that is always today's date is noise, and you have spent effort creating it.

priority and changefreq: the fields to ignore

What Google says it does with them

Nothing. Google's sitemap documentation states flatly that "Google ignores <priority> and <changefreq> values." Both fields were part of the original specification and both have been unused for years.

The implication is straightforward. If you have spent time setting priority values for different page types, that time produced no result. If a previous developer did it, there is no need to unpick it, and there is no benefit in maintaining it.

Why they still appear in most plugins

They persist because they were in the specification the plugins were written against, and because setting them feels like doing something useful. Regenerating them on a schedule also makes a sitemap look active, which is worth nothing if the URLs inside it are wrong.

Leave them alone and spend the attention on the URL list instead. Removing a hundred redirected URLs from a file does more for crawl efficiency than any priority value ever did.

Submitting and monitoring the file

Search Console and the robots.txt line

Submit the file in Google Search Console, and reference it from robots.txt with a sitemap directive. Both routes work, and doing both is sensible because the robots.txt line is useful to other crawlers that read it.

Keep the location stable. Moving a sitemap between paths, or changing its name during a redesign, breaks the submission and the historical data attached to it. If the file has to move, add a redirect and resubmit deliberately.

What the ping retirement changed

The old approach of pinging a search engine each time the sitemap changed is gone. Google's announcement on the subject states that "The sitemaps ping endpoint deprecation is complete." The stated reason was that unauthenticated submissions were largely spam.

Nothing needs replacing. Submit through Search Console, or leave the robots.txt reference in place and let the file be discovered when it changes. Some plugins still call the retired endpoint. It is harmless, and it also does nothing.

Sitemap mistakes that waste crawl attention

Mixed signals between sitemap, robots and canonical

The three signals have to agree. A URL listed in the sitemap, blocked in robots.txt and pointing at a canonical elsewhere is being told three different things at once. Crawlers resolve the conflict by picking one, and it is rarely the one you wanted.

Where the signals disagree, fix the page rather than the sitemap. If a page should not be indexed, the sitemap entry should go and the page should be handled properly on its own terms.

Files that 404, redirect or time out

Check that the sitemap URL returns a 200 with the correct content type. A sitemap referenced in robots.txt that returns a 404 wastes a crawl and reports nothing useful, and a sitemap behind a redirect is being read at third hand.

Then check the size of the response. A large file served slowly, or one generated on demand, will time out for some crawlers. A static file that is regenerated when content changes is faster and more reliable than one assembled on every request.

The wider discipline here is a crawl and index review done properly once or twice a year, which is what an SEO audit that starts with crawl and index health covers, and what run through the technical SEO checklist is designed to work through in order. If a technical problem has been sitting unresolved for months because it is hard to pin down, it is usually cheaper to get a technical issue diagnosed properly than to keep guessing at it. There are more plain English SEO explanations on the blog if you want to work through the rest yourself.

XML sitemap questions I get asked most

Do I even need a sitemap for a small business website?

Often you do not, strictly speaking. Google's own guidance is that if your pages are properly linked it can usually discover most of your site, and that a small site of about 500 pages or fewer, comprehensively linked internally, may not need one. In practice a sitemap is cheap to keep and useful for new sites, larger sites, and any site with images, video or news pages worth surfacing.

Should every page on my site be in the sitemap?

No. Include the canonical, indexable URLs you want found and leave out the rest. If a page is noindexed, redirected, blocked or a duplicate created by filters and parameters, listing it creates a contradiction between your sitemap and your page, and mixed signals are worse than a smaller file.

My plugin adds every product and tag. Is that a problem?

It can be, particularly if those URLs are thin, duplicated or noindexed. A sitemap should describe the pages you consider worth indexing. If your product tag archives add nothing, either improve them or leave them out. When a URL appears in a sitemap and as a noindex page at the same time, the page is not going to be indexed because of it.

How honest does lastmod need to be?

Honest enough to match reality. Google states it uses the lastmod value if it is consistently and verifiably accurate, and the guidance is that it should reflect a significant change to the page, such as main content, structured data or links, rather than a trivial tweak. If a page changed years ago and the field says yesterday, the signal stops being trusted.

What happened to the sitemap ping I used to run?

The endpoint was retired. Google announced in June 2023 that the sitemaps ping endpoint deprecation was complete, and the reason given was that unauthenticated submissions were largely spam. You submit through Search Console or reference the file in robots.txt instead. Some plugins still call the old endpoint; harmless, but also useless.

At what point do I need a sitemap index file?

Once a single sitemap would exceed the limits. The documented ceiling for one sitemap is 50MB uncompressed or 50,000 URLs, so larger sites split into several sitemaps and list them in a sitemap index. You can submit up to 500 sitemap index files per site in Search Console, and the referenced sitemaps must sit in the same directory as the index file or lower.