The sitemap that quietly dropped four pages
A generator owned one block of a shared file. Every time it ran it rewrote that block from its own list, and four pages it had never heard of disappeared. Nothing failed. Nothing warned.
A site had two kinds of page under the same folder. Thirty were produced by a script from a list in its source. Four were written by hand, one per product, and lived in the same folder because that is where they belonged for the reader.
The script also maintained the sitemap. Specifically, it rewrote the section of the sitemap covering that folder, from the only list it had: its own thirty. The four hand-written pages were not on that list, so every run removed them.
Why nobody noticed
Removing a URL from a sitemap is not an error. The file stays valid, the build succeeds, the pages themselves keep working and keep being linked from the homepage. The only symptom is slower discovery in search, weeks later, for exactly the four pages you most wanted found.
It was found by accident, during an unrelated change, when someone read the diff of a routine rebuild and asked why four lines were red.
The fix is about ownership, not about a list
The tempting repair is to hard-code the four names into the script. That works until someone adds a fifth, which is the same bug with a longer fuse.
The repair that holds is narrower deletion. The generator removes only the URLs it can prove it generated, and leaves every other URL in that section untouched. It now keys on "not mine" rather than on a list of exceptions, so a fifth hand-written page needs no change at all.
- A generator may delete what it created; it may not delete what it merely found
- Shared files need an owner per section, written down, or every tool assumes it owns the whole thing
- Silent removal is worse than a crash, because a crash gets fixed the same day
Worth checking on your own site this week: open your sitemap and count. If the number does not match the pages you believe you have, something is editing that file with a shorter list than yours.
The bug cost nothing to fix once seen. It cost four pages several weeks of search visibility before anyone saw it, and that is the part worth remembering about failures that do not announce themselves.