Can ChatGPT Read Your Website? I Checked Mine and Two Pages Were Blank

I had been telling people for weeks that my newsletter was built properly. I had checked it once, months earlier, and everything came back clean. So when I sat down to check it again, it was meant to take five minutes and confirm what I already knew.

Twenty two of my twenty four articles were fine.

The two that were not were my newest two. They were the ones I had put the most work into.

What came back was not a broken page or an error message. It was worse than that, because it looked fine to me. In a browser both articles loaded normally. The headline was there, the text was there, everything sat exactly where it should. The problem only showed up when I looked at what my server was sending out before any of that happened.

Here is the part that took me a while to understand, and it is worth sitting with, because it is the whole thing.

When you open a web page, two separate things happen. First your browser downloads a file from the server. Then your browser runs the code inside that file, which fills in anything that was not already there. To you those two events happen so fast they feel like one. To a machine reading your site they are completely separate, and most machines only do the first one.

Vercel studied real crawler traffic to see what these tools do when they visit a site. GPTBot pulled JavaScript files on about eleven percent of its visits and never ran them. ClaudeBot pulled them on about twenty four percent and never ran them. Perplexity does not run them either. Google’s Gemini is the only one in the group that does, and only because it borrows Google’s own system to do it.

They take the file, read whatever text is sitting inside it, and leave. There is no second pass.

My two newest articles pulled their words in a slightly different way than the other twenty two did. The other twenty two had their text written into the page from the start. The two newest went and fetched their text after the page had already been saved out. In a browser that difference is invisible. To anything that does not run code, those two articles had no headline, no text, and my site’s generic description sitting in the spot where the article title should have been.

That was my own doing. Nobody did it to me. I built it that way and then did not check.

While I was in there I decided to find out something else, which is whether any of this was reaching ChatGPT at all.

It turns out ChatGPT does not use Google. It runs on Bing. Every page it considers quoting was found through Bing’s index first, which means if Bing has never seen your page, ChatGPT cannot see it either. What surprised me more is that your position in Bing barely matters. Being in the index at all is the thing.

So I opened Bing Webmaster Tools, which I had connected months ago and never looked at again. It had almost nothing on me.

When I went looking for why, the answer was dull and slightly embarrassing. Several of my pages were carrying the exact same title, because the per page title and description fields had never been filled in. To a search engine those pages all claimed to be the same thing, so it had no particular reason to show any of them.

Google had never mentioned this once. Everything on the Google side was green. Indexed, no errors, nothing to see. Google does not have a screen that tells you about duplicate titles, because when Google does not like your title it rewrites it in the results and never says a word to you about it.

The problem had been sitting in plain view of one tool and completely invisible to the other.

This one cost me an hour, so it is worth passing on.

The recommendations list Bing shows you is not current. It is built from the last time it scanned your whole site, and that can be weeks ago. Mine was listing problems on a page I had deleted two weeks earlier. Two of the three errors it reported on another page turned out not to exist any more. The only screen telling you about today is the Live URL tab, which fetches the page fresh while you watch.

I nearly spent an evening fixing things that were already fine.

There was one more thing I did not expect, and it matters more to some businesses than everything above.

If somebody asks an assistant for a local business, your website is nearly the last thing it looks at. It checks your Google Business Profile first. Then the reviews other people left about you elsewhere. Then any lists and directories you turn up in. Your own site sits behind all three of those.

What other people say about you, and what can be verified about you, counts for more than anything you say about yourself.

So if you run a local business and you have spent the last few months rewriting your homepage, you have been working very hard on the fourth most important thing on the list.

Open one of your pages. Right click and choose View Source. Do not choose Inspect. Inspect shows you the page after all the code has finished running, which is the exact thing you are trying to see past. View Source shows you what the server really handed over.

Then search that page for a sentence from the middle of your article. If it is sitting there, you are fine. If it is not, then as far as anything that does not run code is concerned, your page is blank.

Do it on your newest page rather than your oldest one.

That last line is the mistake I made.

I checked this once, months earlier, on my oldest and safest article. It passed. I then carried that result around for weeks as though it were a property of my whole site, which it was not, because the newest work was built slightly differently from the old work. New work usually is. That is what makes it new.

A test you ran once is not something your site has. It is a photograph of one page on one day.

None of what I found turned out to be a clever new AI trick. It came back to being in Bing, having your titles filled in, and making sure the words on your page are really on your page. The tools changed and the homework did not.