| Nearly all books ChatGPT recommended shared one thing in common — and most authors don’t know it. | |||||
|
|
|||||
|
Hi there, Welcome to a new edition of Reedsy’s bi-weekly marketing newsletter! Again, it’ll be all about the profound shift that’s coming to book discoverability — and online search in general. Before we get to that, though, a bit of admin:
Now, onto the good stuff! How ChatGPT verifies book informationLast time, we broke down how ChatGPT thinks when readers ask for book recommendations. One important — yet often overlooked — part of that process is verification. As you probably know, LLMs can hallucinate (i.e. make things up) when they don’t know an answer. Newer versions — GPT-5 in particular — are much better at this because they now verify uncertain claims through web searches.
Hallucination rate of GPT-5 vs OpenAI o3 (source: Passionfruit) For example, if I ask ChatGPT-5 “Does Throne of Glass contain explicit sex scenes?”, it will run a few searches and cite sources to answer accurately. Once ChatGPT compiles a list of potential recommendations, it then verifies that:
This verification step is crucial: if ChatGPT can’t easily confirm those two points, your book might get discarded in favor of another for which the data is clearer. So how does ChatGPT verify book info — and which sources does it trust? Naturally, it checks Amazon and Goodreads, which list vital details such as title, author, description, price, and reviews. But there’s a third site that ChatGPT seems to trust just as much — one I didn’t expect until I started investigating. My little side project: analyzing 300+ ChatGPT book recommendationsOver the past few months, I’ve been studying how ChatGPT chooses which books to recommend when given specific prompts. To do this, I created 100 prompts across 10 fiction genres (I’ll get to nonfiction later). Each asked for three recommendations that met defined criteria. For example:
At first, I ran these manually, but each prompt took over three minutes — so I built a small script to automate it through the GPT-5 API. (And since I know as much about coding as I do about plumbing, I enlisted ChatGPT’s help!) Now, this took a lot of fiddling to work, but I finally ended up with 300 book recommendations ready for analysis. More importantly, the process was just as instructive as the end result. See,ChatGPT itself suggested that I verify each recommendation by cross-checking them against… the Google Books Library. In other words, ChatGPT’s own go-to method for confirming a book’s existence is to check whether it’s listed in Google Books. I chose not to bias the process, so I instructed my script to let GPT-5 decide how to verify each title on its own. And yet, when I looked at the results, 99% of all recommended books appeared in Google Books. That’s right — 297 out of 300 books ChatGPT recommended to me were in the Google Books Library. And here comes the most important takeaway for authors and publishers: if you want your books to be found by AI search engines, you better make sure that they’re listed in the Google Books Library. We’ll dive into how to do that in the next issue. Until then, happy writing, and happy marketing! Ricardo Links to previous issues: Generative Search for Books – AI Search Part I: Query Fan Outs – AI Search Part II: Vector Embeddings – AI Search Part III: The Age of Personalization – How does ChatGPT pick its book recommendations? |
|||||
|
|||||
|
Copyright © 2026 Reedsy, All rights reserved. |