Nearly all books ChatGPT recommended shared one thing in common — and most authors don’t know it.

Hi there,


Welcome to a new edition of Reedsy’s bi-weekly marketing newsletter! Again, it’ll be all about the profound shift that’s coming to book discoverability — and online search in general.


Before we get to that, though, a bit of admin:

  1. Since this has turned into a long-running series on AI search, several of you have asked for links to previous issues. I’ll include a full list of them at the bottom of each email from now on, so you can easily revisit past editions.
  2. Since our Reedsy Learning masterclasses and conferences have also made the newsletter schedule busier than ever (check those out here!), I’m officially moving this newsletter to every other Saturday.
  3. As a reminder, this is an informational newsletter focused on marketing for authors. Considering how important AI search is becoming for discoverability, all upcoming issues will stay focused on this topic. If it’s not your cup of tea, you can use the “Manage your subscriptions” link at the bottom of this email to unsubscribe from this series.

Now, onto the good stuff!

How ChatGPT verifies book information

Last time, we broke down how ChatGPT thinks when readers ask for book recommendations. One important — yet often overlooked — part of that process is verification.


As you probably know, LLMs can hallucinate (i.e. make things up) when they don’t know an answer. Newer versions — GPT-5 in particular — are much better at this because they now verify uncertain claims through web searches.


Hallucination rate of GPT-5 vs OpenAI o3 (source: Passionfruit)


For example, if I ask ChatGPT-5 “Does Throne of Glass contain explicit sex scenes?”, it will run a few searches and cite sources to answer accurately.

Once ChatGPT compiles a list of potential recommendations, it then verifies that:

  1. The titles actually exist (i.e. weren’t hallucinated);
  2. The titles match the search criteria.

This verification step is crucial: if ChatGPT can’t easily confirm those two points, your book might get discarded in favor of another for which the data is clearer.


So how does ChatGPT verify book info — and which sources does it trust?

Naturally, it checks Amazon and Goodreads, which list vital details such as title, author, description, price, and reviews.


But there’s a third site that ChatGPT seems to trust just as much — one I didn’t expect until I started investigating.

My little side project: analyzing 300+ ChatGPT book recommendations

Over the past few months, I’ve been studying how ChatGPT chooses which books to recommend when given specific prompts.


To do this, I created 100 prompts across 10 fiction genres (I’ll get to nonfiction later). Each asked for three recommendations that met defined criteria. For example:

  • Romantasy: “Can you recommend three romantasy books, released this year, that feature enemies-to-lovers but no explicit sex scenes? They should ideally be well-reviewed and if possible, feature faes or other magical creatures.”
  • Thriller: "Can you recommend three psychological thriller books with a female protagonist and set in the British countryside?"
  • Mystery: "Can you recommend three paranormal cozy mystery set in Southern US? Ideally featuring witches, vampires, and other magical creatures like that. I would like it to be the first book in a series of at least 5."
  • Science Fiction: "Can you recommend three military sci-fi books with space marines fighting off an alien invasion? I want first-in-series only."

At first, I ran these manually, but each prompt took over three minutes — so I built a small script to automate it through the GPT-5 API. (And since I know as much about coding as I do about plumbing, I enlisted ChatGPT’s help!) 


Now, this took a lot of fiddling to work, but I finally ended up with 300 book recommendations ready for analysis. More importantly, the process was just as instructive as the end result. 


See,ChatGPT itself suggested that I verify each recommendation by cross-checking them against… the Google Books Library. In other words, ChatGPT’s own go-to method for confirming a book’s existence is to check whether it’s listed in Google Books.


I chose not to bias the process, so I instructed my script to let GPT-5 decide how to verify each title on its own. And yet, when I looked at the results, 99% of all recommended books appeared in Google Books. That’s right — 297 out of 300 books ChatGPT recommended to me were in the Google Books Library.


And here comes the most important takeaway for authors and publishers: if you want your books to be found by AI search engines, you better make sure that they’re listed in the Google Books Library.


We’ll dive into how to do that in the next issue.


Until then, happy writing, and happy marketing!

Ricardo


Links to previous issues: Generative Search for BooksAI Search Part I: Query Fan OutsAI Search Part II: Vector EmbeddingsAI Search Part III: The Age of PersonalizationHow does ChatGPT pick its book recommendations?