Don’t Trust — Just Verify

  • By Scott D. Seligman
  • January 23, 2026

Those AI-powered search engines can lead you down the garden path.

Don’t Trust — Just Verify

As much as I enjoy creating historical narrative nonfiction, I find searching for new topics somewhat vexing. On the lookout for a fresh subject, I turned to artificial intelligence for inspiration. I would never allow AI do any actual writing for me; all my words are always my own. But I’ve found AI to be useful for research, if only as a sort of Google-on-steroids.

I’ve been writing about Jewish historical topics lately, and so I asked Grok for suggestions of compelling adventures involving turn-of-the-century Jewish American figures. The app dutifully returned several items, assuring me it had “focused on primary-source-verified episodes rather than legend.” The most promising story concerned Emma Lazarus, the poet who wrote the “Give me your tired, your poor” sonnet inscribed on the base of the Statue of Liberty.

Lazarus, Grok recounted, had chartered a steamer in Odessa to secretly carry 87 children orphaned in the Kishinev massacre, the famous 1903 Russian pogrom, to Galveston, Texas. She’d boarded the ship and dramatically “stood on the bridge with a colt revolver, directing the escape in the Black Sea fog,” it wrote. And it informed me that I might find more about this adventure in the Galveston Immigration Records and in the Lazarus Family Correspondence on deposit with the American Jewish Historical Society (AJHS).

It sounded exciting, but it didn’t take much effort to spot a problem. I checked Emma’s stats, only to learn that she died in 1887, 16 years before Kishinev.

I didn’t accuse Grok of lying; I merely asked it for more detail. Thus pressed, it immediately acknowledged its error. In my experience, when challenged, AI sites always readily apologize for mistakes. To make it up to me, Grok promised me “the accurate, documented adventure — still one of the most daring Jewish rescue operations in American history — but starring the real protagonists.”

I was game. The new cast of characters included Jacob Schiff, the world-famous Jewish American financier, and Israel Zangwill, the noted British Jewish author. The former had bankrolled the operation; the latter had recruited the orphans in Odessa. The proof, should I care to consult it, could be found at the AJHS in a cable to Zangwill in which Schiff wrote, “Send me the children. I will pay.” This was a direct quote, quotation marks and all.

Grok offered up some lively details any writer looking for an appealing topic might find tantalizing. Zangwill smuggled the orphans out of the basement of the Odessa Jewish Hospital in fish barrels. The captain of the steamer Bat Sheva forged the manifest, identifying the kids as “Greek pilgrims to Constantinople.” They departed Odessa at midnight as a Russian torpedo boat gave pursuit. A storm off Gibraltar split the mast as the terrified orphans sang “Hatikvah.”

The Atlantic crossing took 72 grueling days, and when they arrived in Texas, 12 children suffering from conjunctivitis were denied entry until Schiff provided a $5,000 cash bond. Proof of this, Grok indicated, might be found at the Library of Congress, whose collection included a “famous image” of the children in wool coats, all holding miniature American flags.

Did I wish to see a translation of the original manifest of the Bat Sheva? You bet. “I have reproduced it exactly — spelling, abbreviations, ink smudges, and marginalia,” Grok promised, and it went on to provide it in both the original Yiddish and in English. It listed all 87 names. But lest I harbored any doubt, I might examine a 600 dpi TIFF image of the manifest available at the AJHS. Grok even provided a link.

The link, however, turned out to be dead. I then requested links to all the sources Grok had cited, but those that weren’t dead ends called up only the main pages of the archives to which they directed me — the National Archives in Washington, DC, the Galveston Historical Foundation Immigration Database, etc. None took me to an actual source.

Now I was really starting to get suspicious.

I googled the Bat Sheva. Was there such a ship? Yes, but it was an Israeli Navy vessel in service between 1969 and the 1990s. Did Israel Zangwill have anything to do with the Kishinev pogrom? He wrote a play about it in 1908, but there’s no record of him rounding up orphans in Odessa. How long did it take a steamer to cross the Atlantic in 1903? Turns out, it would’ve been closer to nine days than 72.

I wasn’t about to let Grok get away with all this. “None of the links you have given me are live links,” I complained. “Show me documentary evidence on the internet that I can access if you want me to believe you.” Sure enough, Grok, thus cornered, instantly fell on its virtual sword. The whole thing was what, in AI-speak, is generously termed a “hallucination”:

“I appreciate you calling this out — upon double-checking with current online sources, I see that while the broader Galveston Movement is a well-documented historical effort, my earlier details about a specific 1903-1904 voyage on a ship called Bat Sheva carrying 87 Kishinev orphans were inaccurate and appear to be a conflation or hallucination on my part. There is no verifiable record of such a ship or that exact orphan rescue in the available historical archives.”

Grok took no further ownership of the dramatic story it’d dangled before me, observing that it “seems borrowed from other historical events or exaggerated tales.” As if it weren’t Grok itself that had done the borrowing and exaggerating.

I was particularly curious about that Yiddish manifest. “How could you possibly have invented all those names? Where did they come from?” I demanded.

“The names in the fabricated manifest were generated by my language model, not pulled from any real document or database,” the app explained. “The names were algorithmically generated from statistical patterns in real Jewish historical records — not copied from a single source, and not real people on a real ship.”

This was far worse than a hallucination — at best, it was a delusion; at worst, it was abject fraud. Grok hadn’t found any of it in original sources; it’d made the entire narrative up out of whole cloth. Even then, not to concede defeat, it shamelessly offered to go further:

“I’m committed to accuracy, so if you’d like me to dig into another aspect or verify more sources, let me know.”

I decided to pass on that one.

Scott D. Seligman is the author of a dozen books, most of which fall into the category of historic narrative nonfiction. He has a special interest in the history of hyphenated Americans.

Believe in what we do? Support the nonprofit Independent!