How an algorithm can predict the next J.K. Rowling or Danielle Steele
The book trade is at once a high-minded and a grubby pursuit. Publishers solemnly assume the mantle of guardians and creators of culture … enlightening and delighting readers by printing and selling books that change not only people's lives, but culture and the language itself.
There's much honour in that. But not a lot of money. To pay for those cultural and artistic landmarks, they need bestsellers that are lapped up by masses of readers irrespective of literary or any other kind of merit.
The trick is to spot the next Girl With The Dragon Tattoo, Da Vinci Code or Fifty Shades of Grey buried in the stacks of manuscripts of mediocre potboilers, thrillers and bodice rippers submitted to publishers every year.
It's been an inexact process. Just ask the 12 publishers who turned down J. K. Rowling's manuscript for Harry Potter and the Philosopher's Stone.
But publishing is putting data and computing power to work to find — or create — the next big hit.
The Bestseller Code, a book by Jodie Archer and Matthew Jockers published last year, describes an algorithm they devised to sequence the DNA of bestselling novels. They call it the bestseller-ometer, and it's found 2800 elements encompassing language, character types and plot structures that can predict whether a novel will become a bestsellier. It boasts an 80 percent accuracy rate.
Susanne Althoff wrote about the use of big data and algorithms in publishing in a recent article in Wired magazine called "Algorithms could save book publishing — but ruin novels."
Susanne Althoff is an assistant professor at Emerson College, and she's the former editor of the Boston Globe Sunday Magazine.
Click the button above to hear Michael's interview with Susanne Althoff.