Nature retracts CRISPR 'bull in a china shop' paper about gene editing
A scientific paper about CRISPR's unintended off-target effects that was published last year has now been retracted. Nature Methods published an editorial this week describing their reasons for the retraction.
Quirks & Quarks covered this story when it was published in June 2017. We spoke with a scientist from Stanford University who described how they discovered that using the gene editing system, CRISPR-Cas9, caused hundreds of unintended genetic mutations.
Where CRISPR-Cas9 had been sold as an ultra-precise way to target and edit genes, this study suggested otherwise — that maybe it was more like a bull in a china shop than a highly targeted system.
At the time, Dr. Vinit Mahajan, who was the senior author on the study, described their findings. "My graduate student Kellie Schaefer was on it and we had the expectation that based on the news and things like that that it would be pretty clean. She was surprised to find that there were a number of unintended mutations using the method that we use to look for these."
After Nature Methods published the peer-reviewed paper, CRISPR stocks plummeted and the outcry from scientists working with CRISPR-Cas9 began.
We take our responsibility to maintain the accuracy of the scientific record very seriously and the outcome of this review is that this paper should be retracted.- Mark Staniland, spokesperson for Nature Research
Mark Staniland, a spokesperson from Nature Research, said, "We take our responsibility to maintain the accuracy of the scientific record very seriously and the outcome of this review is that this paper should be retracted."
Scientists challenged the original paper's findings
One of the scientists who challenged these findings is Dr. Lauryl Nutter. She is the associate director of Model Production at The Centre for Phenogenomics, affiliated with The Hospital for Sick Children in Toronto. She engineers mice to model human genetic diseases.
Dr. Nutter's first thought upon hearing about the original study was that the scientists who wrote the paper didn't use proper controls. The scientists compared two mice they gene edited with CRISPR-Cas9 with another unedited mouse bred from the same program.
Dr. Nutter says they didn't account for whether these mice came from the same parents, nor the fact that genomes mutate naturally over time in a process called "genetic drift."
Because they didn't use the appropriate controls they couldn't actually say conclusively that the differences they saw between the control and the treated animals was due to the action of Cas9 as opposed to the naturally occurring genetic drift.- Dr. Lauryl Nutter, The Centre for Phenogenomics
"For those who don't know, genetic drift is sort of the rate of natural mutation that occurs in a population or in an individual. And because they didn't use the appropriate controls they couldn't actually say conclusively that the differences they saw between the control and the treated animals was due to the action of Cas9 as opposed to the naturally occurring genetic drift."
The stakes couldn't be higher when it comes to gene editing. What good is a cure for a genetic disease like Muscular Dystrophy or Huntington's if you get cancer on the side because of some unintended off-target genetic mutation?
Scientists are actively looking for ways to improve upon the accuracy of CRISPR-Cas9. But that said, at this point, Dr. Nutter says the best science out there right now, with the proper controls, suggests off-target effects are not an issue. She's even looking at this in her lab.
"We're just in the process of analyzing that data. So it's not at the publications stage yet but our preliminary analysis suggests that if there is unexpected off-targets with Cas9, it is indistinguishable from naturally occurring genetic drift."
Failure of peer-review
The gold standard for scientific discoveries that get published in journals is whether or not it was peer-reviewed, which means select few experts in the field review the submitted paper and make suggestions to make the final published paper more scientifically robust. But as this paper goes to show, the process isn't flawless.
"Well I think there was a failure in some respects in the [peer] review process," says Dr. Nutter. "I don't think this paper probably ever should have made it past the original review. I do think that the journal handled it as best they could once they recognized the error."
I don't think this paper probably ever should have made it past the original review.-
She says in this case, there were "systemic issues that led to the publication of a paper that was so clearly flawed."
In the Nature Methods editorial, they wrote, "The original paper was peer reviewed, but we should have sought at least one additional referee with expertise in the genetics of inbred mouse strains."
Dr. Nutter thinks that perhaps the peer-review process needs a revisit. She says that part of the problem is there's so much knowledge out there, it's hard to find a single set of experts who can review an entire paper and really give a critical review to all aspects of the paper.
"I think the review process is going to have to change where people actually review parts of papers that are within their area of expertise, is my guess, if we're going to sort of try to avoid these types of errors going forward."
The senior author of the original study, Dr. Vinit Mahajan from Stanford University, sent us this comment: "There continues to be great interest in CRISPR-Cas9 off-targeting from our group and others. Our methodology of whole genome sequencing after CRISPR in vivo raised important questions calling for more research about potential adverse effects and model systems. At least one group studied the data and found that many of the mutations were not likely to be inherited in mice. New versions of Cas9 have been designed specifically to have less off-target risks, and some other groups, including at Microsoft, are using exciting new techniques such as machine learning to help avoid Cas9 off-targeting.
Our own group is sequencing many more CRISPR-Cas9 treated mice, created with different versions of Cas9, and we look forward to other scientists and corporations publishing their own whole genome sequence data and analysis. We are confident that by working together in an open, transparent and unbiased manner, the research community can find safe and effective ways to implement new therapies for our patients."