Overwhelmed by AI-generated content, this magazine had to close its submissions
Editor Neil Clarke talks about the scam-like pattern that has been impacting his magazine
Science fiction and fantasy magazine Clarkesworld had so many close encounters of the ChatGPT-3 kind earlier this year, they had to close their submissions for a month.
Editor Neil Clarke first noticed that AI-generated content was coming in at the end of last year, when ChatGPT became available to the public. Since then, he says that the amount of AI-generated works being submitted grows exponentially with each passing month.
He joins host Elamin Abdelmahmoud to outline how his team dealt with the phenomenon, and where it stems from in the first place.
We've included some highlights below, edited for length and clarity. For the full discussion, listen and follow the Commotion with Elamin Abdelmahmoud podcast, on your favourite podcast player.
Elamin: We don't know exactly how artificial intelligence is going to change the world. But for now, it is causing real-life headaches for people like Neil Clarke. He runs a science fiction magazine, Clarkesworld, which publishes original stories and artwork from real people. But lately, his magazine and many others have been hit with this onslaught of fiction written by AI. Neil, how's it going?
Neil: Well, it's getting a little better, but it's still pretty rough.
Elamin: Let's get right into it. You have a strict no-AI policy at your magazine, which is an unbelievable sentence to have to say out loud. Why do you have that policy?
Neil: Well, it really boils down to who owns that work — we don't know. This is something that is presently unsettled. Is it the person who prompted the story out of the system? Is it the person who wrote the AI? Is it the authors whose work was used to train that database? There's too many questions there, and it feels irresponsible of us to even consider publishing work when we don't know who the author is.
Elamin: Is it a matter of the ethics around this haven't settled yet, but "maybe if they will, then we'll reconsider this?" Or are you just kind of like, "I'm not interested in this, actually?"
Neil: It's a bit of both. We're very concerned about the ethics of the situation as well. We're very much in line with the authors and artists we've worked with over the years. We don't want to see them taken advantage of, and we don't want to contribute to that.
Elamin: So ChatGPT becomes a big story in the news a few months ago, and then I presume relatively quickly, this became a problem for your magazine. Can you just describe the process of how that happened?
Submissions are currently closed. It shouldn't be hard to guess why.
—@clarkesworldNeil: Yeah, so I think it came out in November, and in December we were already noticing this. So we normally get about 1,100 submissions a month. In December, we saw about 50 that clearly came from AI models. In January, it doubled; it went to 116. And then in February, in 20 days it went to 513. So we decided to close submissions because in one morning we had 50.
Elamin: Oh, wow. How quickly can you tell that it's made by AI?
Neil: It depends on the amount of effort they put into it. But a lot of the people who are doing this aren't actually part of the science fiction and fantasy community. They're not writers. They're actually coming in off of side hustle videos on YouTube and TikTok.
Elamin: Let's talk about the quality of these AI stories. Are you seeing them get better as you get more of these submissions?
Neil: Well, ChatGPT-3 was producing some of the worst stories we've ever seen in 17 years. So, it's never been really about the quality. And GPT-4 is up the game to the equivalent of the worst stories we've seen from humans. So we're still a little ways off before we're really worried about the quality.
Received nearly 1950 submissions in total for January. That's combined English and Spanish. Even without the Spanish submissions, it was a well-above normal month. ("AI"-written submissions were very noticeable among them.)
—@clarkesworldElamin: OK, that's honestly a relief to me, that ChatGPT has not replaced the ingenuity of human writing. Let's talk about this idea of TikToks and YouTube influencers kind of driving this trend. How were they involved in this?
Neil: Well, you can go on to some of these services and do a search for "make easy money with ChatGPT," and you'll turn up loads of people trying to scam people into these things. They know that generating a story and sending it to one of these hundred magazines that they're pointing them at isn't going to work.
The whole idea behind it is: send them out, wave some cash, say, "I made easy money doing this." And then they come back and say it didn't work. Now [the magazine is] the secondary victim in this. The primary victim is these people, because now they're going to be approached. "Sign up for my class. I'll show you all the mistakes you made. It's only $100." So it's really their scam impacting us, and it's really a multi-layered problem.
Elamin: That's a sophisticated scam if I've ever heard one. Neil, we've got to talk about the fact that you had to close submissions for a while. What kind of process do you put in place now that you've completely closed submissions?
Neil: So we closed in February and reopened in March. And during that period we were closed, we did a lot of analysis of what the commonalities of these things were. Detection software is spotty at best. There's a number of them out there that'll make claims, "Oh, we're 80 to 90 per cent accurate." They're not — at least, not against the data sets that we're [using]. And we're seeing false positives and false negatives, which means sometimes an author might be labeled as an AI when they're not.
Elamin: That's a bad day at work, if your writing is so simple that a detection software says, "You know what, this might be artificial intelligence."
Neil: Yeah, and we don't want to ban anyone inappropriately. We want to encourage new writers. Everybody starts somewhere and gets better over time. So we're being very careful about it. We've noticed patterns in the way these things write — they make certain mistakes over and over again. They write perfect grammar, perfect spelling — can't tell a story.
But there's other little things about it within the metadata. These are also being submitted by humans, and humans have tells, too. We've actually been able to identify, "OK, they're coming from these videos, how do we intercept the traffic before they get to our site to even prevent them from submitting in the first place?" And we've had some success with that. We've managed to bring it down by a little bit better than 50 per cent. It's still a lot of submission to go through, and they'll eventually figure out how to outwit that.
Elamin: Do you think this is ever going to come to an end?
Neil: No, it's sort of like any other spam problem. It's going to continue. We're going to get better. They're going to get better.
Elamin: Well, I wish you luck as they continue to get better. I hope you get better, too. Neil, thank you so much for your time.
Neil: Thank you.
You can listen to the full discussion from today's show on CBC Listen or on our podcast, Commotion with Elamin Abdelmahmoud, available wherever you get your podcasts.
Interview with Neil Clarke produced by Jane van Koeverden.
ABOUT THE AUTHOR
