Reading through Josh Bernoff’s recent blog post, “Why ChatGPT lies about me: an analysis,” I started thinking about the importance of fact-checking and copyediting. In his insightful piece, he describes having ChatGPT create a bio for him, which ends up rife with errors. Definitely check out his work for the full analysis.
We’re all amateur copyeditors as we scroll through our daily news copy or flip through the pages of the latest novel we’re reading. We spot a typo here or a layout error there. But generally, in works that may range from 75,000 to well over 100,000 words long, that single typo is not enough to make you feel like you’re reading something not to be trusted or something that is the product of an amateur. And, when you visit an online news site, somewhere that you’re going to for facts about either your local government or about national news, clean copy provides a level of professionalism and value that increases your trust in the site’s content.
That’s not just my biased opinion as a copyeditor, that’s also the result of a 2015 study conducted by Fred Vultee, PhD, associate professor at the Department of Communication, Wayne State University. He conducted the study in light of the digital disruption that was transforming newsrooms everywhere. And by “transforming,” I mean getting stories to market as quickly as possible, which usually meant eliminating layers of editing.
In “Audience Perceptions of Editing Quality,” Vultee conducted a lab experiment where researchers showed 119 participants four news articles in both edited and unedited drafts. Did the quality of articles matter to the participants? Would they be willing to pay for higher quality content? In a news article summarizing the results, “Study shows the value of copy editing,” Natalie Jomini Stroud wrote, “Copy editing, his results show, is good for business. Audiences can tell when an article isn’t carefully edited, and it affects their perceptions about the news and their willingness to pay for it.”
The study did not address how much more people would be willing to pay for edited, high-quality content, but I know that I will immediately leave a site the minute I see too many errors. In the past, if I were to do a Google search, say a basic search for a “how to” for some technology issue I was having, I was fairly sure that I would find credible sites to walk me through my problem. Now, I find site after site with content that appears to have only had possibly a quick run through an automated spellchecker. Not only is the content rife with errors, sometimes the sentences make zero sense even if every word is in English.
How much of this content is now being generated by AI or spit out by content farms that are simply dumping in buckets of keywords to game SEO algorithms, I don’t know. I do know that poorly written and edited sites will have me clicking off of them in nanoseconds.
I reached out to Associate Professor Vultee to see if he had updated his thinking on content quality and editing in the world of AI and ChatGPT: “A lot of the material I used came from broadcast sites and wasn't very well prepared for print presentation; if an AI bot finds itself looking among broadcast transcripts rather than more polished output, that could hit some of the buzzers I saw.”
Once those errors start to pile up, you start to question not just the professionalism of such a piece but also its validity. The Internet is awash in content that is unedited, not fact-checked, and just plain full of dreck. I asked Professor Vultee whether he thought that the same lessons his study unearthed in 2015 applied to today. Here’s what he said:
“Things seem likely to still depend on how many humans there are between the output and the audience. One lesson of the 2000-2010 era is that managers apparently decided they could tolerate a lot more risk of crap content before the impact became noticeable, and if the same sorts of assumptions are made about letting AI publish without grownup supervision, we're likely to end up with the same stuff.”
I’d be interested in seeing an updated study taking a look at AI-generated content that received no further revisions, editing, or fact-checking and whether people would be willing to pay for that increase in quality. In the meantime, why not reach out and get your content copyedited -- I’d be happy to help!
*I use copyediting rather than copy editing because I’m a Merriam-Webster gal. It’s also why I use copy editor and not copyeditor. Believe me, the copy editor in me is flinching at the lack of consistency between the two.
Commentaires