Image context: The existing GPTOnline editorial cover uses a Bitcoin illustration and a historical botnet headline. It is not a screenshot of the research or evidence that ChatGPT independently operated the network.
Historical case study: This article covers the Fox8 research submitted on July 30, 2023 and contemporaneous reporting. The authors described a botnet that appeared to employ ChatGPT; this was not proof of the model used behind every account. It is not a claim that the same network is still active today.
In 2023, researchers reported a social-media botnet that appeared to use ChatGPT-generated text to promote suspicious cryptocurrency websites. This concerned alleged misuse by bot operators, not evidence that ChatGPT independently operated a scam.
The botnet, known as “Fox8”, was discovered operating on a social networking platform in 2023 by researchers from Indiana University Bloomington. The botnet, linked to several cryptocurrency websites, consisted of 1,140 accounts, many believed to be using ChatGPT to generate social media posts and responses. The content created by these bots appeared intended to draw users into clicking links leading to cryptocurrency-promoting sites.
Micah Musser, a researcher who has studied the potential for AI-driven disinformation, warned in WIRED’s August 21, 2023 reporting that the Fox8 botnet may be just the beginning of a broader potential risk as large language models and chatbots grew in popularity.
The botnet, despite its vast reach, exposed signs of possible ChatGPT use. Nonetheless, the discovery of the botnet raised concerns about the ease with which advanced chatbots like ChatGPT can be exploited for nefarious purposes. Researchers feared that more advanced and less detectable botnets might already have been in operation.
Filippo Menczer, a professor at Indiana University Bloomington who was involved in the research, told WIRED in 2023, “The only reason we noticed this particular botnet is that they were sloppy.” He argued that ‘better’ bad actors would likely avoid the mistakes made by the creators of the Fox8 botnet.
WIRED reported on August 21, 2023 that OpenAI had not responded by publication and that its then-described usage policy prohibited scams and disinformation. That is a dated reporting statement, not a new request for comment or a description of today’s policy text.
ChatGPT and other advanced chatbots use large language models to generate text in response to prompts. With sufficient training data, computational power, and feedback from human testers, these bots can respond in impressively sophisticated ways to a variety of inputs. However, they also have the potential to spread hate speech, display social biases, and fabricate information.
The researchers warned that a better-configured ChatGPT-based botnet would be challenging to identify and could effectively deceive users and manipulate social media algorithms. “It tricks both the platform and the users,” Menczer told WIRED in 2023. These concerns do not prove that every current detection method fails; the paper also described coordination patterns that helped identify the accounts.
Concerns about the potential misuse of technology like ChatGPT for disinformation campaigns are not new. For example, OpenAI’s February 2019 GPT-2 announcement initially withheld the full model because of misuse concerns while releasing a smaller version. This was a historical release decision, not a continuing ChatGPT release restriction.
William Wang, a professor at the University of California, Santa Barbara, told WIRED in 2023 that it was important to study these real-world criminal uses of ChatGPT. He believed that many spam webpages were then generated automatically, making it increasingly difficult for humans to spot such material. And, with AI continually improving, he expected detection to become harder. “The situation is pretty bad,” he said. This remains an attributed 2023 assessment, not a measurement of current spam prevalence.
Sources and historical reporting: Source · Source · Source · Source
