Most people use Artificial Intelligence (AI) platforms such as ChatGPT to ask a question or generate an image. The vast majority of AI models that the public interacts with are closed-sourced, meaning people are not able to see how the AI generates its response, a safety feature that ensures privacy. However, when models are open-sourced, their safety features can be stripped away and the public is left with a model that has no regard for laws restricting CSAM (child sexual abuse material) and non-consensual sexual deepfakes, especially for women and children.
With closed-source AI, the system’s software is owned by the company. The model is provided as a service, effectively allowing users to “borrow” its functionality but never own the model themselves. This permits security measures to be put in place by the company to prevent liability — ask ChatGPT to generate a nonconsensual sexual image of a woman and it will refuse to do so. Clearly, there are policies and rules put in place to prevent damages.
Open-source models can attempt these same regulations, but they have no power to uphold them. There are no real limitations for users who can download the model and strip off the safety features.
A prime example of this is Meta’s AI models. When the original Llama model was released, the code was exploited to create an uncensored chatbot. Its second model suffered the same fate: Llama 2 was released with a guide detailing responsible usage. Since it was open-sourced, people immediately developed an uncensored knockoff with zero safety features. Once an uncensored version is made, nothing can be done to stop people from using it to generate CSAM.
The response to this has been lackluster by Meta. Executives from the company relied on whataboutism, pointing out that people can already generate large amounts of ‘bad’ content — such as misinformation and hate speech — without AI.
However, that is not to say that more “official” AI models are incapable of these dangers. For example, X users are asking Grok AI to unblur the faces of sexual abuse survivors in the Epstein files, and it complied with their requests. Still, the company quickly put up guardrails, after which Grok did not attempt to unredact any of the images. This is due to the company’s fear of liability, which open-source models do not have.
AI models becoming open-source is not the only danger. Allowing user access to the databases AI is trained on is equally dangerous.
AI-generated CSAM creation increased at the end of 2022, when the LAION-5B database, an open-source collection of more than five billion images that anyone can use to train AI models, was released. Hundreds of CSAM images were included in LAION-5B.
“As soon as these things were open sourced, that’s when the production of AI generative CSAM exploded,” chief technology officer at the Internet Watch Foundation Dan Sexton said.
Open-source AI models live in a legal grey zone. Fringe open-source AI models are maintained by online groups, not large companies. This makes it difficult for them to face accountability or liability. These models are also available for download anywhere around the world, making it difficult to legislate. AI regulations vary by country, making it easier for AI models to dodge legislation when put in place.
While AI regulation is never perfect, open-source AI models are a dangerous and unregulated experiment that harm women and children. AI models should be kept secure, and open-sourcing should not be considered as a viable option.
