The AI backlash begins: artists could protect against plagiarism with this powerful tool

A team of researchers from the University of Chicago has developed a tool intended to help online artists “fight back against AI companies” by, essentially, inserting poison pills into their original work.

The software, called Nightshade after the family of poisonous plants, would introduce poisonous pixels into digital art that mess with how generative AIs interpret it. The way models like Stable Diffusion work is that they scour the internet and collect as many images as possible to use as training data. What Nightshade is doing is taking advantage of this ‘security problem’. As explained by the MIT Technology Review, these “poisoned data samples can manipulate models to” learn the wrong thing. For example, it can see an image of a dog as a cat or a car as a cow.

Poison tactics

As part of the testing phase, the team fed Stable Diffusion infected content and then “challenged it to create images of dogs.” After receiving fifty samples, the AI ​​generated photos of deformed dogs with six legs. After 100 you start to see something that looks like a cat. Once there were 300, dogs became full-fledged cats. Below you can see the other tests.

The report goes on to say that Nightshade also influences “tangentially related” ideas, because generative AIs are good “at making connections between words.” Messing around with the word “dog” confuses similar concepts like puppy, husky, or wolf. This also applies to art styles.

It is possible for AI companies to remove the toxic pixels. But as the MIT post points out, it is “very difficult to remove them.” Developers should “find and remove every corrupted example.” To give you an idea of ​​how difficult this would be, a 1080p image has over two million pixels. As if that wasn’t hard enough, these models are “trained on billions of data samples.” So imagine looking through a sea of ​​pixels to find the handful of stuff with the AI ​​engine.