It has been revealed that a relatively small number of samples can be leveraged to conduct data poisoning on large language models, regardless of their scale. Anthropic's research has set off alarms within the AI security community. Conventional wisdom dictates that to poison a large model boasting hundreds of billions of parameters, one would need to gain control over 0.1% of the training data. However, the latest experiments have unveiled a startling reality: attackers can implant backdoors into a model with 13 billion parameters using a mere 250 malicious documents.

The research team adopted a denial-of-service attack strategy, injecting documents laden with trigger words and random gibberish into the training data. This caused the model to generate meaningless output when confronted with specific phrases. The experiments encompassed four models, with parameter counts ranging from 600 million to 13 billion. After introducing 250 poisoned samples into each model, the attack success rate soared to nearly 100%, irrespective of the model's size or the volume of clean data. For instance, the poisoned samples that the 13-billion-parameter model was exposed to constituted a mere 0.00016% of the training data. Yet, it was successfully compromised.

This research exposes the vulnerability inherent in AI training data. When malicious content is interwoven into publicly accessible online corpora, it can transform into an 'invisible bomb' within the model's cognitive framework. Although the current experiments have not substantiated complex attacks, such as the generation of malicious code, they have unequivocally demonstrated that the threshold for data poisoning is significantly lower than previously assumed. This places greater emphasis on the rigorous review of data sources and the enhancement of model defense mechanisms.