AI Supply Chain Security defines attacks by technique, providing an estimated severity and the rationality for classifying it with that severity.
| Detection Category | Estimated Severity | Definition | Rationality for Severity |
|---|---|---|---|
| Arbitrary Code Execution | Critical | Adversaries can inject malicious code into a model, which will be executed whenever the hijacked model is loaded into memory. This vulnerability can be used to exfiltrate sensitive data, execute malware (such as spyware or ransomware) on the machine, or run any kind of malicious scripts. | Cloudpickle: .pkl, .pickle |
| Arbitrary Read Access | High | Adversaries can craft a malicious model that will exfiltrate sensitive data upon loading. | ONNX: .onnx |
| Control Vector | High | Adversaries can inject a control vector into the computational graph of a model introducing refusal ablation or custom, attacker-defined behaviors. | All model formats |
| Decompression Vulnerabilities | High | Adversaries can exploit vulnerabilities in popular compression formats to cause denial of service or leak sensitive data. | Keras: .keras |
| Denial of Service | Medium | Adversaries can craft a malicious model, or modify legitimately pre-trained model, in order to disrupt the system the model will be loaded on. | Cloudpickle: .pkl, .pickle |
| Directory Traversal | Medium | Adversaries can craft a malicious model, or modify legitimately pre-trained model, in order to gain unauthorised access to sensitive files on the system. | ONNX: .onnx |
| Embedded Payloads | Low | Adversaries can embed malicious payloads (such as backdoors, coin miners, spyware, and ransomware) inside the model’s tensors. Such payloads can be injected in plain text, obfuscated, or embedded using steganography. | HDF5: .h5, .hdf5 |
| Graph Payload | High | Adversaries can inject a computational graph payload, introducing a secret attacker-controlled behavior into a pre-trained model. | ONNX: .onnx |
| Model Sideloading | High | Adversaries can load code or model artifacts from an unexpected location bypassing checks performed on the model. | Pickle: .pkl |
| Network Requests | High | Adversaries can craft a malicious model that will make network requests upon loading. | Cloudpickle: .pkl, .pickle |
| Repository Sideloading | Medium | Adversaries can load code or model artifacts from an unexpected location, bypassing checks performed on the artifacts in the repository. | Repository sideloading is an expected behavior allowed by Hugging Face; however, it can be abused to bypass security checks. |
| Suspicious File Format | Medium | Adversaries can modify data structures and encodings in an attempt to evade detection. | Cloudpickle: .pkl, .pickle |
| Suspicious Functions | High | The presence of these functions themselves is not inherently malicious, but they can be used in conjunction with other functions to create a malicious model. | Cloudpickle: .pkl, .pickle |
| TokenBreak | High | Adversaries can exploit a weakness in the tokenizer to bypass model classifications. | Models susceptible to the tokenbreak vulnerability can have their classifications altered on command by an attacker resulting in a weakness wherever the model is used. |