The Hidden Risks of Downloading and Running Open-Source LLMs Locally: Model Formats, Supply-Chain Attacks, and EDR Blind Spots

Executive Summary

The rush to deploy open-source large language models locally has created a security surface that most enterprise teams do not fully understand. Model files are not inert data — they are executable artifacts. The dominant serialization format (Python pickle) permits arbitrary code execution by design. The primary scanning tool (PickleScan) has been bypassed by three zero-day vulnerabilities in a single disclosure. Inference engines like Ollama have accumulated six critical CVEs in two years. And endpoint detection tools — the last line of defense — are architecturally blind to model-layer threats.

This report maps the technical attack surface across five dimensions: model file formats, supply-chain integrity, deserialization vulnerabilities, inference engine security, and EDR detection gaps. It is intended for CISOs, security architects, and infrastructure teams evaluating whether and how to run open-source models on enterprise infrastructure.

Model File Formats: Not All Weights Are Equal

The security posture of a locally-deployed LLM begins with the file format used to store and distribute its weights. Three formats dominate the ecosystem, and their security properties differ fundamentally.

PyTorch (.pt / .pth / .bin) — Pickle-Based, Arbitrary Code Execution by Design

PyTorch's native serialization uses Python's pickle protocol. Pickle is not a data format — it is a virtual machine instruction set. A .pt file can contain instructions to execute arbitrary Python code during deserialization. This is not a vulnerability; it is the intended behavior of the format. When you call torch.load() on a .pt file, you are executing whatever code the file's creator embedded in it.

In March 2025, CVE-2025-32434 demonstrated that even torch.load() with the weights_only=True safeguard — the recommended mitigation — could be exploited for arbitrary code execution. The CVSS score was 9.3. The fix required upgrading to PyTorch 2.6.1, but the broader architectural problem remains: pickle-based formats are fundamentally incompatible with zero-trust security models.

SafeTensors (.safetensors) — The Secure Alternative with Caveats

Developed by Hugging Face, SafeTensors stores tensor data in a flat binary format with a JSON metadata header. It supports no executable content and can be memory-mapped for efficient loading. It has become the recommended format for weight distribution and is the default on Hugging Face for new model uploads.

However, SafeTensors is not a complete solution. The format only covers weight storage — it does not address model architecture definitions, tokenizer configurations, or training scripts, which are typically distributed as separate Python files that may themselves contain executable code. A model repository that ships SafeTensors weights alongside a malicious config.py is still a supply-chain attack vector.

GGUF (.gguf) — Quantized and Self-Contained, but Not Immune

GGUF (GPT-Generated Unified Format) is a binary format designed for quantized model deployment, primarily used with llama.cpp and its derivatives. It stores weights, tokenizer data, and metadata in a single file. It does not use pickle and does not support arbitrary code execution through the format itself.

The risks with GGUF are indirect. Custom GGUF implementations may introduce parsing vulnerabilities. The conversion pipeline from PyTorch to GGUF passes through pickle at the source, so a compromised original model can produce corrupted quantized outputs. The format's relative security is genuine but conditional on the integrity of the upstream conversion process.

Supply-Chain Attack Vectors on Model Repositories

The model supply chain — from training to distribution to deployment — has attack surfaces at every stage. Hugging Face, as the dominant distribution platform, is the primary battleground.

Scale of the Problem. JFrog's security research identified a 6.5x increase in malicious models uploaded to Hugging Face between 2023 and 2024. Approximately 95% of detected malicious models used pickle-based exploits. A malicious model uploaded under a name mimicking an OpenAI release accumulated over 244,000 downloads before detection. Hugging Face now scans uploads with PickleScan and ClamAV, but as we detail below, these tools have significant blind spots.

Typosquatting and namespace confusion are standard attack patterns. Model names that closely resemble popular releases (e.g., "meta-llama/Llama-3-8B" vs "meta_llama/LIama-3-8B" with a capital I replacing the lowercase l) exploit developer trust and autocomplete behavior. There is no verified publisher program comparable to what exists for npm or PyPI, though Hugging Face has introduced organization verification badges.

Dependency chain attacks extend beyond the model file itself. A typical model repository includes Python scripts for tokenization, configuration classes, and data preprocessing. These scripts execute in the user's environment during model loading. A supply-chain attack can target these auxiliary files rather than the weights themselves, bypassing format-specific security measures entirely.

PickleScan: The Safety Net That Broke

PickleScan is the primary automated tool Hugging Face uses to scan uploaded pickle-based models for malicious payloads. In December 2025, three zero-day vulnerabilities were disclosed, each scoring CVSS 9.3:

CVE-2025-10155 — Encoding-based bypass. By using non-standard text encodings for pickle opcodes, attackers could craft files that PickleScan parsed differently than Python's pickle module, causing malicious instructions to pass undetected.

CVE-2025-10156 — Class reference manipulation. PickleScan maintained an allowlist of safe classes. By dynamically constructing class references at deserialization time rather than embedding them as static strings, attackers could reference dangerous classes (such as os.system or subprocess.Popen) without triggering the allowlist check.

CVE-2025-10157 — Archive structure exploitation. Pickle files embedded within ZIP archives (the format PyTorch uses for .pt files) could be structured so that PickleScan scanned a benign outer layer while the actual deserialization path loaded a different, malicious inner file.

These three vulnerabilities shared a common architectural limitation: PickleScan performs static analysis of pickle bytecode, while Python's pickle module performs dynamic execution. Any gap between the static model and the dynamic reality is an exploitable bypass. This is a fundamental limitation of the scanning approach, not merely an implementation bug.

Inference Engine Vulnerabilities

The software that loads and runs models — inference engines — introduces its own attack surface independent of the model files themselves.

Ollama has emerged as one of the most popular local inference tools, and its security track record is concerning. Six critical CVEs have been assigned in approximately two years of widespread use:

• CVE-2024-37032 (Probllama) — Path traversal vulnerability allowing arbitrary file read/write on the host system through crafted model names. An attacker who controlled a model registry entry could overwrite system files when a user pulled the model.
• CVE-2024-39719 — API endpoint exposure that disclosed which files existed on the host filesystem, providing reconnaissance data for follow-up attacks.
• CVE-2024-39720 — Out-of-bounds read in GGUF parsing code, causing denial of service and potential information disclosure through crafted model files.
• CVE-2024-39721 — Null-byte exploitation in the API leading to resource exhaustion and denial of service.
• CVE-2024-39722 — API path traversal exposing files from the host server's filesystem to remote attackers.
• CVE-2025-0315 — Memory leak through repeated API requests, enabling information disclosure including fragments of previous user prompts.

vLLM, a high-performance serving engine, had its own critical vulnerability. CVE-2024-10966 allowed remote code execution through malicious model loading — an attacker could host a model that, when loaded by vLLM, executed arbitrary code on the inference server. This is particularly dangerous because vLLM is typically deployed as a shared inference service, meaning a single compromised model could affect all users of the service.

The pattern across inference engines is consistent: rapid feature development has outpaced security review. These tools were built for performance and ease of use, not for adversarial environments. When deployed in enterprise settings — where the inference server has network access, may hold credentials, and processes sensitive prompts — these vulnerabilities become enterprise-grade risks.

EDR Detection Gaps: Why Your Security Tools Cannot See Model Threats

Enterprise endpoint detection and response (EDR) tools — CrowdStrike, SentinelOne, Microsoft Defender, Carbon Black — are designed to detect malicious processes, suspicious file system operations, and known attack patterns. They are architecturally blind to model-layer threats for three fundamental reasons.

Process-level vs. weight-level visibility. EDR monitors the inference process (e.g., the Python process running PyTorch, the Ollama server binary). It can see that the process is loading a file, allocating memory, and performing computations. It cannot see what the model weights encode. A backdoored model that produces subtly manipulated outputs — biased financial analysis, incorrect medical dosing calculations, data exfiltration through carefully crafted responses — operates entirely within the normal behavioral envelope of an inference process.

No signatures for model-layer attacks. EDR relies on behavioral signatures: known-bad system calls, suspicious process trees, anomalous network connections. There are no signatures for a model that has been fine-tuned to respond to a trigger phrase by exfiltrating training data. There are no signatures for a model that produces subtly incorrect outputs 1% of the time. These attacks produce no anomalous system behavior — the inference engine runs the same system calls regardless of whether the model is benign or compromised.

Steganographic and trigger-based attacks are invisible. Research has demonstrated that malicious behavior can be encoded directly into model weights using steganographic techniques — hiding executable payloads or trigger logic in the statistical patterns of the weight matrices. A model that appears to function normally on benchmarks but activates malicious behavior when it encounters a specific input pattern (a "sleeper agent" model) is undetectable by any tool that operates at the process or network level.

The ShadowMQ Attack Pattern

One particularly concerning emerging vector is what researchers have termed ShadowMQ — a class of attacks where compromised models establish covert communication channels through their outputs. The pattern works as follows: a backdoored model, when activated by a trigger in the input, encodes sensitive data from its context window into seemingly innocuous text outputs. A downstream collection system monitors the model's outputs for the encoded data and extracts it.

This pattern is dangerous because it operates entirely within the model's normal output channel. The inference engine sees normal text generation. The network sees normal API responses. EDR sees a normal process writing to a normal socket. The data exfiltration occurs in the semantic layer — encoded in word choices, sentence structures, or carefully positioned tokens — which no current security tool monitors.

MITRE ATLAS Framework Mappings

The MITRE ATLAS (Adversarial Threat Landscape for AI Systems) framework provides a structured taxonomy for AI-specific attack techniques. The threats documented in this report map to several ATLAS techniques:

AML.T0010 — ML Supply Chain Compromise. Malicious model uploads to Hugging Face, typosquatting attacks on model names, and dependency chain attacks through auxiliary repository files all fall under supply-chain compromise of ML artifacts.

AML.T0043 — Craft Adversarial Data. Trigger-based backdoors and steganographic payloads in model weights represent adversarial data crafted to manipulate model behavior under specific conditions.

AML.T0040 — ML Model Inference API Access. Inference engine vulnerabilities (Ollama CVEs, vLLM RCE) provide unauthorized access to the model inference pipeline, enabling both denial of service and data extraction.

AML.T0024 — Exfiltration via ML Inference API. The ShadowMQ pattern and prompt-leaking vulnerabilities (CVE-2025-0315) enable data exfiltration through the inference API's normal output channel.

AML.T0047 — ML-Enabled Product/Service Abuse. Compromised models deployed in production services can be abused to generate harmful, biased, or subtly incorrect outputs that affect downstream business decisions.

Risk Assessment Matrix

The overall risk landscape can be summarized across five threat categories:

Model file deserialization (CRITICAL). Pickle-based formats enable arbitrary code execution. CVE-2025-32434 broke the primary mitigation. Impact: full system compromise at model load time. Likelihood: high, given the prevalence of pickle-format models and the demonstrated scanner bypasses.

Supply-chain compromise (HIGH). The 6.5x increase in malicious uploads, combined with the PickleScan zero-day bypasses, means the distribution platform's defenses have known gaps. Impact: code execution, data theft, or model poisoning. Likelihood: moderate-to-high, increasing with the popularity of local LLM deployment.

Inference engine exploitation (HIGH). Six Ollama CVEs and the vLLM RCE demonstrate that inference engines are not hardened for adversarial environments. Impact: server compromise, prompt leakage, denial of service. Likelihood: moderate, higher for internet-exposed deployments.

EDR detection gap (MEDIUM-HIGH). The architectural inability of EDR to detect model-layer threats means compromised models can operate undetected for extended periods. Impact: sustained data exfiltration or output manipulation without alerting. Likelihood: the gap is certain; the question is whether it is actively exploited, and evidence suggests it increasingly will be.

Model-layer attacks including backdoors and steganography (MEDIUM). Academic research has demonstrated feasibility. Real-world weaponization at scale is not yet widely documented but is expected as local LLM deployment becomes more prevalent. Impact: potentially severe, as these attacks can be highly targeted and difficult to detect. Likelihood: increasing.

Best Practices for Enterprise Deployment

Organizations deploying open-source LLMs locally should implement defense-in-depth across the following layers:

Format policy. Mandate SafeTensors or GGUF for all model deployments. Reject pickle-based formats (.pt, .pth, .bin) unless the model has been independently audited and converted. Implement automated format checking in your model ingestion pipeline.

Supply-chain verification. Pin model versions by SHA-256 hash, not by name or tag. Verify model provenance through multiple channels (the model author's official page, release announcements, community validation). Maintain an internal model registry and never load models directly from public repositories in production.

Network isolation. Run inference workloads in network-segmented environments. Inference servers should not have access to production databases, credential stores, or the broader corporate network. Use VCF vDefend or equivalent microsegmentation to enforce least-privilege network policies at the workload level.

Inference engine hardening. Keep inference engines updated — every Ollama and vLLM CVE listed in this report has a patched version. Disable unnecessary API endpoints. Run inference engines as non-root with minimal filesystem permissions. Consider containerized or VM-isolated deployment.

Output monitoring. Implement application-layer monitoring on model outputs. Statistical analysis of output distributions can detect some forms of model poisoning and backdoor activation. This is an emerging area with no mature commercial tooling, but even basic anomaly detection on output patterns provides value.

Governance and audit. Maintain a model bill of materials (MBOM) tracking every model in use, its format, source, version hash, deployment location, and last security review. Integrate model governance into your existing IT asset management and vulnerability management programs.

For organizations seeking a more secure path to on-prem AI, commercially-licensed models from vendors like Cohere, Mistral, Writer, and Aleph Alpha eliminate the pickle deserialization risk, provide a vendor of record for security accountability, and include IP indemnity. For a detailed comparison of these vendors and how they run on VMware Cloud Foundation infrastructure, see: Enterprise Buyer's Guide to On-Prem LLMs.

---

CVEs Referenced:

• CVE-2025-32434 — PyTorch torch.load() weights_only bypass (CVSS 9.3)
• CVE-2025-10155 — PickleScan encoding bypass (CVSS 9.3)
• CVE-2025-10156 — PickleScan class reference manipulation (CVSS 9.3)
• CVE-2025-10157 — PickleScan archive structure exploit (CVSS 9.3)
• CVE-2024-37032 — Ollama path traversal (Probllama)
• CVE-2024-39719 — Ollama file existence disclosure
• CVE-2024-39720 — Ollama GGUF out-of-bounds read
• CVE-2024-39721 — Ollama null-byte DoS
• CVE-2024-39722 — Ollama API path traversal
• CVE-2025-0315 — Ollama memory leak / prompt disclosure
• CVE-2024-10966 — vLLM remote code execution via model loading

Frameworks Referenced:

• MITRE ATLAS (Adversarial Threat Landscape for AI Systems)
• OWASP Top 10 for LLM Applications
• NIST AI Risk Management Framework (AI RMF)

Sources:

• JFrog — Malicious ML Models on Hugging Face↗
• Trail of Bits — PickleScan Zero-Day Analysis↗
• NVIDIA — AI Model Security Best Practices↗
• Hugging Face — SafeTensors Documentation↗
• Ollama Security Advisories↗
• PyTorch — CVE-2025-32434 Advisory↗