Model Fragmentation Is Model Sprawl, and Nobody Has the Inventory

You evaluate a model. It passes: no malicious pickle, publisher you recognise, benchmark numbers you can accept. Then your platform team quantises it to 4-bit for the edge fleet, someone fine-tunes a customer-service variant on internal transcripts, a LoRA adapter gets stacked on top for one business unit, and the original stays in the batch pipeline because rewriting that job was never prioritised.

You now run four artefacts. You evaluated one. Quantising to 4-bit rewrites the weights and changes the arithmetic they run in, so the build on your edge fleet fails in places the model you tested did not. The fine-tuned variant has been through a training run on data you did not audit at the depth you audited the base corpus. And nothing in the process that produced any of them was a security review.

A note on the word

I used model fragmentation for this when I first wrote about it in 2023, and the usage did not take. Searching the phrase in this sense today returns this article and its copy. It appears elsewhere in the literature meaning other things, including the literal fragmentation of models and of data. Model sprawl is the more established term for the problem this article is about, and it is the better word. Fragmentation suggests something breaking apart; this is proliferation.

The slug stays because the URL is the ranking, and if you arrived here on the old term, the current one is the one to search. Everything below uses model sprawl.

Every derivative you build and every materially different deployment is a separate thing to assure, with its own provenance, its own vulnerabilities and its own blast radius, and the first control is an inventory most organisations cannot produce.

Where the variants come from

The old version of this article sorted these into version-based, architecture-based and data-based fragmentation. That taxonomy was invented for the article and it does not carve at any joint a defender uses, so it is gone. What matters is what changed and whether anyone looked.

The effective model changed. Fine-tuning and continued pretraining alter the base weights. LoRA and similar adapters usually leave the base frozen and add separately trained parameters, which is a different artefact shape with the same consequence. Distillation and merging produce something new again. In each case the thing you now run has learned something since you evaluated it, from data carrying its own provenance question.

Numerics changed. Quantisation, pruning, compilation to a target runtime. The weights are nominally the same model and the executed computation is not. Evaluations whose conclusions depend on model behaviour should be rerun, or explicitly accepted as stale. A malicious compilation stage can leave the source weights untouched and put the backdoor into the compiled artefact.

Nothing changed but the deployment did. The same weights behind a new system prompt, new tool access, a new retrieval corpus and new credentials. This is the one security teams miss most often because there is no new file to scan, and it is frequently the one with the largest blast radius.

Nothing changed at all. The old version still running somewhere because the migration stalled. Ordinary version drift, and the most common finding in practice.

Why each variant is its own security question

Three consequences follow, and they are the reason this is a security article rather than an MLOps one.

Provenance does not inherit. A signature on the base model says nothing about the derivative your platform team produced last month. If the answer to “where did this come from” is “we fine-tuned the one we downloaded,” the chain has a gap at exactly the point where training data entered. Neural Trojans covers what fits through that gap. The relevant finding is that in the October 2025 pretraining experiments, attack success tracked the absolute number of poisoned documents rather than their share of the corpus, and the authors report similar dynamics during fine-tuning. Read it the way they do: scale is not a defence at either stage. A small curated fine-tuning set is in fact harder for an outsider to contribute to than the open web, which is why the paper’s own conclusion runs toward poisoning being easier for large models than assumed rather than toward small datasets being the softer target.

Safety properties do not inherit either. Qi and colleagues showed in 2023 that fine-tuning an aligned model can compromise its safety even on datasets assembled with no harmful intent. When a derivative ships with its parent’s model card, the reader gets evaluation results measured on a different model.

Robustness properties do not inherit. Adversarial training buys robustness inside a specified perturbation set, typically at substantial compute cost and often at some cost in clean accuracy. Quantise the result and you have changed the model that robustness was measured on. Whether it survives is an empirical question about your build, and you should not assume anyone re-ran the evaluation.

There is a fourth consequence that is less about the model and more about the estate. Each deployed variant tends to arrive with its own endpoint, its own service account and its own set of credentials, so counting models undercounts the exposure. Count the things that can invoke each variant, and what each of those can reach.

The inventory problem

Every security team can answer three questions for its servers and almost none can answer them for its models. What do we run? Where did each one come from? What has touched it since?

The tooling is arriving. CycloneDX has supported machine learning components since specification 1.5 in June 2023, so an ML-BOM can carry model and dataset entries alongside software dependencies. Signing specifications exist. Discovery scanners that walk a codebase for model references, inference endpoints and framework integrations are a young product category and an increasingly crowded one.

Two cautions before anyone buys.

An inventory tells you that you run forty model variants and nothing about which of them can reach production data. A name and a hash do not answer that. Record the parent artefact, the change and who made it, the date of the last evaluation and what it covered, and what the deployment can invoke.

The payoff is not the inventory itself. The day a published model or dataset is found to be compromised, the first question is which of your systems have it. The organisations that can answer inside a day are the ones that wrote it down beforehand.

What to do on Monday

Try to produce the list. Every model artefact running anywhere, with its parent. Most teams cannot finish the exercise. Where they stop is the finding: they can name the models they chose and not the derivatives their pipelines produced.

Record the delta, not just the artefact. For each variant: what changed from the parent, who made the change, and which evaluations you re-ran afterwards. In most estates that last column is empty, which is the honest state and worth seeing.

Re-evaluate after quantisation and after fine-tuning, or say plainly that you did not. Both produce a different artefact. Inheriting the parent’s results is a choice, and it is defensible for low-consequence deployments as long as it is a decision rather than an assumption.

Treat same-weights-different-deployment as a variant. New system prompt, new tools, new credentials, new retrieval corpus. Each is a new system even though no file changed, and for an agent the tools and credentials attached to that deployment are what set the blast radius.

The reason this subject reads as unglamorous is that the answer is asset management, which is where a great deal of security has always ended up. The models are new, the failure is not: an estate nobody has enumerated, changed by people who were not thinking about security because that was not their job, with assurance documents that describe a version no longer running.