Deel is one team among many doing this. We collected 35 other open-weight models that started from an existing base model and were trained further for one domain or task, and recorded the same facts for each: what it was built on, which training stages it went through, how much data it saw, and what it gained over its starting point. Every figure comes from a model card, paper or launch post, checked on 21 October 2026, and every source is listed at the end.

How much training added

Points gained over the model each one started from, on its own headline benchmark

• Pangaea K1HR / Employment, weights not public
  +51
• Lawma-8BLegal
  +37.7
• Foundation-Sec-8B-ReasoningCybersecurity
  +22.2
• MedGemma 1.5 4BMedical
  +18.4
• Sarvam-MIndic languages
  +16
• DianJin-R1-32BFinance
  +14.1
• MedGemma 27BMedical
  +12.8
• MiMo-V2.6-Distill-Qwen-9BCoding / Agents
  +12.6
• Qwen-Open-Finance-R-8BFinance
  +10
• Fin-R1Finance
  +9.6
• Fino1-8BFinance
  +9.6
• OmniCoder-9BCoding / Agents
  +9
• Foundation-Sec-8BCybersecurity
  +9
• SaulLM-141B-InstructLegal
  +8?
• Intern-S1Science
  +7.6
• Apertus-70B-MeditronFOMedical
  +6.5
• LLama-3-8B-TeleTelecom
  +5
• Index-Translate-9BTranslation
  +4.7
• 0102030405060

Qwen is the default starting point

Seventeen of the 36 models start from Qwen. In the 9B class the pattern is complete: Cloudflare's Clef-Flash, Bilibili's Index-Translate-9B, Xiaomi's MiMo-V2.6 distill and Tesslate's OmniCoder-9B all build on Qwen3.5-9B, which Alibaba publishes under Apache 2.0 in sizes from 0.8B to 122B. Deel has not named the base for Pangaea K1, though it is the same size.

Llama 3 and 3.1 account for five models, nearly all from 2024 and early 2025. Mistral and Mixtral account for six, four of them on Mistral Small 24B. Google's Gemma line carries four: both MedGemma releases, Sarvam-Translate and the Open Telco model, which is built on Gemma 4 31B.

If you are choosing a base today, the practical reason for Qwen's lead is the size ladder. One architecture at every scale lets a team prototype on 9B and move up without changing tokenizer, chat format or tooling.

Which training stages teams use

Three stages appear in the table, and they do different jobs. Continued pretraining keeps the model predicting the next word, on domain text instead of the open web; it is how a model takes in facts. Supervised fine-tuning trains it on example answers, which teaches it how to respond. Reinforcement learning rewards answers that pass a check, so it shows up where an answer can be verified: a calculation, a compliance verdict, a test suite.

Eleven models use fine-tuning alone. Seven follow it with reinforcement learning, and that pairing is now the standard recipe for reasoning in a domain: Fin-R1, DianJin-R1 and Fino1 in finance, II-Medical and Baichuan-M3 in medicine, ether0 in chemistry, Sarvam-M for Indian languages. DeepSWE skipped fine-tuning and used reinforcement learning alone.

Continued pretraining appears where the model has to hold a large body of knowledge: SaulLM on legal text, Intern-S1 on science, Foundation-Sec-8B on security, LLama-3-8B-Tele on telecom standards. Deel's write-up gives the reason in one line, citing a 2024 Google study: fine-tuning on facts the model does not already know makes it learn them slowly and hallucinate more. Facts go in during pretraining, and fine-tuning teaches the model what a good answer looks like.

How much data it takes

The amount depends on whether the model is learning a behaviour or a body of facts.

• A behaviour, in thousands of examples. SWE-agent-LM-32B reached 40.2% on SWE-Bench Verified from 5,000 agent trajectories. DeepSWE reached 42.2% from about 4,500 problems over 200 reinforcement learning steps.
• A domain's reasoning, in tens of thousands. Fin-R1 used about 60,000 worked examples and lifted FinQA from 60.0 to 76.0, ahead of DeepSeek-R1 at 71.0.
• A company's knowledge, in billions of tokens. Deel turned a 69 million token knowledge base into a 1.2 billion token training corpus by restating each fact about a dozen ways, then trained for two passes. Foundation-Sec-8B used 5.1 billion tokens and LLama-3-8B-Tele 2.5 billion.
• A whole field, in hundreds of billions or more. SaulLM trained on more than 500 billion legal tokens. Intern-S1 used about 5 trillion, over half of them scientific.

Deel's corpus is the useful reference point for a company. Its source material was small, and the multiplication came from rewriting: research it cites shows a model can memorise a text and still answer 0% of questions about it unless it has seen the facts restated in varied forms.

What the gains look like

The largest gains come where the task is narrow and the base model was weak at it. Lawma-8B nearly doubled its base on legal classification and beat GPT-4 (62.9%). Cisco's Foundation-Sec-8B-Reasoning gained 22.2 points on threat-intelligence root-cause mapping over Llama-3.1-8B. Sarvam-M gained 16 points on an Indian-language knowledge benchmark and nearly doubled its base on maths word problems written in romanized Indian languages.

Broad domains move less. EPFL's Apertus-70B-MeditronFO gained 6.5 points averaged over five medical benchmarks, and Fin-R1 gained 9.6 averaged over five finance benchmarks. DianJin-R1-32B is the outlier in finance: 14.1 points on average, and 96.0 on a compliance checking benchmark where its base scored 56.5 and DeepSeek-R1 scored 67.5.

Several small models match or beat far larger general ones on their own task. Fin-R1 at 7B beats DeepSeek-R1 on FinQA, though DeepSeek-R1 still leads on the five-benchmark average (78.2 against 75.2). Pangaea K1 answered from memory within five points of the best search-equipped model and in about half the time.

What training costs in general ability

Training hard on one domain can wear away what the model knew before. The table shows it in small amounts. Foundation-Sec-8B dropped from 61.7 to 59.3 on MMLU and slightly on one security benchmark. LLama-3-8B-Tele slipped from 62.1 to 61.6 on MMLU. MedGemma 1.5 improved on most medical tasks but fell below MedGemma 1 on some, including PubMedQA.

The usual guard is replay: mixing general text back into training. Deel kept about 7.5% of its corpus as general text, citing research that even 1% measurably reduces forgetting, and added everyday tasks to its fine-tuning set for the same reason.

How to read these numbers

• Every result is self-reported. None of the benchmark figures in the table has been independently reproduced.
• Each model picks its own benchmarks. A 9-point gain on FinQA and a 9-point gain on Terminal-Bench say nothing about each other. Compare within a domain.
• Agent benchmarks depend on the harness. SWE-Bench scores in the table come from OpenHands, SWE-agent and R2E-Gym setups, so they are close to like for like at best.
• Some sources contradict themselves. Baichuan-M3's card claims 28 points over M2 on HealthBench-Hard where the two cards show about 10. SaulLM's paper gives its legal corpus as 500, 520 and 540 billion tokens in different places. Fino1's text and table give different base scores.
• Disclosure is thin. Eight models do not describe their training method at all, among them Google's MedGemma, Mistral's Devstral and Cloudflare's Clef-Flash.

Where the gaps are

We found no open-weight model trained for HR or customer support with published results, and only one for marketing. Marketeam's Qwen-Marketing was trained on more than 10 billion tokens of marketing data from Qwen3-8B, and its card says no formal benchmark exists yet. Cloudflare's Clef-Flash comes closest on support: it routes and escalates tickets, scores 90.9 macro-F1 on BANKING77, and does not compare itself with its base.

The reason is visible in the table. Law, medicine, finance and code all have large public corpora and public benchmarks. A company's own operations have neither: the knowledge sits in its tickets, policies and systems, and the only test that matters is whether the model gets that company's cases right. Deel solved both halves itself, with tens of thousands of internal pages and a 601-question evaluation built from what its teams ask every day.

Frequently asked questions

All sources (36 models)

• Apertus-70B-MeditronFO: Model card (params, base, data, stages, results); Paper (date)
• Baichuan-M2-32B: Model card (all fields)
• Baichuan-M3-235B: Model card (params, base, results); Paper (date, stages)
• Clef-Flash: Model card (params, base, stages, results); Let's Data Science news (date)
• DeepSWE-Preview: Model card (all fields)
• Devstral-Small-2505: Model card (params, base, results; date inferred from name)
• Devstral-Small-2507: Model card (params, base, results; date inferred from name)
• DianJin-R1-32B: Model card (params, base); Paper (Table 3) (date, data, stages, results)
• eCeLLM-M: Model card (params, base, data, stages); Paper (date, results)
• ether0: Paper (all fields)
• Fin-R1: Model card (all fields)
• Fino1-8B: Model card (params, base, data, stages); Paper (Table 3) (date, results)
• Foundation-Sec-8B: Tech report (all fields)
• Foundation-Sec-8B-Reasoning: Model card (all fields)
• II-Medical-8B: Card mirror (Featherless) (date, params, base, data, stages); Model card (results)
• Index-Translate-9B: Model card (params, base, data, stages, results); PyTorch KR summary (date)
• Intern-S1: Model card (params, base, results); Paper (date, data, stages)
• Lawma-8B: Model card (all fields)
• Ling-3.0-flash-Fin: Model card (params, base, data, stages); sec-api.io (date, results)
• LLama-3-8B-Tele: Model card (params, base); Paper (Table III) (date, data, stages, results)
• MedGemma 1.5 4B: Model card (all fields)
• MedGemma 27B: Model card (all fields)
• MiMo-V2.6-Distill-Qwen-9B: Model card (params, base, data, stages, results); gradually.ai (no link) (date)
• OmniCoder-9B: Model card (all fields)
• OpenMath-Nemotron-32B: Model card (all fields)
• OpenReasoning-Nemotron-32B: Model card (all fields)
• OTel-2.0-LLM-31B-IT: Model card (date, params, base, stages, results); Paper (data)
• Palmyra-Fin-70B-32K: Model card (params, base, data); Press release (date); Writer product page (results)
• Pangaea K1: Deel blog (all fields)
• Qwen-Marketing: Model card (all fields)
• Qwen-Open-Finance-R-8B: Model card (params, base, data, stages); Paper (Table 4) (date, results)
• Qwen3-Coder-Next: Tech report (all fields)
• Sarvam-M: Model card (date, params, base, stages); Sarvam blog (results)
• Sarvam-Translate: Model card (date, params, base); Sarvam blog (results)
• SaulLM-141B-Instruct: Model card (params); Paper (date, data, base, stages, results)
• SWE-agent-LM-32B: Model card (params, base, data, stages); SWE-smith paper (date, results)
• Background: Gekhman et al., Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (2024)
• Background: Allen-Zhu and Li, Physics of Language Models, Part 3.1 (2023)
• Background: Qwen3.5 size ladder and licence: Context Studios self-hosting guide