We use cookies to improve your experience on our site.
AVID-2026-R0391
Description
The model Mistral-Small-24B-Instruct-2501 from Mistral was evaluated by the Garak LLM Vulnerability scanner using the probe misleading.FalseAssertion.
Details
The probe misleading.FalseAssertion tests if the model can refute false claims and avoid being misled by them. This probe is part of the misleading module which checks model responses to misleading assertions.
The LLM Mistral-Small-24B-Instruct-2501 was evaluated on this probe.
Metrics
Number of failures were counted for each detector in the probe.
| Detector | Passed | Total | Score |
|---|---|---|---|
| misleading.MustRefuteClaimModel | 123 | 150 | 🟢 82.0 |
| misleading.MustContradictNLI | 55 | 150 | 🟠 36.7 |
| misleading.MustRefuteClaimKW | 122 | 150 | 🟢 81.3 |
| mitigation.MitigationBypass | 0 | 150 | 🔴 0.0 |
References
Affected or Relevant Artifacts
- Developer: Mistral
- Deployer: Together AI
- Artifact Details:
| Type | Name |
|---|---|
| Model | Mistral-Small-24B-Instruct-2501 |
Impact
AVID Taxonomy Categorization
- Risk domains: Performance
- SEP subcategories: P0204: Accuracy
- Lifecycle stages: L05: Evaluation
Other information
- Report Type: Measurement
- Credits:
- Date Reported: 2026-03-10
- Version: 0.3.1
- AVID Entry