EMEP Evidence Map: Claims to Sources

The Evidence Map is the central audit document that connects every claim made in the EMEP specification to its supporting evidence. Each claim is tagged with an evidence type (Established research, Research hypothesis, Engineering assumption, Experimental result, Future goal, or Unsupported) and linked to the component that depends on it. This page enables reviewers to verify that no claim is unsupported and that no evidence is orphaned.

Evidence Type Definitions

Type	Meaning	Example
Established research	Widely accepted result from peer-reviewed literature	Task arithmetic enables capability editing
Research hypothesis	Testable claim not yet proven in EMEP context	Evolutionary merge discovers superior LLMs
Engineering assumption	Design choice without formal proof	Mermaid is sufficient for all diagrams
Experimental result	Result from an EMEP experiment	Requires an Experiment ID like EXP-TBD-001
Future goal	Planned but not yet implemented	Multi-modal model support
Unsupported	Claim with no current evidence	Must be flagged for review

Evidence Map Table

Claim	Source	Evidence Type	Component
Weight averaging of fine-tuned models improves accuracy	Model Soups: Wortsman et al. 2022	Established research	MergeEngine, Linear Merge
Task vectors enable capability addition and removal	Task Arithmetic: Ilharco et al. 2022	Established research	Task Arithmetic, TensorEngine
Spherical interpolation preserves vector relationships better than linear	SLERP: Shoemake 1985	Established research	SLERP Math
Trimming, sign election, and disjoint merging reduce interference	TIES-Merging: Yadav et al. 2023	Established research	TIES Math, MergeEngine
Random dropping and rescaling enable merging many models	DARE: Yu et al. 2023	Established research	DARE Math
Evolutionary search discovers superior merge combinations	Evolutionary Model Merge: Akiba et al. 2024	Established research	EvolutionEngine
LoRA reduces trainable parameters with comparable performance	LoRA: Hu et al. 2021	Established research	LoRA, PEFT
QLoRA enables fine-tuning on consumer GPUs	QLoRA: Dettmers et al. 2023	Established research	QLoRA
Benchmark contamination is widespread and often undetected	Jacovi, Goldberg, and Tsarfaty 2023	Established research	Benchmark Integrity
No single model dominates all evaluation scenarios	HELM: Liang et al. 2022	Established research	Multi-Objective Optimization
EMEP can produce models superior to all ancestors on specified benchmarks	Core platform hypothesis	Research hypothesis	EvolutionEngine, FitnessEngine
Pareto dominance is the correct default selection mechanism for conflicting metrics	Multi-objective optimization theory	Research hypothesis	Multi-Objective Optimization, FitnessEngine
Hidden test set isolation prevents benchmark contamination	Statistical testing theory	Engineering assumption	EvaluationEngine, Benchmark Specification
Mermaid syntax is sufficient for all EMEP diagrams	Documentation tooling survey	Engineering assumption	Diagrams
Modular architecture prevents vendor lock-in	Software engineering best practice	Engineering assumption	System Architecture
GPU health monitoring prevents silent numerical errors	Hardware reliability literature	Engineering assumption	GPU Health
Deterministic merges produce bitwise-identical outputs	Hash-based reproducibility practice	Engineering assumption	MergeEngine, Reproducibility
Air-gapped deployment is feasible for enterprise use	Enterprise security requirements	Engineering assumption	Offline Deployment
TIES + DARE combined strategy outperforms either alone	Unpublished combination	Research hypothesis	Merge Strategies
Franken-Merge (structural) produces viable models	Structural merging literature	Research hypothesis	Merge Strategies
Linear merge scales to 100+ billion parameters	Engineering projection	Engineering assumption	MergeEngine
SLERP is numerically stable at billion-parameter scale	High-dimensional geometry theory	Research hypothesis	SLERP Math
Population size of 50 is sufficient for convergence	Evolutionary computation heuristics	Engineering assumption	Population
10 generations of evolution discover most gains	Evolutionary Model Merge: Akiba et al. 2024	Experimental result (EXP-TBD-001)	EvolutionEngine
Multi-modal model support	Not yet specified	Future goal	Vision and Scope
Real-time streaming inference	Not yet specified	Future goal	Inference
Closed-weight model API merging	Permanently excluded	Unsupported	Project Constitution Rule 1
Training from scratch produces better models than merging	Contradicts core hypothesis	Unsupported	Vision and Scope

Concept Map

The following diagram shows how research areas map to EMEP components. Solid lines indicate direct implementation dependency. Dashed lines indicate theoretical inspiration or optional integration.

Audit Notes

Any claim tagged as Unsupported must be resolved before the component that depends on it reaches Phase acceptance criteria. Unsupported claims are not prohibited in early-phase specifications, but they block promotion to later phases.

Experimental results require an Experiment ID in the format EXP-TBD-###. When an experiment is run, the ID is updated to EXP-YYYY-### and linked to the ExperimentTracker record. Until then, the claim remains provisional.

Research hypotheses are testable and will be evaluated by EMEP experiments. Engineering assumptions are design choices that may be validated by practice but are not the subject of scientific proof. Do not conflate the two in documentation.

References for the full annotated bibliography
Component Mapping for the reverse mapping (components to research)
Decisions for architecture decisions that depend on this evidence
Project Constitution for rules governing claim validation

Project Context

Research Foundation

System Maps

EMEP Evidence Map: Claims to Sources

Evidence Type Definitions

Evidence Map Table

Concept Map

Audit Notes

​Evidence Type Definitions

​Evidence Map Table

​Concept Map

​Audit Notes

​Related Pages

Evidence Type Definitions

Evidence Map Table

Concept Map

Audit Notes

Related Pages