Skip to main content
Architecture Decision Records (ADRs) capture the context, decision, and consequences of every major design choice in EMEP. Each ADR is immutable once ratified. Reversal requires a new ADR that explicitly supersedes the prior record. This page contains the ten foundational ADRs that govern the platform.

​
ADR Format

Every ADR in this specification follows a standard structure:

​
ADR-0001: Modular Architecture, Not MergeKit-Coupled

Status: RATIFIED Context: Model merging tools such as MergeKit provide concrete implementations of merge strategies. However, coupling the entire platform to a single external library creates vendor lock-in, prevents substitution of individual components, and complicates testing. The platform needs to support multiple merge backends and evolve its own implementations. Decision: EMEP components are independent abstractions with defined interfaces. MergeKit may be used as one implementation of MergeStrategy, but no core component (ModelRegistry, MergeEngine, EvaluationEngine, EvolutionEngine) shall depend on MergeKit-specific APIs. All component boundaries are defined by internal interfaces. Consequences:
  • Positive: Components can be tested in isolation. New merge backends can be added without rewriting the platform.
  • Positive: The specification remains valid even if MergeKit changes its API or license.
  • Negative: Initial implementation requires more boilerplate to bridge interfaces.
  • Neutral: MergeKit remains a recommended reference implementation for several strategies.

​
ADR-0002: Documentation-First Execution

Status: RATIFIED Context: EMEP is a specification for a platform that does not yet exist. Building code before documentation risks undocumented behavior, hidden assumptions, and irreproducible experiments. The project is research-oriented and requires precise communication across research engineers, ML engineers, and evaluators. Decision: No implementation begins before its specification is written, reviewed, and ratified. Documentation is the primary artifact. Code is the secondary artifact. Algorithm specifications must include purpose, inputs, parameters, tensor shapes, mathematical formulation, step-by-step processing, decision points, outputs, failure conditions, validation, complexity, memory requirements, numerical stability, precision considerations, reproducibility, unit tests, integration tests, acceptance criteria, known limitations, and research references. Consequences:
  • Positive: Every behavior is specified before it is coded. Reviews catch design errors early.
  • Positive: New team members can understand the platform without reading source code.
  • Negative: Slower initial velocity. Prototyping requires parallel documentation.
  • Neutral: This ADR applies to the specification phase. Implementations may relax the rule for exploratory spikes, but spikes cannot be merged without retroactive documentation.

​
ADR-0003: Hidden Test Set Isolation

Status: RATIFIED Context: Benchmark contamination is a known risk in LLM evaluation. If the hidden test set is used for model selection, hyperparameter tuning, or evolutionary fitness, the resulting performance estimate is optimistically biased and invalid for generalization claims. The platform must guarantee that hidden test data never influences the training or selection process. Decision: The Hidden Test Set is never used by the EvolutionEngine, FitnessEngine, or any hyperparameter search. It is evaluated exactly once per promoted model, after the candidate has been selected by the Optimization Set and Validation Set. The EvaluationEngine enforces this at the API level: any request to evaluate a candidate on the Hidden Test Set before PROMOTED status is rejected. Consequences:
  • Positive: Generalization claims are statistically valid.
  • Positive: Benchmark integrity is auditable and enforceable by code.
  • Negative: Final model selection cannot use the most challenging test data.
  • Negative: Requires strict access control on the Hidden Test Set dataset.

​
ADR-0004: Fine-Tuning Is Optional

Status: RATIFIED Context: Fine-tuning is a common method for adapting models to specific tasks. However, EMEP’s core hypothesis is that merging alone can produce superior models. Making fine-tuning a required step would change the scientific claim and increase computational cost. At the same time, fine-tuned specialist models may provide useful inputs to the merge process. Decision: Fine-tuning may be used to produce specialist models that are then merged, but no primary loop step depends on fine-tuning. The platform supports LoRA, QLoRA, and full fine-tuning as optional pre-processing steps. If fine-tuning is used, its outputs are treated as base models for compatibility analysis. Consequences:
  • Positive: Core hypothesis remains testable without fine-tuning infrastructure.
  • Positive: Reduces barrier to entry for teams without fine-tuning expertise.
  • Negative: Some model combinations may require fine-tuned intermediates to achieve compatibility.
  • Neutral: Fine-tuning components are specified in full but marked as optional in all pipeline definitions.

​
ADR-0005: RAG Is Downstream

Status: RATIFIED Context: Retrieval-Augmented Generation is a popular technique for improving model outputs with external knowledge. However, RAG does not create new parameter tensors. It is an application-layer technique that consumes models. Including RAG in the primary loop would conflate model engineering with application engineering. Decision: RAG is permanently excluded from the primary model creation loop. EMEP-produced models may be consumed by downstream RAG pipelines, but the RAG pipeline is not part of the EMEP specification except as an optional Phase 9 integration. See RAG Architecture. Consequences:
  • Positive: Clear separation of concerns between model engineering and application engineering.
  • Positive: EMEP can be used by teams that do not use RAG.
  • Negative: Teams that want an end-to-end RAG-plus-model platform must integrate EMEP with a separate RAG system.

​
ADR-0006: Agents Out of Scope

Status: RATIFIED Context: Autonomous agents, tool-use frameworks, and multi-agent orchestration are active research areas. They are application-layer systems built on top of language models. Including them in EMEP would expand scope indefinitely and distract from the core hypothesis. Decision: All agent frameworks, tool-use orchestration, and multi-agent systems are permanently excluded from EMEP scope. This includes chatbot interfaces, autonomous planners, and API-wrapping agents. Consequences:
  • Positive: Scope remains focused and deliverable.
  • Positive: No dependency on rapidly evolving agent frameworks.
  • Negative: EMEP does not specify how its models are used in agent contexts.

​
ADR-0007: Mermaid as Primary Diagram Format

Status: RATIFIED Context: Architecture and algorithm diagrams must be version-controllable, renderable in documentation tools, and editable by text editors. Binary formats (PNG, SVG hand-drawn) and proprietary formats (Visio, Lucidchart) fail one or more of these criteria. Decision: All architecture, algorithm, data flow, state machine, and sequence diagrams are specified in Mermaid syntax. Mermaid is the only required diagram format for EMEP conformance. Other formats may be provided as supplements but are not normative. Consequences:
  • Positive: Diagrams live in version control and are reviewed in pull requests.
  • Positive: No dependency on external diagramming tools or licenses.
  • Negative: Complex visual layouts are harder to achieve in Mermaid than in dedicated tools.
  • Neutral: Mermaid renderers may vary slightly in output style, but the semantic content is preserved.

​
ADR-0008: Every Artifact Hashed and Provenance Tracked

Status: RATIFIED Context: Reproducibility, integrity verification, and security auditing require knowing exactly what data and code produced every model. Without cryptographic hashing and immutable provenance records, it is impossible to detect tampering or verify that an experiment was run as claimed. Decision: Every file, model checkpoint, configuration, log, and evaluation result produced by EMEP must be hashed at creation. The hash, timestamp, creating component, and full parameter set are stored in an immutable provenance record. Provenance records are themselves hashed and linked in a chain. Consequences:
  • Positive: Any artifact can be verified against its claimed origin.
  • Positive: Supply chain attacks on model artifacts are detectable.
  • Negative: Storage overhead for hashes and provenance metadata.
  • Negative: Requires careful key management if cryptographic signing is used.

​
ADR-0009: Compatibility States Are Trinary

Status: RATIFIED Context: Model compatibility is not always binary. Two models may be mergeable with minor adjustments (tokenizer remapping, layer padding) or may be fundamentally incompatible (different architectures). A binary compatible/incompatible split forces implementers to make implicit decisions about edge cases. Decision: Every pairwise model comparison must produce exactly one of three states: COMPATIBLE (merge proceeds without modification), CONDITIONALLY_COMPATIBLE (merge possible with documented shims or parameter adjustments), or INCOMPATIBLE (merge must not proceed). No fourth state exists. The ModelCompatibilityAnalyzer must document the specific conditions for CONDITIONALLY_COMPATIBLE. Consequences:
  • Positive: Explicit handling of edge cases prevents silent failures.
  • Positive: CONDITIONALLY_COMPATIBLE cases can be reviewed and approved by a human curator.
  • Negative: Slightly more complex state machine than binary compatibility.
  • Neutral: The trinary state does not guarantee that CONDITIONALLY_COMPATIBLE merges will succeed; it only permits the attempt.

​
ADR-0010: Multi-Objective Optimization via Pareto

Status: RATIFIED Context: LLM evaluation typically produces multiple metrics (accuracy, perplexity, safety score, latency). Scalar aggregation (weighted sum, product) hides trade-offs and requires arbitrary weight choices. Researchers and evaluators need to see the full trade-off surface. Decision: When multiple benchmark metrics conflict, the platform selects candidates using Pareto dominance. A candidate is selected if no other candidate dominates it on all objectives. Scalar aggregation is permitted only if an explicit scalarization function has been ratified by ADR. The Pareto front is the default selection mechanism. Consequences:
  • Positive: Trade-offs are explicit and auditable.
  • Positive: No arbitrary weight selection is required for default operation.
  • Negative: Pareto fronts can grow large, complicating selection.
  • Negative: Some stakeholders prefer a single “best” model and must be educated on Pareto interpretation.