Skip to main content
Fine-tuning is optional in EMEP. The primary loop is compatibility analysis, tensor merging, evaluation, and evolutionary optimization. Fine-tuning exists to produce specialist models that feed into MergeEngine as additional inputs. You do not need to fine-tune to use EMEP.
Fine-tuning is OPTIONAL and secondary to the primary loop. Do not treat it as a required step.

​
Positioning

The EMEP core loop accepts models from ModelRegistry, analyzes compatibility, merges them, evaluates candidates, and evolves the population. Fine-tuning is a side loop that creates new base models or specialist variants before they enter the registry. A fine-tuned model passes through the same lifecycle states as any other: DISCOVERED, IMPORTED, VALIDATED, REGISTERED, then into the merge loop.

​
When to Fine-Tune

Consider fine-tuning when:
  • No existing model in the registry covers a target domain or language.
  • A specialist adapter (LoRA, prefix-tuning) is cheaper than sourcing a new base model.
  • You need a controlled variant for ablation or regression testing.
Prefer merging over fine-tuning when compatible models already exist in the registry. Merging preserves capabilities from multiple parents; fine-tuning risks catastrophic forgetting (McCloskey & Cohen 1989; Kirkpatrick et al. 2017).

​
Fine-Tuning Loop

The specialist model is registered like any other, then becomes available to ModelCompatibilityAnalyzer and MergeEngine.

​
Supported Methods

EMEP supports the following fine-tuning paths:
  • LoRA (Hu et al. 2021): low-rank adapters. Fast, low memory, mergeable into base weights.
  • QLoRA (Dettmers et al. 2023): NF4 quantization plus LoRA. Further memory reduction.
  • Prefix-tuning / Prompt-tuning: adapter families covered in the PEFT survey (Han et al. 2024).
  • Full fine-tuning: when adapters are insufficient. Higher compute cost, higher forgetting risk.

​
Controls and Safeguards

Every fine-tuned model must pass regression evaluation before promotion. Controls include rehearsal, replay, low learning rate, early stopping, regularization (EWC per Kirkpatrick et al. 2017), distillation, and LoRA (partial mitigation only). See Catastrophic Forgetting Controls for the full protocol.

​
Integration Points

  • DatasetRegistry: provides training, validation, and hidden test splits.
  • ExperimentTracker: logs fine-tuning runs with the same schema as merge experiments.
  • ArtifactStore: stores checkpoints, adapters, and merged weights.
  • ModelRegistry: receives the final model after VALIDATED status.

​
Future Goal

Future releases may support multi-task fine-tuning pipelines and automated adapter selection based on domain gap metrics. These are not in the initial core.