Components
| Term | Definition |
|---|---|
| ModelRegistry | Central catalog for all models, their metadata, versions, and lifecycle state |
| ModelLoader | Responsible for loading model weights, configs, and tokenizers from storage into memory |
| ModelCompatibilityAnalyzer | Determines whether two or more models can be merged based on architecture, shapes, and tokenizers |
| TensorEngine | Low-level tensor operation library: addition, scaling, masking, interpolation, and shape validation |
| MergeEngine | Orchestrates the merge process: selects strategy, executes via TensorEngine, produces candidate weights |
| MergeStrategy | Algorithmic recipe for combining weights: Task Arithmetic, TIES, DARE, SLERP, Franken-Merge |
| CandidateGenerator | Produces new model candidates from merge results, ready for evaluation |
| EvolutionEngine | Drives multi-objective search over merge strategies, hyperparameters, and model combinations |
| EvaluationEngine | Executes benchmarks and collects metrics for candidate models |
| BenchmarkEngine | Manages benchmark datasets, splits, and execution protocols |
| FitnessEngine | Computes scalar or vector fitness scores from evaluation results |
| ExperimentTracker | Records experiment configuration, state transitions, and results in an append-only log |
| ArtifactStore | Persistent storage for weights, configs, manifests, and signatures |
| DatasetRegistry | Catalog of datasets available for fine-tuning, evaluation, and benchmark construction |
| QuantizationEngine | Applies post-training quantization to candidate models before deployment |
| InferenceBackend | Executes inference on merged or quantized models for validation and serving |
| DeploymentManager | Packages, signs, and deploys models to target environments |
Compatibility States
| State | Definition |
|---|---|
| COMPATIBLE | Models share identical architecture, tensor shapes, and tokenizer vocabulary. Merge permitted without modification. |
| CONDITIONALLY_COMPATIBLE | Models differ in minor respects (e.g., tokenizer size) but can be merged after alignment or padding. |
| INCOMPATIBLE | Fundamental mismatch in architecture or shapes prevents merge. Requires architecture-level intervention. |
Candidate Status
| Status | Definition |
|---|---|
| PASS | Candidate meets or exceeds all fitness thresholds and regression constraints. |
| FAIL | Candidate falls below fitness thresholds or violates safety constraints. |
| REGRESSION | Candidate improves target metrics but degrades base model capabilities. |
| INVALID | Candidate could not be evaluated due to corruption, shape mismatch, or runtime error. |
| INCOMPLETE | Evaluation is still in progress or was interrupted. |
Model Lifecycle
| Stage | Definition |
|---|---|
| DISCOVERED | Model identified in external source (Hugging Face Hub, local path, enterprise mirror). |
| IMPORTED | Model weights, config, and tokenizer copied into ArtifactStore. |
| VALIDATED | Compatibility analysis and integrity checks passed. |
| REGISTERED | Model entered into ModelRegistry with full metadata and lineage. |
| CANDIDATE | Model produced by MergeEngine or CandidateGenerator, awaiting evaluation. |
| EVALUATED | Benchmark results recorded and signed. |
| PROMOTED | Candidate selected for further evolution or release. |
| RELEASED | Model packaged, versioned, and published to enterprise or public registry. |
| DEPLOYED | Model active in inference backend or production serving. |
| DEPRECATED | Model superseded by newer release. Still available but not recommended. |
| ARCHIVED | Model retained for reproducibility but removed from active serving. |
Experiment States
| State | Definition |
|---|---|
| CREATED | Experiment defined with configuration and benchmark splits. |
| PREPARING | Resources allocated, models imported, environment validated. |
| RUNNING | Evolution or merge operations actively executing. |
| EVALUATING | Candidate models undergoing benchmark evaluation. |
| COMPLETED | All operations finished, results recorded, artifacts stored. |
| FAILED | Unrecoverable error terminated the experiment. |
| CANCELLED | Operator or policy halted the experiment before completion. |
| ARCHIVED | Experiment retained for audit but no longer active. |
Technical Terms
| Term | Definition |
|---|---|
| Task vector | The difference between fine-tuned and pre-trained weights for a specific task. Used in Task Arithmetic (Ilharco et al. 2022). |
| Density | The fraction of non-zero elements retained after sparsification in DARE (Yu et al. 2023). |
| Sparsification | Zeroing out low-magnitude weight differences to reduce interference during merging. |
| SLERP | Spherical linear interpolation for merging weights while preserving angular relationships (Shoemake 1985). |
| TIES | Trimming, Electing Sign, and Disjoint merging: a three-step strategy to reduce parameter interference (Yadav et al. 2023). |
| DARE | Drop And REscale: randomly dropping task vector elements and rescaling survivors (Yu et al. 2023). |
| Franken-Merge | Architecture-level merge combining layers from different models. Not a tensor-level operation. |
| Pareto front | Set of solutions where no objective can be improved without degrading another. Used in multi-objective optimization. |
| Elitism | Preservation of top-performing individuals across generations in evolutionary search. |
| Tournament selection | Selection mechanism where subsets of the population compete, and winners reproduce. |
| Quantization | Reducing numerical precision of weights (e.g., float16 to int8) to reduce memory and latency. |
| Catastrophic forgetting | Degradation of previously learned capabilities when models are modified or merged (McCloskey & Cohen 1989). |
| Benchmark contamination | Test data appearing in training or optimization data, invalidating generalization claims (Sainz et al. 2023). |