Fitness Engine and Candidate Scoring

The FitnessEngine converts evaluation metrics into comparable fitness values for the evolutionary loop. It supports both single-objective scalar aggregation and multi-objective vector fitness. This page specifies the aggregation formulas, weighting schemes, and the critical isolation rule: fitness is computed only from the Optimization Set.

Fitness Computation Flow

Single-Objective Scalar Aggregation

The FitnessEngine combines multiple metrics into a single scalar value using weighted summation or target-relative scoring.

Weighted Sum

fitness = sum_i(w_i * normalized_metric_i)
sum(w_i) = 1.0

Weights are configurable per experiment. Default weights balance accuracy, efficiency, and safety:

Metric Category	Default Weight	Normalization
Accuracy	0.5	Best on Optimization Set
Efficiency	0.2	Inverse latency, max 1.0
Safety	0.2	Pass rate on safety suite
Robustness	0.1	Variance across prompt variants

Target-Relative Scoring

Each metric is scored relative to a target value:

score_i = metric_i / target_i
score_i = min(score_i, 1.0)  # cap at 100%
fitness = sum(w_i * score_i)

Target values are set per benchmark or inherited from baseline models. Target-relative scoring makes fitness interpretable: 1.0 means all targets met.

Multi-Objective Vector Fitness

For multi-objective optimization, fitness is a vector rather than a scalar. The EvolutionEngine passes this vector to the non-dominated sorting algorithm (NSGA-II, Deb et al. 2002).

fitness_vector = [accuracy, efficiency, safety, robustness]

Each component is independently normalized. The Pareto front contains candidates where no other candidate dominates on all objectives.

Example Aggregation Formulas

Standard Accuracy-Weighted Fitness

fitness = 0.6 * normalize(accuracy) + 0.25 * normalize(1 / latency) + 0.15 * normalize(safety_pass_rate)

Balanced Multi-Objective Vector

fitness_vector = [
  normalize(perplexity),      # lower is better, inverted
  normalize(exact_match),     # higher is better
  normalize(1 / memory_gb),    # efficiency
  normalize(refusal_rate)     # safety
]

Target-Relative for Instruction Following

targets = {exact_match: 0.75, rouge_l: 0.70, safety: 0.95}
scores = {k: min(v / targets[k], 1.0) for k, v in metrics.items()}
fitness = mean(scores.values())

Metric Normalization

All metrics are normalized to [0, 1] before aggregation:

For metrics where higher is better: norm = (value - min) / (max - min)
For metrics where lower is better: norm = 1 - (value - min) / (max - min)
If max equals min: norm = 0.5

Min and max are computed from the current population’s Optimization Set results. This makes fitness relative to the current search progress.

Isolation Rule

Fitness is computed ONLY from the Optimization Set. The Validation Set and Hidden Test Set are never used during fitness calculation, ranking, or selection. This rule is enforced at the data layer. Any attempt to compute fitness from non-optimization splits triggers an error and halts the experiment.

Integration with EvolutionEngine

The FitnessEngine exposes:

FitnessEngine.compute_scalar(metrics, weights) -> float
FitnessEngine.compute_vector(metrics) -> List[float]
FitnessEngine.normalize(metrics, population_stats) -> Dict[str, float]

All results are logged to the ExperimentTracker with the full metric dictionary and computed fitness values.

Evolution Engine

Evaluation

Tracking & Reproducibility

Fitness Engine and Candidate Scoring

Fitness Computation Flow

Single-Objective Scalar Aggregation

Weighted Sum

Target-Relative Scoring

Multi-Objective Vector Fitness

Example Aggregation Formulas

Metric Normalization

Isolation Rule

Integration with EvolutionEngine

​Fitness Computation Flow

​Single-Objective Scalar Aggregation

​Weighted Sum

​Target-Relative Scoring

​Multi-Objective Vector Fitness

​Example Aggregation Formulas

​Metric Normalization

​Isolation Rule

​Integration with EvolutionEngine

Fitness Computation Flow

Single-Objective Scalar Aggregation

Weighted Sum

Target-Relative Scoring

Multi-Objective Vector Fitness

Example Aggregation Formulas

Metric Normalization

Isolation Rule

Integration with EvolutionEngine