Evaluate
Determine whether a candidate model meets task-specific quality, safety and reliability thresholds.
- EvaluateAvailable in pilots
Model evaluation, optimisation and selection
Kordane Forge
Prove which permitted model is sufficient.
What is the smallest permitted model that can perform this task reliably?
Evaluate candidate models against each task’s acceptance threshold, train or specialise where useful, and select the smallest permitted model that passes, escalating only when the threshold fails.
Current maturityTraining and evaluation available in pilots · selection and runtime planned
Once a task is permitted, Forge evaluates candidate models against its acceptance threshold and selects the smallest permitted model that passes. Lower model cost is the outcome of proven sufficiency, not an assumption. Optimisation never overrides policy.
What Forge does
Evaluate. Optimise. Select.
Controlled model sufficiency: prove what a candidate model can do, make smaller models sufficient where useful, and choose the approved model and fallback for each task. Each outcome shows the status of the modules behind it.
Optimise
Train, distil, quantise or specialise models when a general-purpose model is unnecessarily expensive or slow.
- DataPlanned
- TrainAvailable in pilots
- DistilPlanned
- RuntimePlanned
Select
Choose an approved model and fallback route based on task, quality, privacy, latency and cost policy.
- RouterPlanned
Forge Evaluate
Evaluation result: choose the right model
Candidate model classes compared against a task-specific quality threshold.
Best modelkordane-invoice-small-v2
F1 score0.94
Avg latency0.6 s
Cost classLow
| Model | Environment | F1 | Latency | Cost class | Result |
|---|---|---|---|---|---|
| local-baseline | Local | 0.78 | 0.4 s | Low | Below threshold |
| kordane-invoice-small-v2Selected | Private | 0.94 | 0.6 s | Low | Passes |
| private-general-mediumFallback | Private | 0.93 | 0.9 s | Medium | Passes |
| approved-frontier | Approved cloud | 0.97 | 1.6 s | High | Passes |
- Selected
- kordane-invoice-small-v2
- Reason
- Smallest approved model that meets the configured quality threshold.
- Fallback
- private-general-medium
Run evidence retained for audit · reference EV-2231 (synthetic)
Forge evaluates quality, latency and cost against your threshold and recommends the smallest approved model that meets the requirement. Run evidence is retained for audit.
Illustrative evaluation run · Synthetic data · Representative of current product capability
The balance Forge manages
Route each task by threshold, not habit.
Selection weighs quality · cost · latency · privacy · deployment location · fallback requirement, designed to send each task to the smallest approved model that clears the thresholds you set, with Boundary deciding whether and where the execution may happen at all.
Synthetic execution decision.Illustrative policy and model-selection logic.Not measured benchmarks.
Deterministic rules illustrate how Boundary (permission and environment) and Forge (model sufficiency) are designed to decide together. Nothing you select is transmitted.
Decision Small local model
- Task classification
- ·
- Information class
- ·
- Allowed environment
- ·
- Selected model class
- ·
- Fallback route
- ·
- External transfer
- ·
- Decision reason
- ·
- Evidence status
- ·
Three route classes. One threshold.
Every task is evaluated against the smallest class that clears it, with an approved fallback above it.
| Route class | Runs | Typical work | When it wins |
|---|---|---|---|
| Local small | On your metal, no egress | Classification, extraction, routine summaries | Repeatable tasks with a clear threshold |
| Specialised private | Your VPC or on-prem | Domain drafting, structured analysis | Trained or tuned for your material |
| Approved frontier | Allowlisted external route | Complex reasoning, high-stakes quality | Only when the threshold demands it |
Fallback always points to the next approved class up, never to an unapproved route.
Lifecycle
Data → Train → Distil → Evaluate → Route → Runtime.
One lifecycle behind the three outcomes, stage by stage.
Forge Data
PlannedDatasets prepared with provenance and leakage prevention.
- Dataset preparation, cleaning and deduplication
- Labelling and synthetic-data generation
- Provenance and dataset versioning
- Train/validation/test splits with leakage prevention
Forge Train
Available in pilotsTask-specific small models, tuned for the job.
- Fine-tuning and instruction tuning
- Adapters and LoRA
- Domain adaptation
- Specialised classifiers and entity extractors
- Prompt-to-model migration where appropriate
Forge Distil
PlannedLarger models reduced to focused task models.
- Teacher/student workflows and knowledge distillation
- Model compression and quantisation preparation
- Task specialisation
Forge Evaluate
Available in pilotsCandidates compared against the task, with release gates.
- Benchmark corpora and candidate comparison
- Accuracy and task metrics, regression testing
- Safety, leakage and refusal tests
- Adversarial cases
- Release gates, evidence packages and rollback criteria
Forge Router
PlannedSmall-model-first routing with quality thresholds and fallback.
- Task-complexity classification
- Model cascading with cost, latency and quality thresholds
- Local/private/cloud route choice
- Fallback to larger models
- Routing-effectiveness monitoring
Forge Runtime
PlannedSpecialised models served on infrastructure you control.
- Local model serving with CPU/GPU optimisation
- Quantisation, batching and caching
- Edge and private-infrastructure deployment
- Runtime monitoring
Boundary decides whether and where a task may run. Forge decides which permitted model should perform it. Together they are designed to produce controlled execution: approved destinations only, sensitive values transformed before any external crossing, every model decision recorded.