Context, memory, and model selection

Storage is not a prompt

A large knowledge base cannot be sent wholesale to every model call. The ContextManager sits between selected memory and model serving: it receives system instructions, optional methodology, task state, recalled episodes, live-case entries, and beliefs, then assembles a bounded set of messages. This is especially important for smaller local models, but a smaller prompt is not by itself proof of equivalent answer quality.

Why three context tiers?

Core holds high-priority system instructions, optional skill body, and task state; Recall holds selected material relevant to the current question; Archival holds less urgent background. ContextItem carries tier, role, and content. FittedContext records estimated tokens, dropped, summarized, and demoted counts, and budget_exceeded so callers can see the trade-off rather than silently pretending nothing was lost.

• Selection and hierarchical demotion trade recall breadth for room for essential instructions.
• Summarize has a simple synchronous compression path and an optional async SummaryProvider; neither guarantees that all details survive.
• Error signals an exceeded budget. The engine also contains a work-decomposition strategy, but the published chart routing enum is only Summarize, Truncate, or Error.

Memory components and their roles

The separate kaigents-memory crate implements MemoryManager. When configured, Qdrant stores searchable short-term records scoped by workspace; RethinkDB stores episodes and beliefs when its feature is enabled; optional NebulaGraph edges represent dependencies and consolidation lineage. The ContextManager fits the retrieved slice into the model window. S3-compatible storage holds large artifacts; it is not the retrieval index. These services are optional integrations requiring their own deployment and access controls.

From one Run to useful recall

• Record permitted observations with workspace and Run identifiers; avoid storing secrets or unrelated customer content.
• Retrieve only evidence scoped to the active workspace. MemoryManager.recall combines Qdrant results with keyword matches on stored episodes when configured.
• Fit candidate memory and task instructions with ContextManager.fit_to_budget or fit_to_budget_tiered; inspect FittedContext counters and budget_exceeded.
• Pass fitted messages to the chosen model. RoutingPolicy.select_model_for_context prefers a local model with sufficient advertised window if configured, or falls back according to policy.
• For repeat work, consolidate Run observations into an Episode with provenance. Evaluate whether the summary is faithful before making it available to future Runs.

Beliefs are revisable, not facts

A Hypothesis stores content, assumptions, confidence, and pending/confirmed/falsified state. Closing an experiment can invalidate dependent hypotheses through a NebulaGraph traversal; without NebulaGraph a RethinkDB-based fallback exists. Neither path turns model output into independently verified truth. Human review, workspace filters, deletion policies, and test cases remain necessary.

Key Rust interfaces

These are library interfaces, not HTTP routes or Kubernetes fields. fit_to_budget consumes the chosen context candidates and policy; select_model_for_context chooses among supplied candidates. The caller still owns retrieval, model invocation, and inspection of budget_exceeded.

ContextManager::fit_to_budget(
    &self, system_prompt: &str, skill_body: Option<&str>,
    task_state: &str, episodes: Vec<String>,
    case_file_entries: Vec<String>, beliefs: Vec<String>,
    budget: u32, strategy: ContextBudgetStrategy
) -> FittedContext

RoutingPolicy::select_model_for_context(
    &self, context_token_count: u32, candidates: &[ModelCandidate]
) -> Result<String, String>

Source and readiness

Read engine/crates/kaigents-core/src/context_manager.rs for ContextManager, FittedContext, and RoutingPolicy; engine/crates/kaigents-memory/src/lib.rs for MemoryManager and its optional backends. The chart MemoryPolicy is declarative and does not itself provision Qdrant, RethinkDB, or NebulaGraph. Do not infer automated hybrid CPU/GPU/NPU scheduling merely from context-based model choice.