> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reilabs.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Learn and refine state values

> Fit values from observed outcomes, reconsider legal alternatives, continue from retained parameters, and verify candidate adoption for planning.

Use Adapt-1's value learner to estimate the usefulness of an unfinished state. Start with recorded outcomes, fit an initial evaluator, and use that evaluator to assess legal continuations. Continue learning from these assessments, comparisons between alternatives, and additional observed outcomes.

The application supplies the state representation, legal transitions, and measured outcomes. Adapt-1 fits the value parameters, constructs continuation targets, evaluates candidates, and retains the installed model. Select the installed model or a candidate as the evaluator for scoring unfinished states in native planning.

## Choose this workflow

Use this path when you can supply states and their outcomes, and your application needs to compare alternatives before a final outcome is available. Exact transition rules and learned values serve different roles: the rules determine the successor, while the value learner estimates its utility.

For action values learned from ordered transitions and feedback, see [sequential learning](https://docs.reilabs.org/docs/neuroadapt/sequential-learning.md). For predictions of the next observable state, see [transition learning](https://docs.reilabs.org/docs/neuroadapt/design-transition-domains.md). These interfaces have distinct observation, training, and state contracts.

> [!NOTE]
>
> Request tables describe schema version 5, and behavior notes describe observed API responses. Check `/version` at the API host root and confirm compatibility with your deployment before integration. Use the [value API reference](https://docs.reilabs.org/docs/neuroadapt/value-api.md) for request contracts.

## Establish the task contract

Choose a Domain, a value relation and policy, a state representation, and an outcome scale. Keep the meaning and perspective of a value consistent across observations, terminal scores, continuation targets, and application comparisons.

The `projected` representation reduces objects and relations to counts. The `object_graph` representation preserves their connections. The documented comparison-group workflow uses `object_graph`.

The value schema accepts numeric maps in `conditions`, `context`, and `policy_features`. [Verify which entries participate in projected inputs](https://docs.reilabs.org/docs/neuroadapt/value-observations.md#verify-projected-inputs) before collecting a larger dataset. Use [one state encoder across observations, prediction, continuations, and planning](https://docs.reilabs.org/docs/neuroadapt/value-observations.md#reuse-one-state-encoding-across-calls), and choose a [compatible training setup](https://docs.reilabs.org/docs/neuroadapt/value-training.md#choose-a-compatible-training-setup).

Decide how you will judge a replacement model. Prediction error measures agreement with value targets. Application utility measures the consequences of the decisions made with those values. A candidate can improve one measure without improving the other.

For a complete Python client using your own observations, see [Run the value API workflow](https://docs.reilabs.org/docs/neuroadapt/value-client-workflow.md). It connects upload, fitting, job polling, prediction, candidate comparison, and explicit promotion, tracking fitting and installation separately.

## Run the learning cycle

**1. Configure the value learner**

Call `value/configure` with the task's relation, policy, representation, sample retention, and promotion settings. Require a response with `status: configured`. The [API reference](https://docs.reilabs.org/docs/neuroadapt/value-api.md#configure) includes the confirmed request shape.

**2. Record observed outcomes**

Upload observations through `value/observe`. Give each row a stable `record_id`, an `episode_id`, and a `step`. Supply exactly one supervision source: a provided numeric `target`, an observed `outcome`, or a model-based `continuation`. Begin with targets derived from measured outcomes, or with observed outcome rows, so the first reference has outcome evidence.

Batch by [serialized bytes as well as row count](https://docs.reilabs.org/docs/neuroadapt/value-observations.md#batch-by-serialized-size-and-row-count). Verify that `accepted + duplicates` accounts for all submitted rows. Keep the original observation content for exact retries. See [value observations](https://docs.reilabs.org/docs/neuroadapt/value-observations.md).

**3. Fit and install the first reference**

Set `options.warm_start: false` explicitly for an independent first fit. Use `options.model_type: object_value` for the graph example and top-level `install: true` to request installation. The train route returns HTTP 202; follow the [polling procedure](https://docs.reilabs.org/docs/neuroadapt/value-training.md#poll-until-the-fit-finishes) until that fit has finished, then confirm which model is installed. Save its SHA-256 identity.

For nonterminal continuations, confirm that the fitted reference is installed before proceeding. `install: true` requests installation through the configured adoption gate; `value/state` reports the installed model.

**4. Reconsider recorded states**

For each selected recorded state, generate legal successors through the task's transition rules. Submit a continuation observation containing those successors. Give a completed successor its exact `terminal_value`; the reference model evaluates unfinished successors.

Retain provided or outcome observations in both the fitting and validation partitions. Continuation training refuses a snapshot without this evidence. Limit the number of successors per fit; the exact deployment cap must be confirmed from its contract.

**5. Declare comparisons between alternatives**

When using the graph path, represent each compared alternative as its own observation. Give related alternatives the same `metadata.comparison_group` value. In this workflow, these observations are sibling child states, each with its own continuation target.

A continuation's `successors` array does not declare a comparison group. Both fitting and validation partitions need groups when comparison training is enabled.

**6. Continue fitting from retained parameters**

Train a candidate with `options.warm_start: true`, top-level `warm_start_source: installed`, and `install: false`. Set `expected_initial_model_sha256` to the retained reference identity. Select the earlier and new observations through top-level `observation_ids` so the fit retains evidence from previous rounds. Use a new `request_id` for the intended fit.

The example configuration uses `comparison_weight: 1`, `comparison_loss: squared`, and `selection_objective: prediction`. A finished uninstalled fit can report `training.status: candidate`; treat that as completion when staging a candidate. Inspect the [training report](https://docs.reilabs.org/docs/neuroadapt/value-training.md) for the selected checkpoint and the contribution of continuation targets and comparisons.

**7. Compare application decisions and promote**

Evaluate the candidate and incumbent on paired cases under the same application conditions. Preserve the model identities and measured utility for each case. For continuation-trained candidates, first check the [reported installation restriction](https://docs.reilabs.org/docs/neuroadapt/value-promotion.md#check-continuation-candidate-installation). Submit the [paired-canary request](https://docs.reilabs.org/docs/neuroadapt/value-promotion.md#submit-a-canary-with-model-guards) to `value/promote` with both expected model hashes, and handle an installation refusal as described there.

Inspect the promotion response and read back the installed model identity. Advance the installed reference after state confirms adoption; after a refusal, keep the confirmed incumbent as the reference and retain the candidate's evaluation result. See [value promotion](https://docs.reilabs.org/docs/neuroadapt/value-promotion.md).

**8. Repeat with the adopted reference**

Advance the reference only after `value/state` confirms the evaluated candidate is installed. If installation or promotion was refused, retain the existing reference and the candidate evaluation as separate results. Add new observed outcomes and retain the earlier evidence selected for the fit. Record the reference, candidate, and installed model identities at each boundary.

## Confirm what each stage changed

| Stage | Evidence to inspect |
| - | - |
| Observation admission | `accepted`, `duplicates`, and `target_sources` |
| Continuation refresh | `training.report.target_refresh.continuation_records`, successor counts, and `reference_model_sha256` |
| Comparison training | `comparison_validation.groups`, `informative_groups`, `top_choice_agreement`, and `mean_target_regret` |
| Warm continuation | `warm_start.applied`, `continuation.initial_model_sha256`, and `warm_start_source` |
| Checkpoint selection | `selected_epoch`, `optimization_applied`, and any checkpoint rejection counts |
| Adoption | Promotion result followed by the installed model identity from `value/state` |

`selected_epoch: 0` identifies the incoming checkpoint as the selected parameter state. Inspect the full fit report for optimization and checkpoint diagnostics. Check target construction, checkpoint selection, and installation separately. Use the model SHA to identify the artifact and [compare predictions on fixed inputs](https://docs.reilabs.org/docs/neuroadapt/value-training.md#record-reproducibility-without-assuming-it) to measure functional changes.

## Use the retained value in planning

Set request-level `evaluator_source: value_model` to select the installed value learner, or `value_candidate` to evaluate a candidate. Check [evaluator readiness](https://docs.reilabs.org/docs/planning/options.md#check-evaluator-readiness) before the first planning request. Bind the expected model identity using the planning contract and check the returned evaluator identity. See [planning integration](https://docs.reilabs.org/docs/planning/integration.md) and [planning options](https://docs.reilabs.org/docs/planning/options.md).

Supply exact `terminal_value` scores only for completed outcomes. An unfinished state's estimate should retain its model provenance. Record search evaluations and transitions separately from admitted observations and executed episodes.

Use an `outcome` target with a terminal-outcome planner by aligning utility, neutral value, discount, perspective, and step units. Include intermediate rewards and path costs in that objective where the task requires them. [Align the value objective](https://docs.reilabs.org/docs/planning/options.md#align-the-value-objective) before combining learned frontier estimates with terminal values.

During frozen evaluation, stop observation and training writes, call `value/freeze` with `session_id` and `expected_model_sha256`, and compare model identities before and after planning. Freeze every other participating Domain learner through its own controls and verify its state separately.

## Keep observed outcomes and assessments distinct

A continuation target is a model-based assessment. An observed outcome is evidence from an executed episode. Preserve both sources in your application record, along with episode, step, and model identity.

The [Coupled Learning and Planning article](https://reilabs.org/blog/coupled-learning-and-planning) describes targets that combine an assessment with an observed outcome. The API accepts one supervision source per row. For a per-row blend, compute the scalar in your application, submit it as `target`, and retain its inputs and provenance. Outcome rows and continuation rows supplied separately remain separate supervised examples. See [target construction](https://docs.reilabs.org/docs/neuroadapt/value-observations.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.