> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reilabs.org/llms.txt
> Use this file to discover all available pages before exploring further.
# Discover sequential policy structure
> Learn action values and delayed credit from ordered states, actions, and measured outcomes.
Use this when actions affect later states and later outcomes should revise earlier choices. Preserve the legal action set and the ordered episode record.
Sequential Discovery lets Adapt-1 form state-dependent action values from ordered interaction. Earlier choices can receive credit from later or terminal outcomes, so later selections can use what happened across the episode.
1. current state -> selected action -> next state
2. later reward or terminal outcome
3. earlier action value is revised
4. a later selection can change
The application declares the legal actions and the meaning of observations and rewards, including episode and reset boundaries. Adapt-1 uses admitted interaction to learn action values and assign delayed credit.
Choose the acquisition schedule separately. Sequential Discovery supports learning across episodes from empty state, separate acquisition before frozen use, and continued learning from an acquired checkpoint. State carried from earlier episodes inside one zero-start run remains within-run state. The wider Domain can use discovered, authored, or mixed structure.
## When to use it
Use Sequential Discovery when all of these conditions hold:
| Observable task property | Why it matters |
| ----------------------------------------------------------------- | ---------------------------------------------- |
| The application repeatedly chooses among permitted actions | Adapt-1 needs a stable public policy space |
| An action changes the state observed later | Current choices affect later evidence |
| Reward can arrive after several steps or at the end of an episode | Earlier choices need delayed credit |
| Events have a real episode ID and ordered step | Credit must stay inside the correct trajectory |
| Executed actions can be linked to measured outcomes | The learner must know what actually happened |
Use direct policy feedback for independent choices with an immediate attributable outcome.
## How the Discovery paths fit together
A Domain can use one path or combine compatible paths inside the same public task boundary.
| Discovery path | Structure formed from admitted evidence |
| ------------------------------------------------------------------------ | -------------------------------------------------------------------------------- |
| [Transition Discovery](/docs/neuroadapt/discovery-transition-projection) | Executable input projection and compatible before/after causal bindings |
| [Structure Discovery](/docs/neuroadapt/discovery-structure) | Outcome-useful fields, combinations, bounded temporal lags, and predictive rules |
| **Sequential Discovery** | State-dependent action values and delayed credit across ordered interaction |
The paths can share observations while keeping their targets and diagnostics distinct. A transition result predicts what follows. Sequential policy state changes which action receives support because of later reward.
## Learning contract
Each eligible sequential transition supplies this logical tuple:
```text theme={null}
episode_id
+ step
+ current observable context
+ selected and executed policy
+ next observable context
+ step reward
+ terminal flag
```
The normal loop uses a query followed by feedback:
1. query current state
2. Core selects one permitted policy
3. application executes that exact policy
4. application observes next state and reward
5. feedback binds the consequence to the decision
6. later queries use retained sequential state
A `decision_id` gives the strongest binding to the earlier selection.
## Starting declaration excerpt
Declare policies as public action identities. Adapt-1 learns action quality and trajectory value from interaction. Numeric settings below are example parameters.
```json theme={null}
{
"hypotheses": [
{
"name": "action A",
"relation": "controls",
"policy": "action_a",
"predicts": ["execute action A"],
"weight": 1.0
},
{
"name": "action B",
"relation": "controls",
"policy": "action_b",
"predicts": ["execute action B"],
"weight": 1.0
}
],
"learning": {
"enabled": true,
"context": {
"feature_paths": [],
"max_samples": 4096
},
"sequential": {
"enabled": true,
"episode_path": "metadata.episode_id",
"step_path": "metadata.step",
"next_context_path": "values.next_state",
"reward_path": "values.step_reward",
"terminal_path": "values.terminal",
"discount": 0.95,
"n_step": 5
},
"training": {
"enabled": true,
"min_samples": 64,
"retrain_interval": 16
}
}
}
```
An empty `learning.context.feature_paths` list admits eligible structured leaves. Review that surface for identifiers, post-outcome values, and target leakage. Use explicit paths when the approved state representation must stay fixed.
## Feedback shape
Return the observed successor state and reward after executing the selected policy:
```json theme={null}
{
"session_id": "ignored",
"decision_id": "DECISION_ID",
"feedback_kind": "execution",
"outcome": "native_reward",
"relation": "controls",
"policy": "action_a",
"values": {
"next_state": {
"values": {
"load": 0.42,
"capacity": 0.60
}
},
"step_reward": 0.25,
"terminal": false
},
"metadata": {
"episode_id": "episode-001",
"step": 14
}
}
```
Keep one learner scope across compatible episodes. Start a new `episode_id` at the real reset boundary. Steps must be ordered inside each episode.
## Opaque-action sequential environment
A Domain can expose public observations and legal opaque actions while leaving their useful task structure unresolved:
1. public observation fields
2. an action from the declared legal set
3. observable next state and native reward
4. retained transition and sequential evidence
5. later action selection
Transition Discovery can form the executable input projection and causal bindings. Sequential Discovery can form state-dependent action value from the ordered reward stream. Structure Discovery can also be enabled when the task needs induced fields, combinations, lags, or predictive rules. Each path remains part of Discovery even when a Domain uses only a subset.
## Inspect the result
Inspect the feedback response for policy admission, then check sequential sample counts and learner versions alongside the policy model report. When the sequential candidate trains and passes validation, the feedback-policy report can expose `model_type: "sequential_q_mlp"`.
Bounded transition scoring can operate in the same Discovery Domain through `learning.sequential.bound_transition`. It forms action-conditioned state-change evidence for a declared bounded objective. Preserve its score and attribution separately from the sequential return model.
Configure the public boundary and choose the Discovery paths that fit the task.
Configure rewards, exploration, training, validation, attribution, and frozen evaluation.
Discover stable transition inputs and compatible causal bindings.
Discover useful fields, combinations, lags, and predictive rules.