Discover sequential policy structure

Learn action values and delayed credit from ordered states, actions, and measured outcomes.

Use this when actions affect later states and later outcomes should revise earlier choices. Preserve the legal action set and the ordered episode record.

Sequential Discovery lets Adapt-1 form state-dependent action values from ordered interaction. Earlier choices can receive credit from later or terminal outcomes, so later selections can use what happened across the episode.

1. current state -> selected action -> next state
2. later reward or terminal outcome
3. earlier action value is revised
4. a later selection can change

The application declares the legal actions and the meaning of observations and rewards, including episode and reset boundaries. Adapt-1 uses admitted interaction to learn action values and assign delayed credit.

Choose the acquisition schedule separately. Sequential Discovery supports learning across episodes from empty state, separate acquisition before frozen use, and continued learning from an acquired checkpoint. State carried from earlier episodes inside one zero-start run remains within-run state. The wider Domain can use discovered, authored, or mixed structure.

When to use it

Use Sequential Discovery when all of these conditions hold:

Observable task property	Why it matters
The application repeatedly chooses among permitted actions	Adapt-1 needs a stable public policy space
An action changes the state observed later	Current choices affect later evidence
Reward can arrive after several steps or at the end of an episode	Earlier choices need delayed credit
Events have a real episode ID and ordered step	Credit must stay inside the correct trajectory
Executed actions can be linked to measured outcomes	The learner must know what actually happened

Use direct policy feedback for independent choices with an immediate attributable outcome.

How the Discovery paths fit together

A Domain can use one path or combine compatible paths inside the same public task boundary.

Discovery path	Structure formed from admitted evidence
Transition Discovery	Executable input projection and compatible before/after causal bindings
Structure Discovery	Outcome-useful fields, combinations, bounded temporal lags, and predictive rules
Sequential Discovery	State-dependent action values and delayed credit across ordered interaction

The paths can share observations while keeping their targets and diagnostics distinct. A transition result predicts what follows. Sequential policy state changes which action receives support because of later reward.

Learning contract

Each eligible sequential transition supplies this logical tuple:

episode_id
+ step
+ current observable context
+ selected and executed policy
+ next observable context
+ step reward
+ terminal flag

The normal loop uses a query followed by feedback:

1. query current state
2. Core selects one permitted policy
3. application executes that exact policy
4. application observes next state and reward
5. feedback binds the consequence to the decision
6. later queries use retained sequential state

A decision_id gives the strongest binding to the earlier selection.

Starting declaration excerpt

Declare policies as public action identities. Adapt-1 learns action quality and trajectory value from interaction. Numeric settings below are example parameters.

{
  "hypotheses": [
    {
      "name": "action A",
      "relation": "controls",
      "policy": "action_a",
      "predicts": ["execute action A"],
      "weight": 1.0
    },
    {
      "name": "action B",
      "relation": "controls",
      "policy": "action_b",
      "predicts": ["execute action B"],
      "weight": 1.0
    }
  ],
  "learning": {
    "enabled": true,
    "context": {
      "feature_paths": [],
      "max_samples": 4096
    },
    "sequential": {
      "enabled": true,
      "episode_path": "metadata.episode_id",
      "step_path": "metadata.step",
      "next_context_path": "values.next_state",
      "reward_path": "values.step_reward",
      "terminal_path": "values.terminal",
      "discount": 0.95,
      "n_step": 5
    },
    "training": {
      "enabled": true,
      "min_samples": 64,
      "retrain_interval": 16
    }
  }
}

An empty learning.context.feature_paths list admits eligible structured leaves. Review that surface for identifiers, post-outcome values, and target leakage. Use explicit paths when the approved state representation must stay fixed.

Feedback shape

Return the observed successor state and reward after executing the selected policy:

{
  "session_id": "ignored",
  "decision_id": "DECISION_ID",
  "feedback_kind": "execution",
  "outcome": "native_reward",
  "relation": "controls",
  "policy": "action_a",
  "values": {
    "next_state": {
      "values": {
        "load": 0.42,
        "capacity": 0.60
      }
    },
    "step_reward": 0.25,
    "terminal": false
  },
  "metadata": {
    "episode_id": "episode-001",
    "step": 14
  }
}

Keep one learner scope across compatible episodes. Start a new episode_id at the real reset boundary. Steps must be ordered inside each episode.

Opaque-action sequential environment

A Domain can expose public observations and legal opaque actions while leaving their useful task structure unresolved:

1. public observation fields
2. an action from the declared legal set
3. observable next state and native reward
4. retained transition and sequential evidence
5. later action selection

Transition Discovery can form the executable input projection and causal bindings. Sequential Discovery can form state-dependent action value from the ordered reward stream. Structure Discovery can also be enabled when the task needs induced fields, combinations, lags, or predictive rules. Each path remains part of Discovery even when a Domain uses only a subset.

Inspect the result

Inspect the feedback response for policy admission, then check sequential sample counts and learner versions alongside the policy model report. When the sequential candidate trains and passes validation, the feedback-policy report can expose model_type: "sequential_q_mlp".

Bounded transition scoring can operate in the same Discovery Domain through learning.sequential.bound_transition. It forms action-conditioned state-change evidence for a declared bounded objective. Preserve its score and attribution separately from the sequential return model.

Discovery overview

Configure the public boundary and choose the Discovery paths that fit the task.

Sequential learning, advanced

Configure rewards, exploration, training, validation, attribution, and frozen evaluation.

Transition Discovery

Discover stable transition inputs and compatible causal bindings.

Structure Discovery

Discover useful fields, combinations, lags, and predictive rules.