Learn from feedback and outcomes

Bind observed outcomes to the decisions that produced them so later Adapt-1 results can improve.

Bind each observed outcome to the decision that produced it, then query again so later results can reflect the update.

Default path: query a Domain, execute the selected action or policy, and submit the observed outcome as feedback. Ordinary one-decision feedback requires no sequential configuration or learner tuning.

For specialized evidence layouts, see Domain learning patterns.

Default feedback loop

observe state -> commit decision -> execute exact action -> record transition -> bind consequence/feedback -> query again

Feedback is part of a continuing reasoning-and-learning loop. It is not a separate training job. Generated language is not required.

Keep explicit feedback, observable consequences, and application-only context separate. An observed transition can supply transition evidence without a success label. An observable consequence is not an application judgment or a feedback write.

Transition evidence and feedback

Information	Where it belongs
Observations available before the consequence and the proposed intervention	Configured transition learner input
Observable consequence at the declared event boundary	Transition target or ordinary Domain event, as specified by the Domain contract
Hidden rule, correct action, solver output, or application metric	Application-only context; never a transition input or observation target
Feedback sent to change a scoped policy or relation	POST /api/v1/domains/{domain_id}/feedback

A transition target describes the observable consequence. It does not state if the consequence was good. Feedback can later assign application value to a committed decision. Record feedback as a separate operation.

See Design Domains for transition learning for the separation between the operational record, learner view, and application-only context.

Decision binding

Keep an application interaction record beside the API requests:

{
  "run_id": "run1",
  "trial_id": "run1-trial-004",
  "relation": "in_front_of",
  "policy": "footprint_edge_projection",
  "selected_action": "move_up",
  "executed_action": "move_up",
  "before_state_id": "state-003",
  "after_state_id": "state-004",
  "native_outcome": "level_advanced",
  "outcome_visible_to_adapt_1": "positive"
}

This ledger belongs to the surrounding application and keeps the full reasoning-to-action loop inspectable. Submit only the fields accepted by the selected API route.

A policy is the context-dependent rule or preference being updated. An application can use it to select an external action, forward a returned action identifier directly, or apply its own commitment rule; preserve the returned policy or selection beside the exact executed action.

If a native outcome such as advancement is declared to mean progress, treat it as positive evidence for the bound transition. If only application-private logic assigns that value, keep the judgment outside Adapt-1 unless the integration explicitly returns it as feedback.

Feedback horizon

Feedback updates support with respect to the outcome signal it encodes. Immediate outcomes can strengthen locally useful behavior, while terminal or delayed outcomes let Adapt-1 organize that learning around a longer objective.

For goal-driven tasks, record which outcome horizon Adapt-1 receives:

• local state improvement;
• binary step success;
• reward magnitude;
• terminal goal completion;
• delayed episode return; or
• only the next observable consequence.

Ordinary decision-bound feedback is the default. Do not enable sequential learning merely because records arrive in order.

Choose only the relationship the task needs. These patterns can coexist inside one Domain or remain isolated in separate Domains. Configure more than one only when the application needs more than one relationship.

What the task needs to learn	Use	Configuration
Which policy works in the current context, followed by an immediate measured outcome	Ordinary feedback	Set learning.enabled: true. Leave sequential and bounded transition learning disabled when this is the only relationship.
A structured observable target from configured inputs, with an action included only when the relationship requires one	Structured transition learning	Set learning.transition.enabled: true, declare input and target paths, and send eligible events. Do not enable sequential learning merely because events are ordered.
How a candidate action changes an explicit bounded public state relative to a numeric objective	Bounded transition scoring	Set learning.sequential.bound_transition.enabled: true and provide the state, action, and objective mappings.
Which action improves a delayed or terminal outcome by changing later state	Sequential learning	Set learning.sequential.enabled: true and send the ordered transition fields described below.

An environment can contain long episodes and still use ordinary feedback. If every decision receives its own immediate outcome and the action does not determine the later state, episode length alone is not a reason to enable sequential learning.

Sequential feedback for delayed outcomes

Enable this path only when the application has a real episode boundary, ordered state transitions, and delayed credit across actions. Independent samples, immediate-only labels, and chronological records whose actions do not affect later state should remain on ordinary feedback.

Enable both learning switches in the Domain:

{
  "learning": {
    "enabled": true,
    "sequential": {
      "enabled": true
    }
  }
}

Then repeat this loop:

query current state -> execute one policy -> return the observed next state and outcome -> query the new state

With the default mappings, each sequential feedback item supplies:

Field	Request path
Episode identifier	metadata.episode_id
Ordered step	metadata.step
Observed next state	values.next_state
Sequential reward	values.step_reward
Episode boundary	values.terminal

All five fields are required for the transition to enter sequential learning. Use a JSON number for step and step_reward, an object for next_state, and a Boolean for terminal. The feedback must also identify the executed relation and policy, and include either the query’s decision_id or the explicit decision-time context.

Send both reward fields when the same feedback participates in ordinary and sequential learning:

{
  "session_id": "agent-17",
  "outcome": "measured",
  "decision_id": "DECISION_FROM_QUERY",
  "relation": "controls",
  "policy": "safe_controller",
  "values": {
    "reward": 0.0,
    "step_reward": 0.0,
    "next_state": {
      "load": 0.71,
      "temperature": 0.43
    },
    "terminal": false
  },
  "metadata": {
    "episode_id": "episode-0007",
    "step": 38
  }
}

values.reward supplies the measured outcome for immediate contextual learning. values.step_reward supplies the ordered reward used for delayed return. They may contain the same value when that matches the task.

Keep the same Domain and learner continuity across a trajectory, but query each step with the newly observed state. Use a new episode ID at the real episode boundary, restart step from zero, and set terminal: true on the final transition. Reset the Domain only for an intentional cold-start condition or deliberate replay.

Sequential settings that may need adjustment

Use the defaults unless the task contract requires one of these changes:

Setting	Change it when
learning.context.max_samples	Long episodes would otherwise be discarded before enough complete trajectories are retained. Size it above episode length × episodes retained.
learning.credit_assignment.neutral_reward	The environment’s neutral value differs from the default 0.5. Use 0.0 when intermediate zero means that reward has not arrived yet.
learning.sequential.discount	The real outcome horizon is substantially longer or shorter than the default horizon.
Sequential field paths	The application does not use the default payload paths shown above.

Leave training parameters, validation thresholds, model weights, and n-step settings at their defaults for ordinary use. Adapt-1 validates learned estimates before they affect selection.

Request learning_state and policy_diagnostics when verifying the integration. A sequential estimate may not appear immediately because the learner first needs enough completed episodes and held-out validation. Inspect the returned model status and reason, then confirm that sequential_expected_reward is present before treating delayed estimates as part of selection.

Bounded transition scoring

Bounded transition scoring is separate from both delayed-return learning and the general structured-transition learner. learning.transition predicts declared observable targets from event inputs. Bounded transition scoring instead learns action-conditioned changes in an explicit public state and projects candidate policies onto a declared numeric objective during selection. Its configuration is nested at learning.sequential.bound_transition because it consumes state transitions, but it does not turn every feedback task into a sequential-return task.

Use it only when:

• the relevant state is public and numerically bounded;
• each candidate identifies the state and action it affects;
• feedback returns the observed successor state; and
• one declared state feature is the objective.

Do not use it for hidden state, an unbounded or undefined objective, or an ordinary context-to-outcome decision.

The required mapping is intentionally small:

Mapping	Function
state_selector_feature	Selects the current state object from candidate policy features
state_features	Maps logical state names to public fields under that object
objective_feature	Identifies the numeric state feature to optimize
objective_min and objective_max	Define the public normalization bounds
transition_values	Identify state-changing candidate actions
terminal_values	Identify terminal candidate actions

Add state_bounds only for real public availability or validity limits. Add action selectors, action features, or signature features only when the action representation requires them. Candidate policies must carry the selector and category values used by the mapping, and feedback must identify the action that actually executed and its observed successor state.

Leave minimum_support, model_weight, novelty_bonus, and transition_cost at their defaults unless the task contract gives a concrete reason to change them. A wrong state, action, or objective binding can materially distort selection.

Feedback-driven policy scoring

For the currently documented relation/policy workflow:

Step	Route
Create a domain	POST /api/v1/domains
Submit the outcome	POST /api/v1/domains/{domain_id}/feedback
Read the updated learned signal	POST /api/v1/domains/{domain_id}/query

Store the relevant relation, policy, and run_id as structured metadata when the API contract calls for them. Keep the relation, executed policy, and relevant decision-time context aligned between the query and its observed feedback. Keep trial, action, transition, and outcome provenance in the surrounding ledger. Do not hide the scoping values only inside free text.

{
  "metadata": {
    "run_id": "run1",
    "relation": "in_front_of",
    "policy": "footprint_edge_projection"
  }
}

Sealed-decision feedback

When a Domain query returns a decision_id, persist it and bind later feedback or an historical explanation to that exact result. Send feedback to:

POST https://rei-neuroadapt-api.reilabs.org/api/v1/domains/{domain_id}/feedback

{
  "session_id": "<session-id>",
  "decision_id": "<decision-id>",
  "feedback_kind": "execution",
  "outcome": "success",
  "relation": "causes",
  "policy": "inspect_feed_path",
  "values": { "reward": 0.9 }
}

Use feedback_kind: "execution" for policy-outcome quality and feedback_kind: "semantic" when the feedback evaluates whether a hypothesis is applicable or supported. Without a declared reward contract, numeric feedback is read in this order: values.reward or metadata.reward, score, utility, Boolean correct/success/accepted, then the outcome label. error_distance is not a reward unless the Domain declares its normalization and minimization.

Accepted or stored feedback supplies evidence; it does not guarantee that a learner was applied or that the same policy will control a later selection. Inspect policy_update, eligibility, attribution, and credit_assignment when returned.

For competing policies, set allow_exploration: true before a non-default selection_mode can apply. Supported modes are exploit, ucb, thompson, and auto; these affect Domain policy selection, not memory retrieval.

Defaults are sufficient for normal use. Use Domain exploration_mode or exploration_strength only when the application intentionally needs different explore/exploit behavior. Use transfer_strength or action_transfer_strength only when evidence should share more broadly across related contexts or actions, or remain more strictly isolated.

Adaptive result

For this workflow, compare the response-level policy_scores entry for the relation/policy that received positive feedback against the one that received negative feedback. Use an unrelated relation as a control so that a scoped update is not mistaken for global behavior.

For this route, policy_scores is the documented learned signal. supporting_memories, core_support, and ranked-hypothesis fields add context and make the result easier to inspect.

Other adaptation surfaces

policy_scores is one public adaptation surface. Adapt-1 also exposes other adaptive results; the documented interval-policy workflow uses /adapt/* and returns prediction.<output_key>.