RESEARCH / 03Agents & execution frameworks

Methods & systems

Let reasoning
reach beyond action.

An agent’s work extends beyond the next sentence.
It must understand a goal, change a state and show evidence that the task is done.

01 / OBSERVEObserve

Read the goal and the state

02 / PLANPlan

Choose a step and its test

03 / ACTAct

Invoke a concrete capability

04 / VERIFYVerify

Bring the result back to evidence

Every action should leave room for new evidence to change what follows.
REASONING → ACTION → EVIDENCEExplore the execution loop

01From answers to tasks

A convincing answer.
A verifiable outcome.

A language model produces candidate explanations and actions. Task completion happens outside it: a correct file, a satisfied rule, an actual state change. These require different judgments.

QUESTION → RESPONSE

“The proposal is ready.”

This is a statement. It does not establish source validity, complete coverage of constraints or acceptance by a target system.

GOAL → ACCEPTANCE

Give “done” an explicit contract.

Goal
The required artifact or final environment state.
Scope
Resources, allowed actions and execution budget.
Criteria
Executable checks, business rules and evidence requirements.
Exceptions
How to report unfinished work and when to hand it to a person.

02The reasoning–action loop

The next step
starts with new evidence.

ReAct interleaves reasoning and actions so observations can inform the plan. Task systems also need explicit acceptance checks: receiving a tool result does not establish that the goal is met.

ReAct
FIELD NOTE / A

How an outdated brief changes the next step.

Constructed example · local interaction
CONSTRAINT_CONFLICT03 / 05

A version check invalidates an old assumption.

Observed evidence
The document index points to v2. Reading its source reveals a new constraint: no holes may be drilled in the housing.
Action & decision
Reject the candidate and localize the conflict to the mounting method. Keep task information that remains valid.

Sample brief v2 · §3 Non-invasive mounting · version conflict

Illustrative trace: evidence, actions and acceptance outcomes explain the repair process.

↳

A failure should reduce uncertainty. Record which assumption failed, what evidence changed and which step needs repair before attempting the task again.

Reflexion

03Choosing a topology

Independent responsibilities.
Then multiple agents.

Adding agents changes parallelism, context allocation and coordination cost. Choose a structure around task dependencies and verification. These are three designs to compare, not a ranking of performance.

TOPOLOGY / 01Structural illustration
Goal & constraints
A
Executing agentObserve → plan → act
Check against acceptance criteria
Split only when responsibilities can be evaluated independently.

Keep responsibility with one executor.

Tasks with tight dependencies, concentrated context and a manageable tool set. Establish this baseline before testing a split.

Potential benefit
Centralized state reduces handoffs and repeated reads; failures are easier to localize.
Cost to account for
Long tasks can crowd the context; different subtasks share one execution path.

Suggested comparison: hold the model, tools, tasks and budget constant. Test a single-agent baseline, then add parallel work or separate review to see whether the benefit offsets handoff costs.

04Tool and context boundaries

Tools have contracts.
Context has boundaries.

Toolformer studies learning to select APIs and use their results. Engineering must also define inputs, outputs, side effects and failure semantics. Selecting a tool is the beginning of an action.

Toolformer
TOOL CONTRACTConceptual example

Read a versioned design brief

Typed input
Document identity, version and section range.
Typed output
Excerpt, source reference, actual version and read status.
Scope & side effects
Read within document permissions; no implied write access.
Failure semantics
Distinguish denied access, missing data, stale versions and temporary failures.
MCP tool schemas
01 / SELECT

Bring in what the step needs.

Assemble context by role, task and data permissions. A subagent receives a bounded objective and evidence, then returns an artifact with sources, reducing repeated copies of the whole conversation.

02 / VALIDATE

Content cannot promote itself into authority.

Documents, web pages and tool results are external data. Instructions inside them do not expand the task’s authorization. Even after schema validation, services must check access and business conditions.

05Memory and provenance

What to retain.
And where it came from.

Context is finite while tasks may continue. Compaction should preserve goals, constraints, unresolved questions and retrievable references. Shortening a conversation does not make its claims more reliable.

01 / SOURCE

Original source

Preserve identity, version and access scope.

Brief v2 / §3
02 / RETRIEVAL

Relevant excerpt

Bring the source location into this step.

Non-invasive mounting / citation
03 / WORKING MEMORY

A resumable task summary

Separate known constraints, candidates and open checks.

Candidate: clamping / open: load
Missing evidence or changed version → return to the source

Reflexion explores verbal feedback and episodic memory for later trials. The research question here is which feedback to retain and how to avoid preserving a mistaken summary as durable knowledge. Reflexion

Task checkpoints capture current state; cross-task memory retains information for a defined purpose. Specify permissions, expiry and deletion separately. LangGraph’s checkpoint/store distinction offers an engineering reference. LangGraph · Persistence

06Bounded execution

Know how to continue.
Know when to stop.

An execution framework needs conditions for continuing and explicit states for failure, waiting and stopping. Repetition without progress is not a measure of autonomy.

01

Budget

Set limits for tool calls, model usage, elapsed time and retries before starting. At the limit, save partial artifacts and unfinished work.

02

Retry

Retry recoverable failures with new information or explicit backoff. Reconcile uncertain write receipts first; repeating a call does not make it idempotent.

03

Approval

Bind approval to a concrete action, resource scope and change version. Existing authorization persists within its valid scope; changed content requires reassessment.

04

Checkpoint

Keep the state version, input references and tool receipts. Reconcile external state before resuming instead of treating an old snapshot as current fact.

When new evidence is unavailable, a hard constraint cannot be met or an action exceeds authorization, stop with a useful handoff.

07Evaluation design

Define success first.
Then compare methods.

Observe task outcomes, process correctness and execution cost together. SWE-bench evaluates code changes for real software issues; τ-bench combines tools, interaction and domain rules. Their task design informs this framework; their scores are not transferred here.

Task-level evaluation dimensions and reporting definitions
DimensionWhat to establishSuggested reporting
Task completionJudge completion against predefined artifacts and final environment state.Completion rate = accepted tasks / all evaluated tasks. Report aborts and failure causes separately.
Trace correctnessCheck whether actions, arguments, ordering and cited evidence support the result.Allow multiple valid paths. Check critical invariants and events instead of enforcing one exact sequence.
Constraint complianceTest permissions, data scope, write conditions and prohibited actions.Declare hard constraints in advance. A violation disqualifies success; report violation types separately.
Execution costInclude model usage, tools, compute and failed attempts.Report cost per attempt and total cost / successful tasks, with a cost breakdown and pricing date.
Latency & consistencyMeasure from task acceptance to result readiness, across repeated runs.Report median and tail latency, timeouts, retries and consistency across trials. Identify human waiting time separately.
PASS THE GATESValidity→Efficiency

Compare cost and speed after hard constraints are satisfied.

Make the comparison interpretable.

Fix the task set and difficulty strata. Record model and tool versions, prompts, budgets, randomness and evaluators; keep evaluation tasks separate from tuning. Repeat each task, retain failures and uncertainty estimates, and compare retrieval, memory, review and parallelism separately.

Read more: evaluating tool agents

These are evaluation proposals. This page reports no measured IDENIFE scores, benchmark rankings or deployment outcomes.

08From methods to systems

Research frames the question.
Engineering makes it testable.

Within IDENIFE’s technology system, ID Axis connects model reasoning, task orchestration and tool calls. This page examines agent methods; execution foundations, application integration and protocol access each have their own treatment.

Original papers & specifications

Follow the questions further.

External research informs the discussion; it does not imply a partnership with the authors or institutions.

  1. 01
    ReActYao et al. · 2022 / ICLR 2023

    Interleaves reasoning and action, using environmental feedback to update subsequent steps.

  2. 02
    ToolformerSchick et al. · 2023

    Uses self-supervised training to learn API selection, arguments and use of returned results.

  3. 03
    ReflexionShinn et al. · 2023

    Uses verbal feedback and episodic memory to inform later attempts without updating model weights.

  4. 04
    SWE-benchJimenez et al. · 2023 / ICLR 2024

    Uses real repositories and issues to evaluate outcomes through code changes and execution.

  5. 05
    τ-benchYao et al. · 2024

    Combines user interaction, domain rules and tools to examine final states and reliability across trials.

  6. 06
    LangGraph · PersistenceLangChain · Official documentation

    Distinguishes thread checkpoints from cross-thread stores, informing continuity and memory boundaries.

  7. 07
    MCP · ToolsProtocol specification · 2025-11-25

    Defines tool inputs and optional output schemas; declarations do not replace application authorization.

This page presents methods and evaluation designs for agent systems. Interactions are constructed examples; implementation and availability are described in the relevant product and developer documentation.

鲁ICP备2024109755号-2
Drag to move. Right-click, touch and hold, or press Shift+F10 to choose a corner.