Why this matters: This lesson gives you a decision procedure for choosing an AI-agent framework—or deliberately declining one. You will convert stakeholder language into testable architectural requirements, separate framework-native capabilities from ordinary application responsibilities, prototype competing approaches, inject realistic failures, inspect traces and persisted state, and document a reversible decision.

A requirements brief branches toward several stack categories, including a no-framework path.

What you will be able to do

  • Translate a proposed agent feature into explicit execution, state, control, and operational requirements.
  • Distinguish principles and application responsibilities from framework-native and external-platform features.
  • Evaluate direct-SDK, custom-loop, agent-framework, and durable-workflow alternatives using the same acceptance tests.
  • Identify abstraction leaks and exit costs involving persistence, observability, provider dependence, upgrades, and side effects.

1. Frame the decision as a production constraint

Decision point: the procurement proof of concept looks deceptively simple: classify the request, call get_vendor_policy, call create_draft_packet, and return a draft. A direct model API plus ordinary Python or TypeScript can do that cleanly. The architectural pressure arrives later: a manager may approve tomorrow; the worker may restart after the draft is created; a compliance reviewer may need to see why a tool call was proposed; and a retry must not submit two purchase requests.

Do not begin by naming frameworks. Begin by writing an operational claim that can be disproved: “This application needs to suspend a run, show a human the proposed action and state, resume safely after approval, and recover without repeating already completed work.” If the claim is false, framework adoption is probably premature. If it is true, identify precisely which runtime capability—not an attractive agent abstraction—must carry that burden.

Learning objective: make a defensible, reversible architecture choice based on representative behavior and official capability evidence rather than a generic framework comparison.

2. Classify the real work before selecting a runtime

Build a requirement table with five columns: requirement, consequence of failure, owner, evidence needed, and acceptance test. For this scenario, separate the following:

  • Architecture principles: explicit boundaries, least privilege, idempotent effects, observable execution, testability, and an exit path. They are not framework features.
  • Framework-native candidates: agent/tool orchestration, graph routing, checkpoints, resume primitives, session helpers, lifecycle callbacks, or built-in traces. For example, the OpenAI Agents SDK says its Agent and Runner manage turns, tools, guardrails, handoffs, and sessions; it explicitly says to use the Responses API directly when you want to own the loop. ([openai.github.io](https://openai.github.io/openai-agents-python/agents/?utm_source=openai)) LangGraph documents checkpoint-based persistence for fault recovery, memory, human-in-the-loop workflows, and replay/time-travel-style debugging. ([docs.langchain.com](https://docs.langchain.com/oss/python/langgraph/persistence?utm_source=openai))
  • External platform/tooling: identity provider, secrets manager, database, queue, workflow service, telemetry backend, deployment platform, and evaluation dataset store.
  • Custom application logic: vendor-policy rules, approval thresholds, authorization, schemas for purchase requests, idempotency keys, compensation, audit retention, and UI behavior.

This classification prevents a common mismatch: adopting an agent framework for business-state durability while quietly leaving authoritative state inside a chat transcript or a framework checkpoint.

3. Compare alternatives by execution and state semantics

Now score four implementation shapes against your requirements. A score is not a market ranking; it is a hypothesis to test.

  1. Direct SDK + deterministic application code. Best starting point when the tool surface is small, flow is predictable, runs are short, and your team can explicitly manage message history, tool dispatch, schema validation, retries, and telemetry. Modern model APIs can reduce custom parsing work through structured outputs/function schemas, but that does not create durable workflow semantics. Keep the loop explicit: call model; validate proposed action; authorize; invoke a narrowly scoped tool; append a normalized result; stop under a bounded iteration policy.
  2. Custom agent loop. Choose it when you need a nonstandard execution policy or provider-neutral interface but not a reusable graph/persistence runtime. Its cost is ownership: termination rules, concurrency behavior, retry classification, state serialization, trace propagation, and tests all remain yours.
  3. Agent framework. Justify it when reusable orchestration and agent-aware state/human-control primitives remove repeated engineering across several workflows. LangGraph, for example, makes checkpointing central to durable execution and human intervention; its interrupts save state and wait for a later Command resume. ([docs.langchain.com](https://docs.langchain.com/oss/python/langgraph/interrupts?utm_source=openai)) The OpenAI Agents SDK offers sessions and built-in tracing, but sessions preserve conversation history rather than replacing your authoritative domain database. ([openai.github.io](https://openai.github.io/openai-agents-python/sessions/?utm_source=openai))
  4. Durable workflow engine around model calls. Favor this shape when long-running, business-critical process control dominates model autonomy: retries, timers, queues, compensation, and recoverable side effects. Put the model behind a deterministic activity or bounded decision step. This may be stronger than an agent framework when the workflow is principally a business process with occasional model-assisted classification or drafting.

Reject “framework because multi-agent” as a requirement. Multiple specialists may be ordinary functions, separate services, or tools. Add a framework only if its concrete runtime semantics address a measured operational gap.

4. Test the abstraction under failure, not only success

Prototype a thin vertical slice, not a demo. Implement the same purchase-request path twice: first using the direct SDK/custom loop; second using one framework or durable workflow candidate. Keep the model, prompts, tools, fixtures, authorization service, and database identical. The only allowed variable is the execution/runtime layer.

Define state explicitly in a versioned application schema: request_id, actor identity, request facts, policy evidence references, draft identifier, approval status, tool attempts, idempotency keys, and terminal outcome. Persist business facts in your database. A framework checkpoint may hold execution progress and transient state, but it should not become an undocumented system of record.

Then execute six tests: (1) ordinary draft, (2) invalid/missing model tool arguments, (3) tool timeout after no side effect, (4) timeout after a side effect whose response was lost, (5) human rejection and edit, (6) worker restart while awaiting approval. Assert business outcomes, not merely that code completes. For every side effect, prove whether the retry is safe, deduplicated, compensated, or escalated to a human.

Inspect the artifacts. OpenAI’s SDK tracing can record generations, tool calls, handoffs, guardrails, and custom events, but trace configuration and sensitive-data treatment remain an operational decision. ([openai.github.io](https://openai.github.io/openai-agents-python/tracing/?utm_source=openai)) LangGraph’s checkpoint model supports state inspection and recovery, yet its documented interrupt behavior restarts the node, so node boundaries and idempotency are part of your design rather than hidden implementation detail. ([docs.langchain.com](https://docs.langchain.com/oss/python/langgraph/interrupts?utm_source=openai))

5. Decide, document, and preserve reversibility

Use a decision memo with four sections. First, evidence: list requirements and the exact prototype result that passed or failed. Second, fit: state why the selected abstraction removes a specific repeated burden. Example: “checkpointed pause/resume plus state inspection eliminated our custom approval-resumption subsystem in two workflows.” Do not write “more scalable” unless you ran and documented a representative measurement.

Third, costs: identify runtime overhead, new state model, serializer/database requirements, model/provider coupling, telemetry destination, deployment assumptions, dependency policy, and skills gap. Framework abstraction leaks often appear as hidden state mutation, confusing replay behavior, tool wrappers that obscure retries, or a deployment environment that cannot host the framework’s persistence requirements.

Fourth, exit plan: retain domain models, prompts, tool contracts, test fixtures, trace correlation IDs, and persistence adapters outside framework-specific types. Wrap framework invocation behind an application interface such as run_purchase_request(command) -> outcome. Version state and tool schemas. Pin dependencies, review release and migration notes before upgrades, and rerun the acceptance suite on upgrades. If a provider-specific feature is necessary, record it plainly. For example, OpenAI-managed continuation via conversation_id or previous_response_id is documented as Responses-API-specific; it is not a portable conversation-state contract. ([openai.github.io](https://openai.github.io/openai-agents-python/running_agents/?utm_source=openai))

The strongest decision can still be no framework now: retain direct SDK plus deterministic code, add structured observability, and create a trigger for reconsideration—such as the second workflow requiring resumable approval or repeated custom recovery code. That is an intentional architecture, not an incomplete one.

Architecture Decision Lab: Keep the Direct Loop or Adopt a Runtime?

Scenario: You are the solution architect for a B2B procurement-support product. The proposed assistant must answer policy questions, retrieve approved vendor data, draft a purchase-request packet, and request manager approval before submitting it. The first proof of concept used a direct model SDK plus two tools and worked for happy-path questions. Stakeholders now request multi-turn work that may pause overnight for approval, recover from worker failures, preserve an inspectable audit trail, and prevent duplicate submissions. Engineering must decide whether to retain a direct SDK/custom loop, add a stateful agent framework, use a durable workflow engine around model calls, or keep the process predominantly deterministic.

Create a two-page entry in your AI Agent Framework Decision Portfolio. Page 1: define the procurement scenario’s requirements, classify each capability by owner, and state a preliminary no-framework hypothesis. Page 2: run the direct-loop and one-runtime prototypes through the six tests in Section 4. Attach sanitized traces/state snapshots, record learner-run measurements, and write a 250-word architecture decision record: selected option, rejected option, evidence, unresolved risks, upgrade policy, and exit trigger. Defend the choice to a reviewer who asks: “Which custom engineering burden did this runtime actually remove, and what new burden did it introduce?”

Self-review checklist

  • Write a one-page requirement brief with must-have, should-have, and prohibited behaviors.
  • Create a capability ledger: framework-native, external platform/tooling, and custom application logic.
  • Implement the same thin vertical slice using a direct SDK/custom loop and one shortlisted framework or workflow runtime.
  • Run fixed representative cases, including approval pause/resume, tool timeout, malformed tool arguments, duplicate delivery, and process restart.
  • Collect per-run trace identifiers, tool-call records, state snapshots, error/retry behavior, and learner-run latency/cost measurements where permitted by your environment and provider terms. Do not generalize from a handful of runs as a benchmark.

Module capstone: apply the module in the AI Agentic School Decision Lab below. Mastery unlocks the LearnDash completion control for this topic.

AI Agentic School Decision Lab

Please sign in to submit your module decision and receive feedback.

Key takeaways

  • A framework is justified by a demonstrated need for reusable orchestration, durable/resumable state, human intervention, standardized lifecycle behavior, or traceability—not merely because a model can call tools.
  • Keep deterministic policy, authorization, irreversible side effects, and domain invariants in ordinary application code even when a framework manages agent execution.
  • Compare a direct SDK implementation and a framework prototype against the same scenario, fixtures, fault injections, and acceptance criteria before committing.
  • A framework’s abstraction can create new obligations: state schemas, replay/idempotency rules, deployment dependencies, tracing data handling, and upgrade/migration work.
  • Record the decision, rejected alternatives, evidence, measurements, and exit plan in the AI Agent Framework Decision Portfolio.

Useful terms

Direct SDK
Use of a model provider’s API/SDK from ordinary application code, with the application explicitly owning the agent loop and surrounding behavior.
Durable execution
An execution model that persists progress so work can recover or resume after failure, interruption, or delay.
Idempotency key
A stable identifier used to ensure that repeated attempts at an external action produce one intended effect rather than duplicates.
Checkpoint
A persisted snapshot of execution state used to resume, inspect, or recover a run.

Research & review note

This lesson is built from source-traceable research and reviewed before publication. AI Agentic School uses AI as a research and practice tool, not as a substitute for evidence or editorial judgment.

Sources & further reading