Why this matters: This lesson converts ambiguous framework requests into a requirements specification and a small, evidence-led proof of concept. You will distinguish what must be designed in your application and infrastructure from what a framework natively provides, then compare candidates against acceptance tests, exit costs, and the team’s actual operating constraints.

What you will be able to do
- Frame a framework-selection decision in terms of business action, system boundary, operating context, and consequences of failure.
- Translate ambiguous stakeholder language into constraints, weighted preferences, assumptions, and POC questions.
- Distinguish framework capabilities from durable infrastructure, security controls, and application-owned business logic.
- Design representative acceptance tests for approval, persistence, traceability, provider portability, and deployment fit.
1. Begin with the architecture decision, not the vendor list
Decision first: The procurement assistant is not choosing “an agent framework.” It is choosing whether a runtime can safely coordinate a bounded, reviewable workflow under the organization’s operating model. The likely architecture mismatch is subtle: a team may select an agent SDK because it offers sessions and tool calling, then discover that a browser refresh, worker crash, delayed approval, or ticket-system timeout leaves no reliable way to resume the exact business action.
Write this decision statement before opening a comparison page:
“Select an implementation approach for a Python procurement-exception workflow that can retrieve authorized evidence, produce a structured recommendation, stop before any ticket write, resume after an overnight human decision and process failure, and provide an auditable run record. The first release supports one workflow and three read-only tools plus one gated write tool. Existing PostgreSQL, Kubernetes, and central logging are mandatory.”
This wording prevents an early category error. A short-lived assistant that only returns a recommendation may need ordinary application code plus a model API. A workflow that can pause, retry, branch, and commit an external side effect may justify a framework or a separate durable-workflow runtime. OpenAI’s own SDK documentation makes this distinction explicitly: use the lower-level Responses API when you want to own the loop, tool dispatch, and state handling; use its Agents SDK when its runtime should manage turns, tools, guardrails, handoffs, or sessions. That is a product-specific positioning statement, not a general rule. ([openai.github.io](https://openai.github.io/openai-agents-python/?trk=public_post_comment-text&utm_source=openai))
2. Convert ambiguity into requirements that can decide
Turn stakeholder language into four buckets. The classification is more valuable than a long feature list because each bucket has a different treatment during selection.
- Mandatory constraint: a failure disqualifies the option. “No ticket write without an approver identity and decision recorded.” “Deployment must run in the existing Kubernetes environment.” “Sensitive procurement text cannot be exported to an unapproved trace service.”
- Weighted preference: valuable but negotiable. “Python-native developer ergonomics,” “visual run inspection,” or “a small amount of orchestration code.” Assign a weight only after naming the business effect: for example, a 5/5 for recovery correctness, 3/5 for observability effort, 2/5 for local developer setup.
- Assumption: a claim you currently rely on. “PostgreSQL can hold the required workflow state.” “The ticket API supports an idempotency key.” Assumptions are not points on a scorecard; they have owners and dates for validation.
- POC question: behavior that documentation cannot prove for your case. “After a deliberate worker termination during approval, can the application resume once and create exactly one ticket?”
Rewrite vague requests as observable statements. “Needs memory” becomes: retain the current request, retrieved evidence IDs, draft, approval state, and action idempotency key for 24 hours; retrieve only data authorized for this requester; retain the business record under our policy. “Needs human-in-the-loop” becomes: the proposed write is displayed with arguments and evidence; rejection prevents the write; approval is associated with a reviewer; a resumed execution cannot silently skip the gate.
Framework-native mechanisms can inform the candidate list. For example, LangGraph documents checkpointed graph state keyed by a thread and dynamic interrupts that save state, wait for JSON-serializable input, and resume with a command. ([langchain-ai.github.io](https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/breakpoints/?utm_source=openai)) The OpenAI Agents SDK documents tool approvals that pause a run, serialize its RunState, and resume after approve/reject decisions. ([openai.github.io](https://openai.github.io/openai-agents-python/human_in_the_loop/?utm_source=openai)) Those are evidence that the APIs exist; neither source proves your identity, authorization, retention, database, or ticket idempotency requirements.
3. Map requirements to the full stack, not a framework demo
Use four layers in the requirements sheet. This stops a framework from being credited for controls it does not own.
- Architecture principle: explicit state transitions, least-privilege tool access, idempotent side effects, approval before commit, and inspectable execution. These should remain true if you replace the framework.
- Framework-native capability: graph/agent control flow, pause/resume API, session abstraction, typed tool interface, schema-constrained output, built-in trace events, or testing utilities. Record the exact package, language, and version tested.
- External platform or tooling: PostgreSQL durability, secret manager, Kubernetes, identity provider, ticket API, queue, tracing backend, evaluation store, and model provider. A framework may integrate with these; integration is not ownership.
- Custom application logic: authorization checks, policy retrieval filters, approval UI, redaction, evidence citations, idempotency-key creation, business audit event schema, retry policy, and reconciliation job.
Apply the layers to persistence. LangGraph distinguishes checkpointed, thread-scoped graph state from cross-thread stores; its Agent Server documentation describes production persistence and task-queue architecture as deployment concerns. ([langchain-ai.github.io](https://langchain-ai.github.io/langgraph/concepts/time-travel/?h=time+travel&utm_source=openai)) By contrast, the OpenAI Agents SDK documents sessions that retrieve and store conversation history, while durable long-running recovery is described through integrations such as Dapr, Temporal, Restate, and DBOS. ([openai.github.io](https://openai.github.io/openai-agents-python/running_agents/?utm_source=openai)) The decision implication is not “LangGraph is durable and the SDK is not.” It is: define whether you require conversational continuity, recoverable workflow progress, exactly-once-like business handling, or all three—then verify the complete stack that provides each.
4. Shortlist with evidence, then make the missing evidence explicit
Create a shortlist only after constraints are visible. For this scenario, compare: (A) custom Python orchestration plus your existing job/approval mechanisms; (B) a graph-oriented runtime with persisted state and explicit interrupts; and (C) an agent SDK with tool approvals, sessions, tracing, plus an external durable-workflow component where needed. You are comparing architecture fit, not declaring any category superior.
Score each candidate using two columns: documented evidence and representative test result. Documentation can verify public API surface, supported deployment mode, stated session behavior, and configuration knobs. For example, the OpenAI Agents SDK documents built-in traces for generations, tool calls, handoffs, guardrails, and custom events, but also says tracing is enabled by default, potentially captures inputs and outputs, and is unavailable for Zero Data Retention organizations. ([openai.github.io](https://openai.github.io/openai-agents-python/tracing/?utm_source=openai)) That creates a specific security review item rather than an automatic observability point.
Likewise, a framework may offer structured outputs or Pydantic/schema validation, but a valid JSON object is not a valid procurement decision. Keep domain validation and authorization in custom logic. A tool should accept a narrow typed request, derive authorization from trusted server-side identity—not model-provided fields—and return a stable receipt containing the idempotency key and external ticket ID.
5. Prototype the risky path and inspect operational evidence
Build a thin vertical slice, not a showcase. Use one representative request containing an ambiguous policy exception, a retrieval result that must be cited, and a ticket write that needs review. Hold the model, prompts, retrieval corpus, tool stubs, and test inputs constant across candidates where practical. Do not turn this into a broad framework tutorial.
Run these acceptance tests:
- Controlled branch: policy confidence below threshold routes to reviewer; above threshold produces a draft but still cannot write a ticket.
- Approval integrity: submit malformed tool arguments, then reject the proposal. Assert zero ticket writes, a recorded rejection, and a clear terminal or retry state.
- Crash/resume: pause at approval, persist the run identifier, terminate the worker, restart, approve once, and assert exactly one ticket creation. Repeat the resume request to test idempotency and duplicate-delivery behavior.
- Trace reconstruction: from a run ID, reconstruct model calls, tool arguments and results, retrieved evidence IDs, state transitions, approver identity, redactions, latency, and failure reason. Compare the framework trace with the immutable business-audit event you create.
- Provider and deployment swap: run a second supported model/provider or a mocked provider, and deploy in the intended container path. Record code changes, configuration changes, lost diagnostics, and any unavailable feature.
Inspect failure behavior, not only successful final text. The OpenAI SDK provides deterministic, provider-neutral in-memory testing utilities for orchestration it owns, while explicitly directing developers to real adapters or integration environments for externally owned model, network, sandbox, or audio behavior. ([openai.github.io](https://openai.github.io/openai-agents-python/testing/?utm_source=openai)) That distinction is a useful general testing rule. For LangGraph, the local in-memory server is documented as suitable for development and testing; its documentation directs production deployments toward persistent storage. ([docs.langchain.com](https://docs.langchain.com/oss/python/langgraph/local-server?utm_source=openai)) A passing local demo is therefore not proof of production recovery.
Decision Portfolio Exercise: The Approval Gate That Must Survive Tomorrow
Scenario: You are selecting an implementation approach for an internal procurement-exception assistant. It receives a request, retrieves policy and purchase-order data, drafts a recommendation, and may create an approval ticket. It must never create or update a ticket without a reviewer’s approval; paused work can resume the next business day after a service restart; auditors need to reconstruct the policy evidence, proposed action, reviewer decision, tool arguments, and resulting ticket ID. The team already runs Python services, PostgreSQL, OpenTelemetry-compatible logs, and Kubernetes. Stakeholders initially say: “Use an agent framework with memory, human-in-the-loop, and observability.” Your task is to turn that statement into a defensible selection decision—not to choose the most feature-rich framework.
Create a Decision Portfolio entry titled “Procurement Exception Assistant—Framework Requirements v1.” 1) Write the decision statement in 120 words or fewer. 2) Make a table with 12 requirements: four mandatory constraints, four weighted preferences totaling 100 points, two assumptions, and two POC questions. 3) For every constraint, name the ownership layer: architecture principle, framework, platform, or custom application. 4) Shortlist custom orchestration and one or two frameworks that meet your language and deployment constraints; cite the exact official documentation page and version/date you examined. 5) Implement or diagram the approval-before-write vertical slice. Execute the five tests in this lesson, recording pass/fail, artifacts inspected, and unresolved risks. 6) Finish with a one-page recommendation: select, reject, or defer each option; state the strongest evidence, the largest uncertainty, migration/exit plan, and the next proof required before production.
Self-review checklist
- Write a one-page decision statement with system boundary, actor, outcome, and consequence of error.
- Extract at least 12 stakeholder statements and classify each as a constraint, preference, assumption, or question.
- Create five acceptance tests, including one restart/resume test and one rejected-approval test.
- Build the same thin vertical slice in no more than two shortlisted approaches—or compare one framework against custom orchestration if that is the real alternative.
- Capture traces and raw state artifacts for every POC case; redact test data before external export if required by policy.
Key takeaways
- Start with the operational decision and failure consequences, not a framework feature checklist.
- A requirement is decision-useful only when it becomes a disqualifying constraint, a weighted preference, an explicit assumption, or a measurable proof-of-concept test.
- Separate execution semantics from convenient SDK features, and separate both from infrastructure, policy, and application logic.
- Documentation establishes that a capability exists; representative behavior tests establish whether it works acceptably in your architecture and operating environment.
- Treat persistence, approvals, tracing, and deployment as end-to-end properties. A framework API alone does not create a durable or compliant system.
Useful terms
- Acceptance test
- A concrete, observable condition that an implementation must satisfy, such as resuming after a crash without creating a duplicate ticket.
- Constraint
- A non-negotiable requirement whose failure disqualifies an option.
- Durable workflow state
- Persisted progress sufficient to resume a process after interruption or restart; it is distinct from conversational history.
- Idempotency key
- A stable identifier used so that repeating a request does not repeat a consequential external side effect.
Research & review note
This lesson is built from source-traceable research and reviewed before publication. AI Agentic School uses AI as a research and practice tool, not as a substitute for evidence or editorial judgment.