Build Production‑Grade AI Agents for Finance Compliance

Problem

Financial institutions must constantly verify transactions, monitor risk, and stay aligned with ever‑changing regulations. Manual checks are slow, error‑prone, and costly, while a single missed flag can trigger fines or reputational damage. Companies like Stripe have turned to AI agents that can read documents, reason over policy, and take corrective actions automatically. The challenge is turning experimental models into a reliable, auditable service that runs at the scale of millions of daily requests.

Prerequisites

Before you start building, make sure you have the following in place:

• Clear compliance scope. Define the exact tasks – e.g., AML screening, KYC verification, transaction risk scoring – and the regulatory standards they must meet.
• Access to a large language model (LLM) API. Stripe’s solution uses a ReAct‑style prompting pattern that works with any LLM offering chain‑of‑thought reasoning.
• Infrastructure on a cloud platform that supports containerized services. AWS was the host of Stripe’s dedicated agent service, but any provider that offers autoscaling, managed queues, and secure storage will work.
• Team for human‑in‑the‑loop (HITL) oversight. You need subject‑matter experts who can review edge cases and provide feedback to the model.
• Monitoring and logging stack. Compliance work demands immutable audit trails; set up structured logs, metrics, and alerting from day one.

Step‑by‑Step Build

Below is a practical workflow that mirrors the lessons Stripe shared in its recent AWS blog post.

1. Adopt the ReAct prompting pattern. ReAct combines reasoning (think) and acting (act) in a single prompt. Write a template that asks the model to first list relevant regulations, then decide which action to take – such as flagging a transaction, requesting additional documents, or approving automatically. Keep the prompt short enough to fit the model’s context window but detailed enough to guide consistent reasoning.
2. Wrap the prompt in a reusable function. Create a thin service layer (e.g., a Lambda or container) that receives a compliance request, injects the ReAct prompt, calls the LLM, and returns a structured response. This layer isolates model calls from business logic and makes it easy to swap models later.
3. Deploy a dedicated agent service. Stripe chose a separate microservice that runs solely for compliance agents. Deploy the function as a horizontally scalable service behind a load balancer. Use autoscaling policies that trigger on request latency or queue length to keep response times low during spikes.
4. Implement task decomposition. Break complex compliance checks into smaller subtasks that the agent can handle independently – e.g., identity verification, source‑of‑funds analysis, and sanctions screening. Each subtask gets its own ReAct prompt, and a lightweight orchestrator stitches the results together.
5. Orchestrate with a state machine. Use a simple state‑machine library or AWS Step Functions to manage the flow: start → decompose → parallel sub‑agents → aggregate → final decision. This makes the process observable and recoverable if a step fails.
6. Introduce human oversight. Route any request where the model’s confidence falls below a threshold to a compliance analyst. Store the analyst’s decision alongside the model’s output for future training and for audit purposes.
7. Cache frequent prompts. Stripe reduced cost by caching the result of prompts that rarely change, such as the list of applicable regulations for a given jurisdiction. Store the cached response in a fast key‑value store (e.g., Redis) keyed by jurisdiction and request type.
8. Log immutable audit records. Every request, model output, human decision, and cache hit should be written to an append‑only log. Include timestamps, request IDs, and the exact prompt sent. This satisfies regulatory traceability requirements.
9. Monitor cost and performance. Track LLM token usage, latency, and cache hit rate. Set alerts if token consumption spikes unexpectedly – a sign that prompts may have drifted or that new compliance rules are being added.
10. Iterate with feedback loops. Periodically review audit logs with compliance experts, identify failure patterns, and refine the ReAct prompts or add new subtasks. Because the agent service is decoupled, you can redeploy updated prompts without downtime.

Pro Tips

• Start small. Pilot the agent on a single compliance rule before scaling to a full suite. Early wins prove the model’s reliability and give the team confidence to expand.
• Version prompts. Treat each prompt template as code – store it in version control, tag releases, and roll back if a new version introduces regressions.
• Use prompt caching wisely. Only cache static parts of the prompt (e.g., regulation lists). Dynamic data like transaction amounts must remain uncached to avoid stale decisions.
• Keep the human loop fast. Build a UI that surfaces the model’s reasoning alongside the raw data. When analysts see the chain‑of‑thought, they can approve or reject with minimal effort.
• Separate cost centers. Run the LLM calls in a dedicated AWS account or billing tag so you can attribute token spend directly to compliance work.
• Plan for model upgrades. When a newer, cheaper LLM becomes available, switch the underlying API without changing the ReAct wrapper. Your orchestration and audit layers stay untouched.