Razorpay and the business test for AI payment agents

A proposed evaluation of merchant-assisted payment operations, with task completion kept in the scorecard

A payment agent can complete the requested operation and still create more work for the merchant. Someone may need to check the evidence, resolve an exception or discover that the action should never have been taken. Those costs belong in the product evaluation alongside the completed task.

My recommendation for a Razorpay case study is to compare an assistant-assisted workflow with the merchant's existing process. Keep task completion as an operational metric, then examine whether the combined system produces better decisions and acceptable business trade-offs.

This is public-source analysis and a proposed evaluation. I have not tested Razorpay's product, interviewed its merchants or measured an improvement. The design concept accompanying this case is an illustration, not a screenshot of an existing interface.

What the product promises, and what remains untested

Razorpay describes merchant-defined permissions and review-first operation for Agent Studio. That is a useful product promise; it does not establish how well merchants understand the choices presented to them. Razorpay's explanation

The comparison also needs to acknowledge existing alternatives. Stripe documents approval requests and designated reviewers, while Cashfree describes authorisation of client access and tool selection. These controls operate at different points in a workflow. Their documentation does not establish which review experience is easier to use. Stripe approvals, Cashfree builders

The product question is therefore specific: when an agent proposes a consequential action, does the merchant have the information needed to judge it, and does that improve the overall workflow?

Start with the merchant's actual process

The baseline should be what the merchant currently does. It might involve a dashboard, a spreadsheet, another automation tool or several people. Calling it a manual process without observing it would make the comparison too convenient.

A useful study would first document one workflow: the request received, the records checked, the person authorised to decide, the action taken and the evidence that it completed. Preparation, review, corrections and exception handling all consume time. The workflow boundary should include them.

For a first study, use one narrow task rather than a mixture of payment operations. A hypothetical refund-review task can expose the evaluation design, but it does not establish that every provider supports the same agent, interface or approval policy.

Compare equivalent work

Both conditions should use the same business rules, authority limits and evidence requirements. One group of merchants uses its existing process; another uses an assistant-assisted process. Random assignment would be useful where feasible. If participants try both, vary the order and use matched cases so remembering an answer does not create the apparent benefit.

The workload should contain routine valid requests and realistic exceptions. Record the task mix before interpreting results. A set dominated by missing evidence will test caution; it will not estimate performance on an ordinary day's work.

Early work can use fictional cases and a non-transactional prototype. A study of real operations needs a separately agreed scope and appropriate access. Neither activity has been performed for this article.

Keep completion, then inspect what it costs

Task completion remains useful: of the eligible requested actions, how many actually reached the required completed state? A generated recommendation or an approval click should not automatically count as completed execution.

A second measure should assess decision quality across the review cases. Did the reviewer approve a supported action, hold a genuinely uncertain case or escalate an action outside the permitted scope? Refusing everything should not look like excellent performance. Correctness depends on both the decision and its reason.

The business scorecard would include:

| Measure | Proposed definition | Why it matters | |---|---|---| | Task completion | Confirmed completed actions / eligible requested actions | Retains the operational result | | Policy-correct review | Correct decisions / all assigned review cases | Includes justified action and justified restraint | | Unsafe approval | Incorrect approvals / cases where approval should be withheld | Makes the cost of acting too readily visible | | Total operator effort | Preparation + review + correction + exception handling | Captures work moved elsewhere | | Cost per correct resolution | Relevant operating costs / correctly resolved cases | Connects output quality with economics |

For cost, include the people, model/tool calls, communication and exception handling actually used in each condition. Do not count the same review effort twice. Any estimate of downstream error cost needs an explicit assumption and a sensitivity check.

These are proposed measures, not reported results. The business would also need to choose which outcomes matter for the selected workflow and what risk is acceptable. A short study cannot establish every downstream loss or customer effect.

A review card to test, rather than assume

The original concept for this case makes the proposed action, source evidence, authority and execution state visible together. In its fictional example, the matching order record is unavailable, so the request remains on hold.

The accompanying eight scenarios cover missing evidence, stale records, scope mismatch, duplicate actions, uncertain execution and conflicting information. They are a failure-oriented design-review set. They have not been run with merchants and should not be presented as a production benchmark.

The card might help a reviewer notice an important gap. It might also add reading time without improving a decision. Existing interfaces may already supply the same information. In that case, the appropriate next step is to test comprehension or simplify the wording, not invent another screen.

What would justify a product decision?

An assistant-assisted workflow would be worth considering if it improves the chosen business outcome while keeping review effort and error costs acceptable. Higher completion alone would not settle that question. Neither would a lower error count achieved by declining legitimate work.

The result could support a narrower rollout, better evidence presentation, different permission defaults or a decision to keep the existing process. That is the value of the evaluation: it gives the team a way to choose, including choosing not to add automation.

For this case, the next proof is a fair comparison with the existing workflow. Until that work is done, the contribution is an explicit, challengeable test plan.