Case Study · Fintech · Production ML

Case Study: Lifting Payment Success Rates 15% with AI-Driven Routing

How we built an ML-based transaction routing engine at Payism Global — architecture, feature store, online inference, safety guardrails, and the measured business impact.

Role

CTO

Domain

Payments

Impact

+15% success

Stack

Python · FastAPI · AWS

The problem

Payment success rates on multi-acquirer routing are usually optimized with static rule tables — BIN ranges, MCC codes, hard-coded fallbacks. Those rules go stale fast: issuers change risk posture, acquirers rotate cost tiers, and fraud patterns shift weekly. We were leaving double-digit success-rate points on the floor.

The approach

We reframed routing as an online decision problem: for each transaction, pick the acquirer/route with the highest expected success-adjusted profit. That meant an online model, a real-time feature store, sub-100ms inference, and a shadow-mode evaluation loop before any traffic actually moved.

Features were split into three groups — transaction (amount, MCC, currency, device fingerprint), historical (issuer × acquirer success rates over rolling windows), and contextual (time of day, recent decline bursts, acquirer health). Historical features materialised into a Redis-backed feature store; contextual features were streamed.

Architecture

A FastAPI service sat behind the payment gateway. On each request it hydrated features, called a gradient-boosted ranker, and returned the chosen route with a confidence score. Guardrails wrapped every decision: minimum-confidence fallback to the static ruleset, per-BIN circuit breakers, and a hard cap on any single acquirer's share.

Training ran nightly on Airflow. Every prediction and its downstream outcome were logged to a warehouse, so the next training run always saw its own recent behaviour — closing the feedback loop without human labelling.

Rollout and safety

We ran the model in shadow mode for two weeks — scoring live traffic without acting on the decisions — and compared its choices to the rule engine on the same transactions. Once shadow-mode uplift was statistically significant, we ramped from 1% to 100% of traffic over a fortnight, one merchant cohort at a time.

Results

Payment success rates rose ~15% aggregate, with the largest gains in the tail of previously-underserved BIN × acquirer pairs. Latency budget stayed under 80ms p95 at the routing layer. Because we owned the whole loop end-to-end, model refresh moved from "quarterly project" to "cron job".

Takeaways

Production ML wins in fintech aren't about model complexity — they're about the plumbing: reliable features, safe rollout, honest offline evaluation, and guardrails your on-call engineer trusts at 3am. A boring model with a great harness beats a fancy model without one.