Migrating APIs Under Traffic: Lessons from a Marketplace Serving Millions

Rewriting a system nobody uses is a weekend project. Rewriting APIs inside a marketplace serving millions of users, while feature work continues and sale events loom on the calendar, is a different discipline entirely. At a major Japanese e-commerce marketplace I migrated PHP and GraphQL projects to Java REST APIs, cutting latency around 30%. The latency win got the headlines. The migration mechanics were the actual work.

You migrate consumers, not code

The naive plan for a migration is "build the new thing, switch the traffic." The plan that works is consumer-by-consumer, route-by-route:

1. New Java service deployed, serving 0% of traffic
2. Shadow traffic: mirror real requests, diff responses offline
3. One internal consumer moves; watch a full weekly traffic cycle
4. Percentage rollout per route (1% → 10% → 50% → 100%)
5. Old route returns 410 only after logs show zero callers for a month

Step 2 is the one teams skip and regret. Shadow-diffing found the bugs that would have been incidents: fields with subtly different null semantics between the PHP and Java implementations, date formatting that differed only for users in specific timezones, an edge case where the old API returned [] and the new one returned null. None of these appear in unit tests, because the tests encode the same assumptions the code does. Only production traffic tells the truth.

Where the 30% actually came from

People assume a latency win like that comes from "Java is faster than PHP." Runtime helped, but the honest breakdown was mostly architecture.

• Fewer round trips. The legacy path accumulated sequential internal calls over years. The rewrite composed them concurrently, and that was the single biggest win.
• Connection reuse. Pooled, keep-alive connections to downstream services instead of per-request handshakes.
• Doing less. The old API returned everything any consumer had ever needed. The new one returned what this consumer asked for. Serialization time scales with payload whether anyone reads the fields or not.

A rewrite gives you permission to fix accumulated architecture. The language change is mostly the occasion, not the cause.

The BFF that saved the org chart

Part of the same era: I designed a Backend-for-Frontend so the marketplace's frontend team could stop negotiating every data-shape change with platform teams. Cross-team dependencies dropped around 40%, which mattered more than any latency number, because the scarcest resource in a large organization isn't compute. It's coordination. (I wrote up the pattern in more detail in my engineering notes.)

Security debt is migration cargo

I also remediated security vulnerabilities across the marketplace during this period, and migrations are where security work gets leverage. Every route you move is a route you can fix while you're already touching it: auth checks tightened, input validation centralized, dependency trees rebuilt from current versions instead of inherited ones. Migrating a route without paying down its security debt just ships the debt to a new address.

What I'd tell a team starting this

1. Buy the diffing infrastructure first. A response-comparison harness costs a week and repays itself on the first prevented incident.
2. Move a boring consumer first. Your first migration should be low-stakes enough that failure is education, not news.
3. Latency budgets per route, in writing. "Faster" isn't a requirement; "p99 under 200ms" is.
4. Never delete on schedule, delete on evidence. Old routes die when the logs say zero callers, not when the roadmap says Q3.

The unglamorous summary: the migration succeeded because of shadow traffic, percentage rollouts, and a diffing harness. Three things that produce no features and impress nobody in a demo, and they're the entire difference between "we migrated" and "we caused the incident review."