Webhook reliability: retry queues, signed delivery, and safer order reconciliation

A payment integration is only production-ready when it behaves correctly outside the happy path. Buyers close tabs. Bank confirmations arrive late. Shop backends redeploy while a payment is in progress. Firewalls, timeouts, and short database incidents hit the exact endpoint that is supposed to mark an order as paid. If payment state depends on a redirect or one fragile callback, fulfilment, support, and accounting inherit that fragility.

FoxPay treats webhooks as operational delivery of backend truth. The redirect brings the buyer back to the shop. Polling keeps the checkout screen fresh. The merchant order should move only after a signed webhook has been verified, persisted, and processed idempotently. That is where retry queues, delivery headers, attempt counters, and redelivery become commercially useful: they reduce manual recovery and prevent paid orders from getting stuck in the wrong state.

Redirects and polling are UX, not order truth

A redirect only proves that a browser reached a page. It does not prove that the merchant backend durably processed the final payment state. A customer may never reach the return page, a mobile browser may drop the handoff, or the payment flow may confirm after the tab is closed. The redirect is valuable for the buyer journey. It is too weak for order reconciliation.

Polling has a similar boundary. GET /payments/{paymentId}/status can show a frontend whether the payment is pending, processing, completed, failed, cancelled, or expired. That is useful for checkout UX, but it should not be the only thing that releases stock, marks invoices as paid, exports accounting records, or updates support history. A reliable system keeps the split clear: redirects and polling inform; verified webhooks decide.

What the FoxPay retry queue provides

FoxPay documents a persistent outbound queue for transaction webhooks. On the queue path, the first delivery attempt still fires inline from the originating event, so the clean path stays fast. If delivery fails, the job persists and a worker retries it over time instead of losing the event after a single broken POST.

Where the queue path is active for a merchant, FoxPay can attempt delivery up to 12 times across roughly 7 days. The hot phase starts with short intervals such as 30 seconds, 2 minutes, 10 minutes, and 30 minutes, then stretches into hour-scale and multi-day long-tail retries. Jitter of about 20 percent spreads retry load so merchant endpoints are not hit in synchronized bursts.

The rollout caveat matters. The queue is activated per merchant; older shops can remain on the legacy path with up to 3 attempts until they are migrated. Merchant code should not depend on which path is active. The contract is the same: webhooks are at-least-once delivery, so duplicate, delayed, and repeated attempts must be safe.

Signed delivery and headers are the contract

Every custom FoxPay webhook delivery carries a signature. Merchants should verify it against the exact raw request body before changing any order state. Re-stringifying parsed JSON can change the bytes and break a valid signature. Skipping verification is worse because it turns a public endpoint into a payment-state mutation surface.

A strong handler starts with trust checks. Capture the raw payload, verify the signature, then persist the payload, verification result, and relevant headers. Only after durable persistence should the handler return 2xx. API keys and webhook secrets do not belong in frontend code, article examples, or full logs. Operations teams need delivery IDs, attempts, timestamps, response classes, and correlation IDs, not sensitive material.

Two headers deserve special treatment: X-Foxpay-Delivery and X-Foxpay-Attempt. The delivery ID is the idempotency anchor. Store it under a unique constraint, ideally scoped by merchant or shop. If the same delivery ID arrives again, absorb it as a duplicate. It must not release the order twice, send another paid email, or enqueue a second fulfilment job.

X-Foxpay-Attempt is a 1-indexed attempt counter. It is not a payment status, but it is a valuable operations signal. Attempt 1 with a fast 2xx is the normal path. Attempt 5 means something failed earlier: timeout, network issue, 5xx, deployment window, or endpoint instability. Used well, the attempt header helps teams distinguish one bad order from a merchant-wide integration incident.

Idempotency belongs in the merchant backend

Do not dedupe only by transaction_id. One transaction can have multiple legitimate status transitions: pending, later processing, then completed or a terminal failure. Those events are not duplicates simply because they share a transaction. Dedupe exact deliveries by X-Foxpay-Delivery, then let a local order state machine decide whether the new status may move the order.

A robust webhook table stores at least merchant context, transaction_id, order_id, FoxPay status, delivery ID, attempt number, timestamp, verification result, and a raw payload reference. After that, business logic can stay explicit. completed can release fulfilment. failed, cancelled, or expired can move the order into support or retry flows. A late pending replay must not move a completed order backwards.

HTTP responses are part of the design. A 2xx response means the delivery has been durably accepted. It should not require ERP, warehouse, email, or fraud systems to finish first. Persist, acknowledge, then process downstream work asynchronously. Temporary server problems should surface as 5xx. Terminal 4xx classes such as 400, 401, 403, 404, and 410 are not retried and should only be used when the failure is truly permanent.

Redelivery and delivery history close the recovery loop

Merchant-triggered redelivery is what turns webhook recovery from guesswork into an operating process. If a bad handler deploy causes deliveries to fail and the bug is later fixed, the merchant can trigger redelivery from the FoxPay panel where that function is available. The transaction Technical Info area shows delivery status, last attempt, next scheduled retry, and delivery history. Teams that want the same context in their own operations tools can use GET /api/transactions/{transaction_id}/webhook/history.

That history is valuable for support, engineering, and finance. Support can see whether FoxPay delivered the event or whether the merchant endpoint rejected it. Engineering can inspect attempts, response classes, and timing. Finance gets an auditable trail instead of a manually patched order with no context.

A practical reconciliation checklist:

• Verify signatures against the raw request body.
• Store X-Foxpay-Delivery as a unique dedupe key.
• Record X-Foxpay-Attempt for monitoring and incident review.
• Return 2xx only after durable persistence.
• Move downstream work into an internal queue or worker.
• Use terminal 4xx carefully and use 5xx for temporary server failures.
• Allow multiple status transitions per transaction, but ignore exact delivery replays.
• Prevent late events from moving an order backwards.
• Link order ID, transaction ID, payment ID, and finance reference early.
• Use redelivery and delivery history for recovery instead of manual status patches.

Conclusion: build for the second attempt

Webhook reliability is payment infrastructure. It decides whether paid orders fulfil automatically, whether support can resolve cases quickly, and whether accounting can understand why a status changed. FoxPay provides signed delivery, delivery and attempt headers, retry queue behavior with jitter and long-tail recovery, redelivery, and delivery history. The merchant side must wire those signals into its own order system deliberately.

A shop that settles orders from redirects alone is built for the ideal case. A shop that verifies signatures, stores deliveries idempotently, classifies HTTP responses carefully, and plans redelivery into support operations is built for production. That discipline is what turns a checkout into a payment integration merchants can trust when volume grows.