17 days until Shopify stops accepting app script tags. Is your store running one? Check your store free
Get started free
← Blog

How to vet a new e-commerce app's reliability before you install it

Before you install a new app, check three things: how its downtime is shaped (many short incidents versus a few long ones), what breaks in your store when it fails, and what the contract actually promises. A single uptime percentage on a vendor's marketing page tells you almost nothing useful, because two services with nearly identical uptime can fail in completely different ways.

This matters most right now. Every app you add between today and Black Friday is a dependency you have not seen fail yet, and Q4 is the worst time to discover an app's failure mode for the first time.

Why is a single uptime percentage not enough?

Compare two services that sit close together on uptime but behave nothing alike. Over the last 90 days, Plaid recorded 98.75% uptime across 4 major or critical incidents totalling 1,548 minutes of downtime, which works out to an average incident of 387 minutes. Shippo recorded 98.97% uptime across 9 incidents totalling 1,332 minutes, an average of 148 minutes each. These figures come from StatusBird's independent 2-minute monitoring over the 90 days ending September 12, 2026.

Those two profiles demand different plans. Four long outages means you need a genuine fallback that a non-technical staff member can switch on and run for six hours. Nine shorter outages means you need queueing and retry behaviour, plus a support macro your team can send nine times without rewriting it.

Service90-day uptimeIncidentsTotal downtimeAvg incidentGrade
Plaid98.75%41,548 min387 minC
Shippo98.97%91,332 min148 minC
ActiveCampaign99.84%3202 min67 minB+
ShipStation99.87%1166 min166 minB+
Cloudflare99.89%2142 min71 minB+
Shopify99.90%2124 min62 minA
Zapier99.91%2118 min59 minA

ShipStation and Cloudflare are two hundredths of a percentage point apart, but ShipStation's downtime arrived in one 166-minute block while Cloudflare's arrived in two incidents averaging 71 minutes. Ask which pattern your operation can absorb before you ask which number is higher. You can pull the full history for any monitored service from its reliability page, for example ShipStation or Plaid.

Does a 90-day window tell the whole story?

No, and you should say so out loud when you present a vendor recommendation to a client or a founder. Monthly totals swing hard: April 2026 saw 20 major or critical incidents and 122.1 hours of combined downtime across 11 services, while March 2026 saw 2 incidents and 8.3 hours across 2 services. August 2026 landed in between with 16 incidents and 40.4 hours across 9 services.

A vendor evaluated during a March-shaped quarter looks flawless. The same vendor evaluated during an April-shaped quarter looks alarming. Check at least two quarters where you can, and read the outage index for the months surrounding your window so you know whether the whole ecosystem was calm or turbulent during the period you sampled.

What should you ask the vendor before signing?

Send these in one email and treat a vague answer as an answer. Six specific questions:

  • What is your published SLA target, and what triggers a credit? Ask whether the credit is automatic or whether you must file a claim, and within how many days.
  • How do you define "degraded" versus "down"? Many vendors exclude partial degradation from SLA calculations, which is exactly the state that breaks a checkout.
  • What happens to my data during an outage? For an app that receives webhooks, ask whether events are queued and replayed, how long the queue holds, and whether replay is automatic or manual.
  • What is your incident notification channel? Ask for a status page with an RSS or webhook feed, not a Twitter account or an in-app banner you will never see at 2am.
  • Which infrastructure and subprocessors do you depend on? If three of your apps all sit on the same CDN or the same cloud region, you have one dependency, not three.
  • Can I see the last three postmortems? A vendor that writes real postmortems has an engineering culture that treats downtime as a defect. One that has none is telling you something.

How do you test an app's failure mode before it matters?

Install on a duplicate theme or a development store first, then simulate failure rather than waiting for it. Practical steps:

  1. Block the app's domain in your browser using devtools request blocking, then load a product page, add to cart, and complete a test checkout. Note what visibly breaks and whether the page still renders.
  2. Time the page without the app. Record largest contentful paint with the app enabled and with its requests blocked. If blocking it makes the page meaningfully faster, that is your ongoing cost.
  3. Check whether the script is render-blocking. An app that injects a synchronous script in the head can turn its own outage into your storefront outage. An asynchronous script that fails quietly cannot.
  4. Test the uninstall. Remove the app from the test theme and confirm it leaves no orphaned code, no broken liquid, and no dead metafield references. Apps that are hard to remove are hard to remove during an incident too.
  5. Verify webhook replay. Disable the app's endpoint for ten minutes, place two test orders, re-enable, and confirm both orders appear. If they do not, you now know you will be reconciling manually.

Which apps deserve the most scrutiny?

Tier by blast radius, not by monthly price. Three tiers, and treat the vetting depth accordingly:

Tier 1: anything in the checkout or payment path

Payment gateways, buy-now-pay-later providers, tax calculation, subscription billing, and any app that injects script into checkout. A failure here stops revenue immediately. Require a written SLA, a fallback you have actually tested, and monitoring on the live service, for example the Stripe status page or Recharge status page.

Tier 2: anything that touches fulfillment or order flow

Shipping label software, 3PL integrations, order management, and automation platforms that move orders between systems. Zapier's most recent recorded incident in the monitoring window was September 9, 2026, and if an automation platform sits between your store and your warehouse, its downtime becomes your shipping delay. A failure here costs you a day, not a sale, but it compounds during peak.

Tier 3: marketing, reviews, loyalty, analytics

Painful and expensive when they break during a campaign, but they do not stop an order from completing. Vet these with the browser-blocking test and the uninstall test, and skip the contract negotiation unless the spend justifies it.

When should you re-vet apps you already have?

Twice a year plus once before peak. Run a full app inventory in early September, remove anything nobody has opened in 90 days, and confirm the remaining Tier 1 and Tier 2 apps each have a named owner on your team and a status feed you actually receive. Freeze new Tier 1 installs from early November through the second week of January, because an app you install in week one of BFCM has zero observed behaviour under your peak traffic.

For anything you keep, write down the three data points that matter: what breaks when it goes down, who gets the alert, and what the manual workaround is. If you cannot fill in all three lines, the app is not vetted, it is just installed. Details on how uptime is measured are on the methodology page.

If you want the alert to reach you before a customer emails about it, StatusBird checks around 84 e-commerce services every 2 minutes and notifies you by SMS, email, Slack, Teams, or Discord. You can set up monitoring here and pick which of your installed apps you want to watch.

Never find out about an outage from your customers

StatusBird monitors Stripe, Klaviyo, Google Ads, Shopify, and 80+ other services your store depends on. Get an SMS alert within minutes of any outage.

Start monitoring free