Mobile App Testing: Why Skipping QA Costs More Than Doing It

Mobile app testing costs less than a bad release. See the test mix to expect, a pre-release checklist and what to demand in a vendor's quote.

10 November 202611 min read

Row of different smartphones and a tablet on a desk while a tester adjusts one, illustrating mobile app testing

Mobile app testing costs less than the problems it prevents. A defect found while the app is being built is cheap to fix. The same defect found after customers have downloaded the app means rework, support calls, a rushed update and sometimes a bruised store rating. Cutting QA from a quote lowers the price on paper and moves the bill to a worse moment.

This article sets out what a sensible test mix looks like for a business app, where automation helps, and what to ask for in a vendor's quote. It is written for the person signing off the budget, not the person writing the code.

Key Takeaways

A US government-commissioned study found that defects cost more the later they are found, with its illustrative example putting a post-release fix at 30 times the cost of one caught at requirements stage.
A business app needs layered testing (unit, integration, interface, device, accessibility, performance and security), not one round of tapping through screens before launch.
Automated regression testing protects what already works on every release, while manual testing finds the usability and edge-case problems scripts miss.
Apple states it rejects apps that crash or show obvious technical problems, and Google Play requires closed testing for some new developer accounts.
A credible quote lists what will be tested, on which devices and OS versions, by whom, and what happens when a bug is found.

Why Does Mobile App Testing Cost Less Than Fixing Problems After Release?

Because the cost of a defect grows the longer it stays in the product. A flaw in how a feature was specified is a conversation. The same flaw inside a released app means rewriting code, retesting everything around it and pushing an update through store review while users wait.

The best-known evidence is a 2002 study for the US National Institute of Standards and Technology (NIST), prepared by RTI. It includes an example of relative repair costs: a defect costing 1 unit to fix at the requirements stage costs 5 during coding and unit testing, 10 during integration testing, 15 during beta testing and 30 after release. The report labels that table "Example Only", so treat it as a pattern and not a price list. It also estimated that inadequate software testing cost the US economy between US$22.2 billion and US$59.5 billion a year.

The study is more than 20 years old, covers US industry and is not specific to mobile apps. We use it for its direction, that late fixes cost more than early ones, and not for the exact multiples.

What Does a Bad Release Cost a Business?

The costs usually described are rework, extra support load and public reviews. We have no reliable cross-industry figures for those, so we will not invent any. Google's guidance says developers who regularly use its testing tools before publishing provide higher quality experiences that can lead to higher ratings. Ratings are visible to every future customer, and a rushed fix does not remove old reviews.

What Does a Sensible Mobile App QA Mix Look Like?

Good mobile app QA is layered. Each layer catches a different kind of problem and none catches everything. Scale each layer to the risk: an internal field-service tool and a customer payments app do not need the same depth.

Unit - What it catches: Errors in single pieces of logic, such as a price calculation; Usual approach: Automated
Integration - What it catches: Breaks between the app, your back end and third-party services; Usual approach: Automated
Interface (UI) - What it catches: Broken journeys such as login or checkout; Usual approach: Automated for key journeys, manual for the rest
Device and OS - What it catches: Layout problems and crashes on specific phones and versions; Usual approach: Real devices plus emulators
Accessibility - What it catches: Screens that break with screen readers or large text; Usual approach: Mostly manual
Performance - What it catches: Slow start-up and weak-network behaviour; Usual approach: Measured, checked on older devices
Security - What it catches: Insecure storage and weak authentication; Usual approach: Checked against a recognised standard

For security, the OWASP Mobile Application Security Verification Standard is the reference we would point a non-technical buyer to. It groups checks into areas such as data storage, cryptography, authentication and network communication. For accessibility, W3C publishes guidance on applying WCAG 2.2 to mobile apps, a fair benchmark to name in a brief.

How Many Phones and OS Versions Should Device Testing Cover?

Fewer than every device on the market and more than the two phones on the project team's desks. A competent team starts from your audience: the devices and OS versions your customers or staff actually use, taken from analytics or from your own device fleet. It then agrees a written matrix with you, usually the newest and oldest supported versions on each platform, a small phone, a large one and one older, slower device.

Real devices catch problems emulators miss, such as camera, biometrics and notification quirks. How much can be tested automatically also depends on how the app is built, which our piece on enterprise mobile app architecture covers.

Automated or Manual Testing: Which Do You Need?

Both, for different jobs. Automated tests suit regression testing: confirming that last month's working features still work after this month's change. A computer repeats the same check on every build at almost no extra cost, which keeps frequent releases affordable.

Manual testing handles what a script cannot judge. Is this screen confusing? What happens if the signal drops halfway through a payment? A good tester also tries what a normal user would never think to do. Be wary of a vendor who promises full automation, or who relies on manual checking alone for an app that will keep changing.

How Do Beta Tracks Reduce App Store Rejection Risk?

They put real builds in real hands before the public sees them. On iPhone, Apple's TestFlight allows up to 100 internal team members and up to 10,000 external testers, and each tester can install the beta on up to 30 devices. External testing needs the first build approved by App Review, so it also gives an early read on how reviewers see the app.

On Android, Google Play offers internal, closed and open testing tracks. Its testing requirements for new personal developer accounts say accounts created after 13 November 2023 must run a closed test with at least 12 testers opted in for 14 continuous days before applying for production access. That rule is stated for personal accounts, so confirm what applies to your account type before fixing a launch date.

Apple's App Review Guidelines are blunt about quality. Guideline 2.1 asks developers to test on-device for bugs and stability, remove placeholder text and supply demo login details with the back-end service switched on. It says Apple will reject incomplete app bundles and binaries that crash or show obvious technical problems. An app store rejection delays your launch date, and it is a cheap mistake to avoid.

What Should a Vendor's Quote Say About Testing?

It should say it in words, not leave it as a line called "QA". Ask your vendor to show the following in writing:

Which test layers are included and which are excluded.
The device and OS matrix, agreed with you.
What is automated, and whether those tests are handed over to you.
Who tests: the people who wrote the code or someone independent.
How bugs are ranked and fixed, and whether fixes after launch are covered.

Our approach to building apps is described on our mobile application development page. Whoever you choose, hold them to the same standard.

A Short App Testing Checklist Before Release

Use this as a final gate before submitting a build. It does not replace the layers above. It checks that none was skipped.

All automated tests pass on the exact build being submitted.
Key journeys (sign-up, login, payment, core task) pass manual testing on the agreed devices.
Regression testing is complete after the last code change.
Text size, contrast and screen reader labels pass accessibility checks.
The app copes with a weak or lost connection without losing data.
Placeholder text, test data and debug settings are removed.
A demo account works, the back end is switched on and beta testers have signed off.
Crash reporting is switched on so problems are visible from day one.

Related Resources

Common Questions About Mobile App Testing

1. How much of an app budget should go on testing?

There is no reliable single percentage, and anyone quoting one without context is guessing. Scale testing to risk, since an app handling payments or personal data needs deeper security and regression coverage than a simple internal tool. Ask for testing to appear as its own scope in the quote.

2. What is the difference between QA and testing?

Testing is the act of checking software for defects. Mobile app QA is the wider process that decides what to test, when, on which devices and what counts as good enough to release.

3. What is regression testing in a mobile app?

It is re-checking existing features after a change to confirm nothing has broken. It is the main reason to automate, because the same checks must run on every release for the life of the app.

4. Why do apps get rejected from the App Store?

Common reasons include crashes, obvious technical problems, incomplete builds, placeholder content and missing demo login details, all covered by Apple's guideline 2.1. Other sections cover safety, business, design and legal issues, so read the full list before submitting.

5. Can my team do user acceptance testing ourselves?

Yes, and you should. Your staff know the real tasks and will spot workflow gaps a vendor misses. Treat it as the final layer after the vendor's testing, using a TestFlight or Google Play test track so everyone tests the same build.

Test Before Release, Not After Complaints

Testing is not a cost to trim when a budget tightens. It decides whether you learn about a problem from your own testers or from your customers. Ask for the test mix, the device matrix and the release checklist in writing before you sign.

If you are planning an app and want a quote that shows its testing clearly, send 9Byte a brief and we will respond with a scoped plan.