How to Tell If an Educational App Works: Run a Two-Week Transfer Test

“Teaches math” and “builds reading” are too broad to test. Write an observable target: identify letter sounds, compare quantities to ten, blend simple words, explain a science relationship or use a calming strategy without the app.

Then define the population and setting. An app that helps a preschooler practice counting with an adult may not help an older child transfer multiplication facts independently. Evidence from a school routine may not predict unsupervised home use.

Finally, decide what improvement would matter and what the app must not replace. Entertainment, motivation and learning can all be legitimate goals, but they require different evidence.

A meta-analysis of 36 educational-app intervention studies and 285 effect sizes found positive average effects in early literacy and mathematics. Many included apps were embedded in school contexts, not simply handed to children at home.

A 2025 Harvard CEPR summary emphasizes that effects vary by skill, learner and measure: gains were larger for constrained skills and researcher-developed assessments, while standardized measures showed smaller effects. This does not invalidate app learning; it means near tests can overstate broad transfer.

Ask whether a claim comes from a randomized comparison, how many children participated, who delivered the activity, what the comparison group did, whether the measure resembled the app and whether results persisted.

Do not practice the exact transfer items beforehand. Otherwise the test becomes another lesson rather than evidence of what generalized.

An app may legitimately build only a narrow subskill. If letter-sound retrieval improves but reading does not yet change, describe that result accurately rather than calling the app either a miracle or failure.

“Adaptive” can mean the software changes difficulty, repeats missed items or simply changes rewards. Observe what happens after an error. Does the app diagnose a misconception, offer a meaningful representation and retest the idea? Or does it highlight the correct button until the child taps it?

Good feedback leaves a cognitive job for the child. After a cue, the learner should attempt a new item and explain the correction. Endless celebration can obscure whether the child knows why an answer is right.

For AI-powered apps, ask what inputs drive adaptation and whether generated explanations are reviewed for age, accuracy and curriculum alignment. Personalization is not evidence of effectiveness.

Twenty minutes of app use may replace passive waiting, or it may replace conversation, sleep, movement, outdoor observation, handwriting, manipulatives or shared reading. The same learning gain has a different value in those two cases.

Record what the app displaced during the two weeks. Also observe transition cost: bargaining before stopping, continued preoccupation, irritability or a longer bedtime routine. Engagement is useful only if it remains governable.

Keep a strong app as one component of instruction. The evidence base does not support assuming that a device should replace responsive adults, physical materials or broad real-world practice.

An app can teach effectively and still be a poor choice if it collects unnecessary voice, location, contacts, identifiers, behavioral profiles or advertising data. Use the least-data version, avoid real child details in open text fields and review deletion and sharing controls.

The U.S. Federal Trade Commission finalized COPPA Rule amendments in 2025, including separate parental consent for certain third-party disclosures. Legal compliance is a floor, not a complete quality test, and rules vary by place.

Schools should disclose who contracted the product, what data flow occurs and what non-digital alternative exists. Families should not have to trade avoidable data for basic access to learning.

• Keep: independent off-screen performance improves, the target matters, use remains bounded and the data practices are acceptable.
• Modify: learning appears only with prompts, transfer is narrow or adult co-use could connect the screen to real objects and language.
• Remove: progress is cosmetic, errors teach little, stopping is consistently disruptive, privacy is unacceptable or the app displaces a clearly better activity.

Record the decision and evidence in two sentences. Re-evaluate after the child outgrows the target; an effective app can become empty repetition.

The same app can produce different results by child, goal and context. A product list ages quickly and invites brand competition; the transfer test remains useful when interfaces and AI features change.

Consult a teacher or specialist when the child needs targeted intervention, accommodations or progress monitoring. Consumer apps should not delay evidence-based instruction or evaluation.

The decisive question is simple: after the prompts, animations and rewards disappear, what new capability does the child still own?