App Analytics: Build a Diagnostic System, Not a Click Logger

Instrumentation is not tagging every button

I do not think of analytics as installing cameras inside an app. A camera metaphor makes it feel as if we can see everything. Product work needs something narrower and more useful: a diagnostic system. Where does the user fail to get the expected value? What are the likely causes? What should we change next?

The common failure is to ask engineering, right before release, to tag all clicks. The team gets many events, inconsistent names, no exposure denominator, no failure states, and finally a dashboard that can only show click counts. More clicks are not always good; users may be clicking repeatedly because they cannot find the next step. Lower conversion is not always bad; an entry point may have become more precise and filtered out low-intent users. Instrumentation should serve questions before it serves charts.

A practical framework: question tree -> journey -> event contract -> action

1. Write the question tree before event names

I start with the stage question. Do we need to know whether new users reach first value, whether existing users keep returning, or whether the payment flow has visible friction?

Break it into three layers:

• Outcome question: what result should improve, such as completion, retention, payment, or sharing.
• Behavior question: what key behaviors must happen before the result can happen.
• Cause question: if the behavior does not happen, is it because the user did not see it, did not understand it, did not trust it, could not complete it, hit a technical failure, or did not find enough value?

Only questions that can lead to decisions deserve tracking.

2. Map the journey

A feature journey can usually be described as exposure, understanding, attempt, success, and repeat use. The number of events can be small, but the states must be covered.

Basic events may include screen_view, feature_exposure, feature_click, task_success, error_show, and payment_result. Properties matter more than the raw number of events. I usually want to see entry, source, screen, module, status, error_code, app_version, os, network, latency_bucket, and experiment_id.

Events describe actions. Properties describe context. Do not create a separate event for every entry point if one event with a stable entry property can do the job.

3. Turn it into an event contract

An event contract should include the event name, trigger timing, reporting side, required properties, optional properties, deduplication rule, owner, validation method, and retirement condition.

The client is good at reporting user behavior, such as clicks, swipes, exposures, and dwell time. The server should confirm business outcomes, such as successful payment, entitlement delivery, and task settlement. I avoid designs that rely only on client events to define payment success. Network failures, delayed callbacks, and backgrounding can all pollute the data.

4. Validate trust before release

I would rather read fewer metrics than read untrusted charts. Before release, run the full journey with test accounts, check whether required properties are empty, cross-check key outcomes with server logs, and confirm how duplicates, delayed events, and old app versions are handled.

Instrumentation checklist

• Which product question does this event answer, and who acts when the metric changes?
• Is there a denominator? A click without exposure cannot explain conversion.
• Are cancel, fail, timeout, back, and empty states covered?
• Can the data be grouped by entry, version, device, network, channel, and experiment?
• Does the success event mean the user received value, or only clicked a button?
• Is the event name stable and independent of UI copy changes?
• Are unnecessary personal data, such as phone numbers, emails, and precise location, excluded?
• Are critical outcomes confirmed by the server?
• Is there a data health view for missing events, duplicates, delays, and abnormal spikes?
• If nobody reads this event for three months, should it be removed?

Analysis: work backward from the result, not from clicks

I usually look at six views, in this order.

First, the funnel. A funnel shows where loss happens, but the largest drop is not automatically the most important problem. Some steps are intentional filters. Judge the drop against the product intent.

Second, segmentation. New and returning users, organic and paid traffic, low-end and high-end devices, different languages or regions may behave differently. Segmentation is useful only when different segments imply different actions.

Third, cohorts. Whether users return after their first successful experience often says more about value than the first click. If we only look at same-day conversion, we may mistake a short-term stimulus for durable growth.

Fourth, frequency and depth. Averages often hide the issue. A small group of heavy users can lift the average while most users try the feature once. Look at the distribution, not only the mean.

Fifth, performance and errors. Slow pages, recording failures, payment callback exceptions, and rejected permissions are product issues, not only engineering issues.

Sixth, paths and feedback. Data can locate the abnormal step, but it does not always explain why. Path review, search terms, support tickets, app store reviews, and small usability tests can help interpret the cause.

Five layers for diagnosing product problems

When a metric looks weak, I check five layers:

1. Reach: the user never saw it. Check entry placement, exposure rate, notification permission, and recommendation rules.
2. Motivation: the user saw it but did not want to click. Check copy, value expression, and audience fit.
3. Understanding: the user clicked but left quickly. Check information hierarchy, terminology, expectation setting, and guidance.
4. Ability: the user wanted to proceed but could not. Check forms, permissions, network, performance, error codes, and payment flow.
5. Value: the user succeeded but did not return. Check result quality, reward feedback, content supply, and usage cadence.

This order keeps me from changing the UI too early. Low conversion is not always a button problem. It may come from a wrong entry point, a permission failure, or a feature that is not valuable enough.

Example: a shadowing feature in a language learning app

This is an example, not a real project and not real data.

Assume a language learning app has a shadowing feature. I would track key events such as lesson_enter, shadowing_exposure, shadowing_start, permission_result, record_complete, score_result, retry_click, and next_lesson.

If shadowing_exposure is high but shadowing_start is low, I would first check the entry copy, value proposition, and timing. If shadowing_start is acceptable but record_complete is low, I would inspect microphone permission, recording length, network, and crashes. If record_complete is normal but score_result fails often, the speech evaluation service, timeout handling, and error message need attention. If scoring succeeds but users do not continue to the next lesson or come back later, the problem may not be the flow. The feedback may not create enough learning motivation.

The point is not the number of events. The point is that every break in the chain maps to a concrete action.

Keep the dashboard small

A healthy core dashboard usually contains four parts: one stage objective, three to five leading metrics, one quality metric, and one data health module. A dashboard is not a place to show effort. It is a tool to reduce arguments.

Ownership also matters. Product owns questions and definitions. Engineering owns the event contract and reporting quality. QA owns journey verification. Operations or data colleagues own the analysis cadence. In a small team, one person may wear several hats, but the definitions and contracts still need to be written down.

How I would execute

Step one: write a one-page tracking plan with only the current business question, core journey, and decision scenarios. Step two: define naming and property conventions. Step three: put the event contract into the requirement instead of adding it before release. Step four: validate data during gray release and keep untrusted metrics out of the core dashboard. Step five: review data weekly and discuss only anomalies, cause hypotheses, and next actions.

The value of instrumentation is not proving that we were right. It is finding out earlier where we were wrong. Data without a decision attached will eventually become noise.

PaxLee