How to tell which growth idea actually worked

The month nobody could answer a simple question

A founder I worked with in Karlín had shipped eleven growth changes in six weeks. New onboarding, a pricing page rewrite, a referral prompt, three lifecycle emails, a Product Hunt launch, some paid tests. Signups were up 34%. He asked me which of the eleven had done it.

We spent a Saturday on it and the honest answer was that nobody could tell. He had 340 distinct event names, most of them fired from one place and never queried. He had no stable identifier before signup, so every anonymous visitor who came back a week later and converted looked like a brand new person arriving direct. The referral prompt had shipped in the same deploy as the onboarding change. The Product Hunt spike overlapped the pricing test.

That is the normal state of things, and it is not a tooling problem. He had a perfectly good analytics vendor. It is a design problem, and it takes roughly a day to fix.

Twenty events, not three hundred

The instinct with event tracking is to log everything, on the theory that you cannot analyse what you did not capture. In practice a large event catalogue is worse than a small one, because nobody remembers whether checkout_started, begin_checkout or cart_submit is the live one, and queries quietly undercount.

Start from the questions instead. Write down the five you will be asked at the end of every month. For most B2B products they are close to: how many qualified people arrived, how many started, how many reached first value, how many paid, how many were still there at day 30. Then instrument exactly the events those questions need, plus the two or three diagnostic events for the step you are currently trying to fix.

That lands at fifteen to twenty five events. My default spine looks like this.

• page_viewed with path, referrer and the full query string preserved
• signup_started and signup_completed
• activation_reached, defined as the single action that correlates with retention, not the one that flatters you
• invite_sent, invite_accepted
• subscription_started, subscription_cancelled with a reason property
• one feature_used event with a feature property, rather than thirty named events

That last one matters more than it looks. Properties are cheap and renameable; event names are load bearing and end up hardcoded in dashboards, warehouse models and someone's Zapier automation. Push variation into properties and keep the name count low.

Name them object_verb, past tense, snake case, and write the convention into a file the linter checks. I use a small events.ts module that exports typed functions, so nothing in the codebase calls the analytics SDK directly. Free text event names in a React component are how you get 340 of them.

If an event has never appeared in a query, it is not data. It is exhaust.

Identity stitching is where the money leaks

The single largest source of wrong growth conclusions in small companies is broken identity. Someone reads your essay on a phone at the weekend, comes back on a laptop on Tuesday, signs up, and your funnel records a direct-traffic signup with no prior touch. Every content and community channel is systematically undercredited, and paid, which lands people on a conversion page inside one session, is systematically overcredited. I have seen teams cut the channel that was actually driving 40% of pipeline because of exactly this.

The fix is three pieces of plumbing, none of them clever.

First, set your own first party anonymous id in a cookie on your own domain, server side, with a one year expiry. Do not rely on the vendor SDK's client side cookie; Safari caps client set cookies at seven days, which quietly truncates any consideration cycle longer than a week. A Set-Cookie from your own backend on a first party domain is not subject to that cap.

Second, call identify at every point where you learn who someone is, not just at signup. Email link click, login, support conversation, invite acceptance. Each call merges the anonymous id into the known user.

Third, store first touch attribution as a user property that is set once and never overwritten. Capture first_utm_source, first_referrer, first_landing_path and the timestamp on the first ever pageview of that anonymous id, and set them with a set-once operation. Keep the last touch separately. Then look at both. The gap between them is the interesting part: last touch tells you what closed, first touch tells you what created demand, and for a founder-led business the two are almost never the same channel.

One more thing that costs nothing: attach a deploy or experiment identifier to every event. If you ship a change behind a flag, the flag value belongs on the event as a property. Without it you are comparing calendar weeks, and calendar weeks contain holidays, outages and other people's launches.

The three dashboards

Most teams have eleven dashboards and read none of them. Three is the right number, because three can be read in ten minutes on a Monday and anything not read is not a dashboard, it is a screensaver.

One: the funnel by cohort of arrival. Arrived, started, activated, paid, retained at 30 days, split by week of first touch. Not by week of the event. This is the difference between a chart that tells you whether last month's cohort was better than the one before, and a chart that mixes six months of users into one bar and hides everything. Add absolute counts next to the percentages. A conversion rate that improved from 4% to 7% on eleven visitors is not a result.

Two: the channel table. Rows are first touch source. Columns are visitors, signups, activated, paid, revenue, and cost where there is a cost. Sorted by paid, not by visitors. This is the table that stops you optimising a channel that produces enormous traffic and no customers. I keep a second copy sorted by last touch and read them side by side once a month.

Three: the change log with a marker per shipped thing. A simple time series of your one north star metric, with vertical markers for every growth change you shipped and a one line note. It sounds trivial. It is the dashboard that actually gets used, because it is the only one that connects a number moving to a human decision. Keep the log in the repo as a markdown file with a date, a description and the flag name, and render markers from it.

That is it. No cohort heatmaps, no path analysis, no session recordings on the dashboard. Those are investigation tools you reach for when one of the three raises a question.

What this rules out, and what it does not

Be honest about the limits. With twenty events and clean identity you can tell whether a cohort improved and roughly where it improved. You cannot cleanly separate two changes shipped in the same week to the same audience. At a few hundred signups a month you do not have the volume for a statistically clean A/B test on anything but the top of the funnel: a test on a 3% conversion rate needing a 20% relative lift wants somewhere north of 15,000 visitors per arm. Most people reading this do not have that in a quarter.

So do the thing that works at your size. Ship one substantial change per week, not eleven. Put a marker in the change log. Watch the arrival cohort for four weeks. Where the change is genuinely reversible and the effect should be immediate, hold back 20% of users as a holdout rather than running a formal split, and compare. A holdout is cruder than a test but it survives low volume, because you are comparing two groups that experienced the same week.

Do this on Monday

Open your analytics tool and export the list of event names with their volume over the last 90 days. Sort ascending. Everything with fewer than 100 events, or that appears in no dashboard, gets deleted. Then write events.ts with the fifteen that survive, typed, and grep the codebase for direct SDK calls until there are none left. Then set the first party cookie server side and add the set-once first touch properties.

By Friday you will know something you did not know on Monday: which channel your paying customers actually first heard about you from. In every engagement I have run, that number surprises the founder. Usually it is the thing they were about to stop doing.