Evidence Based Idea Validation: What Counts as Proof

You describe your idea to four friends. Two of them say they would definitely use it. One asks when it launches. You close the laptop feeling validated, and three months later nobody signs up.

That gap is the whole problem with how most people do validation. A friendly yes costs nothing to give. Evidence has to cost the person something, or it is not evidence at all.

What Counts as Evidence

Evidence based idea validation means one thing: every claim you make about the problem is tied to something a real person wrote or did, and you can point to it on demand.

That rules out a lot of what passes for research.

• Not evidence: friends agreeing with you, a poll of your own followers, a post you wrote and people liked, a market size figure pulled from a report you did not read.
• Weak evidence: someone saying “that sounds useful”, a single complaint with no workaround behind it, a feature request with no situation attached.
• Real evidence: a person describing how they handle the problem today, money they already spend, a workaround they built and maintain, or a sentence where they ask for a better option.

The test is simple. Can you quote the line and link to it? If the answer is no, you are holding an opinion, not proof.

How Strong Each Piece of Evidence Is

Not all proof weighs the same. When two findings point in different directions, the stronger one should win.

• Behavior beats opinion. Someone paying for a half solution outranks ten people saying they would pay.
• A specific story beats a general complaint. Details date the problem to a real workflow.
• Repeated evidence from separate people beats one detailed thread. One person can be unusual.
• Evidence from more than one community beats evidence from one. Different rooms, same complaint.
• Evidence that survives your attempt to disprove it beats evidence you only collected to confirm. This is where counter evidence does its work.

A finding that only exists because you went looking for it should be held loosely.

A Worked Example: Monthly Reconciliation for Bookkeepers

Say your idea is a small tool that reconciles bank statements for solo bookkeepers. Here is how a validation pass looks from start to finish.

1. Write your claim in one sentence: “Solo bookkeepers lose hours every month matching bank lines to invoices by hand.”
2. List the terms a frustrated person would actually type, not the terms a vendor would use. Try “bank reconciliation takes forever”, “matching transactions by hand”, “how do you reconcile multiple accounts”, “statement to invoice matching”.
3. Search each term in the communities you can reach and save every thread that looks relevant. Copy the raw text somewhere you control.
4. Pull out the lines that describe a situation. A line like “I do this for eleven clients and it eats my first week of every month” is evidence. A line like “would be cool if it synced” is not.
5. Group the lines by the underlying problem, not by the wording. Three different phrasings can be one problem.
6. Write down the strongest three quotes with their links. If you cannot find three, that itself is a result.
7. Spend one hour trying to break your own claim, using the counter evidence pass below.
8. Only then decide. Either the claim holds with quotes attached, or it changes.

The whole pass fits into an evening. It is not glamorous, but it is the difference between a guess and a position you can defend.

Recording Evidence So It Survives

Evidence that lives in your head disappears the moment you get excited about a different idea. Keep a single file per idea and store the same fields every time.

• The claim you are testing, in one sentence.
• Each quote, word for word, with a link to where it came from.
• Which community it came from.
• Whether it describes behavior or only an opinion.
• What would have to be true for the quote to be misleading.

That last field matters more than it looks. Writing down the weak spot while you are calm is much easier than noticing it while you are building.

When Evidence Contradicts Itself

It usually will. You will find one thread full of people desperate for a fix and another where someone says the manual way is fine, thanks.

Do not average the two. Ask instead whether the disagreement maps to a difference in the people. Freelancers and agencies are not the same buyer. A solo bookkeeper with three clients may find the monthly task trivial; one with eleven clients may dread it.

If the split follows a line you can name, your idea is probably fine but your audience is too broad. If the split follows no line at all, you have not understood the problem yet.

FAQ

How many pieces of evidence do I need before building?

There is no magic count. What matters is whether the evidence describes behavior and whether it repeats across separate people. Ten opinions are weaker than three stories about a workaround someone built.

Is a landing page test evidence based idea validation?

Only partly. A signup is a weak signal, because it costs one click and no thought. It becomes stronger when the person tells you what they do today and why the current option fails them.

What if I can only find weak evidence?

That is a useful answer. It usually means you are asking about a problem people tolerate, or you are looking in the wrong community. Try a different room before you try a different idea.

If you want the evidence gathered and checked for you, the research tool will do the boring part: it collects posts from Reddit’s public archive, Hacker News, Stack Overflow, and GitHub, groups the duplicates, and throws away any quote it cannot match against the source text. A report takes two to four minutes and costs 2.99 EUR once. To go deeper, read How to Score Pain Intensity in Customer Research and Counter Evidence: The Step Most Founders Skip.