mockcrowd

io.github.phanisaimunipalliPublished by GitHub account phanisaimunipalliv0.3.0Updated Nov 8, 2026

Test a product idea on a crowd of AI personas grounded in real posts. Returns a verdict.

VerifiedSTDIODesktop onlyData & AnalyticsBusiness & CommerceAI & ML

Overview

AI-generated overview

Runs a simulated crowd of AI personas built from real social posts to test a product idea and return a verdict on who would pay.

What you can do
  • Run a validation on an idea with a price or free framing, returning verdict, would-pay and would-try shares, lift over a control pitch, confidence, top objections and the full report
  • Retrieve a past run's report and list earlier runs with their verdicts
  • Record what actually happened after a run so it becomes a backtest case
  • List which AI backends are available to power the simulation
When to use it
Use it before building something, to rank ideas, surface objections and decide what to test for real. It suits founders and product teams who want a quick, evidence-grounded read on demand rather than a revenue forecast.
Requirements
Local Python 3.10+ install (pip or uvx) with the mcp extra; runs as a local stdio process. Needs an AI backend: Claude Code, Codex CLI, Code Puppy, a local Ollama server, or an OpenAI-compatible endpoint configured with MOCKCROWD_API_BASE, MOCKCROWD_API_KEY and a model. Reddit voices need REDDIT_CLIENT_ID and REDDIT_CLIENT_SECRET; GITHUB_TOKEN is optional for higher rate limits. Network access for post collection.
Before you install
Runs make many LLM calls (roughly 100 to 150 for 500 personas), which can cost money on paid API plans. Reddit credentials are required for Reddit voices and free access is non-commercial only. Codex runs in a read-only sandbox that still allows reads, so avoid it on machines with sensitive files. It is a simulation, not a revenue forecast, and famous backtest products may be recognized from model memory. Runs are saved locally under the user's home directory.

Installation

In SourceWeft

  1. Open mockcrowd in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

[mockcrowd: validate your product idea before you build. A crowd of colored dots drifts as opinions shift]

[ci] [python] [license]

[Replay of a real mockcrowd run: persona characters debate a vinyl scanning app in a chat, votes flip from Try free to Pass, ending in a weak demand verdict]

Validate your product idea before you build. mockcrowd builds a crowd of hundreds of AI personas from real posts on Reddit, Hacker News and Bluesky, lets them argue about your idea in a simulated Reddit thread, and tells you how many would really pay, why the rest would not, and how far to trust the answer.

It runs on your machine, on the AI you already have: Claude Code, Codex, Code Puppy, Ollama, or any OpenAI compatible API.

bash
mockcrowd validate "An app that scans your vinyl records with the phone camera to catalog your collection and track prices" --price "\$3.99/month"

Real output from a small test run (20 personas, 1 discussion round, Claude, Hacker News and Bluesky voices):

[mockcrowd discussion: personas debate a vinyl scanning app in a simulated r/vinyl thread; would try free falls from 60% to 35%, would pay from 5% to 0%, with the top objections]

[Terminal recording of a real 20 persona mockcrowd run, sped up 9x, ending in a weak demand verdict]

Alone, 60% said they would try it free and 5% would pay. After the discussion, 35% would try it and nobody would pay. That is the kind of answer you want before you spend a month building.

Why it is different

Problem with asking an LLM "would people buy this?"What mockcrowd does
Models are agreeable and say yes to everythingEvery persona also rates a bland control pitch in the same blind vote. The verdict is the lift over the control, not the raw yes rate.
Invented personas sound like marketing copyEvery persona is built from real posts and cites them. The stance mix of the crowd (positive, neutral, skeptical, negative) is matched to the mix measured in those posts.
One voice, no social dynamicsA Reddit style discussion: threads of about 20 personas, a ranked feed, and a fixed action space (comment, reply, upvote, downvote, do nothing), following the design of OASIS. You see the vote before and after people talk.
No way to know if the tool is rightEvals and a backtest on products with known outcomes (crowdfunding records and well known flops), plus your own recorded outcomes.

Install

bash
pip install mockcrowd# or: uv tool install mockcrowdmockcrowd brains        # shows which AI it can use

Python 3.10 or newer. One small dependency (certifi).

Once a day mockcrowd checks PyPI and prints one line when a newer version is out. It never installs anything. Upgrade with pip install -U mockcrowd or uv tool upgrade mockcrowd. Turn the check off with MOCKCROWD_NO_UPDATE_CHECK=1.

Pick the brain

mockcrowd uses the first one it finds, or the one you pass with --brain.

BrainHow it runsSetup
claudeclaude -p on your Claude Code planClaude Code installed
codexcodex exec on your ChatGPT or API planCodex CLI installed
code-puppycode-puppy -p with your configured modelCode Puppy installed
ollamaLocal models, free and private, slowerollama serve, then --model <name>
openaiAny OpenAI compatible endpoint (OpenAI, OpenRouter, LM Studio, vLLM)MOCKCROWD_API_BASE, MOCKCROWD_API_KEY, --model

Claude has been run live end to end. The Codex, Code Puppy, Ollama and OpenAI adapters are tested against the flags and API each tool documents, but not yet in a live run; reports are welcome. Claude and Code Puppy run with tools off. Codex runs in its read only sandbox, which blocks writes but not reads, so prefer the others if your machine holds sensitive files.

--model is passed through exactly as you write it. A 500 persona run makes roughly 100 to 150 LLM calls, because one call steps a whole batch or thread of personas. A faster, cheaper model makes runs quicker. OPENAI_API_KEY is only used with api.openai.com; other endpoints need MOCKCROWD_API_KEY.

Real voices

SourceKey neededNotes
RedditYes, freeCreate a "script" app at reddit.com/prefs/apps and set REDDIT_CLIENT_ID and REDDIT_CLIENT_SECRET. Reddit blocks keyless requests. Free access is for non-commercial use.
Hacker NewsNoComments through the Algolia search API
BlueskyNoPublic post search
GitHubNoIssues and their top comments, only when the audience is developers. Set GITHUB_TOKEN for a higher rate limit.
App StoreNoOpt in with --source appstore. Apple's public review feed returned no reviews when last tested.
Anything elseX, Threads, Instagram, Discord, surveys, support tickets, interview notes: put the posts in a file and pass --voices posts.jsonl (JSONL, CSV or text).

Each source that fails is listed in the report, and the confidence score drops when evidence is thin.

Use it from your coding agent

bash
mockcrowd install-skill                    # Claude Code: ~/.claude/skills/mockcrowdmockcrowd install-skill --dir <skills dir> # any agent that reads SKILL.md files

Then ask: "validate my idea for a vinyl collection app at $3.99 a month". The agent runs mockcrowd, and when a source is blocked it can gather posts with its own web tools and pass them in.

Use it from any AI agent (MCP)

bash
pip install "mockcrowd[mcp]"claude mcp add mockcrowd -- mockcrowd mcp   # Claude Code

Any other MCP client (Cursor, Codex, Claude Desktop, your own agent) can start it with the command mockcrowd mcp over stdio:

json
{"mcpServers": {"mockcrowd": {"command": "mockcrowd", "args": ["mcp"]}}}

To always run the newest release without installing, use uvx: {"command": "uvx", "args": ["--python", "3.12", "mockcrowd[mcp]@latest", "mcp"]}.

ToolWhat it does
validate_ideaRuns a crowd on an idea and returns the verdict, would pay and try free shares, lift, confidence, top objections and the full report
get_reportThe report of a past run
list_runsPast runs with their verdicts
record_outcomeWhat really happened after a run, which becomes a backtest case
list_brainsWhich AI backends are available

Agent runs default to 60 personas and 1 round to keep them quick.

Free ideas and open source features

For something nobody pays for, such as an open source feature or a free app, add --free. The crowd is asked whether they would use it instead of whether they would pay, and the verdict comes from interest after the discussion compared with a bland control. To backtest on a real feature request, add --exclude github.com/owner/repo/issues/123 so the crowd never sees the issue it is being scored against.

Commands

bash
mockcrowd validate "idea" [--price P | --free] [--audience A] [--agents 500] [--rounds 2]                          [--voices file.jsonl] [--no-live] [--source reddit|hn|bluesky|github|appstore]                          [--exclude TEXT]                          [--question "choice:first:Which feature first?:scan|alerts|wishlist"]                          [--brain auto|claude|codex|code-puppy|ollama|openai] [--model M]mockcrowd collect "idea" -o voices.jsonl   # only gather real postsmockcrowd runs                             # list past runsmockcrowd report <run-id> [--json]mockcrowd outcome <run-id> --demand high|medium|low --note "what really happened"mockcrowd eval --suite quick|full|backtest [--gate]

Runs are saved in ~/.mockcrowd/runs/<run-id>/ as report.md and run.json (every voice, persona, vote and comment).

How a run works

[Plan, collect, label, design, blind vote, discuss, report]

  1. Plan. The brain reads your idea and lists the segments (including people who would not buy), subreddits and search queries.
  2. Collect. Real posts from the sources above.
  3. Label. Each post gets a segment, relevance, pain level and stance. Irrelevant posts are dropped.
  4. Design. Personas are written in batches from samples of real posts, in the segment shares and stance mix those posts show. Each persona cites its posts.
  5. Blind vote. Every persona answers alone: interest, would try free, would pay, biggest objection, plus your own typed questions. The real idea and a bland control appear side by side in random order.
  6. Discuss. Threads of about 20 mixed personas, one action each per round on a ranked feed.
  7. Final vote and report. Shares with 95% intervals, lift over control, shift after discussion, by segment results, objection themes and a confidence score.

Evals and continuous improvement

mockcrowd eval scores the simulator itself and appends to ~/.mockcrowd/evals/history.jsonl. With --gate it exits 1 when any metric falls more than 5 points below its best.

MetricWhat it checks
grounding, citation_validPersonas are built from real posts, and their citations exist
mix_fidelityThe crowd's stance mix matches the real posts
diversityPersonas are not copies of each other
control_lift, agreeablenessThe crowd does not reward a bland idea
persuasion_driftHow much the discussion moved the vote
stabilityRerunning the vote gives the same answer
realismA judge cannot tell real posts from persona comments
backtest_accuracy, backtest_aucKnown hits score above known flops
leakageHow many backtest products the brain recognizes from memory

The backtest ships with 13 cases: six with record crowdfunding or user growth (for example Pebble Time, Exploding Kittens, Flow Hive) and seven well known flops (for example Juicero, Quibi, Fire Phone). Each case cites its source. Pitches are anonymized, and the leakage metric reports how often the model recognizes them anyway.

Your outcomes are the strongest signal. After you test or launch, run mockcrowd outcome <run-id> --demand .... The run becomes a backtest case, and future confidence scores include how well mockcrowd did on cases like yours.

Honest limits

  • It is a simulation. Use it to rank ideas, find objections and choose what to test for real, not to forecast revenue.
  • Personas reflect the people who post online, who are louder and more opinionated than most buyers.
  • Backtest products are famous, so models may know how they ended. Watch the leakage metric.
  • The verdict thresholds are starting points. The backtest reports how well they separate hits from flops on your brain.

Credits

Simulation design follows OASIS by CAMEL-AI (Apache 2.0). The idea of building a world from seed material and simulating it was inspired by MiroFish. No code from either project is included. See NOTICE.

License

Apache 2.0

Source: README.md at commit 92785bb

Tools

0
SourceWeft has not started this package in its sandbox yet, so its tools are not known.

Version history

1
  1. v0.3.0LatestNov 8, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts