Iphone Mirroir Mcp

io.github.jfarcand由 GitHub 账号 jfarcand 发布v0.4.1更新于 Oct 27, 2026

Control a real iPhone through macOS iPhone Mirroring: screenshot, tap, swipe, type.

已验证STDIO仅桌面浏览器自动化开发者工具

概览

AI 生成的概览

通过 macOS 的 iPhone 镜像控制真实 iPhone,让 AI 助手截屏、读取界面文字并执行点击、滑动和输入。

可以做什么
  • 观察镜像的 iPhone 屏幕:截图、文字与图标识别并给出点击坐标、方向与健康检查
  • 操作手机:点击、滑动、输入文字、按键、多点触控手势、主屏、返回、应用切换器和聚焦搜索
  • 导航与管理应用:滚动直到目标文字出现、启动或重置应用、切换网络、测量操作耗时
  • 编写与运行技能:录制步骤、通过探索应用生成 SKILL.md、编译并回放流程、校准组件
  • 管理目标与技能:列出并切换自动化目标、列出和读取已保存的技能
适用场景
适合在 Mac 上让助手操作一台真实 iPhone,用于应用 QA 冒烟测试、教程演示、脚本化演示或重复性的应用内任务,无需手写坐标或脚本。
运行要求
需要 macOS 15 或更高版本、iPhone 镜像功能以及已连接的 iPhone。以本地 npm 包(npx mirroir-mcp)、Homebrew 或源码方式安装。首次截图时需授予 macOS 的屏幕录制与辅助功能权限。可选的 AI 视觉与诊断功能使用已有的 Copilot 或 Claude Code 登录,或 OPENAI_API_KEY、ANTHROPIC_API_KEY 等密钥。
安装前请注意
助手可以代你发送消息、创建日历事件、更改设置并在应用中输入内容,因此写入类工具默认隐藏,需在 permissions.json 中开启;可用 blockedApps 保护敏感应用。启用第三方视觉或诊断模式时,屏幕内容(含截图)会发送给这些模型。可关闭 iPhone 镜像作为紧急停止手段。安装脚本会把远程脚本直接交给 bash 执行。

安装

在 SourceWeft 中

  1. 打开 控制台中的 Iphone Mirroir Mcp,将其添加到工作区。
  2. 为需要使用其工具的对话启用该服务。

Desktop only,通过 STDIO。 STDIO 服务会启动本地进程,因此需要 SourceWeft 桌面宿主。

其他 MCP 客户端

参照 仓库 中的启动说明。

README

[mirroir-mcp]

mirroir-mcp

[npm version] [Build] [Install] [Installers] [MCP Compliance] [License] [macOS 15+] [Discord]

Give your AI eyes, hands, and a real iPhone. An MCP server that lets any AI agent see the screen, tap what it needs, and figure the rest out — through macOS iPhone Mirroring. Experimental support for macOS windows. 38 tools, any MCP client.

Vision: agents that test — and play

mirroir is growing archetype-aware coverage for games. A game splits into two surfaces. The shell — menu, shop, loadout, settings, win/lose screens — is ordinary text-labelled UI that today's skills already drive. The scene — the playfield — is a vision-driven loop an agent follows one touch at a time. The honest targets are game QA smoke tests, tutorial walkthroughs, and scripted demos, not real-time play: iPhone Mirroring exposes a single pointer, so twin-stick and twitch games stay out of reach (see Games).

Token reductions make the scene loop affordable. Screenshots are ~90% of an agent's cost, so text-rich shell screens read through describe_screen with no image at all, and a region/scale capture knob trims the per-frame image cost where a screenshot is unavoidable. On the playfield, describe_screen in vision mode sends the frame to an LLM through embacle; measured on a live Brawl Stars match, one frame costs ~24k prompt tokens and 3–14 s, which GitHub Copilot bills as a single premium request on Claude Sonnet 5.5.

The rollout is tracked as Gaming Surface A–E — shell coverage ships first; the scene loop is measured on a real device before it widens.

Requirements

Install

bash
/bin/bash -c "$(curl -fsSL https://mirroir.dev/get-mirroir.sh)"

or via npx:

bash
npx -y mirroir-mcp install

or via Homebrew:

bash
brew tap jfarcand/tap && brew install mirroir-mcp

The first time you take a screenshot, macOS will prompt for Screen Recording and Accessibility permissions. Grant both.

Per-client setup
Claude Code
bash
claude mcp add --transport stdio mirroir -- npx -y mirroir-mcp
GitHub Copilot (VS Code)

Install from the MCP server gallery: search @mcp mirroir in the Extensions view, or add to .vscode/mcp.json:

json
{  "servers": {    "mirroir": {      "type": "stdio",      "command": "npx",      "args": ["-y", "mirroir-mcp"]    }  }}
Cursor

Add to .cursor/mcp.json in your project root:

json
{  "mcpServers": {    "mirroir": {      "command": "npx",      "args": ["-y", "mirroir-mcp"]    }  }}
OpenAI Codex
bash
codex mcp add mirroir -- npx -y mirroir-mcp

Or add to ~/.codex/config.toml:

toml
[mcp_servers.mirroir]command = "npx"args = ["-y", "mirroir-mcp"]
Install from source
bash
git clone https://github.com/jfarcand/mirroir-mcp.gitcd mirroir-mcp./mirroir.sh

Use the full path to the binary in your .mcp.json: <repo>/.build/release/mirroir-mcp.

How it works

Every interaction follows the same loop: observe, reason, act. describe_screen gives the AI every text element with tap coordinates (eyes). The LLM decides what to do next (brain). tap, type_text, swipe execute the action (hands) — then it loops back to observe. No scripts, no coordinates, just intent.

Beyond the basics, the server exposes higher-level navigation and lifecycle tools: press_back navigates back by OCR-tapping the "<" chevron (with a canonical-position fallback), press_home/press_app_switcher/spotlight drive system gestures, scroll_to scrolls until a target text becomes visible, reset_app force-quits an app via the App Switcher, set_network toggles connectivity through Settings, and measure times how long an action takes to surface a target element. Multi-target setups use list_targets/switch_target to move between window automation endpoints. Skill authoring is served by record_step and save_compiled (compiled replay) and calibrate_component (test a component definition against the live screen). See the Tools Reference for all 38 tools.

Describe Your App

mirroir can explore any iOS app blindly, but it works better when you tell it what to expect. Write an APP.md file and mirroir reads it before exploration starts:

markdown
---app: Santéarchetype: dashboardobstacle_mode: auto---
## StructureDashboard with 4 tabs: Résumé, Partage, Parcourir, Profil.
## Résumé Tab- Summary cards for health metrics that drill down to charts- Cards often show "Aucune donnée" on test devices
## Obstacles- Health Access permission → tap "Autoriser"- Notification permission → tap "Ne pas autoriser"
## Skip- Supprimer les données de Santé- Réinitialiser

What the code actually uses today: archetype overrides recipe auto-detection; obstacles are auto-dismissed when obstacle_mode: auto; skip merges with permissions.json.skipElements; tabs (inline or as a section) are injected as high-priority targets; Structure + tab body + Tips become AI context in generated skills.

See the APP.md specification for the complete field list, loader resolution rules, and the permission-system bridge. Three levels of patterns work together — elements (what rows look like), screens (what the page layout means), and apps (what the developer knows). Patterns & Skills covers the full system.

Examples

Paste any of these into Claude Code, Claude Desktop, ChatGPT, Cursor, or any MCP client:

Open Messages, find my conversation with Alice, and send "running 10 min late".
Open Calendar, create a new event called "Dentist" next Tuesday at 2pm.
Open my Expo Go app, tap "LoginDemo", test the login screen with[email protected] / password123. Screenshot after each step.
Start recording, open Settings, scroll to General > About, stop recording.

Games

Games get three controls through iPhone Mirroring: touch holds a finger across calls for virtual joysticks and press-and-hold (begin, move, end); hold_keys holds keys such as w while dragging the right mouse button, for games that switch to keyboard and mouse controls like Roblox; pinch and rotate drive two-finger zoom and rotation in gesture-recognizer UIs. Mirroring exposes a single pointer touch to apps, so a touch-only game cannot receive two independent fingers — see Games and Multi-Touch for what was measured.

Screen Intelligence

describe_screen is the AI's eyes. Three backends work together to give the agent a complete picture of what's on screen — text, icons, and semantic UI structure.

Apple Vision OCR (default)

The default backend uses Apple's Vision framework to detect every text element on screen and return exact tap coordinates. This is fast, local, and requires no API keys or external services.

Icon Detection (YOLO CoreML)

Text-only OCR misses non-text UI elements — buttons, toggles, tab bar icons, activity rings. Drop a YOLO CoreML model (.mlmodelc) in ~/.mirroir-mcp/models/ and the server auto-detects it at startup, merging icon detection results with OCR text. The AI gets tap targets for elements that text-only OCR cannot see.

ModeocrBackend settingBehavior
Auto-detect (default)"auto"Uses Vision + YOLO if a model is installed, Vision only otherwise
Vision only"vision"Apple Vision OCR text only
YOLO only"yolo"CoreML element detection only
Both"both"Always merge both backends (falls back to Vision if no model)

AI Vision Mode (embacle)

Instead of local OCR, describe_screen can send the screenshot to an AI vision model that identifies UI elements semantically — cards, tabs, buttons, icons, navigation structure — not just raw text. This produces richer context for the agent, especially on screens with complex layouts.

The embacle runtime is embedded directly into the mirroir-mcp binary via Rust FFI. describe_screen calls the embedded runtime in-process — no separate server, no network round-trip, no additional setup. The FFI layer (EmbacleFFI.swift → libembacle.a) handles initialization, chat completion requests, and memory management across the Swift/Rust boundary.

embacle routes vision requests through already-authenticated CLI tools (GitHub Copilot, Claude Code) so there is no separate API key to manage. If you have a Copilot or Claude Code subscription, you already have access.

Install
bash
brew tap dravr-ai/tapbrew trust --tap dravr-ai/tap # recent Homebrew refuses to load untrusted tapsbrew install embacle          # CLI tools (embacle-server, embacle-mcp)brew install embacle-ffi      # Rust FFI static library (libembacle.a)

Both are optional. Without embacle-ffi the build links against Apple Vision and describe_screen uses local OCR.

Then rebuild mirroir-mcp from source (or reinstall via Homebrew) so the binary links against libembacle.a:

bash
# From sourceswift build -c release
# Or via Homebrew (rebuilds automatically)brew reinstall mirroir-mcp
Zero-config activation

When the embacle FFI is linked into the binary, screenDescriberMode defaults to "auto" which automatically resolves to vision mode. No settings change required — install embacle-ffi, rebuild, and describe_screen starts using AI vision.

To force local OCR even when embacle is available, explicitly set "ocr":

json
// .mirroir-mcp/settings.json{  "screenDescriberMode": "ocr"}

See Configuration for all available settings.

Skills

When you find yourself repeating the same agent workflow, capture it as a skill. Skills are SKILL.md files — numbered steps the AI follows, adapting to layout changes and unexpected dialogs. Steps like Tap "Email" use OCR — no hardcoded coordinates.

Place files in ~/.mirroir-mcp/skills/ (global) or <cwd>/.mirroir-mcp/skills/ (project-local).

APP.md

Describe your app's structure to guide exploration — see Describe Your App above and the full APP.md specification. Place APP.md files in ~/.mirroir-mcp/skills/ or the mirroir-skills repo at patterns/apps/.

markdown
---version: 1name: Commute ETA Notificationapp: Waze, Messagestags: ["workflow", "cross-app"]---
## Steps
1. Launch **Waze**2. Wait for "Où va-t-on ?" to appear3. Tap "Où va-t-on ?"4. Wait for "${DESTINATION:-Travail}" to appear5. Tap "${DESTINATION:-Travail}"6. Wait for "Y aller" to appear7. Tap "Y aller"8. Wait for "min" to appear9. Remember: Read the commute time and ETA.10. Press Home11. Launch **Messages**12. Tap "New Message"13. Type "${RECIPIENT}" and select the contact14. Type "On my way! ETA {eta}"15. Press **Return**16. Screenshot: "message_sent"

${VAR} placeholders resolve from environment variables. ${VAR:-default} for fallbacks.

Skill Marketplace

Install ready-to-use skills from jfarcand/mirroir-skills:

bash
git clone https://github.com/jfarcand/mirroir-skills ~/.mirroir-mcp/skills

From Exploration to CI

The generate_skill tool lets an AI agent explore an app and produce SKILL.md files. It uses breadth-first search (BFS) to traverse the app as a navigation graph — screens are nodes, tappable elements are edges. The explorer describes each screen, matches elements against component definitions to decide what to tap, visits child screens, and backtracks via the back chevron. Duplicate screens are skipped via structural fingerprinting. See Component Detection below for how the explorer interprets raw elements into structured UI components.

The explorer works viewport-by-viewport: after calibrating the page length, it builds a plan from the current viewport, taps elements top-to-bottom, scrolls down to reveal more content, and rebuilds the plan for each new viewport. This approach works with both OCR and AI vision describers. Pass seed for deterministic ordering across runs.

Exploration is bounded — it does not discover every reachable screen in large apps. Depth, screen count, and time limits keep runs practical. For targeted flows, provide a goal to focus the traversal.

mermaid
graph TD    A["Launch App"] --> B["Describe Screen"]    B --> C{"Calibrated?"}    C -- No --> D["Scroll Full Page"]    D --> E{"skip_calibration?"}    E -- No --> F["Component Detect +\nClassify + Validate"]    E -- Yes --> G["Classify Elements\nDirectly"]    F --> H["Build Plan"]    G --> H    C -- Yes --> H
    H --> I{"Untried\nElements?"}    I -- Yes --> J["Tap Element"]    I -- No --> K["Return to Root"]
    J --> M["Describe +\nClassify Edge"]    M --> N{"Transition"}    N -- new screen --> O["Add to Frontier"]    O --> P["Backtrack"]    N -- revisited/dead --> P
    P -- push: tap back --> H    P -- modal: tap close --> H    P -- tab: tap prev --> H
    K --> Q{"Frontier\nEmpty?"}    Q -- No --> R["Next Frontier\nScreen"]    R --> B    Q -- Yes --> S["Generate SKILL.md"]

Generate

Two modes: autonomous exploration (BFS) and guided session (manual step-by-step).

Autonomous BFS exploration — the agent explores on its own:

Explore the Settings app and generate a skill that checks the iOS version.

This calls generate_skill(action: "explore", app_name: "Settings", goal: "check iOS version") under the hood. The explorer launches the app, runs BFS from the root screen, and outputs a SKILL.md for the discovered path.

ParameterDefaultDescription
app_namerequiredApp to explore
goalnoneFocus exploration toward a specific flow (e.g. "check software version")
goalsnoneArray of goals — one SKILL.md per goal
max_depth6Maximum BFS depth
max_screens30Maximum screens to visit
max_time300Maximum seconds before stopping
strategyauto"mobile" (default), "social" (Reddit, Instagram, TikTok), or "desktop" (macOS windows)
explorerbfsExploration algorithm: "bfs" (breadth-first, default) or "dfs" (depth-first)
skip_calibrationfalseSkip component detection during calibration. Scrolling still runs. Useful with AI vision describers that produce clean semantic elements
seedrandomInteger seed for deterministic exploration ordering. Same seed produces identical tap sequences
freshtrueDiscard persisted navigation graph and explore from scratch. Set false for incremental exploration

Guided session — the AI navigates manually, capturing each screen:

  1. generate_skill(action: "start", app_name: "MyApp") — launch app, OCR first screen
  2. Use tap/swipe/type_text to navigate, then generate_skill(action: "capture") to record each screen
  3. generate_skill(action: "finish") — assemble captured screens into a SKILL.md

Test

Run skills deterministically from the CLI — no AI in the loop. A YAML skill runs its own steps. A SKILL.md's body is prose for an AI, so it runs the steps recorded in its .compiled.json when an MCP client first ran it (get_skill, then record_step after each step and save_compiled at the end). An uncompiled SKILL.md is refused with that instruction, one edited since it was compiled is refused as stale, and a run with no skill arguments skips any SKILL.md that has no compiled file:

bash
mirroir test apps/settings/check-aboutmirroir test --junit results.xml --verbose        # JUnit outputmirroir test --dry-run apps/settings/check-about    # validate without executing
OptionDescription
--junit <path>Write JUnit XML report
--report-json <path>Write a Playwright JSON-reporter document — the format mirroir-run ingests
--capture <key>Attach each skill's final-screen OCR text to the --report-json document under cross_surface.<key>, with the structural fingerprint of the start screen and of each step's settled screen under screens (start, 0, 1, …) and the window under device
--compiled-out <path>After the one skill given passes, write its .compiled.json to <path>: each step that ran as a passthrough step carrying the screen it left (arrivalFingerprint), the start screen as screenFingerprint. A failed run, or a pass whose start screen or any step's screen could not be read, writes nothing and exits 1. Needs --compiled-source; refused with --dry-run or more than one skill
--compiled-source <file>The file whose SHA-256 the --compiled-out skill records as its source — the scenario mirroir-run rendered the skill from
--screenshot-dir <dir>Save failure screenshots (default: ./mirroir-test-results/)
--timeout <seconds>wait_for timeout (default: 15)
--verboseStep-by-step detail
--dry-runParse and validate without executing (refused with --report-json: a dry run has no verdict to report)
--confirm-destructiveRun steps that send, remove, spend, or reset something on the device. Without it, a run holding one is refused before any step executes, naming each. reset_app: counts as destructive except when the very next step is a launch: of the same app (exact name, case-sensitive after trimming): that pair only returns the app about to be driven to its root, the way every emitted .ios.yaml flow opens
--no-compiledSkip compiled skills, force full OCR
--agent <name>AI diagnosis of a failed step (see AI-Assisted Diagnosis)

Exit code 0 = all pass, 1 = any failure, 2 = refused arguments (an unknown option is named, never ignored). A skill whose every step was skipped is a failure, and a skill holding a step the parser cannot read (an unknown verb, a malformed target:) or cannot use (a drag: naming neither end, a measure: with no action) is refused before anything runs, --dry-run included. A skill holding an AI-only step (verify, remember, condition, repeat, summarize) fails with not evaluated: <step> needs an AI client: the CLI has no AI to check it, so it never reports that skill as passed.

mirroir test only reads .compiled.json files, never rewrites them; it writes one only to the path --compiled-out names. A compiled file stale for its source, format version or window size is named and the skill runs on live OCR.

Compiled Skills

Compile a skill once to capture coordinates and timing. Replay skips OCR for every step with a recorded hint; assertions still re-check the screen, and steps without a hint run live.

bash
mirroir compile apps/settings/check-about        # compilemirroir test apps/settings/check-about            # auto-detects .compiled.jsonmirroir test --no-compiled check-about            # force full OCR

mirroir compile re-records coordinates by running a skill's steps on the device. For a SKILL.md those are the steps in its existing .compiled.json, so the first compile of a SKILL.md always happens through an MCP client.

An AI agent compiles a skill on its first MCP run by calling record_step after each step and save_compiled at the end. See Compiled Skills for details.

Two different files are named .compiled.json. The one next to a skill, written by save_compiled or mirroir compile, is the speed cache mirroir test replays. The <scenario>.compiled.json next to an .ios.yaml in a consumer repo's .mirroir/ is the screen-drift baseline mirroir-run records itself on a passing run; the runner never replays from it, every ios block runs live. So save_compiled with emit: true writes the skill's .compiled.json and a .mirroir/ scenario, never a .compiled.json into .mirroir/.

AI-Assisted Diagnosis

When a test step fails, pass --agent to get an AI diagnosis of what went wrong and suggested fixes:

bash
mirroir test --agent gpt-5.3 apps/settings/check-aboutmirroir test --agent claude-sonnet-5-5 apps/settings/check-aboutmirroir test --agent ollama:llama3 apps/settings/check-aboutmirroir test --agent embacle apps/settings/check-about

Built-in agents:

AgentProviderAPI Key
gpt-5.3OpenAIOPENAI_API_KEY
claude-opus-5-5, claude-sonnet-5-5, claude-haiku-5-5AnthropicANTHROPIC_API_KEY
ollama:<model>Ollama (local)None
embacle, embacle:claudeembacle-serverCLI agent key

How a request reaches its agent: Anthropic, OpenAI and Ollama agents always call their own HTTP API. An embacle agent whose model is a bare Copilot model id (embacle sends claude-sonnet-5.5) runs on the embedded embacle runtime when libembacle is linked and agentTransport is not "http", and over HTTP to base_url otherwise. An embacle model in embacle-server's runner:model form always goes over HTTP, because the embedded runtime is the Copilot runner only: embacle:claude sends claude:sonnet, which selects embacle-server's Claude Code runner, so it needs embacle-server listening on http://localhost:3000. A bare runner name as the model (claude, copilot, …) means that runner's default model and also goes over HTTP.

Custom agents are defined as YAML profiles in <cwd>/.mirroir-mcp/agents/<name>.yaml (project-local) or ~/.mirroir-mcp/agents/<name>.yaml (global, project-local wins). Pass the file's <name> to --agent.

A profile runs in one of two modes:

mode: api — call a cloud or local HTTP endpoint:

yaml
# ~/.mirroir-mcp/agents/my-gpt.yamlmode: apiprovider: openai          # anthropic | openai | ollama | embaclemodel: gpt-5.3api_key_env: OPENAI_API_KEYbase_url: https://api.openai.comsystem_prompt: "You are a terse iOS automation debugger."max_tokens: 4096

mode: command — run a local CLI process (e.g. an already-authenticated claude -p / copilot -p, or a custom script). The diagnostic payload is delivered two ways: if any args entry contains ${PAYLOAD}, the JSON payload is substituted there; otherwise it is piped to the command's stdin:

yaml
# ~/.mirroir-mcp/agents/claude-cli.yamlmode: commandcommand: claudeargs: ["-p", "${PAYLOAD}"]system_prompt: "Diagnose the failed step and reply with JSON."
KeyModeDescription
modebothapi (default) or command
providerapianthropic, openai, ollama, or embacle
modelapiModel identifier
api_key_envapiEnvironment variable holding the API key
base_urlapiEndpoint base URL
commandcommandExecutable to launch (PATH-resolved, or an absolute path)
argscommandArgument array; ${PAYLOAD} is replaced with the JSON payload
system_promptbothSystem prompt sent with the request
max_tokensbothResponse token budget
No API key? Use embacle

embacle routes requests through already-authenticated CLI tools (GitHub Copilot, Claude Code, etc.) — no separate API key needed:

bash
brew tap dravr-ai/tap && brew trust --tap dravr-ai/tap && brew install embaclemirroir test --agent embacle my-skill

Pattern System

The explorer uses a three-layer pattern system to understand iOS apps — the same declarative concept at different scales:

  • Element patterns (patterns/elements/) — 34 definitions matching row-level UI components (table rows, tab bars, toggles, summary cards). Each specifies match rules, interaction behavior, and grouping logic.
  • Screen patterns (patterns/screens/) — 7 archetype recipes that identify screen-level navigation models from element composition. Auto-detected during calibration, or declared via archetype in APP.md.
  • App patterns (patterns/apps/) — APP.md files with structure, obstacles, skip lists, and archetype declarations. The developer's source of truth.

Built-in archetypes: settings-list, dashboard, social-feed, content-grid, conversation-list, utility-display, detail-form.

Place custom patterns in ~/.mirroir-mcp/components/ (elements), ~/.mirroir-mcp/recipes/ (screens), or the mirroir-skills repo.

Vision Indicators

AI vision describers describe UI elements semantically ("Activité chevron") rather than character-by-character ("Activité" + ">"). A vision-indicators.md file maps these descriptions to OCR-compatible characters so the component pipeline works identically with both backends:

markdown
## Indicators- chevron: >- dismiss: ×- back: <

When a vision element ends with a mapped suffix (e.g. "Entraînements chevron"), the normalizer splits it into two elements: "Entraînements" + ">". Place vision-indicators.md alongside your component definitions.

See Component Detection for the full definition format, match rule reference, and the detection pipeline.

Replay anywhere with mirroir-run

mirroir-mcp drives the iPhone — AX + OCR + BFS exploration. mirroir-run replays .mirroir/ SkillStep scenarios against web, process, HTTP, and iOS surfaces: web through Playwright, subprocesses and HTTP natively, and each target: { kind: ios } block handed to mirroir-mcp test on a Mac with iPhone Mirroring connected. Both read one SkillStep grammar, so a single scenario can drive a browser and a phone and check that they show the same thing. A single Rust binary (runner/ in this repo); web, process and HTTP scenarios run on Linux CI with no macOS dependency. How the two binaries meet: Web and iOS in one scenario.

Drop a .mirroir/ directory in any repo and mirroir-run discovers it from the working directory:

your-app/└── .mirroir/    ├── mirroir.yaml          # the plan: must-pass / nice-to-pass entries    └── apps/<sample>/        ├── SAMPLE.md         # how to boot the app under test        ├── APP.md            # structure, obstacles, skip lists        └── scenarios/*.yaml  # SkillStep flows to replay
bash
cd your-appmirroir-run                          # discover .mirroir/, replay the must-pass planmirroir-run --scenarios all          # include nice-to-pass entries too

Each plan entry either points at a local: sample tree or extends a shared archetype (archetypes: ["<pack>/<name>@<ver>"]) with per-instance vars: and boot:. An archetype captures a reusable app shape — say, an AI chat console — once, and parameterizes it per app.

Where the iOS leg comes from. generate_skill … emit=true (on finish or explore), save_compiled … emit=true (a skill an AI ran step by step) and mirroir record (a walk driven by hand) each write the flow into .mirroir/apps/<app>/ as a runnable iOS leg: a target: { kind: ios } scenario, a SAMPLE.md declaring it, and a must_pass plan entry. mirroir-run hands each ios block to mirroir-mcp test on a macOS host with iPhone Mirroring connected, and refuses it by name on any other host. Run the MCP from your consumer repo (or pass output_dir) so the tree lands in the right .mirroir/; emitting into ~/.mirroir (the runner's pack home) is refused. One scenario can hold both legs: a web block, an ios block, and a cross_surface: step whose captures: record the page's scrape and the phone's final screen live, then compare them — the run that checks the gate produces both halves. See runner/docs/mirroir-dotfile.md.

Web steps compile to a Playwright .spec.ts and run across chromium, firefox, and webkit. Selectors resolve three ways: raw CSS, Playwright engine prefixes (role=button[name="Save"], text=Welcome, xpath=…), or a bare label resolved in Playwright's own priority — role, label, placeholder, data-test, visible text. Every compiled spec also collects uncaught page errors and failed responses, so a page that throws is a failure even when every locator resolved. The spec, its config, and the trace / video / screenshot of a failure persist under target/playwright/<sample>/<scenario>/. Process and HTTP steps dispatch natively; an LLM judge step scores agent responses against expected signals, and drift detection catches output divergence vs. a baseline.

ModeCommand
Validate a scenario against the grammarmirroir-run --validate scenario.yaml
Compile the Playwright spec to diskmirroir-run --emit playwright scenario.yaml
Run one scenario end-to-endmirroir-run --run-scenario scenario.yaml
Boot a sample dir, run its scenariosmirroir-run --sample .mirroir/apps/foo
Hand an iOS block to a specific mirroir-mcpMIRROIR_MCP_BIN=/path/to/mirroir-mcp mirroir-run …
Standalone text drift checkmirroir-run --diff-text a.txt b.txt

Install mirroir-run

bash
# crates.iocargo install mirroir-run
# Homebrewbrew install jfarcand/tap/mirroir-run

Prebuilt binaries for macOS (Intel + Apple Silicon), Linux (gnu + musl), and Windows are attached to each runner-v* release. On Windows, subprocess teardown signals are not implemented, so process scenarios are not supported there. See runner/docs/ for the scenario grammar, SAMPLE.md schema, judge profiles, and CI integration.

Security

Giving an AI access to your phone demands defense in depth. mirroir-mcp is fail-closed at every layer.

  • Tool permissions — Without a config file, only the 12 read-only tools (screenshot, describe_screen, start_recording, stop_recording, get_orientation, status, check_health, list_targets, list_skills, get_skill, calibrate_component, classify_screen) are exposed. Every mutating tool — including press_back — is hidden from the MCP client entirely, so it never sees them.
  • App blocking — blockedApps in permissions.json prevents the AI from interacting with sensitive apps like Wallet or Banking, even if mutating tools are allowed.
  • No root required — Runs as a regular user process using the macOS CGEvent API. No daemons, no kernel extensions, no root privileges — just Accessibility permissions.
  • Kill switch — Close iPhone Mirroring to kill all input instantly.
json
// ~/.mirroir-mcp/permissions.json{  "allow": ["tap", "swipe", "type_text", "press_key", "launch_app"],  "deny": [],  "blockedApps": ["Wallet", "Banking"]}

See Permissions and Security for the full threat model.

CLI Tools

Recorder

Record a walk by hand and emit it as a replayable iOS scenario — the same .mirroir/apps/<app>/scenarios/<flow>.ios.yaml that generate_skill … emit=true writes, declared in SAMPLE.md and the plan:

bash
mirroir record --app Calculator --name calculator-add --output-dir ~/code/my-repo

The app is reset and launched first, so the walk starts where its replay starts. Tap, long-press, swipe and type on the mirrored phone, then press Ctrl+C. Each interaction is stored with the OCR text of the screen it acted on, so taps replay by label and each screen a tap reaches gets a wait_for:.

  • --app and --name are required; --name's slug names the scenario file.
  • --output-dir is the consumer repo root; without it the nearest .mirroir/ above the working directory is used (~/.mirroir is refused).
  • Every session is saved first to ~/.mirroir-mcp/recordings/<app>/<flow>.recording.json. When the emit is refused — a tap OCR could not name, say — the error names the step and the recording is kept.
  • --from-recording <file> re-renders a saved recording without the device (the file names its own app and flow, so --app/--name are refused with it).
  • --no-ocr is refused: a walk without OCR names no element for any tap.

Doctor

Verify your setup:

bash
mirroir doctormirroir doctor --json    # machine-readable output

Configure

Set up your keyboard layout for non-US keyboards:

bash
mirroir configure

iPhone Mirroring forwards physical keys, and the iPhone turns them into characters with its own hardware-keyboard layout. keyboardLayout names that layout, so type_text presses the key that produces each character there — on Canadian-CSA, / is the key left of Z, not US QWERTY's / key (which types é). Set it in settings.json or with the IPHONE_KEYBOARD_LAYOUT environment variable; it must match the layout chosen on the iPhone under Settings › General › Keyboard › Hardware Keyboard:

jsonc
// .mirroir-mcp/settings.json{ "keyboardLayout": "Canadian-CSA" }   // or "com.apple.keylayout.Canadian-CSA"; "" or "US" = US QWERTY

A character the layout types with a dead key (ê is ^ then e on Canadian-CSA) is typed as that key pair. A character the layout has no keys for (emoji, CJK, < on Canadian-CSA) goes through the clipboard, which needs Universal Clipboard.

Updating

bash
# curl installer/bin/bash -c "$(curl -fsSL https://mirroir.dev/get-mirroir.sh)"
# npxnpx -y mirroir-mcp install
# Homebrewbrew upgrade mirroir-mcp
# From sourcegit pull && swift build -c release

Uninstall

bash
# Homebrewbrew uninstall mirroir-mcp
# From source./uninstall-mirroir.sh

Configuration

All settings live in settings.json — project-local (.mirroir-mcp/settings.json) or global (~/.mirroir-mcp/settings.json). Project-local settings override global ones. Every setting also has a corresponding environment variable (e.g. MIRROIR_SCREEN_DESCRIBER_MODE).

json
{  "screenDescriberMode": "auto",  "agent": "embacle",  "ocrBackend": "auto",  "keystrokeDelayUs": 15000,  "explorationMaxScreens": 30}

The AI system prompts are built in; override one by dropping diagnosis.md or screen-describe.md into .mirroir-mcp/prompts/ (project-local) or ~/.mirroir-mcp/prompts/ (global) — see System Prompts.

See Configuration Reference for all 40+ settings covering screen intelligence, input timing, scroll behavior, exploration budgets, AI providers, and keyboard layouts.

Documentation

Tools ReferenceAll 38 tools, parameters, and input workflows
ConfigurationAll settings: screen intelligence, input timing, exploration, AI providers
FAQSecurity, focus stealing, keyboard layouts, embacle/vision mode
SecurityThreat model, kill switch, and recommendations
PermissionsFail-closed permission model and config file
Known LimitationsFocus stealing, keyboard layout gaps, autocorrect
Patterns & SkillsElement patterns, screen recipes, APP.md app descriptions, and the detection pipeline
Exploring a New AppStep-by-step playbook for onboarding a new app — APP.md, permissions, components, exploration goal
YOLO Icon DetectionRecommended YOLO models, CoreML setup, and configuration
Compiled SkillsZero-OCR skill replay
TestingFakeMirroring, integration tests, and CI strategy
Web and iOS in one scenarioHow mirroir-run hands iOS blocks to mirroir-mcp, and how cross_surface compares a browser with a phone
Cross-surface replaymirroir-run scenario grammar, .mirroir/ plan, SAMPLE.md, judge profiles, CI
TroubleshootingDebug mode and common issues
ContributingHow to add tools, commands, and tests
Skills MarketplaceSkill format, plugin discovery, and authoring

Community

Join the Discord server to ask questions, share skills, and discuss ideas.

Contributing

Contributions welcome. By submitting a patch, you agree to the Contributor License Agreement — your Git commit metadata serves as your electronic signature.

Limitation register

This repo enforces its honesty with llm-registre: deferral prose in production code ("not yet wired", "is the follow-up", "for now, return") is banned by CI unless the line carries a LIMITATION(registre#n): marker naming the limited item and pointing at an issue in the project's private register. The same gates enforce the 500-line file cap and the inline clippy allow-list, at pre-push and in CI. rg "LIMITATION\(registre#" is a complete inventory of what the codebase knowingly does not do. After cloning, run git submodule update --init and install ripgrep; see CONTRIBUTING.md for the workflow.


Why "mirroir"? — It's the old French spelling of miroir (mirror). A nod to the author's roots, not a typo.

来源:README.md,提交 6bd1e7f

工具

0
SourceWeft 尚未在沙箱中启动过该软件包,因此其工具未知。

版本历史

1
  1. v0.4.1最新Sep 16, 2026

评分

已满足适用检查项的 50%

该评分衡量此收录条目的完整程度,依据注册中心条目、端点与公开来源计算得出。它不是对服务器本身的评价,也不表示服务器安全或当前可用:可连通性是带日期的检查结果,评分也不包含安装量或用户评价。

  • 近 30 天内可连通25 分

    请保持端点能够响应 MCP 握手;SourceWeft 每 30 天探测一次。

  • 已声明认证方式10 分

    请在注册中心条目中声明凭据请求头,或保持端点能够响应探测,以便判定其认证方式。

  • 已列出工具15 分

    请让服务器在无凭据时响应 tools/list,或发布可在沙箱中无凭据启动的软件包。

  • 描述必需15 分
  • 图标10 分
  • 文档15 分
  • 许可证10 分