Framelet

Let an AI operate software while recording its screen and keyboard/mouse events, then query, extract and trim through MCP. Built for AI-driven software tests, bug reproduction, product showcases and walkthroughs. CLI and MCP are the primary interfaces; a native capture frame is also available. No Python, Node.js, .NET or FFmpeg installation is required.

More than a GIF: an interaction timeline

Framelet records keyboard, mouse and frame samples together by default, sharing UTC and monotonic clocks. A timeline and content-integrity manifest accompany the GIF.

• Unique identities and time: shared observation IDs link input events, original samples and GIF frames, with UTC references and explicit timing offsets.
• Input independent of FPS: several actions between samples can map to one frame. Unsampled visual changes cannot be recovered.
• Review and trim with context: locate actions by ID/time, inspect selected pictures, then trim or change speed while preserving original identities and computer-time references.

Keyboard records contain down/up transitions, key codes and scan codes, not clipboard or IME-committed text. Key events can reveal typed content; visual masks are not automatic password protection. Masked mouse coordinates are omitted. Timestamps describe observed intervals, not hardware synchronization; dwell candidates do not identify UI elements.

Small tool, useful capabilities

Static acceptance can use 1 FPS. Use 10–15 FPS only for explicitly size-first clips where smooth motion is secondary; human-facing demos default to 30 FPS, including README GIFs. Actual sampling depends on workload.

Measured review performance

v0.4.1 removes repeated segment checks, skips unused RGB conversions and builds candidate previews in one analysis pass. No runtime dependencies were added.

Windows, 10 alternating warm runs per version, medians. The group fixture is 64×64; the window fixture is 640×360/600 frames; Bilibili reuses the demo below. Timings exclude model inference and UI operation and depend on input/hardware. Methodology, hashes, memory tradeoffs and raw results.

Example 1 · Real window and recording

[Framelet capture frame]

The original small real recording: 620×300 · 3 seconds · 16 FPS · 15,250 bytes. This mostly static moving-marker scene is an example, not a general compression benchmark.

[Original recording]

Example 2 · Bilibili, operated and recorded through MCP

Start recording → scroll the real browser → stop → probe boundaries → trim idle time.

[Real Bilibili recording]

29.88 seconds → 7.63 seconds. Native region comparisons and interval jumps located boundaries with eight midpoint probes, avoiding intermediate image exports. Read the real walkthrough.

These demos predate synchronized timelines. Current releases record keyboard/mouse/frame timelines by default; the older media illustrates capture and editing.

For viewing GIFs and inspecting them frame by frame, we recommend Minimal Image Viewer.

Made for local workflows

Windows 10 2004+ / Windows 11 x64, visible desktop capture. No built-in cloud uploads or telemetry. GIF/PNG only; no audio or MP4. Agent guide · Timeline details · Changelog.

By Neo / 刘民心. Inspired by GifCam, independently implemented. MIT.