RemoveDuplicates.org

org.removeduplicatesPublished by removeduplicates.orgOfficial publisherv1.0.1Updated Oct 24, 2026

Remove duplicate lines or CSV/TSV rows from text. Stateless: processed per call, never stored.

VerifiedStreamable HTTPWeb executableProductivity & WorkflowData & Analytics

Overview

AI-generated overview

Hosted MCP server that removes duplicate lines or CSV/TSV rows from pasted text and returns the cleaned text with counts.

What you can do
  • Remove duplicate lines from a pasted list, with options for case-insensitive comparison, trimming, dropping empty lines, and keeping the first or last copy.
  • Remove duplicate rows from CSV or TSV text, such as rows copied from Excel or Google Sheets, comparing whole rows or selected columns and optionally treating the first row as a header.
  • Get back the cleaned text together with counts of what was removed, using the same engine as the removeduplicates.org website.
When to use it
Useful when an assistant needs to clean up pasted lists or spreadsheet rows mid-conversation, for example deduplicating a column of names, emails or IDs before further work. It is a single-purpose utility, so it is only worth adding if that cleanup task comes up.
Requirements
A remote Streamable HTTP endpoint; nothing to install and no account or API key. Clients that only support stdio need a bridge such as mcp-remote. Text per call is capped at 128 KiB, and .xlsx workbooks are not handled by this tool.
Before you install
Text sent to the tool leaves the user's machine and is processed server-side for that request; the README states nothing is stored, but ordinary HTTP metadata is still processed by the hosting provider. The tool only returns cleaned text and does not modify the user's files.

Installation

In SourceWeft

  1. Open RemoveDuplicates.org in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Web executable via Streamable HTTP. Remote servers run from the web runtime once configured in a workspace.

Use with an agent

Paste this prompt into Claude Code, Codex, Cursor or another agent — or into SourceWeft chat. The agent reads this server's install guide, shows you its publisher, endpoint and sign-in, and adds it to your client once you agree. Keys never go into a command.

Read https://sourceweft.com/mcp/org.removeduplicates%2Fremove-duplicates/install.md and connect the MCP server it describes to my client. Before changing any config, show me its publisher, endpoint and sign-in method and wait for my OK. Never put an API key in a command or in a file I would commit; ask me where it should come from.

Read the install guide the agent follows

Other MCP clients

Add this to your client's mcpServers config.

{
  "mcpServers": {
    "removeduplicates-org": {
      "type": "http",
      "url": "https://removeduplicates.org/mcp"
    }
  }
}

README

removeduplicates.org

A focused static tool for removing duplicates from lines, lists, pasted spreadsheet data, CSV files, and local XLSX workbooks, deployed as a Cloudflare Worker with Static Assets.

Links:

License: MIT

Use it from an AI agent (MCP)

A hosted MCP server is available at https://removeduplicates.org/mcp (Streamable HTTP). There is nothing to install and no API key or account. It is listed in the Official MCP Registry as org.removeduplicates/remove-duplicates.

Claude Code:

claude mcp add --transport http removeduplicates https://removeduplicates.org/mcp

Cline (cline_mcp_settings.json):

{  "mcpServers": {    "removeduplicates": {      "type": "streamableHttp",      "url": "https://removeduplicates.org/mcp"    }  }}

Cursor (mcp.json) takes the same mcpServers entry without type. VS Code (.vscode/mcp.json):

{  "servers": {    "removeduplicates": {      "type": "http",      "url": "https://removeduplicates.org/mcp"    }  }}

Gemini CLI can install the repository as an extension (gemini extensions install https://github.com/suvadadepolo-blip/remove-duplicates); gemini-extension.json points it at the same endpoint.

Clients that only speak stdio can bridge with npx mcp-remote https://removeduplicates.org/mcp.

The server has one read-only tool, remove_duplicates. It takes text (up to 128 KiB) plus optional format (auto, lines, table), ignoreCase, trim, removeEmpty, keep (first, last), order (preserve, sort), compare (row, a column index, or a list of indices) and header, and returns the cleaned text with counts. Each call is processed in memory and nothing is stored. For .xlsx workbooks, use the Excel tool, which runs only in the browser.

Product boundary

  • Line and table-aware deduplication runs in browser memory; larger inputs are streamed to a same-origin Web Worker in responsive chunks.
  • Excel and Google Sheets rows can be pasted as TSV, while quoted CSV/TSV files can be opened up to the same 5 MB local limit.
  • The Excel tool accepts only macro-free .xlsx files, compares an entire row or a stable combination of selected columns, and creates the cleaned workbook in a dedicated Module Worker.
  • XLSX package limits are 10 MiB compressed, 2,000 ZIP entries, 100 MiB declared expanded data, 500,000 workbook cells, and 250,000 cells in the selected sheet. Formula row shifts and intersecting merged ranges are blocked.
  • SheetJS CE 0.20.3 is pinned under public/vendor/sheetjs-0.20.3/ with its official source URLs, license, and SHA-256 receipt. It is lazy-loaded only after a valid XLSX passes local ZIP preflight and is never requested by the homepage.
  • The website does not upload or persist text or workbook contents a user opens. The only server-side processing is the optional /mcp endpoint, which handles text an AI agent sends for that one request and stores nothing.
  • No account, database, analytics, advertising, external font, or third-party browser runtime is included.
  • Cloudflare still processes ordinary HTTP metadata needed to deliver and protect the site; the Privacy copy states this boundary directly.
  • Terms and Privacy are real direct pages, enhanced into an in-page dialog when JavaScript is available.

Agent and answer-engine surfaces

  • robots.txt names the major AI crawlers and declares Content-Signal: search=yes, ai-input=yes, ai-train=yes.
  • /llms.txt and /llms-full.txt (with /llm.txt redirects) describe the site for language models.
  • Every page has a Markdown twin (/index.md, /excel.md, /privacy.md, /terms.md), also served for Accept: text/markdown. HTML routes send Vary: Accept and RFC 8288 Link headers for the API catalog, llms.txt, the agent-skills index, and the Markdown alternate.
  • /.well-known/api-catalog (RFC 9727), /.well-known/mcp/server-card.json, /.well-known/agent-skills/index.json with remove-duplicates/SKILL.md, and /.well-known/ai-catalog.json.
  • /mcp is a stateless Streamable HTTP MCP server with one tool, remove_duplicates, capped at 128 KiB of text per call. The homepage registers the same tool through WebMCP when the browser exposes navigator.modelContext; that path runs locally in the tab. Both share public/js/agent-tool.js and the site's dedupe.js engine.
  • server.json is the Official MCP Registry entry org.removeduplicates/remove-duplicates. Namespace ownership is proven over HTTP by /.well-known/mcp-registry-auth (Ed25519 public key); the private half lives only in the gitignored local.env.txt.
  • node scripts/build-agent-files.mjs regenerates the server card, the skill digest, and llms-full.txt; npm run check fails when they are stale.

Commands

npm run checknpm testnpm run verifynpm run previewnpm run qa:browsernpm run benchmark:xlsxnpm run check:httpnpm run deploy

Runtime preview, the XLSX benchmark, browser QA, downloaded-file readback, and Lighthouse belong on OpenClaw. Port 3630 is reserved for the managed preview. Deployment uses the project-scoped Cloudflare wrapper one directory above; credentials never live in this project.

Repository hygiene

  • Runtime artifacts stay out of Git: output/, .codex-results/, .wrangler/, logs, and .DS_Store.
  • Secret-bearing files stay out of Git: .env*, local.env.txt, and *.env.txt; only placeholder templates such as .env.example may be committed.
  • Local planning and provider evidence stay out of Git: EXCEL_PLAN.md, IMPROVEMENT_PLAN.md, PROJECT_STATE.md, UTILITY_BASELINE.md, and ops/.
  • public/vendor/sheetjs-0.20.3/ is intentionally vendored with its upstream license, source receipt, and SHA-256 checks.

Cloudflare shape

  • Worker: removeduplicates-org
  • Static assets: public/
  • Worker entry point: src/worker.js
  • Static asset binding: ASSETS, with run_worker_first: true
  • Live Workers.dev staging URL: https://removeduplicates-org.whatmyname.workers.dev/
  • Canonical production domain: https://removeduplicates.org/
  • Redirecting production alias: https://www.removeduplicates.org/

The Worker owns /excel, direct legal routes, public HTTP-to-HTTPS and canonical-host redirects, method handling, branded 404s, cache policy, HSTS, a hash-pinned CSP, and Workers.dev noindex protection. Excel path case and trailing-slash variants canonicalize to /excel in the same one-hop redirect as scheme or host changes. The Worker accepts only GET and HEAD.

Production release

The active Cloudflare Zone and wrangler.jsonc contain exactly these two custom_domain routes:

  1. removeduplicates.org
  2. www.removeduplicates.org

The Worker returns a 308 from www to the apex while preserving path and query. A partial route state is invalid, and no Zone Redirect Rule is used. The production apex, redirect alias, TLS, HTTP matrix, five-viewport browser suite, and Lighthouse are recorded as passing in the project state and production deployment ledger.

Source: README.md at commit 08a522a

Tools

1

remove_duplicates

Remove duplicates

Remove duplicate lines from a text list, or duplicate rows from CSV/TSV text such as rows copied from Excel or Google Sheets. Uses the same engine as removeduplicates.org and returns the cleaned text with counts. Stateless: the text is processed in memory for this call only and is never stored.

ParameterskeeptextrequiredtrimorderformatheadercompareignoreCaseremoveEmpty

Version history

2
  1. v1.0.1LatestSep 24, 2026
  2. v1.0.0Sep 23, 2026

Score

100% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 daysRequired25 pts
  • Auth declaredRequired10 pts
  • Tools listedRequired15 pts
  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts

Other Removeduplicates integrations

Codex Reset

Published by GitHub account suvadadepolo-blip

Read-only MCP server that gives an assistant Codex usage-limit reset forecasts, a dated reset history and current Codex service status.

Streamable HTTPWeb executable
Developer Tools3 toolsNov 2, 2026