Crowdsentinel Mcp Server

Warning This project is in active development and intended for security testing, research, and educational purposes only. It is not production-ready. Do not deploy in production environments. APIs, tool interfaces, and data formats may change without notice. Use at your own risk.

Demo

https://github.com/user-attachments/assets/0d0381f0-5b68-43b2-8630-19ec130885b2

What is CrowdSentinel?

CrowdSentinel transforms traditional SIEM querying into intelligent, framework-driven investigations using natural language. It serves as a unified security intelligence layer that connects large language models to enterprise security data sources, enabling:

• Natural Language Threat Hunting — Query Elasticsearch using plain English
• AI-Guided Investigation Workflows — Built-in prompts guide agents through proper IR methodology
• Persistent Investigation State — Memory-managed IoC tracking, forensic timelines, and cross-query correlation that survives across sessions (8GB FIFO storage)
• Cross-Tool IoC Correlation — IoCs discovered in one tool are automatically available to all others
• Multi-Source Analysis — Elasticsearch, EVTX logs (Chainsaw), PCAP files (Wireshark), live endpoint forensics (Velociraptor), local host forensics (osquery)
• Velociraptor Endpoint Forensics — 25 MCP tools for live artefact collection (processes, network, persistence, execution evidence, NTFS MFT, SRUM, remote EVTX) with automatic IoC extraction
• Encoded-Command Deobfuscation — Automatically peels base64/UTF-16LE/gzip/hex layers off encoded commands (e.g. PowerShell -EncodedCommand) so C2 addresses hidden inside them surface as IoCs
• Adaptive Schema Intelligence — Data-driven schema registry (ECS, Sysmon, Windows Security, auditd, Zeek, Packetbeat, AWS CloudTrail) adapts hunts to each index's field conventions
• DFIR Knowledge Resources — 9 MCP resources exposing investigation playbooks, Pyramid of Pain reference, and cross-correlation guidance directly to connected AI agents
• Standalone CLI — Full threat hunting from the terminal without an MCP client

Installation

Install from PyPI (recommended)

# Install with pippip install crowdsentinel-mcp-server
# Or install with uvuv pip install crowdsentinel-mcp-server
# Download detection rules, Chainsaw, and Sigma rules (one-time)crowdsentinel setup
# Validate connector configuration and readiness (offline, no external calls)crowdsentinel doctor

Optional extras:

# Velociraptor live endpoint forensics (adds 25 MCP tools)pip install 'crowdsentinel-mcp-server[velociraptor]'
# Long-tail deobfuscation coverage via chepy (XOR, exotic encodings).# The stdlib decoder core works without this — chepy only extends it.pip install 'crowdsentinel-mcp-server[deobf]'

Detection rules (6,060 Lucene + EQL + ES|QL) are bundled with the package — no download needed. The setup command downloads additional tools:

• Chainsaw binary for EVTX analysis
• 3,000+ Sigma rules for Chainsaw

Downloaded tools are stored in ~/.crowdsentinel/ and persist across package upgrades.

System dependency for PCAP analysis:

# Required for network traffic analysis and cross-tool IoC correlationsudo apt install tshark    # Debian/Ubuntu/Kalisudo dnf install wireshark-cli  # Fedora/RHELbrew install wireshark     # macOS

Run directly with uvx (no install needed)

# Elasticsearch 8.x (default)uvx crowdsentinel-mcp-server
# Other backendsuvx crowdsentinel-mcp-server-es7   # Elasticsearch 7.xuvx crowdsentinel-mcp-server-es9   # Elasticsearch 9.xuvx opensearch-mcp-server          # OpenSearch 1.x/2.x/3.x

Install from source

git clone https://github.com/thomasxm/CrowdSentinels-AI-MCP.gitcd CrowdSentinels-AI-MCPchmod +x setup.sh && ./setup.sh
# Or install everything, including all optional extras (deobf/chepy, velociraptor)./setup.sh --full

The setup script will:

• Install dependencies (pipx, uv, Claude Code CLI if needed)
• Bundle 6,060 detection rules and download Chainsaw binary
• Prompt for Elasticsearch credentials (never hardcoded)
• Configure the MCP server with Claude Code
• Validate your connection

With --full, every optional extra is installed too, so all 139 MCP tools have their dependencies available — the recommended mode for a production-ready server.

Installed Size

CrowdSentinel bundles 6,060 detection rules and integrates with external analysis tools. Below is the full disk space breakdown so you can plan accordingly.

Core package (via pip or uvx):

Additional tools (via crowdsentinel setup):

System dependency (via package manager):

Full installation summary:

Note: PyPI download size is only 8.9 MB (wheel) thanks to compression of the bundled detection rules.

Quick Start

1. Set environment variables

export ELASTICSEARCH_HOSTS="https://localhost:9200"export ELASTICSEARCH_API_KEY="your_api_key"# Or use username/password:# export ELASTICSEARCH_USERNAME="elastic"# export ELASTICSEARCH_PASSWORD="your_password"export VERIFY_CERTS="false"

2. Connect to an MCP Client

CrowdSentinel works with any MCP-compatible AI agent. Choose your client below:

claude mcp add crowdsentinel \  -e ELASTICSEARCH_HOSTS="https://localhost:9200" \  -e ELASTICSEARCH_API_KEY="your_api_key" \  -e VERIFY_CERTS="false" \  -- uvx crowdsentinel-mcp-server

{  "mcpServers": {    "crowdsentinel": {      "command": "uvx",      "args": ["crowdsentinel-mcp-server"],      "env": {        "ELASTICSEARCH_HOSTS": "https://localhost:9200",        "ELASTICSEARCH_API_KEY": "your_api_key",        "VERIFY_CERTS": "false"      }    }  }}

{  "servers": {    "crowdsentinel": {      "command": "uvx",      "args": ["crowdsentinel-mcp-server"],      "env": {        "ELASTICSEARCH_HOSTS": "https://localhost:9200",        "ELASTICSEARCH_API_KEY": "your_api_key",        "VERIFY_CERTS": "false"      }    }  }}

{  "mcpServers": {    "crowdsentinel": {      "command": "uvx",      "args": ["crowdsentinel-mcp-server"],      "env": {        "ELASTICSEARCH_HOSTS": "https://localhost:9200",        "ELASTICSEARCH_API_KEY": "your_api_key",        "VERIFY_CERTS": "false"      }    }  }}

{  "mcpServers": {    "crowdsentinel": {      "command": "uvx",      "args": ["crowdsentinel-mcp-server"],      "env": {        "ELASTICSEARCH_HOSTS": "https://localhost:9200",        "ELASTICSEARCH_API_KEY": "your_api_key",        "VERIFY_CERTS": "false"      }    }  }}

{  "mcpServers": {    "crowdsentinel": {      "command": "uvx",      "args": ["crowdsentinel-mcp-server"],      "env": {        "ELASTICSEARCH_HOSTS": "https://localhost:9200",        "ELASTICSEARCH_API_KEY": "your_api_key",        "VERIFY_CERTS": "false"      }    }  }}

crowdsentinel-mcp-server --transport sse --port 8001# Connect to: http://localhost:8001/sse/

crowdsentinel-mcp-server --transport streamable-http --port 8001# Connect to: http://localhost:8001/mcp/

3. Or use the CLI directly

# Download rules and tools (one-time)crowdsentinel setup
# Check cluster healthcrowdsentinel health
# Hunt for threatscrowdsentinel hunt "powershell encoded" -i winlogbeat-*
# Run detection rulescrowdsentinel rules -p windows --tactic credential_accesscrowdsentinel detect windows_builtin_win_alert_mimikatz_keywords_lucene -i winlogbeat-*
# Analyse PCAP filescrowdsentinel pcap overview capture.pcapcrowdsentinel pcap beaconing capture.pcap
# Hunt EVTX logs with Chainsawcrowdsentinel chainsaw hunt /path/to/evtx/ --sigma-rules /path/to/sigma/

CLI Usage

CrowdSentinel provides a full CLI for threat hunting from the terminal:

pip install crowdsentinel-mcp-servercrowdsentinel setup    # Download rules, Chainsaw, Sigma (one-time)crowdsentinel --help

Available Commands

Output Formats

All commands support --output/-o with three formats:

crowdsentinel hunt "failed login" -i winlogbeat-* -o json     # Structured JSON (default)crowdsentinel hunt "failed login" -i winlogbeat-* -o table    # Human-readable tablecrowdsentinel hunt "failed login" -i winlogbeat-* -o summary  # Condensed summary

Agent Mode (--mcp)

The analyse --mcp flag replaces deterministic analysis with an AI agent that autonomously uses all 139 MCP tools to investigate. The agent follows the 4-phase IR methodology: hunt, analyse, correlate, report.

Authentication:

# Option 1: Browser sign-in (ChatGPT subscription — no API billing)crowdsentinel auth login
# Option 2: Anthropic (setup-token or API key)crowdsentinel auth login --provider anthropic
# Option 3: Environment variableexport ANTHROPIC_API_KEY="sk-ant-..."   # or OPENAI_API_KEY
# Option 4: Local models (Ollama, vLLM — free)crowdsentinel analyse --mcp --model-url http://localhost:11434/v1 --model llama3.1
# Check auth statuscrowdsentinel auth status

Agent flags:

Pipeline Examples

Deterministic analysis (no API key needed):

# Hunt then analysecrowdsentinel hunt "powershell encoded" -i winlogbeat-* -o json | \  crowdsentinel analyse -c "Encoded PowerShell commands" -o summary
# Investigate failed authentication attemptscrowdsentinel hunt "event.code:4625" -i winlogbeat-* -o json | \  crowdsentinel analyse -c "Failed login brute force investigation" -o summary
# Triage process execution and privilege escalationcrowdsentinel hunt "event.code:4688 OR event.code:4672 OR event.code:1" -i winlogbeat-* -o json | \  crowdsentinel analyse -c "Process execution and privilege escalation" -o summary

AI agent investigation (requires auth):

# Credential dumping investigation — agent hunts, analyses kill chain, checks adjacent stagescrowdsentinel hunt "mimikatz OR lsass OR procdump" -i winlogbeat-* -o json | \  crowdsentinel analyse --mcp -c "Credential dumping tools investigation" --max-steps 15 -o summary
# Encoded PowerShell — full IR workflow with kill chain and adjacent stage huntingcrowdsentinel hunt "powershell -enc OR FromBase64String" -i winlogbeat-* -o json | \  crowdsentinel analyse --mcp -c "Full IR workflow: encoded PowerShell" --max-steps 30 -o table
# Process execution with detection rulescrowdsentinel hunt "event.code:4688" -i winlogbeat-* -o json | \  crowdsentinel analyse --mcp -c "Execute detection rules against process creation" --max-steps 20 -o summary
# PCAP beaconing — agent generates IoCs and maps to kill chaincrowdsentinel pcap beaconing capture.pcap -o json | \  crowdsentinel analyse --mcp -c "Investigate beaconing for C2 infrastructure" --max-steps 10 -o summary
# Anti-forensics investigationcrowdsentinel hunt "event.code:1102" -i winlogbeat-* -o json | \  crowdsentinel analyse --mcp -c "Security log cleared - anti-forensics" --max-steps 10 -o summary
# With external MCP server (e.g., VirusTotal)crowdsentinel hunt "powershell" -i winlogbeat-* -o json | \  crowdsentinel analyse --mcp --mcp-server "vt:uvx virustotal-mcp-server" \  -c "Check IoCs against VirusTotal" -o summary

Key Features

Architecture

┌─────────────────────────────────────────────────────────────────┐│                 LLM Client / Claude Code CLI                    │└─────────────────────────────┬───────────────────────────────────┘                              │ MCP Protocol (stdio/SSE/HTTP)                              │ HTTP: Origin validation (anti DNS-rebinding)                              │       + transport-level audit logging                              ▼┌─────────────────────────────────────────────────────────────────┐│                    CrowdSentinel MCP Server                     ││  ┌───────────────┐ ┌───────────────┐ ┌───────────────────────┐  ││  │  139 Tools    │ │ 6,060 Rules   │ │ Security Frameworks   │  ││  │ - Hunting     │ │ - Lucene      │ │ - Cyber Kill Chain    │  ││  │ - Detection   │ │ - EQL         │ │ - Pyramid of Pain     │  ││  │ - Forensics   │ │ - ES|QL       │ │ - Diamond Model       │  ││  │ - Endpoint    │ │               │ │ - MITRE ATT&CK        │  ││  │ - Network     │ ├───────────────┤ │                       │  ││  │ - Enrichment  │ │ Schema        │ │ 9 MCP Resources       │  ││  │ - Correlation │ │ Registry      │ │ (DFIR knowledge base) │  ││  │ - Deobfuscate │ │ (data-driven) │ │                       │  ││  └───────────────┘ └───────────────┘ └───────────────────────┘  ││  ┌─────────────────────────────────────────────────────────────┐││  │              Investigation State (Persistent)               │││  │    Cross-source IoC sharing, auto-capture, STIX 2.1 export  │││  └─────────────────────────────────────────────────────────────┘│└─────┬──────────┬──────────┬──────────────┬─────────────┬────────┘      │          │          │              │             │      ▼          ▼          ▼              ▼             ▼┌───────────┐ ┌──────────┐ ┌────────┐ ┌───────────┐ ┌───────────┐│Elastic-   │ │Velocirap-│ │osquery │ │ Chainsaw  │ │ Wireshark ││search/    │ │tor       │ │(local  │ │(EVTX/     │ │  (PCAP)   ││OpenSearch │ │(EDR/DFIR)│ │ host)  │ │ Sigma)    │ │ (Network) ││  (SIEM)   │ │(Endpoint)│ │        │ │(Offline)  │ │           │└───────────┘ └──────────┘ └────────┘ └───────────┘ └───────────┘                          │  ┌───────────────────────┼───────────────────────┐  ▼                       ▼                       ▼┌───────────────┐ ┌───────────────┐ ┌───────────────────────────┐│ Shodan        │ │  VirusTotal   │ │  AbuseIPDB + ThreatFox    ││ InternetDB    │ │     (v3)      │ │  + MISP (export/search)   ││  (free/no key)│ │  (free tier)  │ │  (free tier)              │└───────────────┘ └───────────────┘ └───────────────────────────┘                          │                          ▼ (Roadmap)┌───────────────┐ ┌───────────────┐ ┌───────────────┐│    Splunk     │ │     Wazuh     │ │     Zeek      ││               │ │  (EDR/XDR)    │ │   (NSM/IDS)   │└───────────────┘ └───────────────┘ └───────────────┘

What's Included

Tool Categories (139 Tools)

Velociraptor tools register when VELOCIRAPTOR_API_CONFIG is set; osquery tools register when a local osqueryi binary is present; the 10 high-risk write tools are hidden when DISABLE_HIGH_RISK_OPERATIONS=true.

Security Frameworks

Detection Rules (6,060 Rules)

Platforms: Windows, Linux, macOS, Cloud (AWS/Azure/GCP), Network, Identity

Log Sources: PowerShell, Sysmon, Security Events, Process Creation, Audit logs

Encoded-Command Deobfuscation

Real intrusions hide C2 addresses and URLs inside encodings the SIEM stores verbatim — most commonly PowerShell -EncodedCommand (base64 of UTF-16LE), but also FromBase64String, base64→gzip, bare base64, and hex blobs. CrowdSentinel decodes these layers automatically during IoC extraction and via the dedicated decode_command tool, so the inner indicators surface instead of staying buried:

• Zero extra dependencies — the decoder core uses only the Python standard library (base64/zlib/codecs/binascii)
• Optional long tail — install the deobf extra (pip install 'crowdsentinel-mcp-server[deobf]') to add chepy for XOR and exotic encodings; its absence degrades gracefully
• Provenance — every decode records its recipe (e.g. base64->utf-16le) for the IR report
• Safe by design — decode-only (nothing is executed), with depth and output-size bounds so a decompression bomb cannot hang the server

Configuration

Environment Variables

# Connection (required — choose one)ELASTICSEARCH_HOSTS="https://localhost:9200"       # Self-hosted# ORELASTICSEARCH_CLOUD_ID="deployment:base64..."      # Elastic Cloud
# Authentication — choose one (in priority order):ELASTICSEARCH_BEARER_TOKEN="service_token_here"    # Service/bearer tokenELASTICSEARCH_API_KEY="your_api_key"               # API key (recommended)ELASTICSEARCH_USERNAME="elastic"                   # Basic authELASTICSEARCH_PASSWORD="your_password"
# TLS / Certificate verificationVERIFY_CERTS="true"                                # Verify against system CA bundle# VERIFY_CERTS="/path/to/ca.crt"                   # Verify against custom CA certificate# ELASTICSEARCH_CA_CERT="/path/to/ca.crt"          # Explicit CA certificate path# ELASTICSEARCH_CLIENT_CERT="/path/to/client.crt"  # Client certificate (mTLS)# ELASTICSEARCH_CLIENT_KEY="/path/to/client.key"   # Client private key (mTLS)
# OptionsREQUEST_TIMEOUT="30"                               # Request timeout in secondsDISABLE_HIGH_RISK_OPERATIONS="true"                # Block all write operations
# HTTP/SSE transport security (stdio unaffected)# MCP_ALLOWED_ORIGINS="https://app.example.com"    # Extra allowed Origins (comma-separated); loopback always allowed
# Velociraptor endpoint forensics (optional — enables 25 extra tools)# VELOCIRAPTOR_API_CONFIG="/path/to/api_client.yaml"
# Threat Intelligence (optional — Shodan InternetDB works without any key)VIRUSTOTAL_API_KEY="your_vt_key"                   # Free: 500 lookups/dayABUSEIPDB_API_KEY="your_abuse_key"                 # Free: 1,000 lookups/dayTHREATFOX_API_KEY="your_tf_key"                    # Free: unlimited
# MISP Integration (optional — offline JSON export works without a server)MISP_URL="https://misp.example.org"                # MISP instance URLMISP_API_KEY="your_misp_key"                       # MISP API key (40-char hex)# MISP_SSL_VERIFY="true"                           # Set "false" for self-signed (dev only)

Security Warning: Never use VERIFY_CERTS="false" or plain-text passwords in production. Use API keys or service tokens with TLS certificate verification enabled. For self-signed certificates, set ELASTICSEARCH_CA_CERT to your CA certificate path.

Production Configuration Examples

ELASTICSEARCH_CLOUD_ID="my-deployment:dXMtY2VudHJhbC0x..."ELASTICSEARCH_API_KEY="your_cloud_api_key"VERIFY_CERTS="true"

ELASTICSEARCH_HOSTS="https://es-cluster.internal:9200"ELASTICSEARCH_API_KEY="your_api_key"ELASTICSEARCH_CA_CERT="/etc/elasticsearch/certs/ca.crt"VERIFY_CERTS="true"

ELASTICSEARCH_HOSTS="https://es-cluster.internal:9200"ELASTICSEARCH_CA_CERT="/etc/elasticsearch/certs/ca.crt"ELASTICSEARCH_CLIENT_CERT="/etc/elasticsearch/certs/client.crt"ELASTICSEARCH_CLIENT_KEY="/etc/elasticsearch/certs/client.key"VERIFY_CERTS="true"

ELASTICSEARCH_HOSTS="http://localhost:9200"ELASTICSEARCH_USERNAME="elastic"ELASTICSEARCH_PASSWORD="test123"VERIFY_CERTS="false"

Transport Modes

The HTTP transports are hardened at the transport layer:

• Origin validation (anti DNS-rebinding) — browser requests are rejected unless their Origin is loopback or listed in MCP_ALLOWED_ORIGINS (comma-separated; * alone disables the check). Requests without an Origin header — every non-browser MCP client — are unaffected.
• Audit logging — every HTTP exchange is logged at the transport boundary (JSON-RPC method, tool name, body size, path), including requests the tool layer never sees, such as unknown tools or protocol errors. Logs are metadata-only, memory-bounded, and sanitised against log forgery.

Disabling Write Operations

For read-only access (recommended for production):

DISABLE_HIGH_RISK_OPERATIONS="true"

This hides destructive tools: create_index, delete_index, index_document, delete_document, delete_by_query, create_data_stream, delete_data_stream, put_alias, delete_alias, general_api_request

Usage Examples

Example 1: Hunt for Brute Force Attacks

You: Hunt for brute force attacks in the last hour
CrowdSentinel: Searching for failed authentication attempts...[Calls hunt_by_timeframe with attack_types=["brute_force"]]
Found 47 failed login attempts from 3 source IPs targeting the "admin" account.Top sources:- 192.168.1.105: 28 attempts- 10.0.0.42: 15 attempts- 172.16.0.8: 4 attempts

Example 2: Investigate a Suspicious Host

You: Show me all activity on host DESKTOP-ABC123 in the last 24 hours
CrowdSentinel: Generating a forensic timeline for that host...[Calls get_host_activity_timeline]
Timeline shows:- 09:15: User "jsmith" logged in (Event 4624)- 09:17: PowerShell executed encoded command- 09:18: New scheduled task created "WindowsUpdate"- 09:22: Outbound connection to 203.0.113.42:443- 09:25: LSASS memory access detected

Example 3: Kill Chain Analysis

You: We found C2 beaconing. What should we hunt for next?
CrowdSentinel: Hunting adjacent kill chain stages...[Calls hunt_adjacent_stages with current_stage="COMMAND_AND_CONTROL"]
Looking backwards (Installation stage):- Found scheduled task persistence on 3 hosts- Registry Run key modifications detected
Looking forwards (Actions on Objectives):- Lateral movement to 2 additional hosts via SMB- Credential dumping attempts detected

Example 4: Network Traffic Analysis

You: Analyse this PCAP for C2 beaconing
CrowdSentinel: Checking for regular connection intervals...[Calls detect_beaconing on the PCAP file]
Detected beaconing pattern:- Host: 192.168.1.50 → 203.0.113.100:443- Interval: 60 seconds (±2s jitter)- Duration: 4 hours- Confidence: HIGH
Recommend blocking 203.0.113.100 and investigating 192.168.1.50.

Example 5: CLI Threat Hunt

# Hunt for encoded PowerShellcrowdsentinel hunt "powershell -enc" -i winlogbeat-* --timeframe 1440 -o json
# Pipe results to analysiscrowdsentinel hunt "event.code:4625" -i winlogbeat-* -o json | \  crowdsentinel analyse -c "Failed authentication investigation"
# Search detection rules for lateral movementcrowdsentinel rules --tactic lateral_movement -p windows

Compatibility

For Developers

crowdsentinel-mcp-server/├── src/│   ├── server.py                 # MCP server entry point│   ├── version.py                # Version constant│   ├── risk_config.py            # Write operation controls│   ├── transport_security.py     # Origin validation for HTTP transports (anti DNS-rebinding)│   ├── transport_audit.py        # Transport-level audit logging (every HTTP exchange)│   ││   ├── cli/                      # Standalone CLI│   │   └── main.py               # CLI entry point (argparse) — 15 commands incl. doctor│   ││   ├── agent/                    # `analyse --mcp` AI agent mode (multi-provider LLM auth)│   ├── config/                   # Connector configuration│   ││   ├── clients/                  # Backend logic layer│   │   ├── base.py               # Base client, authentication│   │   ├── exceptions.py         # Exception handling decorators│   │   └── common/│   │       ├── client.py         # Unified SearchClient (multiple inheritance)│   │       ├── threat_hunting.py # Threat hunting queries│   │       ├── ioc_analysis.py   # IoC extraction & analysis│   │       ├── deobfuscate.py    # Encoded-command decoding (stdlib core + optional chepy)│   │       ├── cyber_kill_chain.py # Kill chain logic│   │       ├── rule_loader.py    # Detection rule loading│   │       ├── chainsaw_client.py # EVTX/Sigma integration│   │       ├── osquery_client.py # Local osquery execution (sqlglot-validated SQL)│   │       ├── misp_client.py    # MISP export/search│   │       ├── threat_intel.py   # Shodan/VirusTotal/AbuseIPDB/ThreatFox enrichment│   │       └── schemas/          # Data-driven schema registry│   │           ├── ecs.py, sysmon.py, windows_security.py, auditd.py│   │           └── defs/         # JSON schema defs (zeek, packetbeat, aws_cloudtrail, …)│   ││   ├── tools/                    # MCP tool interfaces (25 modules, 139 tools)│   │   ├── register.py           # Dynamic tool registration│   │   ├── threat_hunting.py     # Hunting tool definitions│   │   ├── rule_management.py    # Rule management tools│   │   ├── velociraptor_tools.py # Live endpoint forensics (25 tools)│   │   ├── osquery_tools.py      # Local host forensics (4 tools)│   │   ├── cross_correlation.py  # SIEM ↔ endpoint correlation (5 tools)│   │   ├── chainsaw_hunting.py   # Chainsaw tools│   │   ├── wireshark_tools.py    # Network analysis tools│   │   ├── threat_intel.py       # Enrichment + MISP tools│   │   └── investigation_state_tools.py # State management tools│   ││   ├── storage/                  # Persistent investigation state│   │   ├── investigation_state.py # Core state management│   │   ├── storage_manager.py    # File system storage (8GB FIFO)│   │   └── models.py             # Pydantic models (IoC, Investigation)│   ││   ├── utils/                    # Shared utilities│   ││   └── wireshark/                # Network traffic analysis│       ├── core/                 # TShark execution, PCAP parsing│       ├── hunting/              # Beaconing, lateral movement, IoC hunting│       ├── baseline/             # Traffic baseline creation│       ├── extraction/           # File carving from traffic│       └── reporting/            # NCSC-style reports, timelines│├── rules/                        # 6,060 detection rules (EQL + Lucene)├── chainsaw/                     # Chainsaw binary + 3,000+ Sigma rules├── skills/                       # Claude Code agent skills├── Makefile                      # Test pyramid + release automation└── tests/                        # Tiered test suite (unit / integration / api / e2e / system)

class YourClient(SearchClientBase):    def your_method(self, param: str) -> dict:        # Implementation        return results

class SearchClient(YourClient, OtherClients, ...):    pass

class YourTools:    def __init__(self, client, mcp):        self.client = client        self.mcp = mcp
    def register_tools(self):        @self.mcp.tool()        def your_tool(param: str) -> str:            """Tool description for LLM."""            result = self.client.your_method(param)            return json.dumps(result)

from src.tools.your_tools import YourTools
def _register_tools(self):    # ... existing tools ...    YourTools(self.client, self.mcp).register_tools()

make test-unit          # fast hermetic loop (default developer command)make coverage           # unit tier + coverage policy gatemake check-tests        # policy: every new function vs origin/main has a testmake test-integration   # real ES via testcontainers (Docker required)make test-api           # + real MCP server subprocess over streamable-httpmake test-e2e           # + real MCP client journey over the wiremake test-suite         # the whole pyramid, bottom-up, with a summary table

# Start Elasticsearchdocker-compose -f docker-compose-elasticsearch.yml up -d
# Start OpenSearchdocker-compose -f docker-compose-opensearch.yml up -d

Roadmap

See CHANGELOG.md for detailed version history.

Velociraptor Integration (Live Endpoint Forensics)

CrowdSentinel integrates with Velociraptor for live endpoint forensic artefact collection. When configured, 25 additional MCP tools and 5 cross-correlation tools become available.

Setup

# Install optional dependenciespip install crowdsentinel-mcp-server[velociraptor]
# Set the Velociraptor API config pathexport VELOCIRAPTOR_API_CONFIG="/path/to/api_client.yaml"

The API client config is generated from the Velociraptor server:

velociraptor config api_client --name crowdsentinel --role administrator,api \  --config server.config.yaml /path/to/api_client.yaml

Available Tools

Cross-Source IoC Sharing

IoCs extracted from Velociraptor artefacts are automatically captured into the active investigation alongside SIEM, Chainsaw, and Wireshark findings. Use get_shared_iocs to retrieve the combined indicator set for cross-source correlation.

SIEM (Elasticsearch)          Endpoint (Velociraptor)       │                              │  hunt_by_timeframe()         velociraptor_pslist()       │                              │  auto_capture ──────► Investigation State ◄────── auto_capture                              │                     get_shared_iocs()

DFIR Knowledge Resources

9 MCP resources provide structured investigation reference data directly to connected AI agents:

Documentation

User Guides

Feature Guides

Developer Guides

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

Licence

GNU General Public Licence v3.0 — See LICENSE for details.

Cyber Range

CrowdSentinel ships with a companion crowdsentinel-range — a fully ephemeral AWS lab that generates ground-truth-labeled attack telemetry so you can investigate real detections rather than synthetic data.

Current range (v1) — 2-host AWS lab

┌──────────────────────────── AWS VPC (private subnet) ────────────────────────────────┐│                                                                                        ││  ┌──────────────────────────────┐      ┌──────────────────────────────────────────┐  ││  │   Linux SIEM (Ubuntu 22.04)  │      │   Windows Range Box (Server 2022)        │  ││  │   Elasticsearch 8.x (TLS)   │◄─────│   Sysmon (olafhartong) + Winlogbeat 9    │  ││  │   Auditbeat + Filebeat       │      │   Atomic Red Team + PurpleSharp          │  ││  └──────────────────────────────┘      └──────────────────────────────────────────┘  ││                         SSM VPC endpoints — no inbound ports open                     │└───────────────────────────────────────┬────────────────────────────────────────────--┘                                         │ SSM port-forward :19200                              ┌──────────▼──────────────────────┐                              │  ./range-mcp.sh → CrowdSentinel │                              │  MCP Server → Claude Code       │                              └─────────────────────────────────┘

3 commands to go from zero to hunting:

cd crowdsentinel-range && make up && make attack-windowscd .. && ./range-mcp.sh# then in Claude Code: /mcp → "investigate potential intrusions on the windows machine"

Demo: 73,328 events triaged in ~40 seconds — 2 shellcode payloads decoded, C2 IP extracted from raw bytes, full IR report with 19 MITRE techniques.

Future range (v2) — enterprise AD environment

The next range generation targets full Active Directory kill-chain coverage: initial access → lateral movement → domain compromise, with a network security monitor layer, runtime security instrumentation, centralised vulnerability management, and deception infrastructure. Inspired by DetectionLab, Ludus, and SimuLand.

┌───────────────────────────────────── AWS / Multi-VPC ───────────────────────────────────────────────────┐│                                                                                                           ││  ┌──── Attacker Subnet ────┐   ┌───────────── Corporate Subnet ──────────────────────────────────────┐  ││  │                          │   │                                                                       │  ││  │  Kali Linux (C2 server)  │   │  ┌─────────────────┐  ┌──────────────────┐  ┌──────────────────┐  │  ││  │  Caldera + Metasploit    │   │  │  Domain          │  │  Windows 11      │  │  Windows 11      │  │  ││  │  Covenant C2             │──▶│  │  Controller      │  │  Workstation 1   │  │  Workstation 2   │  │  ││  │                          │   │  │  (AD DS, DNS)    │  │  (domain-joined) │  │  (domain-joined) │  │  ││  └──────────────────────────┘   │  └─────────────────┘  └──────────────────┘  └──────────────────┘  │  ││                                  │  ┌──────────────────────────────────────────────────────────────┐  │  ││  ┌──── DMZ Subnet ─────────┐    │  │   Linux File/Web Server (IIS / Apache, SMB share)            │  │  ││  │                          │    │  └──────────────────────────────────────────────────────────────┘  │  ││  │  Nginx reverse proxy     │    └───────────────────────────────────────────────────────────────────┘  ││  │  Simulated internet SVC  │                                                                            ││  └──────────────────────────┘   ┌───────────── Management / SIEM Subnet ──────────────────────────┐    ││                                  │                                                                   │    ││                                  │  ┌─────────────────────┐   ┌───────────────────────────────┐   │    ││                                  │  │  Elasticsearch +     │   │  Zeek / Suricata              │   │    ││                                  │  │  Kibana (SIEM)       │   │  (NSM — full packet + alerts) │   │    ││                                  │  └─────────────────────┘   └───────────────────────────────┘   │    ││                                  │  ┌─────────────────────┐   ┌───────────────────────────────┐   │    ││                                  │  │  Velociraptor        │   │  Falco (runtime security)     │   │    ││                                  │  │  (live EDR/forensics)│   │  eBPF probe (CO-RE, default)  │   │    ││                                  │  └─────────────────────┘   │  + kernel module fallback     │   │    ││                                  │  ┌─────────────────────┐   └───────────────────────────────┘   │    ││                                  │  │  DefectDojo          │   ┌───────────────────────────────┐   │    ││                                  │  │  (vuln management)   │   │  Honeypots + Canary tokens     │   │    ││                                  │  │  Falco·Zeek·scanners │   │  (deception layer)             │   │    ││                                  │  └─────────────────────┘   └───────────────────────────────┘   │    ││                                  │  ┌─────────────────────────────────────────────────────────┐   │    ││                                  │  │  Grafana — centralized dashboards + alerting             │   │    ││                                  │  │  ES · Falco · Suricata · Velociraptor · DefectDojo       │   │    ││                                  │  └─────────────────────────────────────────────────────────┘   │    ││                                  └───────────────────────────────────────────────────────────────┘    ││                                         VyOS router — inter-subnet ACLs + egress filter               │└───────────────────────────────────────────────────────────────────────────────────────────────────────┘                                                  │ SSM / VPN                                    CrowdSentinel MCP + Claude Code

v2 target capabilities:

Design references: DetectionLab (Sysmon+WEF+Splunk convergence), Ludus (modular Proxmox/Ansible, VLAN segmentation), SimuLand (multi-domain AD forest, cloud-native), SANS FOR508 (artifact-first DFIR workflow), Falco (CO-RE eBPF probe architecture, kernel module fallback).

Acknowledgements

CrowdSentinel stands on the shoulders of outstanding open-source projects. Thank you to all of their maintainers and contributors.

Protocol & server framework

• Model Context Protocol — the protocol connecting LLMs to tools, by Anthropic
• FastMCP — the Python MCP server framework this project is built on

Analysis engines & forensic tools

• Chainsaw — EVTX log analyser by WithSecure Labs
• Velociraptor — live endpoint forensics and DFIR platform by Velocidex/Rapid7
• osquery — SQL-powered host instrumentation by the osquery Foundation
• Wireshark / TShark — network protocol analysis by the Wireshark Foundation
• Chepy — CyberChef-like decoding library powering optional long-tail deobfuscation
• sqlglot — SQL parser used to safely validate osquery queries

Detection content

• Sigma / SigmaHQ — the community detection rule standard and rule corpus
• Elastic detection-rules — Elastic's prebuilt EQL/ES|QL detection rules
• Community-contributed Sigma and custom rules

Data platforms & threat intelligence

• Elasticsearch and OpenSearch Python clients
• MISP / PyMISP — open-source threat intelligence platform and client
• stix2 — OASIS STIX 2.1 library for structured IoC export
• Shodan InternetDB, VirusTotal, AbuseIPDB, and abuse.ch ThreatFox — IoC enrichment services with free tiers

Development & testing

• testcontainers-python — real Elasticsearch containers for the live test tiers
• pytest, Ruff, Bandit — test runner, linting, and SAST

Methodology frameworks

• Cyber Kill Chain (Lockheed Martin), Pyramid of Pain (David J. Bianco), Diamond Model (Caltagirone, Pendergast & Betz), MITRE ATT&CK