Nvidia Nim Mcp

NVIDIA NIM MCP Server

A production-ready Model Context Protocol (MCP) server for consuming NVIDIA NIM (NVIDIA Inference Microservices) models. Supports 50+ LLMs, multimodal models, image generation, embeddings, reranking, function calling, vision, and code-specialized models with rich metadata for intelligent agent selection.

🚀 Features

• 10 MCP Tools: chat completion, text generation, embeddings, reranking, function calling, model listing, model info, image generation, image analysis, multimodal tasks, model comparison
• 50+ Supported Models: Llama 3.1/3.2, Nemotron 3 Ultra (550B), MiniMax M3, Kimi K2.6 (1T), DeepSeek V4 Pro, GLM 5.1, Qwen 3.5 397B, Mistral Large 3 (675B), GPT-OSS 120B, DiffusionGemma, FLUX.1, SDXL, SD3, and more
• Rich Model Metadata: licensing, hardware requirements, benchmarks, image generation specs, reasoning modes, tags for agent selection
• Advanced Filtering: by commercial use, reasoning, vision, function calling, multimodal, context length, tags, hardware
• Production-Grade: automatic retries with exponential backoff, per-minute rate limiting, structured JSON logging
• Type-Safe: full TypeScript, Zod input validation on every tool
• Docker-Ready: multi-stage Dockerfile with non-root user, health checks
• Configurable: all settings via environment variables
• Single Required Env: Only NVIDIA_API_KEY required; all others have sensible defaults

📋 Prerequisites

• Node.js 18+ (for NPM installation) or Docker (for container deployment)
• A NVIDIA NGC API key (nvapi-...)

⚙️ Installation

Option 1: NPM Global Installation (Recommended)

# Install globallynpm install -g nvidia-nim-mcp
# Run directlynvidia-nim-mcp

Option 2: NPM Local Installation

# Initialize your projectnpm init -y
# Install locallynpm install nvidia-nim-mcp
# Run with npxnpx nvidia-nim-mcp

Option 3: From Source

# Clone / download the projectcd nvidia-nim-mcp
# Install dependenciesnpm install
# Build TypeScriptnpm run build

Option 4: Docker

# Pull from Docker Hub (when published)docker pull nvidia-nim-mcp
# Or build locallydocker build -t nvidia-nim-mcp .

🔑 Configuration

Copy .env.example to .env and fill in your API key:

cp .env.example .env

Only NVIDIA_API_KEY is required — all other variables have production-ready defaults:

🚀 Running

NPM Global Installation

# Run the servernvidia-nim-mcp
# With custom environment variablesNVIDIA_API_KEY=nvapi-your-key LOG_LEVEL=debug nvidia-nim-mcp

NPM Local Installation

# Run with npxnpx nvidia-nim-mcp
# Or add to package.json scripts# "scripts": { "start": "nvidia-nim-mcp" }npm start

From Source

# Development mode with auto-reloadnpm run dev
# Production mode (compiled)npm run build && npm start

Docker

# Run with environment variablesdocker run --rm \  -e NVIDIA_API_KEY=nvapi-your-key \  -e LOG_LEVEL=info \  nvidia-nim-mcp
# Run in background with port mapping (if needed)docker run -d \  --name nvidia-nim-mcp \  -e NVIDIA_API_KEY=nvapi-your-key \  nvidia-nim-mcp

Standalone Executable

# Make executable (if not already)chmod +x dist/index.js
# Run directly./dist/index.js
# With environment variablesNVIDIA_API_KEY=nvapi-your-key ./dist/index.js

🔧 MCP Client Configuration

For Global NPM Installation

{  "mcpServers": {    "nvidia-nim": {      "command": "nvidia-nim-mcp",      "env": {        "NVIDIA_API_KEY": "nvapi-your-key-here",        "LOG_LEVEL": "info"      }    }  }}

For Local NPM Installation

{  "mcpServers": {    "nvidia-nim": {      "command": "npx",      "args": ["nvidia-nim-mcp"],      "env": {        "NVIDIA_API_KEY": "nvapi-your-key-here",        "LOG_LEVEL": "info"      }    }  }}

For Direct Executable Path

{  "mcpServers": {    "nvidia-nim": {      "command": "node",      "args": ["/absolute/path/to/nvidia-nim-mcp/dist/index.js"],      "env": {        "NVIDIA_API_KEY": "nvapi-your-key-here",        "LOG_LEVEL": "info"      }    }  }}

🛠️ Available Tools

chat_completion

Multi-turn conversation with any NIM LLM.

{  "model": "nvidia/nemotron-3-ultra-550b-a55b",  "messages": [    { "role": "user", "content": "Explain quantum computing" }  ],  "temperature": 0.3,  "max_tokens": 4096}

text_generation

Single-prompt text generation (simplified interface).

{  "prompt": "Write a haiku about machine learning",  "temperature": 0.5,  "max_tokens": 512}

create_embeddings

Convert text(s) to vector embeddings for RAG/search.

{  "model": "nvidia/nv-embed-v1",  "input": ["NVIDIA makes GPUs", "AI runs on GPUs"],  "truncate": "END"}

rerank_passages

Rerank passages by relevance to a query.

{  "query": "What is CUDA?",  "passages": ["CUDA is a GPU programming platform", "NIM serves AI models"],  "top_k": 3}

function_calling

Use NIM models with tool/function calling.

{  "model": "z-ai/glm-5.1",  "messages": [{ "role": "user", "content": "What's the weather in Paris?" }],  "tools": [{    "type": "function",    "function": {      "name": "get_weather",      "description": "Get current weather",      "parameters": {        "type": "object",        "properties": { "city": { "type": "string" } },        "required": ["city"]      }    }  }]}

generate_image

Generate images from text prompts using FLUX.1, SDXL, SD3, DiffusionGemma.

{  "model": "black-forest-labs/flux.1-dev",  "prompt": "A photorealistic mountain landscape at sunset, 8K",  "width": 1024,  "height": 1024,  "steps": 30,  "cfg_scale": 3.5,  "sampler": "euler_a",  "scheduler": "simple"}

analyze_image

Analyze and describe images using vision/multimodal models.

{  "model": "moonshotai/kimi-k2.6",  "image_url": "https://example.com/image.jpg",  "prompt": "Describe this image in detail",  "detail": "high"}

multimodal_task

Perform multimodal tasks combining text and images.

{  "model": "minimaxai/minimax-m3",  "messages": [    {      "role": "user",      "content": [        { "type": "text", "text": "Analyze this chart" },        { "type": "image_url", "image_url": { "url": "https://example.com/chart.png" } }      ]    }  ],  "max_tokens": 2048}

list_models

List available models with rich metadata and advanced filtering.

{  "category": "code",  "commercial_use": true,  "supports_reasoning": true,  "tags": ["coding", "agentic"],  "include_details": true}

Filter Options:

• category: language, embedding, reranking, vision, code, multimodal, image_generation, all
• commercial_use: Filter by commercial license
• supports_reasoning: Filter by reasoning capability
• supports_vision: Filter by vision capability
• supports_function_calling: Filter by function calling
• supports_multimodal: Filter by multimodal input
• min_context_length: Minimum context window (tokens)
• tags: Filter by use case tags
• hardware: Filter by GPU type (Hopper, Blackwell, Ampere)
• include_details: Include full metadata (benchmarks, image specs, etc.)

get_model_info

Get complete metadata for a specific model.

{ "model_id": "nvidia/nemotron-3-ultra-550b-a55b" }

Returns: licensing, hardware requirements, benchmarks, image gen specs, reasoning modes, tags, supported languages, etc.

compare_models

Compare 2-5 models side-by-side across all decision factors.

{  "model_ids": [    "nvidia/nemotron-3-ultra-550b-a55b",    "deepseek-ai/deepseek-v4-pro",    "moonshotai/kimi-k2.6",    "z-ai/glm-5.1"  ]}

Returns: Structured comparison table with licensing, hardware, benchmarks, capabilities, tags, image generation specs, etc.

📦 Supported Models (50+)

Language Models (Frontier Reasoning)

Code-Specialized Models

Multimodal / Vision Models

Image Generation Models

*Non-commercial default; commercial via contact
**Requires Stability AI membership

Embeddings & Reranking

🏭 Production Checklist

• Environment variable validation on startup
• Exponential backoff retry (configurable)
• Per-minute rate limiter
• Request/response logging with Winston
• Structured JSON logs in production
• Zod input validation for all tools
• Graceful shutdown (SIGINT/SIGTERM)
• Unhandled exception/rejection handlers
• Docker multi-stage build (minimal image)
• Non-root Docker user
• Token cap enforcement
• Single required env var (NVIDIA_API_KEY)
• Feature flags for optional capabilities

🧪 Testing

The project includes a comprehensive test suite:

• Unit Tests: Configuration, logging, model handling, tool validation
• Integration Tests: All 10 MCP tools with various input scenarios
• Error Handling: Validation of edge cases and failure modes
• Schema Validation: Zod-based input validation for all tools

Running Tests

# Run all testsnpm test
# Run tests with coverage reportnpm test -- --coverage
# Run tests in watch modenpm test -- --watch
# Run specific test filenpm test src/handlers.test.ts

Current Test Status: ✅ All tests passing (96 tests)

🛠️ Development

Building the Project

# Install dependenciesnpm install
# Compile TypeScript to JavaScriptnpm run build
# Clean build artifactsnpm run clean
# Development mode with auto-reloadnpm run dev

Code Quality

# Run linternpm run lint
# Run testsnpm test
# Run both linting and testsnpm run check

🤝 Contributing

Contributions are welcome!

1. Fork the Repository
2. Create a Feature Branch: git checkout -b feature/your-feature-name
3. Make Your Changes: Follow the existing code style and patterns
4. Add Tests: Ensure new functionality is properly tested
5. Run Checks: npm run check to verify code quality and tests
6. Commit Changes: Use clear, descriptive commit messages
7. Push to Your Fork: git push origin feature/your-feature-name
8. Open a Pull Request: Describe your changes and their benefits

Code Standards

• TypeScript: Strict type checking enabled
• ESLint: Code formatting and best practices
• Zod: Runtime validation for all external inputs
• Testing: Comprehensive test coverage for new features
• Documentation: Update README.md for user-facing changes

Development Workflow

1. Setup: Follow the installation instructions
2. Development: Use npm run dev for continuous development
3. Testing: Run npm test to verify your changes
4. Building: Use npm run build to compile the project
5. Linting: Run npm run lint to check code quality

📦 Packaging & Distribution

NPM Package

• Published to npm registry for easy installation
• Includes compiled JavaScript and TypeScript definitions
• Global and local installation options
• Runs as a standard CLI tool

Docker Image

• Multi-stage build for minimal image size
• Runs as non-root user for security
• Includes health check endpoint
• Easy deployment to containerized environments

Standalone Executable

• Self-contained JavaScript file with shebang
• Can be run directly on any system with Node.js
• No installation required beyond Node.js

Building Packages

# Build the projectnpm run build
# Create NPM package (.tgz)npm pack
# Build Docker imagedocker build -t nvidia-nim-mcp .
# All checks (lint, test, build)npm run check && npm run build

📄 License

MIT