The Quotient MCP Server is our synchronous API for steering agent behavior in real-time and in production using models we develop. Quotient MCP is available in two ways:

Remote HTTP Server: https://mcp.quotientai.co/mcp/
Local Installation: Installable as a local MCP server using stdio transport

Tools

`evaluate_tool_call`

Evaluates whether an AI agent correctly used available tools in a conversation context. Parameters:

available_tools: Array of tool definitions with schemas
message_history: Array of conversation messages

Returns:

score: "correct", "incorrect_tool", "incorrect_parameter_names", or "incorrect_parameter_values"
reason: Detailed explanation list

Example Usage

Input:

{
  "available_tools": [
    {
      "name": "google-play-developer",
      "description": "Get apps by a developer on Google Play",
      "input_schema": {
        "type": "object",
        "properties": {
          "devId": {"type": "string", "description": "Developer ID"},
          "num": {"type": "number", "default": 60, "description": "Number of results"},
          "lang": {"type": "string", "default": "en", "description": "Language code"},
          "country": {"type": "string", "default": "us", "description": "Country code"}
        },
        "required": ["devId"]
      }
    }
  ],
  "message_history": [
    {
      "role": "user", 
      "content": "Get apps by developer 'com.example.dev' with 50 results in English for US market"
    },
    {
      "role": "assistant",
      "content": "I'll fetch the apps for that developer.",
      "tool_calls": [
        {
          "function": {
            "name": "google-play-developer",
            "arguments": {
              "devId": "com.example.dev",
              "num": 50,
              "lang": "en",
              "country": "us"
            }
          }
        }
      ]
    }
  ]
}

Output:

{
  "score": "correct",
  "reason": []
}

Underlying Model

The evaluation is powered by the Limbic Tool Use Evaluator, a fine-tuned Qwen 2.5 0.5B model specifically designed for MCP function call evaluation.

Integration Examples

Cursor IDE

You can use the Quotient MCP server in Cursor IDE by adding the following to your .cursor/mcp.json:

{
  "mcpServers": {
    "quotient-mcp-server": {
      "url": "https://mcp.quotientai.co/mcp/"
    }
  }
}

Then, add an evaluation rule in .cursor/rules/evaluate-tool-call.mdc to always apply the evaluate_tool_call function after any agent tool call.

---
alwaysApply: true
---
# Evaluate Tool Calls Rule

## Critical Rule

- Always use the `evaluate_tool_call` function after any agent tool call to evaluate proper usage of the tool
- Use this function to validate that tools are being called correctly with proper parameters
- Apply this evaluation when assessing agent performance or troubleshooting tool usage issues

## When to Apply

- After Any Tool Call
- When reviewing agent conversations that involve tool calls
- When validating that an AI agent correctly used available tools
- When checking if tool parameters were set correctly
- When assessing overall agent tool calling performance

## Example Usage

<example>
When evaluating if an agent correctly used tools, always call:
evaluate_tool_call(available_tools, message_history)

This ensures proper validation of tool usage patterns and parameter correctness.
</example>

Amp

You can use the Quotient MCP server in Amp through both CLI and editor extension configurations. Configure the server in your editor extension settings or CLI configuration:

{
  "amp.mcpServers": {
    "quotient-mcp": {
      "url": "https://mcp.quotientai.co/mcp/"
    }
  }
}

CLI Configuration

The CLI configuration file location varies by operating system:

Windows: %APPDATA%\amp\settings.json
macOS: ~/.config/amp/settings.json
Linux: ~/.config/amp/settings.json

Add the MCP server configuration to your platform-specific settings file.

Editor Extension Configuration

For editor extensions that support Amp, add the configuration to your editor settings (e.g., .vscode/settings.json). All settings use the amp. prefix. Then, add evaluation rules in your project directory as AGENT.md:

# Evaluate Tool Calls Rule

## Critical Rule

- Always use the `evaluate_tool_call` function after any agent tool call to evaluate proper usage of the tool
- Use this function to validate that tools are being called correctly with proper parameters
- Apply this evaluation when assessing agent performance or troubleshooting tool usage issues

## When to Apply

- After Any Tool Call
- When reviewing agent conversations that involve tool calls
- When validating that an AI agent correctly used available tools
- When checking if tool parameters were set correctly
- When assessing overall agent tool calling performance

## Example Usage

<example>
When evaluating if an agent correctly used tools, always call:
evaluate_tool_call(available_tools, message_history)

This ensures proper validation of tool usage patterns and parameter correctness.
</example>

<example type="invalid">
Reviewing agent tool usage without using evaluate_tool_call function to validate the behavior.
</example>

Claude Desktop

Complete setup guide for integrating Quotient MCP with Claude Desktop using the stdio transport.

Prerequisites

Install uv: brew install uv (macOS) or follow uv installation guide

Setup Steps

1. Clone the Repository

git clone https://github.com/quotient-ai/quotient-mcp
cd quotient-mcp

2. Configure Claude Desktop Add the following configuration to your Claude Desktop config file: Access your config file:

via UI: Settings → Developer → Local MCP → Edit Config
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json

Add this configuration:

{
  "mcpServers": {
    "quotient-mcp": {
      "command": "uv",
      "args": [
        "run",
        "--with", "fastmcp>=2.10.6",
        "--with", "mcp==1.12.2", 
        "--with", "requests>=2.28.0",
        "fastmcp",
        "run",
        "/absolute/path/to/your/quotient-mcp/server.py"
      ]
    }
  }
}

Important: Replace /absolute/path/to/your/quotient-mcp/server.py with the full absolute path to your server.py file.

3. Configure Personal Preferences (Recommended) To automatically evaluate tool usage, add this rule to your Claude Desktop preferences:

Go to Settings → Profile → Personal Preferences
Copy and paste the following:

After making any tool call always use quotient-mcp evaluate_tool_call to evaluate proper tool use.

4. Restart Claude Desktop Completely quit Claude Desktop and reopen it for the changes to take effect. Once configured, you’ll have access to the quotient-mcp tools in Claude Desktop.

Claude Code

You can use the Quotient MCP server in Claude Code by adding the following to your project:

claude mcp add --transport http https://mcp.quotientai.co/mcp/

Then, add to your project instructions to always use the evaluate_tool_call function after any agent tool call.

# Project Instructions

## Tool Usage Requirements

- Always use `mcp__http-server__evaluate_tool_calls` after every tool call to evaluate the correctness of tool usage

OpenAI Agents SDK

You can use the Quotient MCP server in various agent SDKs such as the OpenAI Agents SDK by adding the following to your agent configuration:

from agents.mcp import MCPServerStreamableHttp

# custom prompt prefix for tool use evaluation
TOOL_USE_EVAL_PROMPT_PREFIX = """
After using any tool, you should use the tool_use_evaluator from quotient_mcp to evaluate the effectiveness and accuracy of the tool usage.
"""

# initialize MCP server connection
quotient_mcp = MCPServerStreamableHttp(
    params={
        "url": "https://mcp.quotientai.co/mcp/",
    },
    name="Quotient MCP Server",
)

# use in your agent configuration
agent = Agent(
    name="customer_service",
    instructions=TOOL_USE_EVAL_PROMPT_PREFIX + "Your agent instructions here...",
    mcp_servers=[quotient_mcp]
)

Open Source

Quotient MCP is open sourced at https://github.com/quotient-ai/quotient-mcp.

Transports

Quotient MCP supports both HTTP and stdio transports:

HTTP Transport (Remote Hosting)

The HTTP transport is used for remote hosting and is available at https://mcp.quotientai.co/mcp/.

Stdio Transport (Local Integration)

The stdio transport is designed for direct MCP client integration where the client manages the server process through standard input/output. Refer to the Claude Desktop section for an example implementation.

Get Started

Data

Tools

Steering

Links

Open Source Projects

Changelog

Quotient MCP

Tools

`evaluate_tool_call`

Example Usage

Underlying Model

Integration Examples

Cursor IDE

Amp

CLI Configuration

Editor Extension Configuration

Claude Desktop

Prerequisites

Setup Steps

Claude Code

OpenAI Agents SDK

Open Source

Transports

HTTP Transport (Remote Hosting)

Stdio Transport (Local Integration)

Get Started

Data

Tools

Steering

Links

Open Source Projects

Changelog

​Tools

​evaluate_tool_call

​Example Usage

​Underlying Model

​Integration Examples

​Cursor IDE

​Amp

​CLI Configuration

​Editor Extension Configuration

​Claude Desktop

​Prerequisites

​Setup Steps

​Claude Code

​OpenAI Agents SDK

​Open Source

​Transports

​HTTP Transport (Remote Hosting)

​Stdio Transport (Local Integration)

Tools

`evaluate_tool_call`

Example Usage

Underlying Model

Integration Examples

Cursor IDE

Amp

CLI Configuration

Editor Extension Configuration

Claude Desktop

Prerequisites

Setup Steps

Claude Code

OpenAI Agents SDK

Open Source

Transports

HTTP Transport (Remote Hosting)

Stdio Transport (Local Integration)