galileo-agent-evals

Endpoint

Hosted endpoint · transport http

Connect

This is a remote MCP server — point an MCP client at the endpoint URL. Clients with native remote support (Claude, Cursor, VS Code, …):

For stdio-only clients (older Claude Desktop, etc.), bridge with mcp-remote:

If the server requires authentication, add the provider's token/header (e.g. an X-API-Key) per its docs. A quick reachability check:

Tool quality observed · not scored

Unhinted state-changing: create_galileo_dataset, create_prompt_template

Heaviest: create_galileo_dataset 1346

From an anonymous initialize + tools/list capture measured 2026-10-20; tokens are chars/3.6 of each tool definition. Recorded for the Kin Score's agent readiness as an observed signal — it contributes no points yet.

Tools

• integrate_galileo_with_langchain — Integrate Galileo observability into LangChain applications. **Supported Languages:** python **Parameters:** - language (optional): Programming language for integration guide. Defaults to "python". **Provides:** - Complete integration code examples and setup instructions - Automatic logging of prompts, responses, model parameters, and token usage - Environment configuration templates - Language-specific best practices Returns working code snippets ready to copy and paste for robust LLM observability.
• integrate_galileo_with_openai — Integrate Galileo observability into OpenAI API applications. **Supported Languages:** python, typescript **Parameters:** - language (optional): Programming language for integration guide. Defaults to "python". **Provides:** - Complete integration code examples and setup instructions - Automatic logging of prompts, responses, model parameters, and token usage - Environment configuration templates - Language-specific best practices Returns working code snippets ready to copy and paste for robust LLM observability.
• get_logstream_insights — Get the insights (also known as signals) for a log stream in a specific project. This tool analyzes log streams and provides insights/signals that identify issues, patterns, and suggest improvements. **When to use this tool:** - User asks for "insights" or "signals" about a log stream - User mentions a project name and wants to analyze a log stream - User asks questions like: "What's wrong with my application?", "Get insights about my log stream", "Show me signals for my logs" **Parameters:** - project (required): The exact name of the Galileo project. Extract this from phrases like: - "in the {project} project" - "for project {project}" - "from {project}" Example: If user says "in the customer-support project", use project="customer-support" - log_stream (required): The exact name of the log stream. Extract this from phrases like: - "log stream {log_stream}" - "logs from {log_stream}" - "the {log_stream} log stream" - If user says "my log stream" without a name, ask them for the specific log stream name Example: If user says "about my log stream in customer-support", you may need to ask for the log stream name, or infer it from context **Example Natural Language Queries:** - "Get insights about my log stream in the customer-support project" → project="customer-support", log_stream={ask user or infer from context} - "Get insights for the api-logs log stream in customer-support" → project="customer-support", log_stream="api-logs" - "What insights do you have for my logs in project customer-support?" → project="customer-support", log_stream={ask user for specific log stream name} **Returns:** - A list of insights/signals about the log stream, including: - Issues and anomalies detected - Patterns and trends - Suggested improvements - Session notes and trace notes if available
• get_logstream_signals — Get the signals (also known as insights) for a log stream in a specific project. This tool analyzes log streams and provides signals/insights that identify issues, patterns, and suggest improvements. **When to use this tool:** - User asks for "signals" or "insights" about a log stream - User mentions a project name and wants to analyze a log stream - User asks questions like: "What's wrong with my application?", "Get signals about my log stream", "Show me insights for my logs" **Parameters:** - project (required): The exact name of the Galileo project. Extract this from phrases like: - "in the {project} project" - "for project {project}" - "from {project}" Example: If user says "in the customer-support project", use project="customer-support" - log_stream (required): The exact name of the log stream. Extract this from phrases like: - "log stream {log_stream}" - "logs from {log_stream}" - "the {log_stream} log stream" - If user says "my log stream" without a name, ask them for the specific log stream name Example: If user says "about my log stream in customer-support", you may need to ask for the log stream name, or infer it from context **Example Natural Language Queries:** - "Get signals about my log stream in the customer-support project" → project="customer-support", log_stream={ask user or infer from context} - "Get insights for the api-logs log stream in customer-support" → project="customer-support", log_stream="api-logs" - "What signals do you have for my logs in project customer-support?" → project="customer-support", log_stream={ask user for specific log stream name} **Returns:** - A list of signals/insights about the log stream, including: - Issues and anomalies detected - Patterns and trends - Suggested improvements - Session notes and trace notes if available
• validate_dataset — Check the status of a synthetic dataset generation job and view dataset contents. This tool allows the user to check the progress of a synthetic dataset generation job that was previously started using the create_galileo_dataset tool. It always shows the first 10 rows of the dataset for preview, regardless of generation status. Users can start by asking a question such as: Check the status of my dataset generation job. **Parameters:** - dataset_id (required): The id of the dataset to check the generation status for. **Returns:** - The current status and progress of the synthetic dataset generation job, including progress message, steps completed, and total steps. - Always includes a preview of the first 10 rows of the dataset with column names and values, whether generation is complete, in progress, or failed. - Total number of rows in the dataset and how many are being shown. **Example:** After creating a dataset with create_galileo_dataset, use this tool to check if the generation is complete and preview the generated data.
• create_galileo_dataset — Create a new Galileo dataset with multiple data source options. **Prerequisites:** - Requires a valid Galileo API key configured in your MCP settings - You must have an active Galileo account with dataset creation permissions **Creation Options:** Ask the user which option they prefer: 1. Synthetic data generation: Automatically generate data based on description 2. CSV file: Provide CSV content to create dataset (columns: input, output, optional metadata) 3. JSON data: Provide structured JSON data **Required User Inputs:** Please ask the user to provide the following information: - description: A description of the dataset they want to create. This will be used as instructions for the synthetic data generation if the user chooses to use the synthetic data source type. - data_source_type: The type of data source to use for the dataset. Valid types: "synthetic", "csv", "json". **Optional User Inputs:** - model: The model to use for generating synthetic data (e.g., "gpt-4.1-mini", "claude-4-sonnet"). - count: Number of synthetic samples to generate (must be between 1 and 100). - data_types: Types of data to generate. Valid types: "General Query", "Prompt Injection", "Off-Topic Query", "Toxic Content in Query", "Multiple Questions in Query". - sample_data: Sample data to use as reference during generation. Can be newline-separated examples. - csv_content: CSV file content as string (for CSV data source type). - json_data: JSON data as string or array of objects (for JSON data source type). - name: Name for the dataset (optional, defaults to "New Dataset"). **Returns:** - Dataset ID for the created dataset - Status of the generation job - Parameters used for generation - Next steps for accessing the dataset **Example Tool Call:** description="Generate customer service queries about product returns",model="gpt-4o-mini",count=20,data_types="General Query,Off-Topic Query",sample_data="How do I return a defective product? What is your return policy?" **What this tool does:** 1. Authenticates using your Galileo API key 2. Creates a new dataset in your Galileo organization 3. Dispatches a background job to generate synthetic data or processes uploaded data 4. Returns the dataset ID for tracking progress 5. Generated data will be available in your Galileo console **Instructions for Agent:** Always ask the user to provide: description and data_source_type before making the tool call. These are required for successful dataset creation. Ask the user to wait for the dataset to be created before using the validate_dataset tool. Continue to ask the user if they want to validate the dataset that was just created.
• create_prompt_template — Create a new global prompt template that can be used across all projects. **Prerequisites:** - Requires a valid Galileo API key configured in your MCP settings - You must have an active Galileo account with prompt template creation permissions **Required User Inputs:** Please ask the user to provide the following information: - name: A descriptive name for the prompt template. - template: The actual prompt template content. Can be plain text or JSON format for chat templates. **Optional User Inputs:** - raw: Boolean flag indicating if this is a raw template (default: false). - output_type: The expected output type for the template (optional). - model_alias: The model to use with this template (default: "gpt-4.1-mini"). - temperature: Controls randomness in the output (0.0 to 2.0, default: 0.7). - max_tokens: Maximum number of tokens in the response (default: 1000). - top_p: Controls diversity of the output (0.0 to 1.0, default: 1.0). - frequency_penalty: Penalty for frequent tokens (-2.0 to 2.0, default: 0.0). - presence_penalty: Penalty for repeated tokens (-2.0 to 2.0, default: 0.0). **Returns:** - Template ID for the created template - Template details including name, version, and settings - Success confirmation with template metadata **Example Tool Calls:** Simple text template: name="Customer Support Response",template="You are a helpful customer support agent. Please respond to the following customer inquiry: {inquiry}",model_alias="gpt-4.1-mini",temperature=0.5 JSON chat template: name="AI Coding Assistant",template="[{"role":"system","content":"You are an Extremely friendly and helpful AI Coding assistant"}]",model_alias="gpt-4.1-mini",temperature=0.3 **What this tool does:** 1. Authenticates using your Galileo API key 2. Creates a new global prompt template in your Galileo organization 3. Sets up the template with the specified settings and configuration 4. Returns the template ID and details for future use 5. Template will be available globally across all projects **Instructions for Agent:** Always ask the user to provide: name and template before making the tool call. These are required for successful template creation. The template will be created as a global template, accessible across all projects in the organization.
• setup_galileo_experiment — Setup and create a Galileo experiment using the Galileo SDK. **Supported Languages:** python **Parameters:** - language (optional): Programming language for experiment setup guide. Defaults to "python". **Provides:** - Complete experiment setup code examples and instructions - Step-by-step guide for creating and running experiments - Dataset preparation and metrics configuration - Integration with existing application code - Advanced usage patterns for agentic/RAG workflows **What this tool helps you do:** - Set up Galileo experiments with datasets (remote or inline) - Create and configure prompts for experiments - Define custom metrics and use built-in scorers - Run experiments with proper logging and tracing - Integrate experiment workflows into existing applications - Inspect and analyze experiment results Returns working code snippets ready to copy and paste for robust experiment setup and execution.
• search_docs — Search across the Galileo documentation to find relevant information, code examples, API references, and guides. Use this tool when you need to answer questions about Galileo, find specific documentation, understand how features work, or locate implementation details. The search returns contextual content with titles and direct links to the documentation pages. **Parameters:** - query (required): The search query to find relevant documentation. **Returns:** - Relevant documentation content and links based on the search query. **Example:** search_docs(query="how to set up data logging")

About MCP

The Model Context Protocol (MCP) is an open protocol Anthropic introduced for connecting LLM-based agents to external tools and data sources. Providers publish MCP servers that expose their API surface as structured, discoverable tools — an MCP-compatible client (Claude Desktop, Cursor, Cline, Continue, etc.) can connect to the server and call its tools without any per-provider integration code.

Browse every MCP server on the APIs.io network or compare with the broader Agent Skill surfaces of the same providers.

Work with this as data

Every MCP server here is available over the APIs.io API and to AI agents over MCP.

curl "https://apis.io/api/v1/mcp/galileo-agent-evals"

curl "https://apis.io/api/v1/mcp?limit=25"