Antfly Query Operations API

Execute queries across tables using full-text search, vector similarity search, or hybrid approaches combining both with Reciprocal Rank Fusion (RRF). ## Query Types ### Full-Text Search Uses Antfly query syntax for powerful text searching: - Field-specific queries: `body:computer` - Boolean operators: `AND`, `OR`, `NOT` - Range queries: `year:>2020` - Phrase queries: `"exact phrase"` ### Semantic Search Natural language queries using vector similarity: - Searches across specified embedding indexes - Returns semantically similar documents - Supports multiple embedding models ### Hybrid Search (RRF) Combines full-text and semantic search using Reciprocal Rank Fusion: ```json { "full_text_search": {"query": "body:computer"}, "semantic_search": "artificial intelligence", "indexes": ["title_body_nomic"] } ``` ## Filtering and Exclusion Refine results with powerful filtering: ### Filter Prefix Only return documents whose keys start with a specific string: ```json {"filter_prefix": "user:"} ``` Returns only keys like "user:123", "user:456" ### Filter Query Apply an Antfly query as an AND condition: ```json {"filter_query": {"query": "+category:technology +year:>2020"}} ``` Documents must match both the main query and filter query. ### Exclusion Query Exclude documents matching an Antfly query (NOT condition): ```json {"exclusion_query": {"query": "category:deprecated OR status:archived"}} ``` **Performance Note**: All filters are applied before scoring and ranking, improving query performance. ## RAG (Retrieval-Augmented Generation) Execute queries and generate summaries using LLMs: - Supports multiple retrieval queries - Streams results as Server-Sent Events (SSE) - Returns markdown summaries with inline citations - Combines results from multiple tables ## Retrieval Agent Intelligent query routing with automatic query generation: - Classifies queries as "question" or "search" - Generates optimal search queries across tables - Returns generated responses (for questions) or document IDs (for searches) - Streams classification, keywords, queries, and results ## Improving Search Relevance Antfly provides several techniques to improve search result quality: ### 1. Hybrid Search (Combine Full-Text + Semantic) Use both `full_text_search` and `semantic_search` together. Results are merged using Reciprocal Rank Fusion (RRF) to balance keyword matching and semantic similarity. ### 2. Reranking Apply a cross-encoder model to re-score results based on query-document relevance. Rerankers are more accurate than embedding models but slower, so use them on already-filtered results (e.g., top 50-100). ```json { "semantic_search": "gaming laptops", "limit": 10, "reranker": { "provider": "antfly", "model": "mixedbread-ai/mxbai-rerank-base-v1", "field": "description", "candidate_count": 100 } } ``` ### 3. Result Pruning Filter out low-quality results by detecting score gaps or setting minimum thresholds: ```json { "semantic_search": "machine learning", "pruner": { "min_score_ratio": 0.5, "max_score_gap_percent": 30.0 } } ``` - `min_score_ratio`: Keep only results scoring at least X% of the top result - `max_score_gap_percent`: Stop when a consecutive score gap exceeds X% of the total score range ### 4. Merge Strategies Choose how full-text and semantic results are combined: - `rrf` (default): Reciprocal Rank Fusion - balanced, works well in most cases - `rsf`: Relative Score Fusion - normalizes and weights scores differently ### 5. Query Filtering Use `filter_query` and `filter_prefix` to pre-filter before scoring, improving both performance and relevance by removing irrelevant documents early. **Best Practice**: Combine techniques for optimal results: 1. Start with hybrid search (full-text + semantic) 2. Apply filters to narrow the domain 3. Use reranker on top results 4. Apply pruner to remove outliers