Search Engine

The search engine in Kinesis API provides powerful and flexible data filtering capabilities through an intuitive query language. It allows you to search through records in any table with sophisticated matching operations and complex filter conditions.

Query Language

The search engine implements a custom query language that supports:

• String operations: contains, equals, startsWith, endsWith, fuzzy matching
• Numeric operations: greater than, less than, equals, between ranges
• Boolean operations: equals, not equals
• Logical operators: AND, OR, NOT
• Grouping with parentheses for complex expressions

Using the Search API

REST API Endpoints

There are two main endpoints for using the search feature:

1. Search with structured filter (POST)

POST /search/
Content-Type: application/json
Authorization: Bearer <token>

{
  "uid": 0,
  "search": {
    "table_name": "users",
    "filter": {
      "String": {
        "field": "name",
        "matcher": {"Contains": ["John", false]}
      }
    },
    "top_n": 10
  }
}

2. Search with query string (GET)

GET /search/?uid=0&table_name=users&query=name contains "John"&top_n=10
Authorization: Bearer <token>

The query string approach provides a more human-readable and concise way to create search filters.

Query Syntax Examples

String Operations

name contains "John"             // Case-insensitive substring match
name equals "John Doe", true     // Case-sensitive exact match
name startswith "J"              // Prefix matching
name endswith "son"              // Suffix matching
name fuzzy "Jon", 2, true        // Fuzzy matching with edit distance 2 and substring enabled

Numeric Operations

age > 30                         // Greater than
age < 20                         // Less than
age = 25                         // Equals
age between 20 and 30            // Range (inclusive)

Boolean Operations

active equals true               // Boolean equality
active equals false              // Boolean equality

Logical Operators

name contains "John" AND age > 30                  // Logical AND
name contains "John" OR name contains "Jane"       // Logical OR
NOT name contains "Admin"                          // Logical NOT

Complex Expressions

name contains "John" AND (age > 30 OR role equals "admin")    // Nested conditions
(name contains "John" OR name contains "Jane") AND age > 30    // Multiple groupings

How It Works

The search engine processes queries through three main steps:

1. Parsing: Converts string queries into a structured filter representation
2. Evaluation: Applies the filter to records, calculating match scores
3. Ranking: Orders results by relevance score and returns the top N results

The engine uses indexes where available for better performance, falling back to full table scans when necessary.

Performance Considerations

• Use indexed fields in your queries when possible
• Prefer exact matches over fuzzy searches for better performance
• Limit results with the top_n parameter for large tables
• Complex queries with multiple conditions may take longer to process