Skip to main content

Elasticsearch Sink

A sink connector that consumes messages from Iggy streams and indexes them to Elasticsearch.

Configuration​

  • url: Elasticsearch cluster URL
  • index: Target index name
  • username/password: Optional authentication credentials
  • batch_size: Bulk indexing batch size (default: 100)
  • timeout_seconds: Request timeout (default: 30s)
  • create_index_if_not_exists: Automatically create index (default: true)
  • index_mapping: Index mapping configuration
[plugin_config]
url = "http://localhost:9200"
index = "events"
username = "elastic"
password = "changeme"
batch_size = 100
timeout_seconds = 30
create_index_if_not_exists = true
index_mapping = """
{
  "mappings": {
    "properties": {
      "timestamp": { "type": "date" },
      "message": { "type": "text" },
      "service_name": { "type": "keyword" },
      "user_id": { "type": "keyword" },
      "level": { "type": "keyword" }
    }
  }
}
"""

Features​

  • Bulk indexing optimization
  • Automatic index creation
  • Error handling and retry mechanisms
  • Metadata field injection
  • Support for multiple data formats