github.com/rulego/streamsql
Package streamsql is a lightweight, SQL-based IoT edge stream processing engine. StreamSQL provides efficient unbounded data stream processing and analysis capabilities, supporting multiple window types, aggregate functions, custom functions, and seamless integration with the RuleGo ecosystem. • Lightweight design - Pure in-memory operations, no external dependencies • SQL syntax support - Process stream data using familiar SQL syntax • Multiple window types - Sliding, tumbling, counting, and session windows • Event time and processing time - Support both time semantics for accurate stream processing • Watermark mechanism - Handle out-of-order and late-arriving data with configurable tolerance • Rich aggregate functions - MAX, MIN, AVG, SUM, STDDEV, MEDIAN, PERCENTILE, etc. • Plugin-based custom functions - Runtime dynamic registration, supports 8 function types • RuleGo ecosystem integration - Extend input/output sources using RuleGo components Basic stream data processing: StreamSQL supports multiple window types: StreamSQL supports two time semantics for window processing: ## Processing Time (Default) Processing time uses the system clock when data arrives. Windows are triggered based on data arrival time: ## Event Time Event time uses timestamps embedded in the data itself. Windows are triggered based on event timestamps, allowing correct handling of out-of-order and late-arriving data: ## Watermark and Late Data Handling Event time windows use watermark mechanism to handle out-of-order and late data: Key concepts: • MaxOutOfOrderness: Affects watermark calculation, delays window trigger to tolerate out-of-order data • AllowedLateness: Keeps window open after trigger to accept late data and update results • IdleTimeout: When data source is idle (no data arrives within timeout), watermark advances based on processing time to ensure windows can close • Watermark: Indicates that no events with timestamp less than watermark are expected StreamSQL supports plugin-based custom functions with runtime dynamic registration: Supported custom function types: • TypeMath - Mathematical calculation functions • TypeString - String processing functions • TypeConversion - Type conversion functions • TypeDateTime - Date and time functions • TypeAggregation - Aggregate functions • TypeAnalytical - Analytical functions • TypeWindow - Window functions • TypeCustom - General custom functions StreamSQL provides flexible log configuration options: StreamSQL provides deep integration with the RuleGo rule engine through two dedicated components for stream data processing: • streamTransform (x/streamTransform) - Stream transformer, handles non-aggregation SQL queries • streamAggregator (x/streamAggregator) - Stream aggregator, handles aggregation SQL queries Basic integration example:
proxy.golang.org
v0.10.7
3 months ago
8
Links
| Registry | proxy.golang.org |
| Source | Repository |
| Docs | Documentation |
| JSON API | View JSON |
| CodeMeta | codemeta.json |
Package Details
| PURL |
pkg:golang/github.com/rulego/streamsql
spec |
| License | Apache-2.0 |
| Namespace | github.com/rulego |
| First Release | 9 months ago |
| Last Synced | 14 days ago |
Repository
| Stars | 46 on GitHub |
| Forks | 4 on GitHub |