proxy.golang.org : github.com/vlab-research/spine
Package spine provides a simple abstraction for processing a Kafka stream as an at-least-once "sink" of any kind. Messages are split into "chunks" which can be processed by a user-provided blocking function that performs some side effect. The Kafka stream is then consumed in "batches", with each batch consisting of one or more chunks of messages. Chunks within the batch are parallelized to the user-provided side effect function in go loops. After each batch is sucessfully processed, the kafka consumer commits the offset, and consumes another batch.
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/vlab-research/spine
License: MIT
Latest release: almost 2 years ago
First release: over 4 years ago
Namespace: github.com/vlab-research
Dependent repositories: 1
Stars: 1 on GitHub
Forks: 0 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 26 days ago