proxy.golang.org : github.com/andreyvit/journal
Package journal implements WAL-like append-only journals. A journal is split into segments; the last segment is the one being written to. Intended use cases: Features: Suitable for a large number of very short records. Per-record overhead can be as low as 2 bytes. Suitable for very large records, too. (In the future, it will be possible to write records in chunks.) Fault-resistant. Self-healing. Verifies the checksums and truncates corrupted data when opening the journal. Performant. Automatically rotates the files when they reach a certain size. TODO: Trigger rotation based on time (say, each day gets a new segment). Basically limit how old in-progress segments can be. Allow to rotate a file without writing a new record. (Otherwise rarely-used journals will never get archived.) Give work-in-progress file a prefixed name (W*). Auto-commit every N seconds, after K bytes, after M records. Option for millisecond timestamp precision? Reading API. (Search based on time and record ordinals.) Segment files: We always set bit 0 of commit checksums, and we use size*2 when encoding records; so bit 0 of the first byte of an item indicates whether it's a record or a commit. Timestamps are 32-bit unix times and have 1 second precision. (Rationale is that the primary use of timestamps is to search logs by time, and that does not require a higher precision. For high-frequency logs, with 1-second precision, timestamp deltas will typically fit within 1 byte.)
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/andreyvit/journal
License: BSD-2-Clause
Latest release: about 1 month ago
First release: 4 months ago
Namespace: github.com/andreyvit
Stars: 0 on GitHub
Forks: 0 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 12 days ago