proxy.golang.org : github.com/ipfs/go-qringbuf

Package qringbuf provides a concurrency-friendly, zero-copy abstraction of io.ReadAtLeast(…) over a pre-allocated ring-buffer, populated asynchronously by a standalone goroutine. It is primarily designed for processing a series of consecutive sub-streams from a single io.Reader, each sub-stream in turn comprised of variable-length records. The buffer object DOES NOT ASSUME exclusive ownership of the supplied io.Reader, never reads more than instructed by an argument to StartFill(…), and exposes a standard sync.Mutex interface allowing pausing all operations when exclusive access of the underlying Reader is desired. In all cases below the background "collector" goroutine reading from the enclosed someIoReader into the ring buffer is guaranteed to: In code the basic usage looks roughly like this (error/flow handling elided): In addition one can operate over individual (sub)regions with "fearless concurrency": The specific technical guarantees made by an object of this package are: Unlike io.ReadAtLeast(…), errors from the underlying reader are always made available on NextRegion(…). As with the standard io.Read(…) semantics, an error can be returned together with a result. One should always check whether the *Region return value is nil first, before processing the error. See the documentation of io.Read(…) for an extended discussion. Changes of the NextRegion(…) "emitter" and collector positions are protected by a mutex on the qringbuf object. Calls modifying the buffer state will block until this lock can be obtained. The same mutex is exposed as part of the API, so one can pause the collector if a direct read and/or skip on the underlying io.Reader is needed. The *Region.{Reserve/Release}() functionality does not use the mutex, ensuring that an asynchronous Release() call can not be affected by the current state of the buffer. Reservation tracking is implemented as an atomically modified list of reservation counts, one int32 per SectorSize bytes of the buffer. The reservation system explicitly allows "recursive locking": you can hold an arbitrary number of reservations over a sector by repeatedly creating SubRegion(…) objects. Care must be taken to release every single reservation obtained previously, otherwise the collector will remain blocked forever. Follows an illustration of a contrived lifecycle of a hypothetical qringbuf object initialized with: Note that for brevity THE DIAGRAMS BELOW ARE DECIDEDLY NOT REPRESENTATIVE of a typical lifecycle. Normally BufferSize is an order of magnitude larger than MinRegion and MaxCopy, and the time spent waiting and copying data is insignificant in relation to all other possible states. Also outstanding async reservations typically trail the emitter very closely, so after a wrap the collector is virtually never blocked, contrary to what is depicted below. Instead the diagrams merely demonstrate the choices this library makes dealing with the "tricky parts" of maintaining the illusion of an arbitrary stream of contiguous bytes.

Registry - Source - Documentation - JSON
purl: pkg:golang/github.com/ipfs/go-qringbuf
License: Apache-2.0,MIT
Latest release: over 1 year ago
First release: over 2 years ago
Namespace: github.com/ipfs
Dependent packages: 6
Dependent repositories: 2
Stars: 10 on GitHub
Forks: 4 on GitHub
Docker dependents: 1
Docker downloads: 15
See more repository details: repos.ecosyste.ms
Last synced: 23 days ago

• Readme
• Versions 4
• Dependent Packages 6
• Maintainers 0