proxy.golang.org : github.com/riverqueue/river
Package river is a robust high-performance job processing system for Go and Postgres. See homepage, docs, and godoc, as well as the River UI. Being built for Postgres, River encourages the use of the same database for application data and job queue. By enqueueing jobs transactionally along with other database changes, whole classes of distributed systems problems are avoided. Jobs are guaranteed to be enqueued if their transaction commits, are removed if their transaction rolls back, and aren't visible for work _until_ commit. See transactional enqueueing for more background on this philosophy. Jobs are defined in struct pairs, with an implementation of `JobArgs` and one of `Worker`. Job args contain `json` annotations and define how jobs are serialized to and from the database, along with a "kind", a stable string that uniquely identifies the job. Workers expose a `Work` function that dictates how jobs run. Jobs are uniquely identified by their "kind" string. Workers are registered on start up so that River knows how to assign jobs to workers: A River `Client` provides an interface for job insertion and manages job processing and maintenance services. A client's created with a database pool, driver, and config struct containing a `Workers` bundle and other settings. Here's a client `Client` working one queue (`"default"`) with up to 100 worker goroutines at a time: ## Insert-only clients It's often desirable to have a client that'll be used for inserting jobs, but not working them. This is possible by omitting the `Queues` configuration, and skipping the call to `Start`: `Workers` can also be omitted, but it's better to include it so River can check that inserted job kinds have a worker that can run them. ## Stopping The client should also be stopped on program shutdown: There are some complexities around ensuring clients stop cleanly, but also in a timely manner. See graceful shutdown for more details on River's stop modes. `Client.InsertTx` is used in conjunction with an instance of job args to insert a job to work on a transaction: See the `InsertAndWork` example for complete code. Batch job insertion for efficiently inserting many jobs at once using Postgres `COPY FROM`. Cancelling jobs from inside a work function. Error and panic handling. Multiple queues to better guarantee job throughput, worker availability, and isolation between components. Periodic and cron jobs. Scheduled jobs that run automatically at their scheduled time in the future. Snoozing jobs from inside a work function. Subscriptions to queue activity and statistics, providing easy hooks for telemetry like logging and metrics. Test helpers to verify that jobs are inserted as expected. Transactional job completion to guarantee job completion commits with other changes in a transaction. Unique jobs by args, period, queue, and state. Web UI for inspecting and interacting with jobs and queues. Work functions for simplified worker implementation. ## Cross language enqueueing River supports inserting jobs in some non-Go languages which are then worked by Go implementations. This may be desirable in performance sensitive cases so that jobs can take advantage of Go's fast runtime. See developing River. Example_batchInsert demonstrates how many jobs can be inserted for work as part of a single operation. Example_completeJobWithinTx demonstrates how to transactionally complete a job alongside other database changes being made. Example_cronJob demonstrates how to create a cron job with a more complex schedule using a third party cron package to parse more elaborate crontab syntax. Example_customInsertOpts demonstrates the use of a job with custom job-specific insertion options. Example_errorHandler demonstrates how to use the ErrorHandler interface for custom application telemetry. Example_globalHooks demonstrates the use of hooks to modify River behavior which are global to a River client. Example_globalMiddleware demonstrates the use of middleware to modify River behavior which are global to a River client. Example_gracefulShutdown demonstrates a realistic-looking stop loop for River. It listens for SIGINT/SIGTERM (like might be received by a Ctrl+C locally or on a platform like Heroku to stop a process) and when received, tries a soft stop that waits for work to finish. If it doesn't finish in time, a second SIGINT/SIGTERM will initiate a hard stop that cancels all jobs using context cancellation. A third will give up on the stop procedure and exit uncleanly. Example_insertAndWork demonstrates how to register job workers, start a client, and insert a job on it to be worked. Example_jobArgsHooks demonstrates the use of hooks to modify River behavior. Example_jobCancel demonstrates how to permanently cancel a job from within Work using JobCancel. Example_jobCancelFromClient demonstrates how to permanently cancel a job from any Client using JobCancel. Example_jobSnooze demonstrates how to snooze a job from within Work using JobSnooze. The job will be run again after 5 minutes and the snooze attempt will decrement the job's attempt count, ensuring that one can snooze as many times as desired without being impacted by the max attempts. Example_periodicJob demonstrates the use of a periodic job. Example_queuePause demonstrates how to pause queues to prevent them from working new jobs, and later resume them. Example_scheduledJob demonstrates how to schedule a job to be worked in the future. Example_subscription demonstrates the use of client subscriptions to receive events containing information about worked jobs. Example_uniqueJob demonstrates the use of a job with custom job-specific insertion options. Example_workFunc demonstrates the use of river.WorkFunc, which can be used to easily add a worker with only a function instead of having to implement a full worker struct.
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/riverqueue/river
Keywords:
background-jobs
, go
, golang
, postgres
, postgresql
, queue
License: MPL-2.0
Latest release: 4 days ago
First release: over 1 year ago
Namespace: github.com/riverqueue
Stars: 3,507 on GitHub
Forks: 94 on GitHub
Total Commits: 369
Committers: 22
Average commits per author: 16.773
Development Distribution Score (DDS): 0.358
More commit stats: commits.ecosyste.ms
See more repository details: repos.ecosyste.ms
Last synced: 2 days ago