Ecosyste.ms: Packages

proxy.golang.org : github.com/kavehmz/queue

Package queue is a simple Queue system written in Go that will uses Redis. Focus of this design is mainly horisontal scalability via concurrency, partitioning and fault-detection Queues can be partitions in to more than one Redis if necessary. Number of redis partitions is set by using Urls function and setting slice of Redis URL connections. Redis partitioning is required in cases that one redis cannot handle the load because of IO, moemory or in rare situations CPU limitations. In case of crash record of all incomplete tasks will be kepts in redis as keys with this format ID will indicate the ID of failed tasks. To use this library you need to use queue struct. Adding tasks is done by calling AddTask. This function will accept an ID and the task itself that will be as a string. ID can be used in a special way. If ID of two tasks are the same while processing AnalysePool will send them to the same analyser goroutine if analyzer waits enough. This feature can be used in analyser to process a set of related tasks one after another. If you are adding ID related tasks and you need to spinup more than one AnalysePool to fetch and distribute tasks you need to insert the tasks into separate queue or separate redis servers. To have separate queues you can set Queues number in the queue strcuture. AnalysePool accepts 3 parameters. One analyzerID that will identify which redis pool this AnalysePool will connect to. AnalysePool need two closures, analyzer and exitOnEmpty. Format of those closure are as follows.

Registry - Source - Documentation - JSON
purl: pkg:golang/github.com/kavehmz/queue
Keywords: golang, queue, redis
License: MIT
Latest release: over 2 years ago
First release: over 2 years ago
Namespace: github.com/kavehmz
Stars: 74 on GitHub
Forks: 3 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 23 days ago

• Readme
• Versions 1
• Dependent Packages 0
• Maintainers 0