786 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Ken Hibino	01eeb8756e	(cli): Update queue inspect cmd to show # of groups and aggregating tasks	2022-04-11 16:55:43 -07:00
Ken Hibino	47af17cfb4	Fix RDB.CurrentStats to report the correct queue size	2022-04-11 16:55:43 -07:00
Ken Hibino	eb064c2bab	Fix AggregationCheck with unlimited size to clear group name from ... all-groups set	2022-04-11 16:55:43 -07:00
Ken Hibino	652939dd3a	Update memory usage redis lua script to account for groups	2022-04-11 16:55:43 -07:00
Ken Hibino	efe3c74037	Show number of groups and aggregating task count in QueueInfo	2022-04-11 16:55:43 -07:00
Ken Hibino	74d2eea4e0	Clear group if aggregation set empties the group	2022-04-11 16:55:43 -07:00
Ken Hibino	60a4dc1401	Add test for DeleteAggregationSet error case	2022-04-11 16:55:43 -07:00
Ken Hibino	4b716780ef	Rewrite test for DeleteAggregationSet function with a new pattern	2022-04-11 16:55:43 -07:00
Ken Hibino	e63f41fb24	Fix DeleteAggregationSet	2022-04-11 16:55:43 -07:00
Ken Hibino	1c388baf06	Implement RDB.ReclaimStaleAggregationSets	2022-04-11 16:55:43 -07:00
Ken Hibino	47a66231b3	Store aggregation set *key* in all aggreationsets zset	2022-04-11 16:55:43 -07:00
Ken Hibino	3551d3334c	Use zset for aggregation set to preserve score	2022-04-11 16:55:43 -07:00
Ken Hibino	8b16ede8bc	Declare ReclaimStaleAggregationSets	2022-04-11 16:55:43 -07:00
Ken Hibino	c8658a53e6	Add aggregator test	2022-04-11 16:55:43 -07:00
Ken Hibino	562506c7ba	Fix client to return error when nil task is passed	2022-04-11 16:55:43 -07:00
Ken Hibino	888b5590fb	Make GroupMaxSize and GroupMaxDelay config optional	2022-04-11 16:55:43 -07:00
Ken Hibino	196db64d4d	Run aggregator on the server	2022-04-11 16:55:43 -07:00
Ken Hibino	4b35eb0e1a	Fix RDB.AggregationCheck when run against an empty group	2022-04-11 16:55:43 -07:00
Ken Hibino	b29fe58434	Implement RDB.ListGroups	2022-04-11 16:55:43 -07:00
Ken Hibino	7849b1114c	Implement RDB.DeleteAggregationSet	2022-04-11 16:55:43 -07:00
Ken Hibino	99c00bffeb	Implement RDB.AggregationCheck	2022-04-11 16:55:43 -07:00
Ken Hibino	4542b52da8	Check for aggregation at an interval <= gracePeriod	2022-04-11 16:55:43 -07:00
Ken Hibino	d841dc2f8d	Add initial implementation of aggregator	2022-04-11 16:55:43 -07:00
Ken Hibino	ab28234767	Update client dependency to base.Broker	2022-04-11 16:55:43 -07:00
Ken Hibino	eb27b0fe1e	Add TaskMessageBuilder type as a test helper	2022-04-11 16:55:43 -07:00
Ken Hibino	088be63ee4	Update forwarder to use time.Timer	2022-04-11 16:55:43 -07:00
Ken Hibino	ed69667e86	Update ForwardIfReady test with group	2022-04-11 16:55:43 -07:00
Ken Hibino	4e8885276c	Update client to store groupKey under TaskMessage	2022-04-11 16:55:43 -07:00
Ken Hibino	401f7fb4fe	Add GroupKey field to TaskMessage	2022-04-11 16:55:43 -07:00
Ken Hibino	61854ea1dc	Update RDB.ForwardIfReady to forward to group if groupKey is specified	2022-04-11 16:55:43 -07:00
Ken Hibino	f17c157b0f	Update Client to add task to group if Group option is specified	2022-04-11 16:55:43 -07:00
Ken Hibino	8b582899ad	Add RDB.AddToGroup and RDB.AddToGroupUnique methods	2022-04-11 16:55:43 -07:00
Ken Hibino	e3d2939a4c	Add helper functions to generate group key	2022-04-11 16:55:43 -07:00
Ken Hibino	2ce71e83b0	Add Group task option	2022-04-11 16:55:43 -07:00
Ken Hibino	1608366032	Add group related configuration options	2022-04-11 16:55:43 -07:00
ashang	3f4f0c1daa	Use explicit types for limit constants	2022-03-29 06:30:10 -07:00
Ken Hibino	f94a65dc9f	Add go1.18 to build workflow matrix	2022-03-22 06:51:28 -07:00
Erwan Leboucher	04d7c8c38c	Add rediss url parsing support	2022-02-24 08:30:55 -08:00
Ken Hibino	c04fd41653	v0.22.1	2022-02-20 06:22:55 -08:00
Ken Hibino	7e5efb0e30	Drop GT option from RDB.ExtendLease ... GT option in ZAdd is supported for redis v6.2.0 or above. This Change fixes redis version compatibility (currently v4.0+)	2022-02-20 06:20:38 -08:00
Ken Hibino	cabf8d3627	Fix changelog	2022-02-19 06:21:56 -08:00
Ken Hibino	a19909f5f4	v0.22.0	2022-02-19 06:20:05 -08:00
Ken Hibino	cea5110d15	Add IsOrphaned field to TaskInfo	2022-02-19 06:15:44 -08:00
Ken Hibino	9b63e23274	Update log messages	2022-02-19 06:15:44 -08:00
Ken Hibino	de25201d9f	Make timeutil.SimulatedClock concurrency safe	2022-02-19 06:15:44 -08:00
Ken Hibino	ec560afb01	Fix processor test	2022-02-19 06:15:44 -08:00
Ken Hibino	d4006894ad	Remove base.DeadlinesKey	2022-02-19 06:15:44 -08:00
Ken Hibino	59927509d8	Remove timeout and deadline fields under task key	2022-02-19 06:15:44 -08:00
Ken Hibino	8211167de2	Update processor to create a lease and watch for expiration	2022-02-19 06:15:44 -08:00
Ken Hibino	d7169cd445	Update heartbeat to extend lease of active workers	2022-02-19 06:15:44 -08:00