go-pkg/asynq
2
0
Fork 0
mirror of https://github.com/hibiken/asynq.git synced 2024-12-27 00:02:19 +08:00
Code Issues Projects Releases Wiki Activity
917 Commits 20 Branches 53 Tags 12 MiB
87375b5534
Commit Graph

362 Commits

Author SHA1 Message Date
Ken Hibino
7849b1114c Implement RDB.DeleteAggregationSet 2022-04-11 16:55:43 -07:00
Ken Hibino
99c00bffeb Implement RDB.AggregationCheck 2022-04-11 16:55:43 -07:00
Ken Hibino
d841dc2f8d Add initial implementation of aggregator 2022-04-11 16:55:43 -07:00
Ken Hibino
eb27b0fe1e Add TaskMessageBuilder type as a test helper 2022-04-11 16:55:43 -07:00
Ken Hibino
ed69667e86 Update ForwardIfReady test with group 2022-04-11 16:55:43 -07:00
Ken Hibino
401f7fb4fe Add GroupKey field to TaskMessage 2022-04-11 16:55:43 -07:00
Ken Hibino
61854ea1dc Update RDB.ForwardIfReady to forward to group if groupKey is specified 2022-04-11 16:55:43 -07:00
Ken Hibino
f17c157b0f Update Client to add task to group if Group option is specified 2022-04-11 16:55:43 -07:00
Ken Hibino
8b582899ad Add RDB.AddToGroup and RDB.AddToGroupUnique methods 2022-04-11 16:55:43 -07:00
Ken Hibino
e3d2939a4c Add helper functions to generate group key 2022-04-11 16:55:43 -07:00
ashang
3f4f0c1daa
Use explicit types for limit constants 2022-03-29 06:30:10 -07:00
Ken Hibino
c04fd41653 v0.22.1 2022-02-20 06:22:55 -08:00
Ken Hibino
7e5efb0e30 Drop GT option from RDB.ExtendLease 
GT option in ZAdd is supported for redis v6.2.0 or above.
This Change fixes redis version compatibility (currently v4.0+)
 2022-02-20 06:20:38 -08:00
Ken Hibino
a19909f5f4 v0.22.0 2022-02-19 06:20:05 -08:00
Ken Hibino
de25201d9f Make timeutil.SimulatedClock concurrency safe 2022-02-19 06:15:44 -08:00
Ken Hibino
d4006894ad Remove base.DeadlinesKey 2022-02-19 06:15:44 -08:00
Ken Hibino
59927509d8 Remove timeout and deadline fields under task key 2022-02-19 06:15:44 -08:00
Ken Hibino
d7169cd445 Update heartbeat to extend lease of active workers 2022-02-19 06:15:44 -08:00
Ken Hibino
dfae8638e1 Update RDB methods to work with lease 2022-02-19 06:15:44 -08:00
Ken Hibino
b9943de2ab Add Lease type to base package 2022-02-19 06:15:44 -08:00
Ken Hibino
871474f220 Update heartbeat goroutine to call ExtendLease on active tasks 2022-02-19 06:15:44 -08:00
Ken Hibino
87dc392c7f Add RDB.ExtendLease method 2022-02-19 06:15:44 -08:00
Ken Hibino
dabcb120d5 Update recoverer to use ListLeaseExpired 2022-02-19 06:15:44 -08:00
Ken Hibino
bc2f1986d7 Update ListDeadlineExceeded to ListLeaseExpired 2022-02-19 06:15:44 -08:00
Ken Hibino
b8cb579407 Update RDB methods to use lease instead of deadlines set 2022-02-19 06:15:44 -08:00
Ken Hibino
d865d89900 Update RDB.Dequeue to insert task ID to lease set 2022-02-19 06:15:44 -08:00
Ken Hibino
852af7abd1 Add base.LeaseKey helper function 2022-02-19 06:15:44 -08:00
Ken Hibino
5490d2c625 Fix tests 2022-02-16 07:08:01 -08:00
Binaek Sarkar
ebd7a32c0f conventions 2022-02-16 06:43:08 -08:00
Binaek Sarkar
55d0610a03 test and changelog 2022-02-16 06:43:08 -08:00
Binaek Sarkar
ab8a4f5b1e review corrections 2022-02-16 06:43:08 -08:00
Binaek Sarkar
d7ceb0c090 first cut 2022-02-16 06:43:08 -08:00
Ken Hibino
dff2e3a336 v0.21.0 2022-01-22 06:15:29 -08:00
Ken Hibino
25832e5e95
Fix bug related to concurrently executing server state changes 2022-01-12 09:10:56 -08:00
Ken Hibino
aa26f3819e
Fix flaky tests 2022-01-05 09:07:42 -08:00
Ken Hibino
8dfabfccb3 Fix build 2021-12-19 07:06:37 -08:00
Ken Hibino
5f20edcbd1 v0.20.0 2021-12-19 07:00:21 -08:00
Ken Hibino
1ddb2f7bce Use math.MaxInt64 instead of custom const 2021-12-19 06:58:12 -08:00
Ken Hibino
82d18e3d91
Record total tasks processed/failed 2021-12-16 16:53:02 -08:00
Ken Hibino
970cb7a606 v0.19.1 2021-12-12 06:16:13 -08:00
Ken Hibino
157e97e72e Update changelog 2021-12-11 10:29:43 -08:00
Ken Hibino
22e6c9d297 Delete "pending_since" under task-key when state changes to active 2021-12-11 10:29:43 -08:00
Ken Hibino
99a6750656 Add Latency field to QueueInfo 2021-12-11 10:29:43 -08:00
Ken Hibino
e7c1c3ad6f Use clock in RDB 2021-12-11 10:29:43 -08:00
Ken Hibino
c9183374c5 Add internal timeutil package 2021-12-11 10:29:43 -08:00
Ken Hibino
6e7106c8f2 Record time when task moved to pending state 2021-12-11 10:29:43 -08:00
Ken Hibino
9f2c321e98
Add EnqueueContext method to Client 2021-11-15 16:34:26 -08:00
Ken Hibino
413afc2ab6 v0.19.0 2021-11-06 15:20:09 -07:00
Ken Hibino
f4ddac4dcc Introduce Task Results 
* Added Retention Option to specify retention TTL for tasks
* Added ResultWriter as a client interface to write result data for the associated task
 2021-11-06 15:18:42 -07:00
Ken Hibino
4638405cbd Fix flaky test 2021-11-06 15:18:42 -07:00
Ken Hibino
dbdd9c6d5f Update RDB Enqueue and Schedule methods to check for task ID conflict 2021-11-06 15:18:42 -07:00
Ken Hibino
2261c7c9a0 Change TaskMessage.ID type from uuid.UUID to string 2021-11-06 15:18:42 -07:00
Ajat Prabha
23c522dc9f
Add asynq/x/rate package 
- Added a directory /x for external, experimental packeges
- Added a `rate` package to enable rate limiting across multiple asynq worker servers
 2021-11-03 15:55:23 -07:00
Ken Hibino
d612a8a9e4 v0.18.6 2021-10-03 05:55:49 -07:00
Jason White
b3ef9e91a9
Upgrade go-redis/redis to version 8 2021-09-02 05:56:02 -07:00
Ken Hibino
05534c6f24 v0.18.5 2021-09-01 06:02:49 -07:00
Ken Hibino
f0db219f6a
Add IsFailure to Config 
With this IsFailure config, users can provide a predicate function to 
determine whether the error returned from Handler counts as a failure.
 2021-09-01 06:00:54 -07:00
Ken Hibino
421dc584ff v0.18.4 2021-08-17 17:12:33 -07:00
Ken Hibino
c197902dc0 v0.18.3 2021-08-09 08:59:35 -07:00
Ken Hibino
e6355bf3f5 Use approximate memory usage for QueueInfo 2021-08-09 08:58:44 -07:00
Ken Hibino
684a7e0c98 v0.18.2 2021-07-15 06:56:53 -07:00
Ken Hibino
46b23d6495 Allow upper case characters in queue name 2021-07-15 06:55:47 -07:00
Ken Hibino
c0ae62499f v0.18.1 2021-07-04 06:39:54 -07:00
Ken Hibino
d02b722d8a v0.18.0 2021-06-29 16:36:52 -07:00
Ken Hibino
99c7ebeef2 Add migration command in CLI 2021-06-29 16:34:21 -07:00
Ken Hibino
e01c6379c8 Fix lua script for redis-cluster mode 2021-06-29 16:34:21 -07:00
Ken Hibino
a0df047f71 Use md5 to generate checksum for unique key 2021-06-29 16:34:21 -07:00
Ken Hibino
68dd6d9a9d (fix): Clear unique lock when task is deleted via Inspector 2021-06-29 16:34:21 -07:00
Ken Hibino
6cce31a134 Fix recoverer test 2021-06-29 16:34:21 -07:00
Ken Hibino
68839dc9d3 Fix lua scripts for redis cluster 2021-06-29 16:34:21 -07:00
Ken Hibino
8922d2423a Define RDB.GetTaskInfo 2021-06-29 16:34:21 -07:00
Ken Hibino
87264b66f3 Record last_failed_at time on Retry or Archive event 2021-06-29 16:34:21 -07:00
Ken Hibino
62168b8d0d Add LastFailedAt field to TaskMessage 2021-06-29 16:34:21 -07:00
Ken Hibino
456edb6b71 Replace RunTaskByKey with RunTask in Inspector 2021-06-29 16:34:21 -07:00
Ken Hibino
b9c2572203 Refactor redis keys and store messages in protobuf 
Changes:
- Task messages are stored under "asynq:{<qname>}:t:<task_id>" key in redis, value is a HASH type and message are stored under "msg" key in the hash. The hash also stores "deadline", "timeout".
- Redis LIST and ZSET stores task message IDs
- Task messages are serialized using protocol buffer
 2021-06-29 16:34:21 -07:00
Ken Hibino
0bf767cf21 Add TaskState type to base package 2021-06-29 16:34:21 -07:00
Ken Hibino
4af65d5fa5 Update RDB methods with new errors package 2021-06-29 16:34:21 -07:00
Ken Hibino
a19ad19382 Update RDB.Dequeue with new errors package 2021-06-29 16:34:21 -07:00
Ken Hibino
8117ce8972 Minor fixes 2021-06-29 16:34:21 -07:00
Ken Hibino
d98ecdebb4 Update RDB.EnqueueUnique and RDB.ScheduleUnique with specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
ffe9aa74b3 Add errors.RedisCommandError type 2021-06-29 16:34:21 -07:00
Ken Hibino
d2d4029aba Update RDB.CurrentStats and RDB.HistoricalStats with specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
76bd865ebc Update RDB.RemoveQueue with specific error types 2021-06-29 16:34:21 -07:00
Ken Hibino
136d1c9ea9 Update rdb.List* methods with specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
52e04355d3 Return QueueNotFoundError from DeleteAll* methods 2021-06-29 16:34:21 -07:00
Ken Hibino
cde3e57c6c Update RDB.RunAll* methods with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
dd66acef1b Return QueueNotFoundError from ArchiveAll* methods 2021-06-29 16:34:21 -07:00
Ken Hibino
30a3d9641a Update tests for RDB.DeleteTask and RDB.ArchiveTask 2021-06-29 16:34:21 -07:00
Ken Hibino
961582cba6 Update RDB.RunTask with more specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
430dbb298e Update RDB.DeleteTask with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
675826be5f Update RDB.ArchiveAll methods with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
62f4e46b73 Update RDB.ArchiveAllPendingTasks with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
a500f8a534 Reorganize test for RDB.ArchiveTask 2021-06-29 16:34:21 -07:00
Ken Hibino
bcfeff38ed Update errors package with detailed comments 2021-06-29 16:34:21 -07:00
Ken Hibino
12a90f6a8d Update RDB.ArchiveTask with custom errors 2021-06-29 16:34:21 -07:00
Ken Hibino
807624e7dd Create internal errors package 2021-06-29 16:34:21 -07:00
Ken Hibino
4d65024bd7 Update rdb.ArchiveTask with more specific error types 2021-06-29 16:34:21 -07:00
Ken Hibino
76486b5cb4 Rename error types 2021-06-29 16:34:21 -07:00
Ken Hibino
1db516c53c Add a list of canonical errors in base package 2021-06-29 16:34:21 -07:00
Ken Hibino
cb5bdf245c Update RDB.ArchiveTask with task state 2021-06-29 16:34:21 -07:00