Pior Bastida
3dbda60333
Improve performance of enqueueing tasks ( #946 )
...
* Improve performance of enqueueing tasks
Add an in-memory cache to keep track of all the queues. Use this cache
to avoid sending an SADD since after the first call, that extra network
call isn't necessary.
The cache will expire every 10 secs so for cases where the queue is
deleted from asynq:queues set, it can be added again next time a task is
enqueued to it.
* Use sync.Map to simplify the conditional SADD
* Cleanup queuePublished in RemoveQueue
---------
Co-authored-by: Yousif <753751+yousifh@users.noreply.github.com>
2024-10-30 08:25:35 +03:00
Harrison Miller
0dc670d7d8
Archived tasks that are trimmed from the set are deleted ( #743 )
...
* fixed trimmed archive tasks not being deleted.
* improved test case.
* changed ZRANGEBYSCORE to ZRANGE with BYSCORE option.
---------
Co-authored-by: Harrison <harrison@Harrisons-MacBook-Pro.local>
Co-authored-by: Harrison Miller <harrison.miller@MBP-Harrison-Miller-M2.local>
2024-10-19 09:18:09 +03:00
Tedja
d04888e748
feature: configurable janitor interval and deletion batch size ( #715 )
...
* feature: configurable janitor interval and deletion batch size
* warn user when they set a big number of janitor batch size
* Update CHANGELOG.md
---------
Co-authored-by: Agung Hariadi Tedja <agung.tedja@kumparan.com>
2024-05-06 14:11:52 +08:00
camcui
b35b559d40
chore: fix function names in comment
...
Signed-off-by: camcui <cuishua@sina.cn>
2024-04-12 13:54:08 +08:00
yeqown
1e0bf88bf3
fix: listLeaseExpiredCmd doesn't ignore possibly empty value of task in lua script
2023-12-10 09:47:55 -08:00
yeqown
d0041c55a3
fix(274): ignore empty data to append to msgs
...
fix issue 274
2023-12-10 09:47:55 -08:00
Emanuel Bennici
0275df8df4
Update redis/go-redis to v9
...
Version v9 implements the support for Redis v7 and has some
other improvements.
2023-04-17 22:30:33 -07:00
cui fliter
cc777ebdaa
fix some typos
...
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-01-05 20:03:02 -08:00
Ken Hibino
bafed907e9
Fix redis script error
2023-01-02 14:53:45 -08:00
Ken Hibino
1acd62c760
Move test helpers to asynqtest package
2022-04-11 16:55:43 -07:00
Ken Hibino
eb064c2bab
Fix AggregationCheck with unlimited size to clear group name from
...
all-groups set
2022-04-11 16:55:43 -07:00
Ken Hibino
74d2eea4e0
Clear group if aggregation set empties the group
2022-04-11 16:55:43 -07:00
Ken Hibino
e63f41fb24
Fix DeleteAggregationSet
2022-04-11 16:55:43 -07:00
Ken Hibino
1c388baf06
Implement RDB.ReclaimStaleAggregationSets
2022-04-11 16:55:43 -07:00
Ken Hibino
47a66231b3
Store aggregation set *key* in all aggreationsets zset
2022-04-11 16:55:43 -07:00
Ken Hibino
3551d3334c
Use zset for aggregation set to preserve score
2022-04-11 16:55:43 -07:00
Ken Hibino
8b16ede8bc
Declare ReclaimStaleAggregationSets
2022-04-11 16:55:43 -07:00
Ken Hibino
888b5590fb
Make GroupMaxSize and GroupMaxDelay config optional
2022-04-11 16:55:43 -07:00
Ken Hibino
4b35eb0e1a
Fix RDB.AggregationCheck when run against an empty group
2022-04-11 16:55:43 -07:00
Ken Hibino
b29fe58434
Implement RDB.ListGroups
2022-04-11 16:55:43 -07:00
Ken Hibino
7849b1114c
Implement RDB.DeleteAggregationSet
2022-04-11 16:55:43 -07:00
Ken Hibino
99c00bffeb
Implement RDB.AggregationCheck
2022-04-11 16:55:43 -07:00
Ken Hibino
d841dc2f8d
Add initial implementation of aggregator
2022-04-11 16:55:43 -07:00
Ken Hibino
ed69667e86
Update ForwardIfReady test with group
2022-04-11 16:55:43 -07:00
Ken Hibino
61854ea1dc
Update RDB.ForwardIfReady to forward to group if groupKey is specified
2022-04-11 16:55:43 -07:00
Ken Hibino
8b582899ad
Add RDB.AddToGroup and RDB.AddToGroupUnique methods
2022-04-11 16:55:43 -07:00
ashang
3f4f0c1daa
Use explicit types for limit constants
2022-03-29 06:30:10 -07:00
Ken Hibino
7e5efb0e30
Drop GT option from RDB.ExtendLease
...
GT option in ZAdd is supported for redis v6.2.0 or above.
This Change fixes redis version compatibility (currently v4.0+)
2022-02-20 06:20:38 -08:00
Ken Hibino
59927509d8
Remove timeout and deadline fields under task key
2022-02-19 06:15:44 -08:00
Ken Hibino
dfae8638e1
Update RDB methods to work with lease
2022-02-19 06:15:44 -08:00
Ken Hibino
87dc392c7f
Add RDB.ExtendLease method
2022-02-19 06:15:44 -08:00
Ken Hibino
bc2f1986d7
Update ListDeadlineExceeded to ListLeaseExpired
2022-02-19 06:15:44 -08:00
Ken Hibino
b8cb579407
Update RDB methods to use lease instead of deadlines set
2022-02-19 06:15:44 -08:00
Ken Hibino
d865d89900
Update RDB.Dequeue to insert task ID to lease set
2022-02-19 06:15:44 -08:00
Ken Hibino
1ddb2f7bce
Use math.MaxInt64 instead of custom const
2021-12-19 06:58:12 -08:00
Ken Hibino
82d18e3d91
Record total tasks processed/failed
2021-12-16 16:53:02 -08:00
Ken Hibino
157e97e72e
Update changelog
2021-12-11 10:29:43 -08:00
Ken Hibino
22e6c9d297
Delete "pending_since" under task-key when state changes to active
2021-12-11 10:29:43 -08:00
Ken Hibino
99a6750656
Add Latency field to QueueInfo
2021-12-11 10:29:43 -08:00
Ken Hibino
e7c1c3ad6f
Use clock in RDB
2021-12-11 10:29:43 -08:00
Ken Hibino
6e7106c8f2
Record time when task moved to pending state
2021-12-11 10:29:43 -08:00
Ken Hibino
9f2c321e98
Add EnqueueContext method to Client
2021-11-15 16:34:26 -08:00
Ken Hibino
f4ddac4dcc
Introduce Task Results
...
* Added Retention Option to specify retention TTL for tasks
* Added ResultWriter as a client interface to write result data for the associated task
2021-11-06 15:18:42 -07:00
Ken Hibino
dbdd9c6d5f
Update RDB Enqueue and Schedule methods to check for task ID conflict
2021-11-06 15:18:42 -07:00
Ken Hibino
2261c7c9a0
Change TaskMessage.ID type from uuid.UUID to string
2021-11-06 15:18:42 -07:00
Jason White
b3ef9e91a9
Upgrade go-redis/redis to version 8
2021-09-02 05:56:02 -07:00
Ken Hibino
f0db219f6a
Add IsFailure to Config
...
With this IsFailure config, users can provide a predicate function to
determine whether the error returned from Handler counts as a failure.
2021-09-01 06:00:54 -07:00
Ken Hibino
99c7ebeef2
Add migration command in CLI
2021-06-29 16:34:21 -07:00
Ken Hibino
68dd6d9a9d
(fix): Clear unique lock when task is deleted via Inspector
2021-06-29 16:34:21 -07:00
Ken Hibino
87264b66f3
Record last_failed_at time on Retry or Archive event
2021-06-29 16:34:21 -07:00
