go-pkg/asynq
2
0
Fork 0
mirror of https://github.com/hibiken/asynq.git synced 2024-12-27 16:13:40 +08:00
Code Issues Projects Releases Wiki Activity
931 Commits 20 Branches 53 Tags 12 MiB
8213e981a5
Commit Graph

119 Commits

Author SHA1 Message Date
fajrifernanda
9261e4318c feature: configurable archive ttl and max archived size 2024-12-04 13:23:04 +07:00
Pior Bastida
3dbda60333
Improve performance of enqueueing tasks (#946) 
* Improve performance of enqueueing tasks

Add an in-memory cache to keep track of all the queues. Use this cache
to avoid sending an SADD since after the first call, that extra network
call isn't necessary.

The cache will expire every 10 secs so for cases where the queue is
deleted from asynq:queues set, it can be added again next time a task is
enqueued to it.

* Use sync.Map to simplify the conditional SADD

* Cleanup queuePublished in RemoveQueue

---------

Co-authored-by: Yousif <753751+yousifh@users.noreply.github.com>
 2024-10-30 08:25:35 +03:00
guoguangwu
e4b8663154 chore: unnecessary use of fmt.Sprintf 2023-07-07 20:45:42 -07:00
Emanuel Bennici
0275df8df4 Update redis/go-redis to v9 
Version v9 implements the support for Redis v7 and has some
other improvements.
 2023-04-17 22:30:33 -07:00
Erwan Leboucher
5c723f597e Correct the error message to cancel an active tasks 2022-04-13 06:08:46 -07:00
Ken Hibino
de139cc18e Update RDB.RunTask to schedule aggregating task 2022-04-11 16:55:43 -07:00
Ken Hibino
74db013ab9 Add RDB.RunAllAggregatingTasks 2022-04-11 16:55:43 -07:00
Ken Hibino
725105ca03 Update RDB.ArchiveTask to archive aggregating task 2022-04-11 16:55:43 -07:00
Ken Hibino
d8f31e45f1 Add RDB.ArchiveAllAggregatingTasks 2022-04-11 16:55:43 -07:00
Ken Hibino
9023cbf4be Update RDB.DeleteTask to handle aggregating task 2022-04-11 16:55:43 -07:00
Ken Hibino
9279c09125 Add RDB.DeleteAllAggregatingTasks 2022-04-11 16:55:43 -07:00
Ken Hibino
bc27126670 Fix memory usage lua script 2022-04-11 16:55:43 -07:00
Ken Hibino
0cfa7f47ba Fix memory_usage lua script 2022-04-11 16:55:43 -07:00
Ken Hibino
71bd8f0535 Add RDB.ListAggregating 2022-04-11 16:55:43 -07:00
Ken Hibino
0149396bae Add RDB.GroupStats for inspecting groups 2022-04-11 16:55:43 -07:00
Ken Hibino
47af17cfb4 Fix RDB.CurrentStats to report the correct queue size 2022-04-11 16:55:43 -07:00
Ken Hibino
652939dd3a Update memory usage redis lua script to account for groups 2022-04-11 16:55:43 -07:00
Ken Hibino
efe3c74037 Show number of groups and aggregating task count in QueueInfo 2022-04-11 16:55:43 -07:00
Ken Hibino
d4006894ad Remove base.DeadlinesKey 2022-02-19 06:15:44 -08:00
Ken Hibino
aa26f3819e
Fix flaky tests 2022-01-05 09:07:42 -08:00
Ken Hibino
82d18e3d91
Record total tasks processed/failed 2021-12-16 16:53:02 -08:00
Ken Hibino
99a6750656 Add Latency field to QueueInfo 2021-12-11 10:29:43 -08:00
Ken Hibino
6e7106c8f2 Record time when task moved to pending state 2021-12-11 10:29:43 -08:00
Ken Hibino
f4ddac4dcc Introduce Task Results 
* Added Retention Option to specify retention TTL for tasks
* Added ResultWriter as a client interface to write result data for the associated task
 2021-11-06 15:18:42 -07:00
Ken Hibino
2261c7c9a0 Change TaskMessage.ID type from uuid.UUID to string 2021-11-06 15:18:42 -07:00
Jason White
b3ef9e91a9
Upgrade go-redis/redis to version 8 2021-09-02 05:56:02 -07:00
Ken Hibino
e6355bf3f5 Use approximate memory usage for QueueInfo 2021-08-09 08:58:44 -07:00
Ken Hibino
99c7ebeef2 Add migration command in CLI 2021-06-29 16:34:21 -07:00
Ken Hibino
e01c6379c8 Fix lua script for redis-cluster mode 2021-06-29 16:34:21 -07:00
Ken Hibino
68dd6d9a9d (fix): Clear unique lock when task is deleted via Inspector 2021-06-29 16:34:21 -07:00
Ken Hibino
68839dc9d3 Fix lua scripts for redis cluster 2021-06-29 16:34:21 -07:00
Ken Hibino
8922d2423a Define RDB.GetTaskInfo 2021-06-29 16:34:21 -07:00
Ken Hibino
b9c2572203 Refactor redis keys and store messages in protobuf 
Changes:
- Task messages are stored under "asynq:{<qname>}:t:<task_id>" key in redis, value is a HASH type and message are stored under "msg" key in the hash. The hash also stores "deadline", "timeout".
- Redis LIST and ZSET stores task message IDs
- Task messages are serialized using protocol buffer
 2021-06-29 16:34:21 -07:00
Ken Hibino
ffe9aa74b3 Add errors.RedisCommandError type 2021-06-29 16:34:21 -07:00
Ken Hibino
d2d4029aba Update RDB.CurrentStats and RDB.HistoricalStats with specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
76bd865ebc Update RDB.RemoveQueue with specific error types 2021-06-29 16:34:21 -07:00
Ken Hibino
136d1c9ea9 Update rdb.List* methods with specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
52e04355d3 Return QueueNotFoundError from DeleteAll* methods 2021-06-29 16:34:21 -07:00
Ken Hibino
cde3e57c6c Update RDB.RunAll* methods with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
dd66acef1b Return QueueNotFoundError from ArchiveAll* methods 2021-06-29 16:34:21 -07:00
Ken Hibino
30a3d9641a Update tests for RDB.DeleteTask and RDB.ArchiveTask 2021-06-29 16:34:21 -07:00
Ken Hibino
961582cba6 Update RDB.RunTask with more specific errors 2021-06-29 16:34:21 -07:00
Ken Hibino
430dbb298e Update RDB.DeleteTask with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
675826be5f Update RDB.ArchiveAll methods with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
62f4e46b73 Update RDB.ArchiveAllPendingTasks with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
12a90f6a8d Update RDB.ArchiveTask with custom errors 2021-06-29 16:34:21 -07:00
Ken Hibino
4d65024bd7 Update rdb.ArchiveTask with more specific error types 2021-06-29 16:34:21 -07:00
Ken Hibino
76486b5cb4 Rename error types 2021-06-29 16:34:21 -07:00
Ken Hibino
cb5bdf245c Update RDB.ArchiveTask with task state 2021-06-29 16:34:21 -07:00
Ken Hibino
267493ccef Update RDB.RunTask with task state 2021-06-29 16:34:21 -07:00