go-pkg/asynq
2
0
Fork 0
mirror of https://github.com/hibiken/asynq.git synced 2024-12-27 00:02:19 +08:00
Code Issues Projects Releases Wiki Activity
887 Commits 20 Branches 53 Tags 12 MiB
f4b56498f2
Commit Graph

109 Commits

Author SHA1 Message Date
Ken Hibino
3c722386b0 Add Deadline option when enqueuing tasks 
Deadline option sets the deadline for the given task's context deadline.
 2020-03-08 17:12:42 -07:00
Ken Hibino
a4e4c0b1d5 Call error handler when task was not processed successfully 2020-02-29 22:09:13 -08:00
Ken Hibino
e3c9aae8c7 Include worker stats to ProcessState 2020-02-22 08:56:51 -08:00
Ken Hibino
830020eb39 Add ProcessState type to base package 2020-02-22 08:56:51 -08:00
Ken Hibino
3d9a222bb3 Use sync.WaitGroup for shutdown 2020-02-16 18:22:59 -08:00
Ken Hibino
2bcaea52ce Make heartbeater goroutine a monitor goroutine for process info. 2020-02-16 18:22:59 -08:00
Ken Hibino
d33ca98648 Use int as priority value type. 
Do not use unsigned int for merely non-negative quantities as it
complicates simple arithmetic.
 2020-02-13 06:23:51 -08:00
Ken Hibino
6685827147 Add redis pubsub subscriber for cancelation 2020-02-13 06:23:51 -08:00
Ken Hibino
39459b4412 Allow user to specify timeout per task 2020-02-13 06:23:51 -08:00
Ken Hibino
6e14062325 Send cancellation signal to all in-progress handlers on shutdown 2020-02-13 06:23:51 -08:00
Ken Hibino
1b41c721b4 Change Handler interface to take context.Context 2020-02-13 06:23:51 -08:00
Ken Hibino
d9327cf24b Avoid creating script struct on every rdb method invocation 2020-02-09 09:34:41 -08:00
Ken Hibino
96f06ac89b Add ps command to asynqmon 2020-02-02 20:56:33 -08:00
Ken Hibino
44c657bec6 Rate limit error logs 2020-01-22 06:36:18 -08:00
Ken Hibino
db8e9d05c3 Add custom logger 2020-01-22 06:02:53 -08:00
Ken Hibino
c29200b1fc Add syncer to retry failed redis commands 2020-01-18 15:07:15 -08:00
Ken Hibino
5a6f737589 [performance] Use BRPOPLPUSH if one queue is used 2020-01-14 20:46:47 -08:00
Ken Hibino
874d8e8843 Add RDB.RemoveQueue method 2020-01-14 06:02:00 -08:00
Ken Hibino
84eef4ed0b Add strict-priority option 2020-01-12 18:39:57 -08:00
Ken Hibino
390eb13149 Remove stale field in processor struct 2020-01-11 10:09:15 -08:00
Ken Hibino
8d9a2d1313 Update processor to query queues based on priority 2020-01-07 21:55:18 -08:00
Ken Hibino
53d0902808 Change RDB.Dequeue to query multiple queues 2020-01-07 21:55:18 -08:00
Ken Hibino
03cb6eef09 Add Queues field to Config 2020-01-07 21:55:18 -08:00
Ken Hibino
f3a23b9b12 Make Task type immutable 
This change makes it impossible to mutate payload within Handler or
RetryDelayFunc.
 2020-01-05 09:55:39 -08:00
Ken Hibino
c62833540c Add license comment to all src files 2020-01-02 18:13:16 -08:00
Ken Hibino
ade97befbc Minor cleanup 2019-12-30 07:14:49 -08:00
Ken Hibino
22b21df884 Allow user to specify retry delay duration 2019-12-30 07:14:49 -08:00
Ken Hibino
6491f46955 Minor cleanup 2019-12-25 20:04:29 -08:00
Ken Hibino
3fd248615b Create base internal package 2019-12-22 07:40:25 -08:00
Ken Hibino
8b98b6e5a0 Rename channel name 2019-12-18 18:57:48 -08:00
Ken Hibino
33e9da953d Log warning and info messages when unfinished tasks get aborted 2019-12-18 18:55:08 -08:00
Ken Hibino
69b46a7f0d Use sync.Once 2019-12-17 20:34:56 -08:00
Ken Hibino
5ddba8ca98 Unblock processor shutdown process if processor is waiting for semaphore 
token
 2019-12-17 20:07:17 -08:00
Ken Hibino
24dd78b31c Stop processing more tasks from the queue once TSTP signal is received 2019-12-17 06:18:22 -08:00
Ken Hibino
eb3216d354 Fix: Use buffered channel of size 1 to avoid goroutine leaks 2019-12-16 06:40:52 -08:00
Ken Hibino
989b2b6d55 Add timeout to worker goroutines when TERM signal is received 
Wait for a certain amount of time to allow for worker goroutines to
finish. If the goroutines don't finish with the timeout duration,
processor will quit the goroutines and restore any unfinished tasks from
the in_progress queue back to the default queue.
 2019-12-15 21:00:09 -08:00
Ken Hibino
afacc31990 Minor improvements 2019-12-04 16:58:07 -08:00
Ken Hibino
318b24b3b8 Rename MoveAll to RestoreUnfinished 2019-12-04 06:50:52 -08:00
Ken Hibino
985018e1b5 Rename rdb Remove to Done 2019-12-04 06:33:05 -08:00
Ken Hibino
39f177dabf Change rdb Dequeue signature 2019-12-04 06:25:58 -08:00
Ken Hibino
d4e442d04f Extract rdb to internal package 2019-12-03 22:13:48 -08:00
Ken Hibino
b0a54cd2b2 Change Background API to take Handler interface 2019-12-02 20:45:40 -08:00
Ken Hibino
d53e5d3350 Add test for processor 2019-11-29 17:12:15 -08:00
Ken Hibino
facdadd7b0 Change shutdown message to use info level logging 2019-11-29 07:14:28 -08:00
Ken Hibino
1617278d86 Rename (*rdb).lrem to (*rdb).remove 2019-11-27 20:05:31 -08:00
Ken Hibino
c9a8f5fabd Minor cleanup 2019-11-27 19:36:56 -08:00
Ken Hibino
0db4b8a34f fix:Remove task from in_progress when worker goroutine finishes 
There was a bug related to the logic of removing task from
"asynq:in_progress" list when worker is finished. The task was not
properly removed from the list when the taskMessage gets modified before
the deferred call. More specifically when task was scheduled for retry
the message was modified and therefore LREM could not properly removed
the message from "asynq:in_progress" list.
 2019-11-27 14:26:04 -08:00
Ken Hibino
d4f136ebc9 Protect handler call against panic 2019-11-27 14:03:04 -08:00
Ken Hibino
e6b1230c36 Update log messages on shutdown 2019-11-27 06:33:04 -08:00
Ken Hibino
60132f3208 Minor cleanup 2019-11-27 06:21:57 -08:00
Ken Hibino
fd80126a67 Add logic to restore unfinished tasks back into the default queue if 
there are any uncompleted tasks
 2019-11-23 15:09:50 -08:00
Ken Hibino
4a327933bd Use command BRPUSHLPOP to move from queue to in_progress in redis 2019-11-23 08:43:41 -08:00
Ken Hibino
3fa867b190 Remove per task heartbeat 2019-11-23 08:21:32 -08:00
Ken Hibino
2e0af11ca0 Add heartbeat goroutine to write heartbeat data to redis for active 
worker
 2019-11-22 06:43:42 -08:00
Ken Hibino
67a9e8aa00 Track in-progress tasks with redis SET 2019-11-21 21:45:27 -08:00
Ken Hibino
c84287d7ab Minor fixes 2019-11-21 20:22:55 -08:00
Ken Hibino
66930970f9 Move retry related logic to its own file 2019-11-20 20:38:49 -08:00
Ken Hibino
3dddcfbb14 Minor cleanup 2019-11-20 20:27:01 -08:00
Ken Hibino
2dd5f2c5ab Rename manager to processor 2019-11-20 20:10:00 -08:00