108 Commits

Author	SHA1	Message	Date
Ken Hibino	a4e4c0b1d5	Call error handler when task was not processed successfully	2020-02-29 22:09:13 -08:00
Ken Hibino	e3c9aae8c7	Include worker stats to ProcessState	2020-02-22 08:56:51 -08:00
Ken Hibino	830020eb39	Add ProcessState type to base package	2020-02-22 08:56:51 -08:00
Ken Hibino	3d9a222bb3	Use sync.WaitGroup for shutdown	2020-02-16 18:22:59 -08:00
Ken Hibino	2bcaea52ce	Make heartbeater goroutine a monitor goroutine for process info.	2020-02-16 18:22:59 -08:00
Ken Hibino	d33ca98648	Use int as priority value type. ... Do not use unsigned int for merely non-negative quantities as it complicates simple arithmetic.	2020-02-13 06:23:51 -08:00
Ken Hibino	6685827147	Add redis pubsub subscriber for cancelation	2020-02-13 06:23:51 -08:00
Ken Hibino	39459b4412	Allow user to specify timeout per task	2020-02-13 06:23:51 -08:00
Ken Hibino	6e14062325	Send cancellation signal to all in-progress handlers on shutdown	2020-02-13 06:23:51 -08:00
Ken Hibino	1b41c721b4	Change Handler interface to take context.Context	2020-02-13 06:23:51 -08:00
Ken Hibino	d9327cf24b	Avoid creating script struct on every rdb method invocation	2020-02-09 09:34:41 -08:00
Ken Hibino	96f06ac89b	Add ps command to asynqmon	2020-02-02 20:56:33 -08:00
Ken Hibino	44c657bec6	Rate limit error logs	2020-01-22 06:36:18 -08:00
Ken Hibino	db8e9d05c3	Add custom logger	2020-01-22 06:02:53 -08:00
Ken Hibino	c29200b1fc	Add syncer to retry failed redis commands	2020-01-18 15:07:15 -08:00
Ken Hibino	5a6f737589	[performance] Use BRPOPLPUSH if one queue is used	2020-01-14 20:46:47 -08:00
Ken Hibino	874d8e8843	Add RDB.RemoveQueue method	2020-01-14 06:02:00 -08:00
Ken Hibino	84eef4ed0b	Add strict-priority option	2020-01-12 18:39:57 -08:00
Ken Hibino	390eb13149	Remove stale field in processor struct	2020-01-11 10:09:15 -08:00
Ken Hibino	8d9a2d1313	Update processor to query queues based on priority	2020-01-07 21:55:18 -08:00
Ken Hibino	53d0902808	Change RDB.Dequeue to query multiple queues	2020-01-07 21:55:18 -08:00
Ken Hibino	03cb6eef09	Add Queues field to Config	2020-01-07 21:55:18 -08:00
Ken Hibino	f3a23b9b12	Make Task type immutable ... This change makes it impossible to mutate payload within Handler or RetryDelayFunc.	2020-01-05 09:55:39 -08:00
Ken Hibino	c62833540c	Add license comment to all src files	2020-01-02 18:13:16 -08:00
Ken Hibino	ade97befbc	Minor cleanup	2019-12-30 07:14:49 -08:00
Ken Hibino	22b21df884	Allow user to specify retry delay duration	2019-12-30 07:14:49 -08:00
Ken Hibino	6491f46955	Minor cleanup	2019-12-25 20:04:29 -08:00
Ken Hibino	3fd248615b	Create base internal package	2019-12-22 07:40:25 -08:00
Ken Hibino	8b98b6e5a0	Rename channel name	2019-12-18 18:57:48 -08:00
Ken Hibino	33e9da953d	Log warning and info messages when unfinished tasks get aborted	2019-12-18 18:55:08 -08:00
Ken Hibino	69b46a7f0d	Use sync.Once	2019-12-17 20:34:56 -08:00
Ken Hibino	5ddba8ca98	Unblock processor shutdown process if processor is waiting for semaphore ... token	2019-12-17 20:07:17 -08:00
Ken Hibino	24dd78b31c	Stop processing more tasks from the queue once TSTP signal is received	2019-12-17 06:18:22 -08:00
Ken Hibino	eb3216d354	Fix: Use buffered channel of size 1 to avoid goroutine leaks	2019-12-16 06:40:52 -08:00
Ken Hibino	989b2b6d55	Add timeout to worker goroutines when TERM signal is received ... Wait for a certain amount of time to allow for worker goroutines to finish. If the goroutines don't finish with the timeout duration, processor will quit the goroutines and restore any unfinished tasks from the in_progress queue back to the default queue.	2019-12-15 21:00:09 -08:00
Ken Hibino	afacc31990	Minor improvements	2019-12-04 16:58:07 -08:00
Ken Hibino	318b24b3b8	Rename MoveAll to RestoreUnfinished	2019-12-04 06:50:52 -08:00
Ken Hibino	985018e1b5	Rename rdb Remove to Done	2019-12-04 06:33:05 -08:00
Ken Hibino	39f177dabf	Change rdb Dequeue signature	2019-12-04 06:25:58 -08:00
Ken Hibino	d4e442d04f	Extract rdb to internal package	2019-12-03 22:13:48 -08:00
Ken Hibino	b0a54cd2b2	Change Background API to take Handler interface	2019-12-02 20:45:40 -08:00
Ken Hibino	d53e5d3350	Add test for processor	2019-11-29 17:12:15 -08:00
Ken Hibino	facdadd7b0	Change shutdown message to use info level logging	2019-11-29 07:14:28 -08:00
Ken Hibino	1617278d86	Rename (*rdb).lrem to (*rdb).remove	2019-11-27 20:05:31 -08:00
Ken Hibino	c9a8f5fabd	Minor cleanup	2019-11-27 19:36:56 -08:00
Ken Hibino	0db4b8a34f	fix:Remove task from in_progress when worker goroutine finishes ... There was a bug related to the logic of removing task from "asynq:in_progress" list when worker is finished. The task was not properly removed from the list when the taskMessage gets modified before the deferred call. More specifically when task was scheduled for retry the message was modified and therefore LREM could not properly removed the message from "asynq:in_progress" list.	2019-11-27 14:26:04 -08:00
Ken Hibino	d4f136ebc9	Protect handler call against panic	2019-11-27 14:03:04 -08:00
Ken Hibino	e6b1230c36	Update log messages on shutdown	2019-11-27 06:33:04 -08:00
Ken Hibino	60132f3208	Minor cleanup	2019-11-27 06:21:57 -08:00
Ken Hibino	fd80126a67	Add logic to restore unfinished tasks back into the default queue if ... there are any uncompleted tasks	2019-11-23 15:09:50 -08:00
Ken Hibino	4a327933bd	Use command BRPUSHLPOP to move from queue to in_progress in redis	2019-11-23 08:43:41 -08:00
Ken Hibino	3fa867b190	Remove per task heartbeat	2019-11-23 08:21:32 -08:00
Ken Hibino	2e0af11ca0	Add heartbeat goroutine to write heartbeat data to redis for active ... worker	2019-11-22 06:43:42 -08:00
Ken Hibino	67a9e8aa00	Track in-progress tasks with redis SET	2019-11-21 21:45:27 -08:00
Ken Hibino	c84287d7ab	Minor fixes	2019-11-21 20:22:55 -08:00
Ken Hibino	66930970f9	Move retry related logic to its own file	2019-11-20 20:38:49 -08:00
Ken Hibino	3dddcfbb14	Minor cleanup	2019-11-20 20:27:01 -08:00
Ken Hibino	2dd5f2c5ab	Rename manager to processor	2019-11-20 20:10:00 -08:00