Ken Hibino
e13122723a
Move all inspector related code to subpackage inspeq
2021-01-31 06:09:40 -08:00
Ken Hibino
eba7c4e085
Record deadline within WorkerInfo
2021-01-31 06:09:40 -08:00
Ken Hibino
afde6a7266
Add MemoryUsage field to QueueStats
2021-01-31 06:09:40 -08:00
Ken Hibino
6529a1e0b1
Fix scheduler
...
* Delete scheduler history data when scheduler stops
* Fix history trimming bug
2021-01-31 06:09:40 -08:00
Ken Hibino
c9a6ab8ae1
Support delete and archive actions on PendingTask
...
* Add `DeleteAllPendingTasks`, `ArchiveAllPendingTasks` to `Inspector`
* `DeleteTaskByKey` and `ArchiveTaskByKey` now supports deleting/archiving PendingTask
* Updated `asynq task` command with support for deleting/archiving pending tasks
2021-01-31 06:09:40 -08:00
Ken Hibino
2884044e75
v0.14.1
2021-01-19 06:22:54 -08:00
Ken Hibino
42c7ac0746
v0.14.0
2021-01-14 06:49:36 -08:00
Ken Hibino
00a25ca570
Rename DeadTask to ArchivedTask and action "kill" to "archive"
2021-01-14 06:43:44 -08:00
Ken Hibino
c5100a9c23
Add a method to list running servers to Inspector
2021-01-14 06:43:44 -08:00
Ken Hibino
196d66f221
Fix ListSchedulerEnqueueEvents to list recent events first
2021-01-14 06:43:44 -08:00
Ken Hibino
f4dd8fe962
Add ListScheduelerEnqueueEvents to Inspector
2021-01-14 06:43:44 -08:00
Ken Hibino
f9c0673116
Add SchedulerEntries method to Inspector
2021-01-14 06:43:44 -08:00
Ken Hibino
e9239260ae
Add DeleteQueue method to Inspector
...
- Added ErrQueueNotFound and ErrQueueNotEmpty type to indicate the kind
of an error returned from the method.
2021-01-14 06:43:44 -08:00
MinJae Kwon
c4dc993241
fix: resolve go vet lint
2020-12-20 06:09:51 -08:00
Ken Hibino
942345ee80
v0.13.0
2020-10-13 06:33:47 -07:00
Ken Hibino
96b2318300
Add EnqueueErrorHandler option to SchedulerOpts
2020-10-13 06:31:47 -07:00
Ken Hibino
8312515e64
Update Option interface
...
- Added `String()`, `Type()`, and `Value()` methods to the interface to
aid with debugging and error handling.
2020-10-13 06:31:47 -07:00
Ken Hibino
50e7f38365
Add Scheduler
...
- Renamed previously called scheduler to forwarder to resolve name
conflicts
2020-10-13 06:31:47 -07:00
Ken Hibino
c43fb21a0a
Minor test updates
2020-09-12 12:59:03 -07:00
Ken Hibino
69d7ec725a
Close redis client after each test run
2020-09-12 12:59:03 -07:00
Ken Hibino
c26b7469bd
Display cluster info in stats command when --cluster flag is passed
2020-09-12 12:59:03 -07:00
Ken Hibino
29e542e591
Rename Enqueue methods in Inspector to Run
2020-09-12 12:59:03 -07:00
Ken Hibino
a891ce5568
Rename InProgress to Active
2020-09-12 12:59:03 -07:00
Ken Hibino
c8c47fcbf0
Rename Enqueued to Pending
2020-09-12 12:59:03 -07:00
Ken Hibino
8076b5ae50
Use different redis db number for rdb package tests
2020-09-12 12:59:03 -07:00
Ken Hibino
a88325cb96
Add ClusterNodes and ClusterKeySlot in Inspector
2020-09-12 12:59:03 -07:00
Ken Hibino
6b96459881
Add test flags to run tests using redis cluster
2020-09-12 12:59:03 -07:00
Ken Hibino
572eb338d5
Fix flaky ProcessorRetry test
2020-09-12 12:59:03 -07:00
Ken Hibino
27f4027447
Add RedisClusterClientOpt to connect to redis cluster
2020-09-12 12:59:03 -07:00
Ken Hibino
ee1afd12f5
Fix done lua script
...
If UniqueKey is an empty string, do not provide the key to Lua script
because that will cause CROSSSLOT error in redis cluster (since it
doesn't have any hash tag).
2020-09-12 12:59:03 -07:00
Ken Hibino
3ac548e97c
Fix dequeue Lua script to use a single hash tag
2020-09-12 12:59:03 -07:00
Ken Hibino
f38f94b947
Restructure CLI commands with subcommands
2020-09-12 12:59:03 -07:00
Ken Hibino
118ef27bf2
Update RemoveQueue in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
96f23d88cd
Add more processor tests
2020-09-12 12:59:03 -07:00
Ken Hibino
2f226dfb84
Update ListServers and ListWorkers methods in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
aa2676bb57
Update Broker interface
2020-09-12 12:59:03 -07:00
Ken Hibino
f59de9ac56
Update all delete methods in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
996a6c0ead
Update all kill methods in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
47e9ba4eba
Update enqueue methods in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
dbf140a767
Update all list methods in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
5f82b4b365
Update HistoricalStats method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
44a3d177f0
Update Pause and Unpause methods in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
24b13bd865
Update CurrentStats method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
d25090c669
Add AllQueues method to RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
b5caefd663
Remove stale benchmark test
2020-09-12 12:59:03 -07:00
Ken Hibino
becd26479b
Update WriteServerState and ClearServerState in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
4b81b91d3e
Minor fix
2020-09-12 12:59:03 -07:00
Ken Hibino
a873d488ee
Update ListDeadlineExceeded in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
650d7fdbe9
Update CheckAndEnqueue method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
f6d504939e
Update Requeue method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
74f08795f8
Update Kill method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
35b2b1782e
Update Retry method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
f63dcce0c0
Update Done method in RDB
2020-09-12 12:59:03 -07:00
Ken Hibino
565f86ee4f
Update Dequeue command in rdb
2020-09-12 12:59:03 -07:00
Ken Hibino
94aa878060
Update Enqueue and Schedule commands in rdb
2020-09-12 12:59:03 -07:00
Ken Hibino
50b6034bf9
Move unique key generator function to base
2020-09-12 12:59:03 -07:00
Ken Hibino
154113d0d0
Update base package to generate redis keys with hashtag
2020-09-12 12:59:03 -07:00
Ken Hibino
a913e6d73f
Add healthchecker to check broker connection
2020-07-28 22:45:57 -07:00
Ken Hibino
92d77bbc6e
Minor comment fix
2020-07-28 22:45:57 -07:00
Ken Hibino
a28f61f313
Add Inspector type
2020-07-28 22:45:57 -07:00
Ken Hibino
8b60e6a268
Replace github.com/rs/xid with github.com/google/uuid
2020-07-06 05:48:31 -07:00
Ken Hibino
486dcd799b
Add version command to CLI
2020-07-06 05:48:31 -07:00
Ken Hibino
199bf4d66a
Minor code cleanup
2020-07-06 05:48:31 -07:00
Ken Hibino
7e942ec241
Use int64 type for Timeout and Deadline in TaskMessage
2020-07-06 05:48:31 -07:00
Ken Hibino
feee87adda
Add recoverer
2020-07-06 05:48:31 -07:00
Ken Hibino
7657f560ec
Add RDB.ListDeadlineExceeded
2020-07-06 05:48:31 -07:00
Ken Hibino
7c7de0d8e0
Fix processor
2020-07-06 05:48:31 -07:00
Ken Hibino
4e8ac151ae
Update processor to adapt for deadlines set change
...
- Processor dequeues tasks only when it's available to process
- Processor retries a task when its context's Done channel is closed
2020-07-06 05:48:31 -07:00
Ken Hibino
08b71672aa
Update RDB.Requeue to remove message from deadlines set
2020-07-06 05:48:31 -07:00
Ken Hibino
92af00f9fd
Update RDB.Dequeue to return deadline as time.Time
2020-07-06 05:48:31 -07:00
Ken Hibino
113451ce6a
Update RDB.Kill to remove message from deadlines set
2020-07-06 05:48:31 -07:00
Ken Hibino
9cd9f3d6b4
Update RDB.Retry to remove message from deadlines set
2020-07-06 05:48:31 -07:00
Ken Hibino
7b9119c703
Update RDB.Done to remove message from deadlines set
2020-07-06 05:48:31 -07:00
Ken Hibino
9b05dea394
Update RDB.Dequeue to return message and deadline
2020-07-06 05:48:31 -07:00
Ken Hibino
6cc5bafaba
Add task message to deadlines set on dequeue
...
Updated dequeueCmd to decode the message and compute its deadline and add
the message to the Deadline set.
2020-07-06 05:48:31 -07:00
Ken Hibino
0527b93432
Change TaskMessage Timeout and Deadline to int
...
* This change breaks existing tasks in Redis
2020-07-06 05:48:31 -07:00
Ken Hibino
5dddc35d7c
Add redis key for deadlines in base package
2020-07-06 05:48:31 -07:00
Ken Hibino
7f30fa2bb6
Fix requeue logic in processor
2020-06-13 06:22:32 -07:00
Ken Hibino
a2abeedaa0
Fix JSON number ovewflow issue
2020-06-12 06:29:36 -07:00
Ken Hibino
06c4a1c7f8
Limit the number of tasks moved by CheckAndEnqueue to prevent a long
...
running script
2020-06-08 06:06:18 -07:00
Ken Hibino
4e800a7f68
Update stats command to show queue paused status
2020-06-08 06:06:18 -07:00
Ken Hibino
363cfedb49
Update Dequeue operation to skip paused queues
2020-06-08 06:06:18 -07:00
Ken Hibino
4595bd41c3
Add Pause and Unpause methods to rdb
2020-06-08 06:06:18 -07:00
Ken Hibino
a38f628f3b
Refactor server state management
2020-05-31 06:41:19 -07:00
Ken Hibino
fa3082e5bb
Change LogLevel to satisfy flag.Value interface
2020-05-16 08:00:35 -07:00
Ken Hibino
d13f7e900f
Allow setting minimum log level for logger
2020-05-16 08:00:35 -07:00
Ken Hibino
b63476ddc8
Simplify Logger interface
2020-05-16 08:00:35 -07:00
Ken Hibino
5161b9368a
Clean up tests
2020-05-02 07:05:26 -07:00
Ken Hibino
7c7f8e5f30
Move Broker interface to base package
2020-04-19 08:51:17 -07:00
Ken Hibino
46ab4417dd
Add test to simulate situation where redis is down
2020-04-19 08:51:17 -07:00
Ken Hibino
c688b8f4f9
Fix test for base package
2020-04-19 08:51:17 -07:00
Ken Hibino
239ef27a6e
Update doc comments
2020-04-19 08:51:17 -07:00
Ken Hibino
b086e88a47
Rename ps command to servers
2020-04-19 08:51:17 -07:00
Ken Hibino
aafd8a5b74
Rename internal ProcessState to ServerState
2020-04-19 08:51:17 -07:00
Ken Hibino
b14c73809e
Refactor server state
2020-04-19 08:51:17 -07:00
Ken Hibino
9b4438347e
Fix comment
2020-03-21 11:44:26 -07:00
Ken Hibino
c33dd447ac
Allow client to enqueue a task with unique option
...
Changes:
- Added Unique option for clients
- Require go v.13 or above (to use new errors wrapping functions)
- Fixed adding queue key to all-queues set (asynq:queues) when scheduling.
2020-03-21 11:40:40 -07:00
Ken Hibino
8d2b9d6be7
Add comments to exported types and functions from internal/log package
2020-03-13 21:04:45 -07:00
Bo-Yi Wu
30b68728d4
chore(lint): fix from gofmt -s
2020-03-13 20:01:39 -07:00
Ken Hibino
0bc6eba021
Allow custom logger to be used in Background
2020-03-12 08:40:37 -07:00
Ken Hibino
d664d68fa4
Extract out log package
2020-03-09 07:17:52 -07:00
Ken Hibino
a425f54d23
[ci skip] Remove todo comment
2020-03-09 06:09:07 -07:00
Ken Hibino
3c722386b0
Add Deadline option when enqueuing tasks
...
Deadline option sets the deadline for the given task's context deadline.
2020-03-08 17:12:42 -07:00
Ken Hibino
26e399bc9c
Add workers command to asynqmon
2020-02-22 21:21:55 -08:00
Ken Hibino
e21fe3bd79
Add ListWorkers to RDB
2020-02-22 21:21:55 -08:00
Ken Hibino
37c6c73d9b
Add test for cancelation pubsub
2020-02-22 15:16:24 -08:00
Ken Hibino
5775a5818d
Update WriteProcessState to write both ProcessInfo and WorkerInfo to
...
redis
2020-02-22 08:56:51 -08:00
Ken Hibino
e3c9aae8c7
Include worker stats to ProcessState
2020-02-22 08:56:51 -08:00
Ken Hibino
9e02a91808
Add workers key in base package
2020-02-22 08:56:51 -08:00
Ken Hibino
830020eb39
Add ProcessState type to base package
2020-02-22 08:56:51 -08:00
Ken Hibino
2bcaea52ce
Make heartbeater goroutine a monitor goroutine for process info.
2020-02-16 18:22:59 -08:00
Ken Hibino
d33ca98648
Use int as priority value type.
...
Do not use unsigned int for merely non-negative quantities as it
complicates simple arithmetic.
2020-02-13 06:23:51 -08:00
Ken Hibino
cd55a0e316
Add cancel command to asynqmon cli
2020-02-13 06:23:51 -08:00
Ken Hibino
6685827147
Add redis pubsub subscriber for cancelation
2020-02-13 06:23:51 -08:00
Ken Hibino
39459b4412
Allow user to specify timeout per task
2020-02-13 06:23:51 -08:00
Ken Hibino
d9327cf24b
Avoid creating script struct on every rdb method invocation
2020-02-09 09:34:41 -08:00
Ken Hibino
96f06ac89b
Add ps command to asynqmon
2020-02-02 20:56:33 -08:00
Ken Hibino
d03fa34eaf
Add hearbeater
2020-02-01 09:35:49 -08:00
Ken Hibino
6fbaa2ed6c
(fix): RestoreUnfinished to select correct queue
2020-01-26 16:05:46 -08:00
Ken Hibino
166497748b
(fix): Requeue to select correct queue
2020-01-26 16:05:46 -08:00
Ken Hibino
31123fd42a
Paginate tasks with asynqmon ls command
...
Changes:
* Added --page and --size flags to ls command
* By default, the command will show first 30 tasks from the specified
queue
2020-01-26 13:12:01 -08:00
Ken Hibino
fd8eb51440
[ci skip] Add badges to readme
2020-01-16 21:13:34 -08:00
Ken Hibino
cb2ebf18ac
[performance] Skip the overhead of json decoding when scheduling to one
...
queue
2020-01-14 20:46:47 -08:00
Ken Hibino
5a6f737589
[performance] Use BRPOPLPUSH if one queue is used
2020-01-14 20:46:47 -08:00
Ken Hibino
858b0325bd
Add rmq command to asynqmon
2020-01-14 06:02:00 -08:00
Ken Hibino
874d8e8843
Add RDB.RemoveQueue method
2020-01-14 06:02:00 -08:00
Ken Hibino
97316d6766
Fix flaky tests
...
Some tests were failing due to mismatch in Score in ZSetEntry.
Changed ZSetEntry Score to float64 type so that we can use
cmpopts.EquateApprox to allow for margin when comparing.
2020-01-11 10:09:15 -08:00
Ken Hibino
2631672575
Allow filtering results of asynqmon ls enqueued
by providing queue
...
name
2020-01-11 10:09:15 -08:00
Ken Hibino
cf78a12866
Add "Queue" column to the output of asynqmon ls
2020-01-11 10:09:15 -08:00
Ken Hibino
c5b215e3b9
Update RDB.ListEnqueued to list tasks from all queues
2020-01-11 10:09:15 -08:00
Ken Hibino
89843ac565
Change RDB.CurrentStats to be multi-queue aware
2020-01-11 10:09:15 -08:00
Ken Hibino
67f381269a
Maintain a set of queue names in redis set
2020-01-11 10:09:15 -08:00
Ken Hibino
390eb13149
Remove stale field in processor struct
2020-01-11 10:09:15 -08:00
Ken Hibino
718336ff44
Update RDB.Enqueue* methods to be multi queue aware
2020-01-11 10:09:15 -08:00
Ken Hibino
4f5d115b3e
[ci skip] Upgrade github.com/google/go-cmp to v0.4.0
2020-01-07 21:55:18 -08:00
Ken Hibino
53d0902808
Change RDB.Dequeue to query multiple queues
2020-01-07 21:55:18 -08:00
Ken Hibino
2af9eb2c88
Delete stale code
2020-01-07 21:55:18 -08:00
Ken Hibino
28d698c24e
Update CheckAndEnqueue to enqueue tasks to specified queue
2020-01-07 21:55:18 -08:00
Ken Hibino
ca78b92078
Add Queue option to allow user to specify queue from client
...
Added base.QueueKey method to get redis key for given queue name.
Changed asynqtest.GetEnqueuedMessages to optionally take queue name.
2020-01-07 21:55:18 -08:00
Ken Hibino
f3a23b9b12
Make Task type immutable
...
This change makes it impossible to mutate payload within Handler or
RetryDelayFunc.
2020-01-05 09:55:39 -08:00
Ken Hibino
294a695b98
Add history command to asynqmon tool
2020-01-04 12:22:00 -08:00
Ken Hibino
c62833540c
Add license comment to all src files
2020-01-02 18:13:16 -08:00
Ken Hibino
4376dc1e9d
Add benchmark tests for rdb
2019-12-31 08:24:03 -08:00
Ken Hibino
b7c0c5d3aa
Handle mutated task in RDB's Done, Retry, Kill methods
...
It is possible that user mutates the task's payload in Handler
(Although doc says the task in handler is read-only). Prevent
ending up in an inconsistent state by handling the case where
user mutates the task.
2019-12-31 08:24:03 -08:00
Ken Hibino
ae0c2f9ca5
Use asynqtest helpers in asynq package tests
2019-12-29 10:05:02 -08:00
Ken Hibino
807729d36a
Create internal asynqtest package for test helpers
2019-12-29 10:05:02 -08:00
Ken Hibino
49d6ab5df0
Clean up rdb package
2019-12-29 10:05:02 -08:00
Ken Hibino
4ceb49cfd1
Minor improvements
2019-12-28 06:41:39 -08:00
Ken Hibino
8d3d30da8f
Add KillAllRetryTasks and KillAllScheduledTasks method to RDB
2019-12-27 15:17:45 -08:00
Ken Hibino
5b98b8eb62
Add KillRetryTask, KillScheduledTask methods to RDB
2019-12-27 15:17:45 -08:00