2
0
mirror of https://github.com/hibiken/asynq.git synced 2024-12-24 23:02:18 +08:00
golang基于redis的异步队列
Go to file
Ken Hibino 742ed6546f Add ServeMux type
Allow user to use ServeMux type to be used as a Handler.
ServeMux API is design to be similar to net/http.ServeMux API.
2020-03-01 15:53:18 -08:00
.github/ISSUE_TEMPLATE Update issue templates 2019-12-27 10:45:45 -08:00
.travis Run and compare benchmarks after successful ci-build 2020-01-30 21:38:16 -08:00
docs/assets [ci skip] Update readme 2020-02-05 22:12:26 -08:00
internal Add workers command to asynqmon 2020-02-22 21:21:55 -08:00
tools Update redis package to v7.2.0 2020-02-22 21:21:55 -08:00
.gitignore [ci skip] Allow config file to set default values for flags 2020-01-19 09:10:48 -08:00
.travis.yml Run travis build with go v1.14.x 2020-02-27 08:28:35 -08:00
asynq_test.go Fix benchmark tests 2020-01-18 15:07:15 -08:00
asynq.go Add ServeMux type 2020-03-01 15:53:18 -08:00
background_test.go Change Client APIs 2020-02-23 20:40:40 -08:00
background.go Call error handler when task was not processed successfully 2020-02-29 22:09:13 -08:00
benchmark_test.go Change Client APIs 2020-02-23 20:40:40 -08:00
CHANGELOG.md Add ServeMux type 2020-03-01 15:53:18 -08:00
client_test.go Change Client APIs 2020-02-23 20:40:40 -08:00
client.go Change Client APIs 2020-02-23 20:40:40 -08:00
CONTRIBUTING.md [ci skip] Add contribution doc 2020-02-08 09:48:04 -08:00
doc.go Change Client APIs 2020-02-23 20:40:40 -08:00
go.mod Update redis package to v7.2.0 2020-02-22 21:21:55 -08:00
go.sum Update redis package to v7.2.0 2020-02-22 21:21:55 -08:00
heartbeat_test.go Add ProcessState type to base package 2020-02-22 08:56:51 -08:00
heartbeat.go Update WriteProcessState to write both ProcessInfo and WorkerInfo to 2020-02-22 08:56:51 -08:00
LICENSE Add MIT License 2019-11-30 10:21:25 -08:00
logger_test.go Add custom logger 2020-01-22 06:02:53 -08:00
logger.go Add custom logger 2020-01-22 06:02:53 -08:00
payload_test.go Add test for payload key not exist 2020-01-05 09:55:39 -08:00
payload.go [ci skip] Update docs 2020-01-16 21:04:46 -08:00
processor_test.go Add ErrorHandler type to changelog 2020-02-29 22:09:13 -08:00
processor.go Call error handler when task was not processed successfully 2020-02-29 22:09:13 -08:00
README.md Add ServeMux type 2020-03-01 15:53:18 -08:00
scheduler_test.go Use sync.WaitGroup for shutdown 2020-02-16 18:22:59 -08:00
scheduler.go Use sync.WaitGroup for shutdown 2020-02-16 18:22:59 -08:00
servemux_test.go Add ServeMux type 2020-03-01 15:53:18 -08:00
servemux.go Add ServeMux type 2020-03-01 15:53:18 -08:00
subscriber_test.go Fix data race in subscriber test 2020-02-16 18:22:59 -08:00
subscriber.go Include worker stats to ProcessState 2020-02-22 08:56:51 -08:00
syncer_test.go Fix data race in syncer test 2020-02-16 21:25:10 -08:00
syncer.go Use sync.WaitGroup for shutdown 2020-02-16 18:22:59 -08:00

Asynq

Build Status License: MIT Go Report Card GoDoc Gitter chat codecov

Asynq is a simple Go library for queueing tasks and processing them in the background with workers.
It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

Important Note: Current major version is zero (v0.x.x) to accomodate rapid development and fast iteration while getting early feedback from users. The public API could change without a major version update before v1.0.0 release.

Gif

Installation

To install asynq library, run the following command:

go get -u github.com/hibiken/asynq

Quickstart

First, make sure you are running a Redis server locally.

$ redis-server

To create and schedule tasks, use Client and provide a task and when to enqueue the task.

func main() {
    r := &asynq.RedisClientOpt{
        Addr: "127.0.0.1:6379",
    }

    client := asynq.NewClient(r)

    // Create a task with task type and payload
    t1 := asynq.NewTask("email:signup", map[string]interface{}{"user_id": 42})

    t2 := asynq.NewTask("email:reminder", map[string]interface{}{"user_id": 42})

    // Process immediately
    err := client.Enqueue(t1)

    // Process 24 hrs later
    err = client.EnqueueIn(24*time.Hour, t2)

    // Process at specified time.
    target := time.Date(2020, time.March, 6, 10, 0, 0, 0, time.UTC)
    err = client.EnqueueAt(target, t2)

    // Pass options to specify processing behavior for a given task.
    //
    // MaxRetry specifies the maximum number of times this task will be retried (Default is 25).
    // Queue specifies which queue to enqueue this task to (Default is "default").
    // Timeout specifies the the timeout for the task's context (Default is no timeout).
    err = client.Enqueue(t1, asynq.MaxRetry(10), asynq.Queue("critical"), asynq.Timeout(time.Minute))
}

To start the background workers, use Background and provide your Handler to process the tasks.

Handler is an interface with one method ProcessTask with the following signature.

// ProcessTask should return nil if the processing of a task
// is successful.
//
// If ProcessTask return a non-nil error or panics, the task
// will be retried after delay.
type Handler interface {
    ProcessTask(context.Context, *asynq.Task) error
}

You can optionally use ServeMux to create a handler, just as you would with "net/http" Handler.

func main() {
    r := &asynq.RedisClientOpt{
        Addr: "127.0.0.1:6379",
    }

    bg := asynq.NewBackground(r, &asynq.Config{
        // Specify how many concurrent workers to use
        Concurrency: 10,
        // Optionally specify multiple queues with different priority.
        Queues: map[string]int{
            "critical": 6,
            "default":  3,
            "low":      1,
        },
        // See the godoc for other configuration options
    })

    mux := asynq.NewServeMux()
    mux.HandleFunc("email:signup", signupEmailHandler)
    mux.HandleFunc("email:reminder", reminderEmailHandler)
    // ...register other handlers...

    bg.Run(mux)
}

// function with the same signature as the ProcessTask method for the Handler interface.
func signupEmailHandler(ctx context.Context, t *asynq.Task) error {
    id, err := t.Payload.GetInt("user_id")
    if err != nil {
        return err
    }
    fmt.Printf("Send welcome email to user %d\n", id)
    // ...your email sending logic...
    return nil
}

For a more detailed walk-through of the library, see our Getting Started Guide.

To Learn more about asynq features and APIs, see our Wiki pages and godoc.

Requirements

Dependency Version
Redis v2.8+
Go v1.12+

Command Line Tool

Asynq ships with a command line tool to inspect the state of queues and tasks.

To install, run the following command:

go get -u github.com/hibiken/asynq/tools/asynqmon

For details on how to use the tool, refer to the tool's README.

Contributing

We are open to, and grateful for, any contributions (Github issues/pull-requests, feedback on Gitter channel, etc) made by the community. Please see the Contribution Guide before contributing.

Acknowledgements

  • Sidekiq : Many of the design ideas are taken from sidekiq and its Web UI
  • RQ : Client APIs are inspired by rq library.
  • Cobra : Asynqmon CLI is built with cobra

License

Asynq is released under the MIT license. See LICENSE.