Asynq

Asynq is a simple Go library for queueing tasks and processing them in the background with workers.
It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

Important Note: Current major version is zero (v0.x.x) to accomodate rapid development and fast iteration while getting early feedback from users. The public API could change without a major version update before v1.0.0 release.

Quickstart

First, make sure you are running a Redis server locally.

$ redis-server

To create and schedule tasks, use Client and provide a task and when to enqueue the task.
A task will be processed by a background worker as soon as the task gets enqueued.
Scheduled tasks will be stored in Redis and will be enqueued at the specified time.

func main() {
    r := &asynq.RedisClientOpt{
        Addr: "127.0.0.1:6379",
    }

    client := asynq.NewClient(r)

    // Create a task with task type and payload.
    t1 := asynq.NewTask("email:signup", map[string]interface{}{"user_id": 42})

    t2 := asynq.NewTask("email:reminder", map[string]interface{}{"user_id": 42})

    // Enqueue immediately.
    err := client.Enqueue(t1)

    // Enqueue 24 hrs later.
    err = client.EnqueueIn(24*time.Hour, t2)

    // Enqueue at specific time.
    err = client.EnqueueAt(time.Date(2020, time.March, 6, 10, 0, 0, 0, time.UTC), t2)

    // Pass vararg options to specify processing behavior for the given task.
    //
    // MaxRetry specifies the max number of retry if the task fails (Default is 25).
    // Queue specifies which queue to enqueue this task to (Default is "default" queue).
    // Timeout specifies the the task timeout (Default is no timeout).
    err = client.Enqueue(t1, asynq.MaxRetry(10), asynq.Queue("critical"), asynq.Timeout(time.Minute))
}

To start the background workers, use Background and provide your Handler to process the tasks.

Handler is an interface with one method ProcessTask with the following signature.

// ProcessTask should return nil if the processing of a task is successful.
//
// If ProcessTask return a non-nil error or panics, the task will be retried after delay.
type Handler interface {
    ProcessTask(context.Context, *asynq.Task) error
}

You can optionally use ServeMux to create a handler, just as you would with "net/http" Handler.

func main() {
    r := &asynq.RedisClientOpt{
        Addr: "127.0.0.1:6379",
    }

    bg := asynq.NewBackground(r, &asynq.Config{
        // Specify how many concurrent workers to use
        Concurrency: 10,
        // Optionally specify multiple queues with different priority.
        Queues: map[string]int{
            "critical": 6,
            "default":  3,
            "low":      1,
        },
        // See the godoc for other configuration options
    })

    // mux maps a type to a handler
    mux := asynq.NewServeMux()
    mux.HandleFunc("email:signup", signupEmailHandler)
    mux.HandleFunc("email:reminder", reminderEmailHandler)
    // ...register other handlers...

    bg.Run(mux)
}

// function with the same signature as the ProcessTask method for the Handler interface.
func signupEmailHandler(ctx context.Context, t *asynq.Task) error {
    id, err := t.Payload.GetInt("user_id")
    if err != nil {
        return err
    }
    fmt.Printf("Send welcome email to user %d\n", id)
    // ...your email sending logic...
    return nil
}

For a more detailed walk-through of the library, see our Getting Started Guide.

To Learn more about asynq features and APIs, see our Wiki and godoc.

Command Line Tool

Asynq ships with a command line tool to inspect the state of queues and tasks.

Here's an example of running the stats command.

For details on how to use the tool, refer to the tool's README.

Installation

To install asynq library, run the following command:

go get -u github.com/hibiken/asynq

To install the CLI tool, run the following command:

go get -u github.com/hibiken/asynq/tools/asynqmon

Requirements

Dependency	Version
Redis	v2.8+
Go	v1.12+

Contributing

We are open to, and grateful for, any contributions (Github issues/pull-requests, feedback on Gitter channel, etc) made by the community. Please see the Contribution Guide before contributing.

Acknowledgements

Sidekiq : Many of the design ideas are taken from sidekiq and its Web UI
RQ : Client APIs are inspired by rq library.
Cobra : Asynqmon CLI is built with cobra

License

Asynq is released under the MIT license. See LICENSE.

5.7 KiB Raw Permalink Blame History