Go Hypercube: a backend framework case study

The problem

Backend modules need a clear agreement about when their services are available. A plugin can depend on another plugin, so the order in which an application adds them cannot be the only thing deciding startup order.

Background jobs have a similar boundary. An application needs to decide when to retry a failed job, while a queue backend needs to store and deliver messages. Mixing those responsibilities would make a change of backend a change of application policy too.

My contribution

I’m the founder and maintainer of Go Hypercube, an evolution of my earlier Go Light Framework. This case study focuses on two parts of that work: the core plugin lifecycle and the queue contract I built with implementations for SQL, Redis, Valkey, Amazon SQS, RabbitMQ, and memory.

The design separates the decisions an application owns from the operations an adapter performs. The core repository and the driver repositories in the Go Hypercube organization make those boundaries available to inspect. You can explore the source on GitHub.

Make startup order explicit

Plugins declare their dependencies. Before registration begins, the framework checks that required plugins exist and their versions are compatible, then resolves a dependency-first order. Missing or incompatible dependencies are reported together. A dependency cycle stops setup with the cycle path in the error.

Registration and startup are separate passes. Setup() calls each plugin’s Register method, making its services and registrations available. Only after setup succeeds can Boot() run the startup pass. Plugins cannot be added after setup, which keeps that set stable during initialization.

The dependency resolver shows the ordering and validation rules; the application lifecycle shows where registration ends and boot begins. The public plugin tests provide concrete examples of accepted and rejected configurations.

Keep job policy with the worker

The queue package has no dependency on the application, job, or plugin packages. Its main interface exposes five transport operations. The worker decides whether a delivery succeeded, needs another attempt, or should move to the failed-message store.

Worker                 Contract              Driver
Decide job policy  ->  Queue operations  ->  Backend execution

The worker runs the job and applies its attempt limit and backoff. The contract expresses the transport operations, and the selected driver executes them using SQL, Redis, Valkey, SQS, RabbitMQ, or memory.

type Queue interface {
    Push(ctx context.Context, msgs ...*Message) error
    Pop(ctx context.Context, queueName string, maxMessages int) ([]*Message, error)
    Ack(ctx context.Context, msgs ...*Message) error
    Retry(ctx context.Context, msgs ...*Message) error
    DeadLetter(ctx context.Context, msgs ...*Message) error
}

This interface is an excerpt from the public queue contract. Its behavioral rules matter as much as its method signatures. A driver snapshots message data, returns an independently owned delivery, and uses delivery-specific state to settle it. A message ID alone does not establish that a worker still owns that delivery.

Batch operations can partly succeed. Per-item errors preserve the original input positions, so a caller can handle failed items without blindly repeating the successful ones. Backend request limits stay inside the driver.

Keep the tradeoffs visible

A portable interface does not make every backend behave identically. Delivery is at least once within each driver’s documented durability guarantees, so job handlers must tolerate repeats. Finishing a job and acknowledging it are separate operations; a worker can stop between them.

The memory driver loses its messages when the process exits. The RabbitMQ driver keeps deliveries on an AMQP channel rather than giving them a visibility deadline. Visibility extension, redrive, and statistics are optional capabilities, so a caller can check what its selected driver supports.

Separating registration from boot also does not provide automatic recovery from every startup failure. Both passes can return an error that the application has to handle.

The result

The public implementation provides a dependency-checked plugin lifecycle and one queue contract across six backend implementations. Retry policy stays with the worker, while delivery ownership, settlement, and optional capabilities have explicit rules.

The source links show the implementation and its tests. I’ve written more about the decisions in the journal articles below. The framework continues to evolve, and this case study follows the linked source snapshot.