Retries, Timeouts & Workers

Workers

The number of worker threads determines how many jobs from this queue can run concurrently.

using namespace framework::jobs;

queue_config config;
config.workers_ = 5;  // 5 concurrent workers

Recommendations:

• CPU-bound jobs: workers_ = number of CPU cores.
• I/O-bound jobs (HTTP calls, DB queries): higher values, e.g., 10–50.
• Start with 3–5 and tune based on latency and throughput.

---

Retries

When a job's handle() throws an exception, the job is retried according to the queue configuration.

config.max_attempts_ = 3;           // Try up to 3 times
config.retry_delay_ms_ = 5000;      // Wait 5s before the first retry
config.backoff_exponential_ = true; // Double the delay each attempt

Retry behavior:

• Attempt 1 fails → wait retry_delay_ms_ (5s).
• Attempt 2 fails → wait retry_delay_ms_ * 2 (10s) if exponential.
• Attempt 3 fails → job marked as failed_, no more retries.

Detecting retries inside a job:

class SendEmailJob : public job {
    void handle() override {
        try {
            // risky operation
        } catch (const std::exception& e) {
            // Log and re-throw to trigger retry
            LOG_ERROR_SVC(config, "Attempt {} failed: {}", attempts(), e.what());
            throw;
        }
    }
};

---

Timeouts

If a job runs longer than max_job_timeout_ms_, it is cancelled and marked as timed_out_.

config.max_job_timeout_ms_ = 30000; // 30 seconds max
config.max_job_timeout_ms_ = 0;     // 0 = no timeout (default)

When to set timeouts:

• External API calls: 10–30s.
• Database operations: 30–60s.
• File processing: 5–300s depending on file size.

---

Retention

Completed and failed jobs are kept in memory for inspection.

config.retention_seconds_ = 86400; // 24 hours (default)

After the retention period, jobs are automatically removed to free memory.

---

Monitoring

auto& queues = app.get_state()->of_jobs();

// Queue-level metrics
auto metrics = queues.get_queue_metrics("emails");
// metrics.jobs_completed_, metrics.jobs_failed_,
// metrics.jobs_dispatched_, metrics.jobs_cancelled_,
// metrics.jobs_timed_out_, metrics.jobs_retried_

// Individual job status
auto entry = queues.get_entry(job_id);
if (entry) {
    auto status = entry->status_;  // pending, running, completed, failed, cancelled, expired
    auto attempts = entry->attempts_;
}