QueueResource Technical Reference
Overview
QueueResource is the Activate resource type used to queue, retry, suspend, resume, and throttle work that should not fail immediately when a downstream dependency is unavailable, busy, or rate-limited.
Unlike a simple rate limiter, a QueueResource provides a complete queue-backed workflow replay mechanism. It allows workflows to pause safely, be retried later by the Orchestrator, and resume from the point at which they were suspended.
Typical use cases include:
- Service desk integrations
- REST API integrations
- Vendor platforms with rate limits
- Systems that experience transient outages
- Long-running synchronisation processes
- Workflows that should retry automatically rather than fail immediately
How QueueResource Works
A QueueResource owns an ActivateBackgroundQueue.
Internally, the queue is created or retrieved using:
The queue is keyed by the resource path, ensuring that each queue resource maintains its own independent workload.
The queue is responsible for:
- Maintaining queued jobs
- Retrying failed work
- Suspending workflows
- Resuming workflows later
- Applying rate limits
- Tracking queue depth
- Processing retry schedules
At a high level:
Why Use QueueResource?
A QueueResource allows Activate to:
- Prevent external systems from being overwhelmed
- Retry transient failures automatically
- Suspend workflows safely
- Resume processing later through the Job Service
- Maintain visible queue depth
- Handle vendor outages without generating immediate workflow failures
- Apply rate limits consistently across integrations
Resource Structure
A QueueResource is configured as a resource node within Activate.
A common pattern is:
The parent queue resource typically contains shared defaults while child queue resources represent individual integration queues.
Parameter | Example | Purpose |
|---|---|---|
Class | Innovation.Activate.QueueResource | Defines the resource as a QueueResource |
Queue_RetrySequence | 1,1,1,1,1,5 | Retry timing sequence |
Queue_SubmitDelay | 1000 | Delay between queue submissions (ms) |
TransactionLimit | 100/min | Queue rate limit |
Queue_Count | 0 | Current queue depth |
Queue_MaxAge | 1 day | Maximum queue item age |
Queue_Timeout | 10 minutes | Idle queue thread timeout |
TransactionsPerSecond | 2.0 | Present in some configurations; verify provider usage before relying on it |
Queue Creation and Startup
When a queue is first required, GetQueue() requests a queue instance from the background thread manager.
If one does not already exist:
creates a new ActivateBackgroundQueue.
TryGetQueue() performs the same lookup without creating a new queue.
Queue Initialisation
When a queue starts, Activate loads any queued jobs that were previously persisted:
Each loaded job ID is added back into the queue.
This allows queued work to survive:
- Orchestrator restarts
- IIS recycles
- Server reboots
- Service interruptions
This process is commonly referred to as queue rehydration.
Queue Count
Queue_Count represents the current queue depth.
When the Orchestrator is running:
- Queue count comes from the active background queue
When the Orchestrator is not running:
- Queue count falls back to the stored Queue_Count parameter
The background queue updates this value automatically.
Important
Queue_Count is operational state, not deployment configuration.
Do not manually configure it.
Understanding ShouldTry
ShouldTry is the primary gate used before calling external systems.
This behaviour is often misunderstood.
Internally, the logic behaves approximately as follows:
Why?
This prevents new work from bypassing jobs that are already waiting in the queue.
A queued retry should always get the next opportunity to run before new work is introduced.
Key Concept
QueueResource does not simply rate-limit requests.
New jobs are blocked while work is already queued, but jobs already inside the queue are allowed to continue retrying.
This ensures:
- Fair processing order
- Consistent retry behaviour
- No queue starvation
- No bypassing queued work
Example Integration Pattern
A typical service desk integration follows this pattern:
Design principles:
- Resolve the queue from the resource tree
- Check ShouldTry immediately before submission
- Submit work only when allowed
- Retry when the queue is busy
- Separate retryable failures from permanent failures
This pattern is widely used in service desk integrations.
QueueJob()
QueueJob(job) is used when a workflow explicitly needs to move into the retry queue.
The method:
- Checks whether the job is already queued
- Prevents duplicate queue entries
- Adds the job ID to the queue
- Suspends the workflow
- Moves the job into WaitingAction
- Allows the queue to resume it later
The practical effect is:
Background Processing
When a queued job is processed:
Activate:
- Refreshes organisation context
- Sets the current context name
- Loads the job
- Marks the session as queue-driven
- Creates a JobWorkflow
- Runs the workflow
- Continues until completion or waiting state
If the workflow returns:
the queue logs the retry and processes it again later.
If an exception occurs:
- The queue logs the failure
- Attempts to mark the job as failed
- Prevents the queue thread from terminating
Rate Limiting
QueueResource includes a limiter used to control outbound throughput.
Connectors typically use:
This prevents requests from exceeding the queue's configured transaction rate.
The limiter is configured primarily through:
If no limit is configured, the current implementation defaults to approximately:
Understanding Burst Capacity
Rate limiting controls long-term throughput.
Burst capacity controls short-term spikes.
Think of it as a bucket:
When requests arrive:
When the burst is exhausted:
For example:
After a quiet period:
might be allowed immediately.
After the burst is consumed:
becomes the sustained processing rate.
This allows:
- Fast processing after idle periods
- Protection against sustained overload
- Better throughput during provisioning spikes
Burst capacity is not unlimited parallelism.
It simply allows temporary acceleration before returning to the configured rate.
Retry Timing
Retry timing is controlled by:
Example:
The sequence is interpreted as:
Rules:
- Comma or semicolon separated
- Values are seconds
- Empty sequence defaults to one minute
- Final value repeats indefinitely
This allows aggressive retrying for short-lived failures while backing off for longer outages.
Queue Processing Parameters
Parameter | Purpose |
|---|---|
Queue_RetrySequence | Retry timing sequence |
Queue_SubmitDelay | Delay between queue submissions |
Queue_MaxAge | Maximum age of queue entries |
Queue_Timeout | Idle thread timeout |
Queue_Count | Current queue depth |
TransactionLimit | Rate limiting configuration |
Connector Pattern
For API connectors, the recommended pattern is:
This ensures:
- Queue ordering is respected
- Rate limits are enforced
- Busy systems are protected
- Retries are handled consistently
Operational Guidance
Keep Queues Specific
Use one queue per external system.
Good:
Avoid:
unless a shared throttle is intentional.
Treat Queue_Count as State
Queue_Count should never be treated as deployment configuration.
It is runtime state maintained by the queue.
Tune Retry Timing Carefully
Example:
is aggressive.
Useful for:
- Short locks
- Temporary API delays
Less useful for:
- Vendor outages
- Extended maintenance windows
For unstable APIs, use longer retry intervals.
Monitor Queue Logs
Queue logs show:
- Queue startup
- Submission attempts
- Retry processing
- Queue depth changes
- Limiter statistics
These logs are often the fastest way to diagnose queue issues.
Troubleshooting
Symptom | What to Check |
|---|---|
Jobs are not retrying | Confirm the Orchestrator is running |
Queue count remains non-zero | Review pending jobs for the queue |
Jobs retry too quickly | Increase Queue_RetrySequence |
Jobs retry forever | Check workflow return values |
External API overloaded | Lower TransactionLimit |
Duplicate submissions | Ensure ShouldTry is checked immediately before submission |
Summary
QueueResource is Activate's queue-backed retry, replay, suspension, and throttling mechanism.
It allows workflows to:
- Pause safely
- Retry automatically
- Resume later
- Respect external rate limits
- Avoid overwhelming downstream systems
The typical pattern is:
- Define a QueueResource
- Configure retry and rate-limiting settings
- Resolve the queue in code
- Check ShouldTry
- Optionally acquire limiter capacity
- Call the external system
- Retry or queue work when necessary
- Allow the Orchestrator to replay the workflow later
This makes QueueResource the preferred mechanism for service desk integrations, vendor APIs, and any workflow where temporary failure should not immediately become job failure.