QueueResource Technical Reference

overview queueresource is the activate resource type used to queue, retry, suspend, resume, and throttle work that should not fail immediately when a downstream dependency is unavailable, busy, or rate limited unlike a simple rate limiter, a queueresource provides a complete queue backed workflow replay mechanism it allows workflows to pause safely, be retried later by the orchestrator, and resume from the point at which they were suspended typical use cases include service desk integrations rest api integrations vendor platforms with rate limits systems that experience transient outages long running synchronisation processes workflows that should retry automatically rather than fail immediately how queueresource works a queueresource owns an activatebackgroundqueue internally, the queue is created or retrieved using provisioningbackgroundthreads instance getorstart( ) the queue is keyed by the resource path, ensuring that each queue resource maintains its own independent workload the queue is responsible for maintaining queued jobs retrying failed work suspending workflows resuming workflows later applying rate limits tracking queue depth processing retry schedules at a high level workflow | v queueresource | + > queue shouldtry | + > limiter tryacquire() | + > external system why use queueresource? a queueresource allows activate to prevent external systems from being overwhelmed retry transient failures automatically suspend workflows safely resume processing later through the job service maintain visible queue depth handle vendor outages without generating immediate workflow failures apply rate limits consistently across integrations resource structure a queueresource is configured as a resource node within activate a common pattern is queues ├─ servicenow ├─ jira ├─ freshservice └─ vendorapi the parent queue resource typically contains shared defaults while child queue resources represent individual integration queues parameter example purpose class innovation activate queueresource defines the resource as a queueresource queue retrysequence 1,1,1,1,1,5 retry timing sequence queue submitdelay 1000 delay between queue submissions (ms) transactionlimit 100/min queue rate limit queue count 0 current queue depth queue maxage 1 day maximum queue item age queue timeout 10 minutes idle queue thread timeout transactionspersecond 2 0 present in some configurations; verify provider usage before relying on it queue creation and startup when a queue is first required, getqueue() requests a queue instance from the background thread manager if one does not already exist provisioningbackgroundthreads instance getorstart( ) creates a new activatebackgroundqueue trygetqueue() performs the same lookup without creating a new queue queue initialisation when a queue starts, activate loads any queued jobs that were previously persisted jobcollection loadqueuedjobs(this) each loaded job id is added back into the queue this allows queued work to survive orchestrator restarts iis recycles server reboots service interruptions this process is commonly referred to as queue rehydration queue count queue count represents the current queue depth when the orchestrator is running queue count comes from the active background queue when the orchestrator is not running queue count falls back to the stored queue count parameter the background queue updates this value automatically important queue count is operational state, not deployment configuration do not manually configure it understanding shouldtry shouldtry is the primary gate used before calling external systems this behaviour is often misunderstood internally, the logic behaves approximately as follows if current job is already in this queue allow processing else only allow processing if queue count is zero why? this prevents new work from bypassing jobs that are already waiting in the queue a queued retry should always get the next opportunity to run before new work is introduced key concept queueresource does not simply rate limit requests new jobs are blocked while work is already queued, but jobs already inside the queue are allowed to continue retrying this ensures fair processing order consistent retry behaviour no queue starvation no bypassing queued work example integration pattern a typical service desk integration follows this pattern queueresource queue = evaluator getobject("=//resources/integration/queues/example") as queueresource; if (queue shouldtry) { if (createincident( )) { return jobworkflow\ result nextstep; } } return jobworkflow\ result retry; design principles resolve the queue from the resource tree check shouldtry immediately before submission submit work only when allowed retry when the queue is busy separate retryable failures from permanent failures this pattern is widely used in service desk integrations queuejob() queuejob(job) is used when a workflow explicitly needs to move into the retry queue the method checks whether the job is already queued prevents duplicate queue entries adds the job id to the queue suspends the workflow moves the job into waitingaction allows the queue to resume it later the practical effect is running job | v queuejob() | v waitingaction | v background queue | v resume workflow later background processing when a queued job is processed backgroundprocess(jobid) activate refreshes organisation context sets the current context name loads the job marks the session as queue driven creates a jobworkflow runs the workflow continues until completion or waiting state if the workflow returns waitaction the queue logs the retry and processes it again later if an exception occurs the queue logs the failure attempts to mark the job as failed prevents the queue thread from terminating rate limiting queueresource includes a limiter used to control outbound throughput connectors typically use if (!queue limiter tryacquire() succeed) { response isretryable = true; response shouldretry = true; return response; } this prevents requests from exceeding the queue's configured transaction rate the limiter is configured primarily through transactionlimit if no limit is configured, the current implementation defaults to approximately 10 transactions per second understanding burst capacity rate limiting controls long term throughput burst capacity controls short term spikes think of it as a bucket capacity builds up while idle when requests arrive capacity is consumed when the burst is exhausted requests fall back to the steady rate for example transactionlimit = 5/sec burst = 20 after a quiet period 20 requests might be allowed immediately after the burst is consumed 5 requests per second becomes the sustained processing rate this allows fast processing after idle periods protection against sustained overload better throughput during provisioning spikes burst capacity is not unlimited parallelism it simply allows temporary acceleration before returning to the configured rate retry timing retry timing is controlled by queue retrysequence example 1,1,1,1,1,5 the sequence is interpreted as retry 1 after 1 second retry 2 after 1 second retry 3 after 1 second retry 4 after 1 second retry 5 after 1 second retry 6 and onwards after 5 seconds rules comma or semicolon separated values are seconds empty sequence defaults to one minute final value repeats indefinitely this allows aggressive retrying for short lived failures while backing off for longer outages queue processing parameters parameter purpose queue retrysequence retry timing sequence queue submitdelay delay between queue submissions queue maxage maximum age of queue entries queue timeout idle thread timeout queue count current queue depth transactionlimit rate limiting configuration connector pattern for api connectors, the recommended pattern is if (queue != null && !queue shouldtry) { return retryresult(); } if (queue != null && !queue limiter tryacquire() succeed) { return retryresult(); } sendrequest(); this ensures queue ordering is respected rate limits are enforced busy systems are protected retries are handled consistently operational guidance keep queues specific use one queue per external system good queues ├─ servicenow ├─ jira ├─ workday avoid queues └─ sharedexternalsystems unless a shared throttle is intentional treat queue count as state queue count should never be treated as deployment configuration it is runtime state maintained by the queue tune retry timing carefully example 1,1,1,1,1,5 is aggressive useful for short locks temporary api delays less useful for vendor outages extended maintenance windows for unstable apis, use longer retry intervals monitor queue logs queue logs show queue startup submission attempts retry processing queue depth changes limiter statistics these logs are often the fastest way to diagnose queue issues troubleshooting symptom what to check jobs are not retrying confirm the orchestrator is running queue count remains non zero review pending jobs for the queue jobs retry too quickly increase queue retrysequence jobs retry forever check workflow return values external api overloaded lower transactionlimit duplicate submissions ensure shouldtry is checked immediately before submission summary queueresource is activate's queue backed retry, replay, suspension, and throttling mechanism it allows workflows to pause safely retry automatically resume later respect external rate limits avoid overwhelming downstream systems the typical pattern is define a queueresource configure retry and rate limiting settings resolve the queue in code check shouldtry optionally acquire limiter capacity call the external system retry or queue work when necessary allow the orchestrator to replay the workflow later this makes queueresource the preferred mechanism for service desk integrations, vendor apis, and any workflow where temporary failure should not immediately become job failure