What Happens When Eight Workers Share One Odds API Key?

A worker gets a 429 response, waits for the requested delay, and tries again. That sounds like sensible client behaviour.

Now give the application eight workers.

Take a backend that retrieves bookmaker prices from an odds api. Each worker refreshes a snapshot for a different event. The URLs differ, but the workers use the same API key and compete for the same request allowance.

One worker waiting does not make the other seven wait. If they all receive the same delay, they can also wake up together and repeat the collision.

That is where a retry loop that looks correct on its own starts creating unnecessary work. The application needs to coordinate when requests start, when the group pauses, and how many extra attempts a failure can produce.

Those are separate decisions:

1. Every worker waits, and the group still exceeds the limit

For a small reproduction, give a fake server a token bucket with capacity two and a refill of two requests per second. A rejected request returns a one-second retry delay.

These are teaching settings. They are not an Odds API plan limit.

Start eight logical workers at time zero. The first two requests succeed. The other six get a 429 and wait one second.

At one second, all six retry. Two succeed, four get rejected. At two seconds, those four try again. Two succeed and two get rejected. The final two succeed at three seconds.

Every rejected worker respected its delay. The batch still needed 20 attempts to finish eight jobs.

Adding jitter would spread those retries out. It would not establish a shared ceiling on how many requests the workers can make. A concurrency limit answers a different question too: two requests in flight can still produce a high request rate if they complete quickly.

The decision belongs with the allowance the workers share. An event ID or a component instance is usually the wrong place to keep that state when several events use one credential and bucket.

2. Reproduce the collision without a live API

A virtual clock makes the sequence easy to check. The next item in a queue determines the current time. Waiting means scheduling that item later, so the test runs without sleeping.

Save the following as retry-model.mjs. It includes the fake server, the shared gate, and the scheduler used for both versions.

// Synthetic teaching settings: burst 2, refill 2 requests/second.
class Bucket {
  constructor() {
    this.tokens = 2;
    this.last = 0;
  }
  refill(now) {
    this.tokens = Math.min(2, this.tokens + (now - this.last) / 500);
    this.last = now;
  }
  take(now) {
    this.refill(now);
    if (this.tokens < 1) return false;
    this.tokens--;
    return true;
  }
}
 
export class Gate extends Bucket {
  blockedUntil = 0;
  defer(now, delayMs) {
    if (!Number.isFinite(delayMs) || delayMs < 0) {
      throw new RangeError('A finite, non-negative delay is required');
    }
    this.blockedUntil = Math.max(this.blockedUntil, now + delayMs);
  }
  admitAt(now) {
    if (now < this.blockedUntil) return this.blockedUntil;
    this.refill(now);
    if (this.tokens < 1) return now + Math.ceil((1 - this.tokens) * 500);
    this.tokens--;
    return now;
  }
}
 
export function simulate({
  shared = false,
  separateGates = false,
  alwaysReject = false,
  maxAttempts = 10,
  retryBudget = null,
} = {}) {
  const server = new Bucket();
  const gate = new Gate();
  const twoGates = [new Gate(), new Gate()];
  const queue = [];
  const timeline = [];
  let order = 0, steps = 0, completed = 0, abandoned = 0;
  let retriesLeft = retryBudget;
  const schedule = (at, worker, count) => {
    queue.push({ at, worker, count, order: order++ });
  };
 
  for (let worker = 0; worker < 8; worker++) schedule(0, worker, 0);
 
  while (queue.length) {
    if (++steps > 1000) throw new Error('Scheduler did not terminate');
    queue.sort((a, b) => a.at - b.at || a.order - b.order);
    const { at, worker, count } = queue.shift();
    if (count > 0 && retriesLeft !== null && retriesLeft <= 0) {
      abandoned++;
      continue;
    }
 
    const activeGate = shared ? gate : separateGates ? twoGates[worker % 2] : null;
    if (activeGate) {
      const next = activeGate.admitAt(at);
      if (next > at) {
        schedule(next, worker, count);
        continue;
      }
    }
 
    if (count > 0 && retriesLeft !== null) retriesLeft--;
    const allowed = !alwaysReject && server.take(at);
    timeline.push({ at, worker, status: allowed ? 200 : 429 });
    if (allowed) {
      completed++;
      continue;
    }
    if (count + 1 >= maxAttempts) {
      abandoned++;
      continue;
    }
 
    // The fake server's 429 has Retry-After: 1.
    if (activeGate) activeGate.defer(at, 1000);
    schedule(at + 1000, worker, count + 1);
  }
 
  return {
    attempts: timeline.length,
    rejected: timeline.filter(x => x.status === 429).length,
    completed,
    abandoned,
    lastAttemptMs: timeline.at(-1)?.at ?? 0,
    timeline,
  };
}

The at value is virtual time in milliseconds. Equal-time items have a stable order so the same inputs produce the same trace. The eight workers are logical jobs in this model, rather than eight operating-system threads.

With shared: false, each worker schedules its own retry. With shared: true, every attempt passes through one gate, including the initial requests. That last detail matters: slowing retries while letting fresh work bypass the gate leaves another way to exceed the allowance.

The gate consumes a token only when it admits an attempt. A worker told to return later goes back into the queue without consuming a retry attempt.

For this model, the shared version admits requests at 0, 0, 500, 1,000, 1,500, 2,000, 2,500 and 3,000 milliseconds.

Both finish at three seconds. The shared version avoids 12 unnecessary attempts under the selected settings. That is a result from this deterministic example, not a production throughput measurement or a promise that coordinated clients never receive 429s.

3. A shared delay still needs a controlled restart

There is an easy halfway fix: keep one shared blockedUntil value and make every worker wait for it.

That stops requests during the cooldown. It can still release the whole queue at the same instant afterwards.

In the model, Gate.admitAt checks the cooldown first, then checks the token bucket. If eight workers return at the deadline, only two can use the recovered burst capacity. The other six wait for tokens.

There is another edge case when responses arrive at different times. One response might ask for five seconds, while a later response asks for one. The shorter delay should not move the group's existing deadline backwards.

That is why defer takes the maximum of the existing deadline and the newly observed one. A longer instruction can extend the pause. A shorter one leaves it intact.

A local backoff cap should not shorten the provider's requested wait either. If the server asks for 30 seconds and the job only has 10 seconds left, the job should stop or defer to a later batch. It should not turn that instruction into a 10-second retry.

These controls affect new admissions. Requests already in flight may still complete or return more 429s after the gate closes.

4. Keep the rate-limit feedback before the client hides it

An Odds API snapshot request uses the event route GET /v1/events/{event_id}/odds/snapshot. Its public contract describes Retry-After and the X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Bucket headers when supplied.

There is a practical integration detail in the public TypeScript SDK. Its request method makes one fetch and throws an OddsApiError for a failed response. That error exposes the status and parsed body, but does not expose the response headers.

A catch block receiving only that error cannot read Retry-After from it. The constructor's fetchImpl option provides a place to observe the response before the SDK consumes its body.

Save this small observer as observe-throttle.mjs:

// Observes delay-seconds headers; admission and retries belong to the caller.
export function observeThrottle(fetchImpl, gate, now) {
  return async (...args) => {
    const response = await fetchImpl(...args);
    if (response.status === 429) {
      const raw = response.headers.get('Retry-After')?.trim();
      if (raw && /^\d+$/.test(raw)) {
        const delayMs = Number(raw) * 1000;
        if (!Number.isSafeInteger(delayMs)) {
          throw new RangeError('Retry-After exceeds the supported delay');
        }
        gate.defer(now(), delayMs);
      }
    }
    return response;
  };
}

The observer returns the original response and leaves its body readable. It handles the delay-seconds format documented by this API. An unrepresentable delay fails explicitly instead of becoming a shorter wait.

This helper only records cooldown feedback. It does not perform admission or retry the request. The caller still owns those decisions. When a delay is missing or invalid, use a bounded fallback backoff with jitter; do not interpret an unchanged cooldown as permission for an immediate retry loop.

The gate and the header observer must also use the same clock basis. A running application can use a monotonic clock for elapsed waits; the tests supply virtual milliseconds.

Keep one coordinator for the credential and request bucket being used. The bucket header can help identify the provider's scope, but its remaining count is approximate. Several responses reporting one remaining request do not give several workers a reserved slot.

The capacity header is not a complete refill policy either. Configure the request rate from the provider's documented allowance. Monthly credits and maximum snapshot page sizes describe different limits.

5. Give retries a shared allowance of their own

Pacing requests controls when they leave the application. It does not decide how much work should continue during a persistent failure.

Suppose each of eight workers gets three attempts. That is a possible 24 calls, even though the application only wanted eight snapshots.

The model's retryBudget adds another bound. With a shared allowance of four retries, eight initial jobs can create at most 12 attempts. Every extra attempt consumes that same allowance, whichever worker makes it.

The per-worker cap still has a purpose. It prevents one job from retrying indefinitely. The group allowance limits the amplification across jobs.

Here the allowance lasts for one batch and is not replenished. Creating a fresh allowance for every failing job would remove the group bound. A long-running service needs an explicit replenishment policy and a limit on queued work as well.

Time needs a bound too. An event snapshot can stop being useful while a job waits. Give the batch a deadline, abandon work that cannot be admitted in time, and distinguish an abandoned refresh from a successful one. The demonstration counts abandoned jobs; it does not implement a production queue's expiry or cancellation policy.

Limit automatic replay to requests whose semantics permit it. These workers fetch snapshots with GET. A write endpoint needs its own idempotency and failure rules.

6. Run the checks against the failure, not just the happy path

Save this as retry-demo.test.mjs beside the other two files:

import assert from 'node:assert/strict';
import test from 'node:test';
import { Gate, simulate } from './retry-model.mjs';
import { observeThrottle } from './observe-throttle.mjs';
 
test('independent delays repeat the collision; shared admission avoids it in this model', () => {
  const independent = simulate();
  const shared = simulate({ shared: true });
  assert.deepEqual([independent.attempts, independent.rejected, independent.completed], [20, 12, 8]);
  assert.deepEqual([shared.attempts, shared.rejected, shared.completed], [8, 0, 8]);
  assert.equal(independent.lastAttemptMs, 3000);
  assert.equal(shared.lastAttemptMs, 3000);
  assert.deepEqual(shared.timeline.map(x => x.at), [0, 0, 500, 1000, 1500, 2000, 2500, 3000]);
});
 
test('cooldown blocks every worker, then admission still limits the burst', () => {
  const gate = new Gate();
  gate.defer(0, 2000);
  assert.ok(Array.from({ length: 8 }, () => gate.admitAt(1999)).every(t => t === 2000));
  const resumed = Array.from({ length: 8 }, () => gate.admitAt(2000));
  assert.equal(resumed.filter(t => t === 2000).length, 2);
  assert.equal(resumed.filter(t => t > 2000).length, 6);
});
 
test('later responses can extend the cooldown but cannot shorten it', () => {
  const gate = new Gate();
  gate.defer(0, 5000);
  gate.defer(100, 1000);
  assert.equal(gate.blockedUntil, 5000);
  gate.defer(1000, 8000);
  assert.equal(gate.blockedUntil, 9000);
});
 
test('two independent gates still exceed the one server allowance', () => {
  const result = simulate({ separateGates: true });
  assert.deepEqual([result.attempts, result.rejected, result.completed], [14, 6, 8]);
});
 
test('one batch retry allowance bounds extra calls across all eight workers', () => {
  const perWorker = simulate({ alwaysReject: true, maxAttempts: 3 });
  const shared = simulate({ alwaysReject: true, maxAttempts: 3, retryBudget: 4 });
  assert.deepEqual([perWorker.attempts, perWorker.abandoned], [24, 8]);
  assert.deepEqual([shared.attempts, shared.abandoned], [12, 8]);
  assert.equal(shared.completed, 0);
});
 
test('the observer preserves a long server delay and leaves the body readable', async () => {
  const gate = new Gate();
  const response = new Response('{"detail":"Synthetic throttle"}', {
    status: 429, headers: { 'Retry-After': '30' },
  });
  const transport = observeThrottle(async () => response, gate, () => 100);
  assert.equal(await transport('https://example.invalid/snapshot'), response);
  assert.equal(gate.blockedUntil, 30100);
  assert.equal(gate.admitAt(10000), 30100);
  assert.equal((await response.json()).detail, 'Synthetic throttle');
});
 
test('missing or invalid delay headers do not become invented cooldowns', async () => {
  for (const raw of [null, 'later', '-1', '1.5']) {
    const gate = new Gate();
    const headers = raw === null ? {} : { 'Retry-After': raw };
    const response = new Response(null, { status: 429, headers });
    const transport = observeThrottle(async () => response, gate, () => 0);
    assert.equal(await transport('https://example.invalid/snapshot'), response);
    assert.equal(gate.blockedUntil, 0);
  }
});
 
test('an unrepresentable server delay fails instead of shortening the requested wait', async () => {
  const gate = new Gate();
  const response = new Response(null, { status: 429, headers: { 'Retry-After': '999999999999999999999' } });
  const transport = observeThrottle(async () => response, gate, () => 0);
  await assert.rejects(transport('https://example.invalid/snapshot'), RangeError);
});

The examples were tested with Node.js 24.14.1. They use built-in modules and make no network requests. Run:

node --test retry-demo.test.mjs

All eight checks pass. They cover the repeated collision, paced recovery, overlapping delays, separate coordinators, aggregate retry counts and header observation without consuming the response body.

The persistent-failure test also makes an important distinction visible: exhausting the retry allowance produces abandoned jobs. It does not make the batch successful merely because requests have stopped.

For the public SDK behaviour described here, the source review used commit 2f15370 of odds-api/odds-api, specifically sdks/typescript/src/index.ts. The scheduler and observer are separate teaching code.

7. Know when waiting cannot solve the problem

A 429 does not always mean a short request window has filled up. Odds API's public contract also describes monthly API-credit exhaustion.

Backoff does not replenish that quota. Use available quota information to decide whether work can continue, and keep a terminal path when it cannot.

The diagnostic work needs coordination too. If eight rejected workers each start polling /usage, the application has added another group of requests to the same failure. Make a bounded diagnostic decision for the group, and allow for that diagnostic request to fail.

The same caution applies when the application grows beyond one process. A JavaScript object only coordinates the tasks that use that object. Starting another process with another gate creates another independent allowance in memory.

The model's two-gate version makes 14 attempts, including six rejections, to finish the same eight jobs. It demonstrates competing local state; it is not a distributed-system test.

Several processes need a shared admission authority, such as a request dispatcher or an atomic shared-store design. That adds costs: another dependency, queue delays, failure handling and fairness between callers. A Redis counter by itself does not settle those questions. Admission, cooldown updates and retry accounting have to remain coordinated when processes race or disappear.

For one backend process, a shared gate can be enough to start. As the system grows, keep the boundary explicit: which callers share an allowance, and where is the decision made?

Eight workers can all follow their own retry rules while the application keeps exceeding its limit. When the API sees one shared allowance, the caller needs a shared place to decide what happens next.

Disclosure: I run Odds API, the service used as the integration example.

AI assistance: Codex assisted with repository research, drafting and the example code. The included examples were tested locally with Node.js 24.14.1.