Jobs, Queues, and Batch Processing

An ERP is a batch system with an interactive UI attached. Most of the value — MRP, depreciation, settlement, payroll, billing, interfaces — happens in scheduled jobs, usually at night, usually unattended.

The job landscape

Frequency	Jobs
Every few minutes	Interface polling, IDoc/message processing, output/print, workflow timers
Hourly	ATP recalculation, replication, cache refresh
Nightly	MRP net change, billing due list, dunning, ATP rebuild, index maintenance, backups, data quality checks
Weekly	MRP regenerative, reorganisation, statistics, archiving
Monthly	Depreciation, settlement, allocations, WIP, revenue recognition, FX revaluation, close reports, payroll
Annually	Balance carryforward, number ranges, budget rollover, standard cost roll, calendar creation

The seven properties every ERP job must have

1. Idempotent

Running it twice for the same scope produces the same result as running it once.

Depreciation run for period 03/2026, run twice
  → second run posts nothing (already posted for the period)

Implement with a run registry:

job_run(
  job_name, scope_key,       -- e.g. 'DEPRECIATION' + 'CC1000|2026|03'
  status,                    -- RUNNING | COMPLETED | FAILED
  started_at, completed_at,
  records_processed, records_failed,
  parameters JSONB,
  UNIQUE(job_name, scope_key, status) WHERE status = 'COMPLETED'
)

2. Restartable

A failure at record 80,000 of 100,000 resumes from 80,001.

job_checkpoint(run_id, last_processed_key, records_done, updated_at)

Checkpoint every N records, in the same transaction as the work. Then a crash loses at most one chunk.

3. Test / simulation mode

Produce the numbers, write nothing. Non-negotiable for anything financial — the controller will run it in test mode, review, then run it for real.

4. Chunked commits

Not: one transaction for 5,000,000 records   (blocks cleanup, brutal rollback, huge locks)
Not: one transaction per record              (commit overhead dominates)
Yes: commit every 1,000–10,000 records

5. Parallelisable

Split by a natural key range — company code, plant, material range, asset number, employee number.

Depreciation:  4 parallel jobs, asset numbers 1–250k, 250k–500k, 500k–750k, 750k–1M

Ensure the partitions don’t contend on the same rows. Splitting by a key that shares a hot aggregate row just moves the contention.

6. Observable

[2026-03-01 02:14:33] DEPRECIATION_RUN started, period 03/2026, companies [1000,2000]
[2026-03-01 02:14:35] Company 1000: 84,231 assets selected
[2026-03-01 02:19:12] Company 1000: 40,000/84,231 (47%) elapsed 4m37s, ETA 5m10s
[2026-03-01 02:24:51] Company 1000: complete. 84,198 posted, 33 errors
[2026-03-01 02:24:51] Errors written to job_error, run_id 88213

A silent job that has been running for three hours at 3am is indistinguishable from a hung one. Progress output is an operational requirement, not a nicety.

7. Graceful error handling

Wrong:  one bad asset → exception → whole run rolls back → nothing posted
Right:  one bad asset → log it → continue → report "84,198 posted, 33 failed" with reasons

Then provide a reprocessing path for the failures. A run that succeeds for 99.96% and reports the rest is far more useful than an all-or-nothing run that never completes.

Job scheduling

job_definition(
  id, name, program, parameters JSONB,
  schedule_cron, calendar_id,           -- factory/business calendar aware
  depends_on[],                          -- predecessor jobs
  max_runtime_minutes, retry_count, retry_delay,
  on_failure,                            -- ALERT | CONTINUE | STOP_CHAIN
  enabled, owner, business_criticality
)

Dependencies matter more than time. “Run depreciation at 02:00” fails when the prerequisite job runs late. “Run depreciation after asset acquisition posting completes” is correct. Model the chain, not the clock.

Nightly chain:
  interface_inbound
     └─▶ goods_movement_posting
            └─▶ inventory_valuation
                   ├─▶ mrp_net_change
                   └─▶ costing_update
                          └─▶ analytics_extract

Calendar awareness: a job scheduled for “the last working day of the month” must consult the factory/business calendar, not compute month-end minus weekends.

Queues and asynchronous processing

Producer ──▶ Queue ──▶ Consumer(s)
              │
         Retry with backoff
              │
        Dead letter queue ──▶ human review

The transactional outbox

The single most important pattern for reliable ERP integration:

BEGIN;
  INSERT INTO goods_receipt (...);
  UPDATE inventory SET ...;
  INSERT INTO journal_line (...);
  INSERT INTO outbox (event_type, payload, created_at) VALUES ('GoodsReceiptPosted', {...}, now());
COMMIT;

-- separate relay process
SELECT * FROM outbox WHERE published_at IS NULL ORDER BY id LIMIT 100;
  → publish to broker
  → UPDATE outbox SET published_at = now()

Why: publishing to a broker inside the business transaction gives you two failure modes — an event published for a transaction that rolled back, or a committed transaction with no event. The outbox eliminates both. Nothing else does.

Dead letter handling

Messages that fail repeatedly must go somewhere a human looks at, with:

The original payload
The error and the stack
Retry count and history
A reprocess action after the underlying issue is fixed
An owner and an alert. A DLQ nobody monitors is a data-loss mechanism with extra steps.

Concurrency in batch

The classic problems:

Problem	Mitigation
Batch job vs interactive users on the same rows	Run in a low-usage window; chunk commits; shorter transactions
Two instances of the same job	Advisory lock or a unique constraint on the run registry
Deadlock between parallel workers	Process keys in a consistent order; partition so ranges don’t overlap
Long transaction blocking cleanup/vacuum	Chunked commits
Batch saturating the connection pool	A separate pool for batch, capped

The batch window

18:00  Users go home (mostly)
20:00  Interfaces settle
22:00  ─────── BATCH WINDOW OPENS ───────
       Backups
       Data quality checks
       MRP net change                    ← the long one
       Costing update
       Billing due list
       Dunning proposal
       Analytics extract
       Index maintenance
05:00  ─────── WINDOW CLOSES ───────
06:00  Early shift logs in

The window shrinks as the business grows and the data grows. Track every job’s duration as a time series — an MRP run creeping from 90 minutes to 4 hours over a year tells you months in advance that you have a problem.

For global companies there is no window: someone is always working. This forces online-capable batch — jobs that can run alongside users, with smaller chunks, lower priority, and throttling.

Print and output jobs

Output determination generates documents (invoices, delivery notes, purchase orders) and routes them:

Trigger (document posted)
  → determine output type, medium, and recipient from config
  → render (form template + data)
  → dispatch: printer / email / EDI / portal / e-invoicing API / archive
  → record status; retry on failure

Common failure modes: printer offline, mailbox full, e-invoicing API down, template error on an edge-case document. Every output needs a status and a resend action. “Did the customer get the invoice?” must be answerable from the system.

Monitoring and alerting

Alert on:

Condition	Severity
Critical job failed	Page someone
Job exceeded 2× normal duration	Warn
Job did not start when scheduled	Warn
Dead letter queue depth > threshold	Warn
Interface error queue growing	Warn
Chain broken (a predecessor failed)	Page during close
Batch window overrun	Warn, trending to critical
Zero records processed when records were expected	Warn — silent failure is the worst kind

That last one catches the nastiest bug class: a job that “succeeds” while selecting nothing because a parameter or a date range was wrong.