2026-03-22 Engineering

The Signal Engine is not a cron job. Here is why.

Most 'compliance engines' are a cron that checks a checklist once a day. Ours is not a cron. Here is why.

So. The competition has a “compliance engine.” You look under the hood; a 24h cron, a list of SQL conditions, an email blast. That is not an engine. That is rust.

TL;DR

Cron = batch run at 3 AM; signal arrives 23 hours too late
Event-driven on every mutation is too expensive (1,000 signals/sec)
Our solution: signal scheduler with reactive deps at the model level plus cron fallback
Median latency: 14 seconds; max: 4 minutes

The problem; cron is not enough

Context: a client logs an incident at 10:00. They should get a NIS2-001 signal (incident >24h without notification) at 10:00 the next day, as an early warning. Cron runs every 24h at 03:00; signal arrives 5 hours late. How it ended; a missed deadline, an administrative fine, client looking for another vendor.

The decision; event-based signal scheduler plus per-type intervals

Each signal type has its own interval (tickRate). Some are reactive (on every mutation), some are cron (once an hour). DPIA-MISSING fires reactively because it changes on every RcpEntry update. ASV-EXPIRED fires once a day because the last scan date rarely changes.

Decision: per-signal-type configuration. Reactive or cron, not both at once. Configured in code, not in the database; because a change requires a code review.

The solution; two paths

Reactive path

Every mutation on RcpEntry / Supplier / Contract / Incident passes through middleware that publishes an event to BullMQ. A worker picks it up, asks the scheduler “which signal types react to this event,” fires the checks. The result lands in the Signal table.

Cron path

For types that have no natural trigger (e.g. “contract expires in 90 days”) it runs once a day at 04:00. For “ASV scan overdue”; once a week on Sunday.

Pitfalls we hit

Pitfall 1: recursion. A signal generates a notification, a notification generates an audit log entry, an audit log entry generates a signal "audit log entry." Loop. Fix: the scheduler has a blacklist of entities from which it does not pull back-events.

Pitfall 2: bulk import kills the system. A client imports 4,000 RcpEntry rows at once. 4,000 events. Worker explodes. Fix: a bulk-aware emitter that groups N events into a single batch event; the scheduler processes them as a batch.

Outage: once BullMQ lost 80 jobs after a Redis restart. A client missed a signal about a transfer outside the EEA. Fix: every enqueued job is logged to PostgreSQL before it is enqueued; a recovery job on scheduler startup re-queues whatever Redis lost.

Numbers

Median latency in production: 14 seconds from mutation to Signal write. P99: 4 minutes (when the worker is busy). Cron-only fallback: 1h. That is enough for the 22 signal types we currently handle.

Written by someone on the RealSec team. Want to talk specifics ; get in touch.

← All notes