Event
One source callback with its payload and intake result. An accepted event starts recipient matching; a duplicate or rejected event does not.
Incoming event processing and fan-out delivery service
Receives HTTP events, selects recipients with configurable rules, creates a separate delivery for each match and keeps delivery history.
Project overview
What the service does
The service accepts HTTP events from advertising, analytics and product platforms and checks the source key and required fields. For each brand, it finds enabled Watchers and evaluates their Rules. Rules set event conditions, recipient details and delivery limits; a configured macro can transform an event field before a rule checks it. Every matching recipient gets a separate HTTP request. The same event key is treated as a duplicate for 24 hours. The service uses two PostgreSQL databases: one stores platform, brand, source, Watcher, Rule and macro settings; the other transactionally stores accepted events and the processing and delivery queues. ClickHouse receives each event after intake, routing decisions after recipient selection, and deliveries with their attempts after a final status is reached. PostgreSQL records are removed only after the ClickHouse copy is confirmed.
One event may need to reach several systems, each with its own conditions and request format. The service selects the right recipients, retries temporary delivery failures and keeps the results available to operators.
Who uses it
Teams integrating advertising, analytics or product systems through event callbacks; operations staff who configure rules and investigate delivery failures.
Input
- HTTP callback with the source key and event fields.
- Platform, brand and source settings.
- Watcher conditions, Rules, configured macros and recipient URLs.
Output
- An intake result: accepted, duplicate or rejected.
- A separate outbound request and status history for each matching recipient.
- ClickHouse history for event intake, routing decisions and finalized deliveries.
Event processing rules
- Each incoming source is checked against its platform, brand and key settings.
- An accepted event is deduplicated for 24 hours before recipient matching.
- Watcher, Rule and configured macros determine whether and how each recipient is called.
- Events, routing decisions and final delivery results are synchronized to ClickHouse at different stages.
01 / Architecture
Main entities
Watcher
An enabled recipient configuration for a brand. Its conditions decide whether the event should be considered for delivery.
Rule
Defines event conditions, request fields and delivery limits for a route. A match creates a separate Postback for that recipient.
Macro
The operator chooses a payload field, a supported text transform and its parameters. The resulting value can be used in a Rule condition and previewed before the rule is applied.
Postback / delivery attempt
A Postback stores the request prepared for one recipient; its delivery task tracks attempts, responses, retry time and final status.
From one incoming event to separate recipient deliveries
Four stages show what is stored, which rules run, how delivery is retried and when data reaches ClickHouse.
Receive and deduplicate
Every request is recorded; only the first accepted copy starts routing.
Event platform
Sends a callback with a platform key and event payload.
Validate the source
Checks platform and brand settings, key, and required fields using the admin configuration.
Commit event + dedup decision
One transaction uses an advisory lock and a 24-hour idempotency window. The event row and acceptance result are committed together.
Separate admin PostgreSQL: platforms, brands, source keys, watchers, rules and macros.
Choose routes and create deliveries
The fan-out worker turns one accepted event into one delivery row for every matching recipient.
Claim a pending event
A worker claims work with PostgreSQL SKIP LOCKED; other workers can process other events at the same time.
Load active watchers
A brand links the event to the watchers configured to inspect it.
Evaluate conditions
Rules check the event and delivery limits. A named macro can read a payload field, transform text with a configured delimiter function and provide that value to a rule condition.
Create one row per match
Matched recipients get separate requests. The rule result and reason are stored in the same transaction.
Send, record the response, retry if needed
The HTTP call runs outside the database transaction; a lease protects which worker may save its result.
Reserve a delivery
A worker reserves a pending request for a limited time. An expired worker cannot overwrite a newer attempt.
Call the recipient
Send the rule-built request and retain the status and response for this attempt.
Recipient response2xxMark delivered 429, 5xx, network errorSchedule another attempt with backoff Permanent error or retry limit reachedFinish with an error and keep the attempt history
Archive and clean up
Each record is synchronized when its state is ready; processing rows are removed only after ClickHouse confirms the copy.
ClickHouse
Versioned event, decision and delivery history for filtering and reports.
Verify before deleting
Cleanup checks that the ClickHouse row exists before deleting the corresponding PostgreSQL record.
02 / System architecture
Main components and their roles
Validate the source
Check the platform, brand, source key and required event fields.
How it works
The API stores the intake result in the processing PostgreSQL database. Source and brand configuration lives in the separate admin database.
Record and deduplicate
Store each request. If the same accepted event appears again within 24 hours, mark it duplicate and do not route it again.
How it works
A transaction-scoped advisory lock protects the idempotency check while the event is written.
Choose recipients
Check each enabled Watcher and its Rules against the event and brand.
How it works
A configured macro can read a payload field, apply a supported text transform and pass its value into a Rule condition. A matching Rule builds the recipient request and checks its limits.
Create one delivery per match
Create a separate Postback for every recipient whose Rule matches.
How it works
The worker claims events with SKIP LOCKED. Postbacks, routing decisions and event completion are saved in the same transaction.
Send the request
Claim a pending Postback and call its recipient URL.
How it works
The HTTP call runs outside the database transaction. A lease token prevents an expired worker from overwriting a newer result.
Record result or retry
Store the response and attempt number. Retry temporary HTTP 429, 5xx and network failures after a delay.
How it works
Successful, permanent-error and exhausted-retry deliveries keep their final status and attempt history. If a worker stops after sending but before saving the response, the service may send the request again.
Archive by processing stage
Copy an event after intake, its Watcher/Rule decisions after fan-out, and the Postback with attempts after delivery is finalized.
How it works
Synchronization runs asynchronously and retries failed copies. A worker marks PostgreSQL rows after a successful copy; cleanup checks for the corresponding ClickHouse row before removing operational data.
03 / Implementation details
Engineering decisions
PostgreSQL owns active work
PostgreSQL transactions protect accepted events, routing decisions, delivery jobs and their current state.
HTTP calls use leases
Workers reserve a delivery, make the network call outside the transaction and save the response only while they still own the lease.
ClickHouse keeps finalized history
Events, rule decisions and delivery attempts are copied when each part reaches its sync stage; PostgreSQL rows are cleaned up only after the copy is visible.
Software development for business
Have a project in mind?
Tell me what the application should do, how the team works today and which requirements or deadlines matter.