PortfolioData integration
In-house integration application for client data
Dystributr · SaaS · Data integration
Dystributr is an application we built from the ground up and run for clients. It receives a payload, checks the client, and delivers that payload to the systems that the client uses. We currently operate it in-house.
- Application
- Integrations
- Infrastructure
- Security
- Operations
01 · Story
The story
Dystributr is an integration application we built for client work and operate in-house. When a client sends data, the system checks who they are, records the payload, and delivers it to the systems that client already uses. Intake, routing, credentials, and delivery stay in one place we run with the engagement.
Clients rarely send data to one destination forever. Some routes need parallel delivery to several systems at once. Others need steps in order, such as creating a record and then notifying a downstream tool with that record. The application chooses the path from each client's configured routes instead of hard-coding every connection.
Shared tables, queues, and topics hold the routing data and the work in flight. Each destination is a small deployable handler, so adding a new system for a client is registration work, not a rewrite of intake.
The operations console is a web application where we manage accounts, routes, tokens, templates, intake history, errors, and releases. Handler credentials stay in a managed secrets store. The public site at dystributr.com introduces the product.
02 · Constraints
Challenges
- 01
Each client has a different set of destinations
One intake has to serve many destinations. Routes are data, keyed to the client and the operation, so a new destination is a registered handler. The intake function stays the same.
- 02
Some deliveries can run together, and some have to wait
A notice that can go to several systems at once uses parallel fan-out. When a step needs the previous result, such as creating a record and then sending a message that includes that record, then the route uses a pipeline. Fan-out chooses the path from the route record.
- 03
We have to be able to operate every client
Dystributr is in-house. Tokens, usage, failed intake, and template delivery need an operations interface, logs, and a way to redrive a failed message. Those controls are part of the application.
03 · Delivery
What we did
- Built intake, authorization, parallel delivery, and ordered pipelines as separate serverless services, with shared infrastructure for tables, topics, and queues.
- Stored clients, handlers, and routes as data so a new destination is configuration, not a new intake path.
- Built the operations console for accounts, routes, tokens, templates, intake, logs, errors, and releases.
- Routed failed work to dead-letter queues and surfaced intake errors in the console for redrive.
- Exercised routes in a local AWS-compatible environment before client deployments.
04 · Results
Outcomes
- Client data moves through one application we deploy and operate.
- A new destination is a registered handler, and the intake function stays in place.
- Accounts, routes, tokens, failures, and deploys are managed in the operations application.
Stack
Next
Planning a site like this?
If you want a site that fits your industry and can grow without a rewrite in a year, tell us what the business needs to do online. We will respond with a clear first version and what should wait.