Desktop app for data pipelines and rules
Data pipelines and rule sets, built as a graph.
Place components on a canvas, connect them and run. Delta turns the graph into a Go program you can build and deploy on its own.
How it works
From a drawing to a running program.
- 1
Draw the graph
Pick components from a palette and place them on a canvas: read a file, filter, join, look up, decide, write a file. Drag from an output to an input to connect them.
- 2
Run it and watch
Delta turns the graph into a Go program, builds it and runs it, and shows each component’s record counts and timings as it goes. A trace also records what every component put out.
- 3
Build what you ship
Build an executable that runs the graph without Delta: for Windows, Linux or macOS, on x64 or ARM, whichever machine you are building on.
Speed
Millions of records a second.
A run is not Delta working through your graph. It is a program compiled from the graph, and after a run Delta shows the time each component took, so you can see where a graph spends it.
| Graph | Records a second | Time | Most memory |
|---|---|---|---|
| DeDupe Keep the first order of each customer | 9.0 million | 1.1 s | 44 MB |
| Filter Keep the shipped orders of ten or more | 7.0 million | 1.4 s | 31 MB |
| Rollup Total the quantity of each product in each region | 4.1 million | 2.4 s | 39 MB |
| Validation, Lookup and Decision Table Validate, look each customer up, price with a decision table | 3.3 million | 3.0 s | 135 MB |
| Join Join every order to its customer | 2.8 million | 3.5 s | 139 MB |
| Sort Sort by date and then customer | 1.0 million | 9.3 s | 3.4 GB |
Measured by us on a 16-core AMD Ryzen 9 7950X3D with 64 GB of memory, running Windows 11: the middle of three runs of each program, from start to finish, with the file already in the system's cache. Your files, graphs and machine will give other figures.
Compiled, not interpreted
Delta writes the graph as a Go program and compiles it for the machine it will run on. What processes your records is native code, with no interpreter reading the graph as they go by.
Every component works at once
Each component runs on its own and hands records on in batches as they arrive, so reading, filtering and writing happen at the same time. It does not take a big machine: held to four processor threads, these programs ran about as fast.
Memory that does not grow with the file
Most components take a record and pass it on, holding nothing: filtering ten million orders took 31 MB, about what one million took. Join and Lookup hold their reference data, and a Sort holds every record, which is why its figure is the large one.
Components
Everything on the palette.
Each kind of component has its own color on the canvas. Delimited files, JSON and MySQL are read and written directly.
Input
- File Input
- Read a delimited file
- JSON Input
- Read a JSON file
- Multi-File Input
- Read a file stored in parts
- MySQL Input
- Read the result of a query
- Random Input
- Generate records
- Rejects
- The records another component could not process
Output
- File Output
- Write a delimited file
- JSON Output
- Write a JSON file
- Multi-File Output
- Write a file in parts
- MySQL Output
- Write records to a table
Logic
- DeDupe
- Keep the first record of each key
- Filter
- Keep records that match
- Join
- Match records of two streams
- Rollup
- Group and total
- Sort
- Order records
Rules
- Decision Table
- Rows of conditions and outcomes
- Router
- Send records down the first matching route
- Validation
- Separate valid from invalid records
- Set
- Add or replace columns
- Score
- Add up points per rule
- Lookup
- Add columns from reference data
- MySQL Query
- Run a query for each record
- Sub-graph
- Run another graph as a step
- Loop
- Run another graph until its output stops changing
Stream
- Merge
- One stream after another
- Transform
- Reshape each record
- Split
- Copy a stream to two branches
In the app
Made for work that has to be right.
Rules as well as pipelines
Decision tables, routers, validation, scoring and lookups sit on the same canvas as filters, joins, sorts and rollups, so the rules and the data they work on are one graph.
Tests that travel with the graph
A graph can carry its own tests: named cases with expected output files. Run them from the app and see which cases no longer match.
Development and production settings
File components have a development location and a production one. A location written as env:NAME is read from that environment variable when the program runs.
File names that fill themselves in
Put the date of the run, a variable of the graph or the value of a column into a file location, and get a file per day, per region or per value.
Graphs made of graphs
Run another graph as a step with a sub-graph, or as a loop that goes round until its output stops changing.
Bad records have somewhere to go
Say whether a record that cannot be processed stops the run or is rejected, and send the rejects down their own branch of the graph.
Deploy
What you build runs without Delta.
A built program needs no license and no Delta beside it. It goes on working whatever becomes of your license.
An executable
One program for the machine you choose. It needs neither Delta nor Go where it runs, and takes variables from its command line or the environment.
An HTTP service
A graph with one JSON Input and one JSON Output becomes a service: post records to it and the response is what the graph wrote. Each request is its own run.
A folder, a Docker image or an AWS Lambda function
A package puts the files the program reads beside it, with a README saying how to run it. Copy the folder, build the image, or upload the zip.
POST / run the graph on the JSON in the request GET /health say that the service is there
The service has no authentication or TLS of its own, so it belongs behind something that does. Asked to stop, it finishes the requests in progress first.
Assistant
An assistant that keeps your data on your computer.
Ask how Delta works, or describe a pipeline and have the graph made for you. Unless you choose otherwise the language model runs on your own computer: nothing you type and none of your data is sent anywhere, and it costs nothing to use.
Ask and Build
Ask answers questions about Delta and about the graph in front of you. Build makes a graph from a description, or changes the one that is open; Undo puts it back.
An ordinary laptop is enough
16 GB of memory and no graphics card will do. The model is downloaded once, about 5 GB, and after that no network is used. A small model gets simple pipelines right and makes mistakes on long ones, so check a graph before you run it.
Online models, if you choose them
Claude and GPT are better at building graphs. Pick one and enter your own key, and Delta shows what is sent: your message, column names and the plan of the graph. Never your data files.
From Claude and other AI apps
Delta has an MCP server, so Claude Desktop, Claude Code and other apps that support MCP can build graphs with their own model: you ask there and the graph opens in Delta.
Accessibility
Built for the keyboard, and for the way you read.
Delta's defaults are made to WCAG 2.2 AA, and its Settings let each person change how the window looks and moves.
Graphs without a mouse
Move cards with the arrow keys, connect an output to an input with Enter, and hear what happened from your screen reader.
Text and type that suit you
Text and controls up to 200%, and the Atkinson Hyperlegible Next typeface, which keeps letters such as I, l and 1 apart.
Contrast, focus and motion
Higher contrast, a stronger focus outline, underlined links, and animations off when you or your system ask.
Licensing
One license per company, not per seat.
A license covers as many of your company's computers as you like. The graphs you make and the programs Delta builds from them are yours.