Spec¶

The examples below use pump measurements to show how entities, datasets, features, mappings, and functions describe an analytical system.

Entity¶

The analytical subject

An entity is an analytical subject. Here, it is pump telemetry: measurements describing pumps over time.

P-01 identifies a real pump. Pressure and flow measurements connect its physical behavior to the analytical model.

PRESSURE6 bar

FLOW12 L/s

P-01 / 09:00

Dataset¶

The entity's observations at a defined row scope

An entity can have several datasets, each describing a different row scope. This dataset contains minute-by-minute observations of the pump entity above.

Other datasets could represent:

Dataset	What one row represents	Example features
Pump reference data	One pump	Rated flow, installation date
Minute-level telemetry	One pump at one timestamp	Pressure, flow
Daily pump summaries	One pump on one day	Maximum pressure, average flow
Maintenance events	One maintenance event for a pump	Event type, downtime

Note that this is analytical representation. Data might be stored physically in different way.

GRAINOne pump at one timestamp

FILTERINGP-01, 09:00 to 09:02

COVERAGEEvery minute in this interval

DIMENSIONS / observation keys		TELEMETRY / mapping A		LIMITS / mapping B	CALCULATED
pump	timestamp	pressure	flow	limit	ratio
P-01	09:00	6	12	8	0.75
P-01	09:01	8	14	8	1.00
P-01	09:02	4	10	8	0.50

Color key for all examples

Blue: Dimension keyspump + timestamp identify an observation, not a measured value.
Teal: Telemetry featurespressure and flow are outputs of mapping A.
Violet: Limit featureslimit is an output of mapping B and keeps its color when aligned or expanded.
Amber: Calculated featuresratio and max_pressure are produced by vectorize or collapse functions.

Colors identify column roles throughout the page, not which cells changed. Headers and labels carry the same meaning without color. Neutral cells describe arguments or operations.

The combined dataset is a conceptual view, not a requirement to store all intermediate columns in one physical table.

Functions¶

A function template defines reusable code and argument types.

In the compact tables, time abbreviates timestamp. Pressure and limits are in bar; tables without a pump column still describe P-01. Column colors follow the dataset legend above, including on both sides of each transformation.

Read¶

Bring observations in

Telemetry sourceDatabase or file

pump = P-01
09:00 through 09:02

retrieve

OUTPUT / 3 observations

pump	time	pressure
P-01	09:00	6
P-01	09:01	8
P-01	09:02	4

Boundary crossed: source to spec. The returned dataframe contains dimension keys and declared features.

Vectorize¶

Add a feature, keep the rows

BEFORE / input features

time	pressure	limit
09:00	6	8
09:01	8	8
09:02	4	8

divide

AFTER / one new feature

time	pressure	ratio
09:00	6	0.75
09:01	8	1.00
09:02	4	0.50

Exactly one output feature. No new rows. Here, ratio = pressure / limit. The existing limit column is omitted from the after-view only for space.

Conform¶

Align with the target row scope

SOURCE / wider time selection

pump	time	limit
P-01	08:59	8
P-01	09:00	8
P-01	09:01	8
P-01	09:02	8

align

TARGET / telemetry keys

pump	time	limit
P-01	09:00	8
P-01	09:01	8
P-01	09:02	8

The target scope is the contract. This example excludes an out-of-scope observation. The binding's implementation defines how alignment is performed.

Expand¶

Broaden dimensional coverage

SOURCE DATASET / pump grain

pump	limit
P-01	8

One value for the pump.

extend

TARGET FRAGMENT / pump + time

pump	time	limit
P-01	09:00	8
P-01	09:01	8
P-01	09:02	8

One observation becomes three in this example. Unlike conform's target-scope alignment, expand explicitly broadens dimensional coverage.

Collapse¶

Move to a coarser grain

collapse is the operation that groups source observations and returns a dataset at the target grain. The engine runs the grouping; collapse analytical functions calculate output features within each group.

SOURCE / pump + time grain

pump	time	pressure
P-01	09:00	6
P-01	09:01	8
P-01	09:02	4

collapse

TARGET / pump grain

pump	max_pressure
P-01	8

One row summarizes the three moments.

Each collapse function binding produces one feature per group. The engine evaluates multiple bindings together to calculate multiple summary features.

Write¶

Deliver or evaluate results

INPUT / selected features

pump	time	ratio
P-01	09:00	0.75
P-01	09:01	1.00
P-01	09:02	0.50

deliver

Your destinationPersist to a database or file.
Print a report.
Evaluate the results.

Boundary crossed: spec to use. Keep external I/O in read and write functions, not analytical transformations.

Escape¶

Run one opaque external workflow

Use escape only when an external operation does not fit the standard operations. Prefer read, vectorize, collapse, and write when their explicit dataflow and runtime checks fit the work.

ESCAPE / ONE FUNCTION External workflow Read source calculate write result

Declared features track dependencies; outputs are not dataframe columns.

Mapping¶

Connect feature dependencies and outputs to a function binding

A mapping declares which features a function needs and produces, then binds that function to concrete arguments. In this example, divide receives the current dataset and feature identifiers, and returns pressure_ratio.

INPUT DATAFRAME / declared features

time	pressure	limit
09:00	6	8
09:01	8	8
09:02	4	8

MAPPING / JSON

{
  "binding": {
    "function": "divide",
    "args": {
      "dataset": "$datasets.@.dataframe",
      "numerator": "$features.pressure.id",
      "denominator": "$features.limit.id",
      "output": "$features.pressure_ratio.id"
    }
  },
  "features": {
    "inputs": ["$features.pressure", "$features.limit"],
    "outputs": ["$features.pressure_ratio"]
  }
}

OUTPUT DATAFRAME / declared feature

time	pressure	limit	pressure_ratio
09:00	6	8	0.75
09:01	8	8	1.00
09:02	4	8	0.50

Two complementary declarations. Feature inputs and outputs describe data dependencies. Binding arguments provide the dataframe and feature identifiers the function uses; an identifier argument alone does not declare an input dependency.