ProVega
Add a provega block to a Vega-Lite specification and the chart
stops waiting for all the data: it draws what has arrived, keeps drawing as more
comes in, and lets the viewer pause, step back, or watch how far along it is.
Data chunking, process chunking, or both, and a backend that follows the spec.
ProVega: A Grammar to Ease the Prototyping, Creation, and Reproducibility of Progressive Data Analysis and Visualization Solutions
Modern data analysis requires speed for massive datasets. Progressive Data Analysis and Visualization (PDAV) emerged as a discipline to address this problem, providing fast response times while maintaining interactivity with controlled accuracy. Yet it remains difficult to implement and reproduce. To lower this barrier, we present ProVega, a Vega-Lite-based grammar that simplifies PDAV instrumentation for both simple visualizations and complex visual environments. Alongside it, we introduce Pro-Ex, an editor designed to streamline the creation and analysis of progressive solutions. We validated ProVega by reimplementing 11 exemplars from the literature, verified for fidelity by 39 users, and demonstrating its support for various progressive methods, including data-chunking, process-chunking, and mixed-chunking. An expert user study confirmed the efficacy of ProVega and the Pro-Ex environment in real-world tasks.
Want to go deeper?
Start here
Four short specs, each opening straight in the editor. Change a value, press Run Visualization, watch what moves.
A backend that follows the spec
The data stays on the server, where ProgressiVis loads it: 60,000
Uber pickups in New York. When the chart connects, it sends the backend its
chunking.reading, and the backend sends exactly that: chunk_size
rows every frequency milliseconds. Change them here and run again; the backend's
code is never touched, and however fast it can produce, Vega only gets what the spec
asked for.
{
"data": {"name": "pickups", "url": "sio:/progressivis"},
"provega": {
"progression": {
"chunking": {
"type": "data",
"reading": {"chunk_size": 1500, "frequency": 250}
},
"monitoring": {"progressbar": true}
}
},
"width": 360,
"height": 220,
"autosize": {"type": "pad", "resize": true},
"transform": [
{"calculate": "hours(toDate(datum.Date_Time))", "as": "hour"},
{
"calculate": "['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'][day(toDate(datum.Date_Time))]",
"as": "weekday"
}
],
"mark": {"type": "rect", "cornerRadius": 2},
"encoding": {
"x": {
"field": "hour",
"type": "ordinal",
"title": "Hour of day",
"axis": {"labelAngle": 0}
},
"y": {
"field": "weekday",
"type": "ordinal",
"title": null,
"sort": ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
},
"color": {
"aggregate": "count",
"type": "quantitative",
"title": "Pickups",
"scale": {"scheme": "purples"}
}
},
"config": {"view": {"stroke": null}}
}
One Python file, the same backend this page uses (provega_backend.py, requirements.txt, sample data):
pip install -r requirements.txt
python provega_backend.py my-data.csv # listens on ws://localhost:8765then point a spec at it with "url": "ws://localhost:8765".
Press Run it: the backend streams the pickups, and each chunk deepens the week's pattern.
- Nothing yet.
The shape of a spec
Ordinary Vega-Lite, plus one block. This one reads 25 rows every 400 ms and gives the viewer the full set of controls.
{
"data": { "url": "cars.json" },
"mark": "point",
"encoding": { ... },
"provega": {
"progression": {
"chunking": {
"type": "data",
"reading": { "chunk_size": 25, "frequency": 400 }
},
"control": { "mode": "exploration" },
"monitoring": { "progressbar": true, "aliveness": true }
}
}
}The editor completes these keys as you type and flags the ones that do not exist, which matters because a misspelt key is otherwise silently ignored.
What can progress
Every example in the gallery demonstrates one of these. Filter by them over in the browser.
Community gallery
Progressive charts made by people who use ProVega. Build yours in the editor and press Submit to gallery: it appears here once the ProVega team has reviewed it.
Nothing here yet. The first one could be yours.
Help us extend ProVega
The grammar is open, and so is the platform. A chunking strategy,
an indicator, a control the literature uses and provega cannot express
yet: fork the repository, add it, and send a pull request. Bugs and examples that
break the grammar are just as welcome.
