User Interface

The current user interface has been seriously overhauled in version 0.9, and we will continue working on it to make it the best possible UI experience. And we have these two "big ticket" items.

Data management

Data management is the core business of the tool; we will complete the existing tool set to achieve brilliance in this area, as time and budget permit.


The current postgres-based storage system works very well and provides a fast transactional storage with good density. It will in remain the default for a long time. However we have a plan for something with better performance (less storage space, lower latency) and scalability (big-data scalability).

Version 0.9 (shipped 2024-06-15)

Developped between September 2023 and June 2024, this release contains an important number of features and significant fixes.

Whole new packages will also be provided with the "Pro" (commercial customers only) version: a tool named "tswatch" to monitor the time series status (we often want to know if series are in a resonnable state of freshness) and a separate complete Dashboarding tool with a low-code/slick UI to create the graphs.

Experimental support for identification using OAuth2 + OpenIdConnect has been added and covers the API and UI. This is also part of the Commercial offering.

API points

User Interface changes

This release is chock full of changes. First of all, the landing (home) page has been completely redesigned and now provides much more clarity and crisp information about where to go from there. We have added a general menu on the left, which eases navigation amongst views a lot. Lastly, a number of views have been enhanced, re-designed and sometimes completely rewritten, with drastic effects on performance and interaction quality.

New Tools

The commercial offering now comes with two new powerful packages: a time series "monitoring" tool and a full-fledged Dashboard system. While only distributed to commercial subsribers, these packages are Open Source.

Technical Changes

Version 0.8 (shipped 2023-09-07)

Developped between January and August 2023, this release contains a number of new powerful features and also some internal changes. In this report we will separate these clearly.

API points

User Interface changes

Technical changes

The prehistory

Genesis of a time series cache

The TimeSeries Refinery started in 2017 as an experiment to plug a logistical hole between on one hand, a "big data" enterprise time series silo, and on the other hand people doing analysis using Excel (and sometimes Python).

The Excel sheet could receive data from the big data silo but there were a number of downsides working like this:

So we designed a "simple cache" for the silo's versioned time series and another Excel client to talk to our cache. The benefits became quickly clear:

Adding computations

That successful initial success opened the road to the next step: moving computations out of Excel. After months of observing analysts' workflows with Excel it became clear a notion of computed series had to be added to the "cache". When that was done, around 2019, using an elegant domain specific language for time series under a clean and simple API, the Time Series Refinery was truly born.

We chose the simplest syntax available for the formula language: Lisp. This was immediately picked up by analysts (it is after all simpler than the Excel formula language or the ubiquitous VBA) and they started to build very sophisticated formulas made of formulas ... down to stored series of course. We added features to track formula dependencies and show and edit formulas from within the browser. A low-code platform was born.

Lastly, we also coupled to the timeseries (stored and computed) system with a task manager fit for the purpose of managing scraping and models tasks. Simple and lightweight, it provides the maintainers of the Information System a great deal of overview of the health of the system and again, a lot of autonomy.

Towards a Universal Time Series Information System

At that point in time though, people from other commodities or topical activities (e.g. hydrology, meteorology) had started to use it by setting up their own "cache". Quickly enough, it was understood that some "caches" would be interested in the data of another (the meteo time series are typically a cross-interest item as they are used as inputs in a variety of forecast models), and that duplicating data would be a bad idea. We came up with a straightforward implementation of the "data mesh" concept and soon we had a web of connected refineries instances. It turned out it would then be possible to aggregate many of them into some kind of "data hub" for further downstream usage. This is the basis of the current EnergyScan commercial offering. While doing so we also made a number of things better:

It is on top of these robust foundations and years of hard work on the ground that we are confidently bringing the TimeSeries Refinery to the commercial open source sector. Its pupose is to reduce the often tiresome Analysts / IT back and forth communications by giving the maximum amount possible of autonomy to the former, while discharging the later from many chores. We hope it will be a resounding success !