Context
A good amount of data in organisations is maintained in tables with multiple columns. Typically you can think of a multitude of Excel or CSV tables. This data often has a dimension which describes the time, sometimes other dimensions for classifying the data, and in most cases some actual observed values or counts.
Such tables are holding the data in a structured form, but most of the time, the information to understand the columns and also the necessary metadata enabling the creation of use-full representations in charts and visualisations is missing.
With the creation of Cubes you as data provider and domain specialist of the data are able to augment and annotate your data with everything necessary to understand the input data – directly in the to be published dataset. Finally can fully annotated Cubes also be used to visualize your data with tools like https://visualize.admin.ch .
The Cube Creator allows us to transform data provided as clean CSV into a standardised RDF Cube format. At a second step in the Cube Creator – the Cube Designer – it is possible to annotate the Cube with the necessary descriptive and technical metadata. Further is it possible to map common values to known concepts (e.g. Cantons, Municipalities, Companies, Departement) which further augments the data at hand. Finally the Cube Creator allows to manage the publishing of the Cube on LINDAS which allows it to be consumed through https://visualize.admin.ch (for end-users) and queried on https://lindas.admin.ch/sparql/ (for developers).
Another way of thinking about Cubes are multi-dimensional representations of your input tables. (Also compare to https://statistics.gov.scot/help/data_cubes for a in-depth illustration of the data cube concept.)
In the image above we see a cube with 3 dimensions: Year, Location and Season. In the cube is for each of this combinations the average temperature reported.
The source of this cube might be provided though following table:
| Year | Location | Season | Average Temperature |
|---|---|---|---|
| 2019 | Bern | Summer | 22 °C |
| 2020 | Bern | Summer | 23 °C |
| 2021 | Bern | Summer | 24 °C |
| 2019 | Zürich | Summer | 21 °C |
| 2020 | Zürich | Summer | 22 °C |
| 2021 | Zürich | Summer | 23 °C |
| 2019 | Bern | Winter | 12 °C |
| 2020 | Bern | Winter | 13 °C |
| 2021 | Bern | Winter | 14 °C |
| 2019 | Zürich | Winter | 11 °C |
| 2020 | Zürich | Winter | 12 °C |
| 2021 | Zürich | Winter | 13 °C |
For every combination of this three dimensions we have a value which we call an observation. It is possible to have more than three dimensions for which every combination provides an observation. It is also possible to have multiple observations per combination of dimensions.
A cube needs to have at least two different types of dimensions. The Key Dimension ans the Measurement Dimension.
One or multiple key dimensions together uniquely define an observation. In the example above the key dimensions to identify an observation are year, location and season combined. It is also possible to have only one key dimension to identify an observation, often the point in time the observation was made.
An observation can also be a statistical aggregation. (How many people live in one place in a period of time.) Such an aggregation is often seen in the incoming data for one specific period of time. (E.g. the year 2021, or Q2). It is advised to add a key dimension representing this period even if the incoming data is only valid for one period to start with.
The measurement dimension is the actual value of the observation. This is often one, but can be multiple values for one observation. In the example above we have the Temperature. Possible would be also if measurement at the same location the Humidity.
Every measurement dimension needs to provide one unified unit of measurement. E.g °C in the example above. If your input data has multiple units in one dimension, this is a strong indicator that these are probably multiple dimensions.
Further we also see in the example that we have an Average Temperature which indicates the measurement can also already be an aggregate of some sort for the period of time.
A dimension itself can have multiple forms (we will introduce it later in details with scale of measures). It might be numerical, e.g. 22,23,24 / 2001,2002 or texts (strings), e.g. Summer, Winter. Because cubes should be build multilingual, the goal is to move from textual descriptions to concepts, which in turn can have attached multi-lingual textual descriptions.
So instead of having a text saying Summer. The cube will have concepts which has the labels Sommer, Été, Estate, Summer attached. Further can a description, other attributes or even links to other concepts be attached to such a concept. To build these concepts, we need to have translations as inputs for our cubes ready.
Finally we have concepts which are used over and over in many cubes. These concepts can be reused inside your data cubes.
- This allows you to profit from translations and descriptions.
- Further the provided data can be connected to other data (cubes) through the common use of the same concepts.
To profit from these effects, a one time mapping from your texts/strings to already defined concepts need to be provided.