Data Cube Example
Previously we introduced data cubes, also known as OLAP cubes. Though data cubes have largely been replaced by data warehouses and columnar databases, one case where we’ve seen data cubes, so our data cube example in this article, involves software which builds the cubes dynamically.
The application
The whole purpose of this application is analysis. Using its sourced data, it supports drill down, slicing and dicing and pivoting.
For this case, the data that the application consumes comes from many different sources. There is no central data warehouse, so the application makes service calls to different systems to get the required data. The data is stored as a list of detail objects – the objects that are stored are the most fine-grained element.
The data is then used to create a data cube, whose structure is determined by the way the user has configured the UI. If the user changes the UI, for example, drills down into a category or adds a new pivot column, the data cube is recalculated based on the detail objects.
The data cube itself is maintained completely in memory on the application server, which means that clients have a server session that is sticky – the client needs to always call to the same server, where its session is maintained.
The diagram below shows the application architecture:
If this application were being built from scratch, a data warehouse or columnar database could be populated with detail records, when the client application starts up, instead of building a data cube.
Once the data warehouse was populated with the detail records, as the user interacted with the UI, the application server could perform queries against the database to provide the aggregated data based on the detail records. By using a data warehouse to save the detail data pulled from the source systems, clients would no longer require a sticky server session because the analysis functionality would come from the data warehouse instead of a custom data cube, which was saved in server memory.
The diagram below shows the new proposed application architecture:
Up next
Columnar databases.