BI cubes will no longer be developed
The development of internal BI cubes in ERP.net started in 2008, when the technology was the only way to process and analyze huge amounts of data. BI cubes is a great tech and allowed us to achieve UX not possible in any other way back in the day.
However, from 2012 and onward, the Business Intelligence industry started a transition away from the "BI cubes" technology. New and better ways to do business intelligence were devised, including:
- In-memory analytics
- Real-time operational analytics
Both ways to do BI are easier to implement and are in some way or another better than the old "BI cubes" technology. Based on these changes, we decided to stop further development of the internal BI cubes and focus on more contemporary technologies.
What will happen to the existing cubes?
The existing cubes will remain accessible for the foreseeable future. While we do not plan to improve them, we also do not plan to immediately take them down. We will support them for at least one more major version.
However, if we see a trend of (nearly) all customers abandoning them, replaced by new and better BI technologies, we might finally decide to stop their support altogether. BI cubes as a technology has its quirks and supporting it is becoming more and more of a burden. All while the future of the technology is very dim.
What are the new technologies?
Both new techs, described here, have one important advantage in common: Simplicity.
During the years, it turned out that finding high quality data analysts became more and more problematic. All new technologies are much closer as technology to other contemporary development stacks and hence, they are easier to be developed with existing knowledge.
In-memory analytics (IMA) is a non-real-time BI tech. While it still requires refreshing (usually during off-peak hours), it has the great advantage of UX speed. It is fast for the end-user. Really fast. Like 10's of times faster than BI cubes. Most of the time it is nearly instantaneous to the human eye.
IMA has its cons also. It requires lots of RAM and still needs refreshes. So, at the end of the day, it is still a balance.
Real-time operational analytics (RTOA) is a real-time BI tech. Its main advantage is the freshness of the data. It always contains the latest data and there is no need to refresh anything. Refreshes were commonly the biggest support burden for BI cubes; so having no refreshes might also cut heavily on the support side of the equation.
For many years, implementing fast enough RTOA was an elusive matter. It worked for small business. But once the data volume increases, it starts to become slower and not so usable. However, with the "invention" and maturing of the so called "column-store" databases, RTOA finally became a mainstream technology.
What is our strategy?
In the recent years, providers of BI client technologies, like Power BI, have developed much more robust and mature tech stacks. While we can support the internal BI tools, we decided that strategically it would be best to better support external BI tools. We intend to do the following:
- Support both IMA and RTOA workflows.
- Make all the necessary adjustments, so external BI tools fit better in our UI.
- Create and improve the API's, so external BI tools can refresh and query directly better and faster.
- Create RTOA views, which can be used as basis for external BI tools.
Implementing the strategy will take some time. This blog post serves the purpose to inform all parties to plan their roadmaps with respect to our strategy.