Business Intelligence must be Automated

Sunday, December 06, 2009

Business intelligence is now taking off with companies like Endeca and NetSuite fighting for market share along with more established companies such as IBM and Microsoft. The idea of real time integration of all types of business data so that one can make more informed predictions and decisions is simple and obvious. Yet pulling this off in a business environment, with enough data format acronyms to make a reverse engineer cringe, is complex.

The November 26th FT article The final frontier of business advantage shares experiences indicating that it is "the automatic roll-up of data to a central repository" where most missteps in implementing a business intelligence plan are made. As a result, many business intelligence initiatives turn into costly time sinks. One company mentioned in the article says that their "answer for its customers is to convert all the data to one consistent type and store it in one repository." The problem we see with this approach is efficiency, and transparency.

For this approach to work the data must be converted, and they probably mean ALL the data. Then, I assume, all new data must only be collected in this new format. This process is extremely likely to be costly and onerous but, suppose it's not. We've converted decades of business data to a new format and are placing all our new data in this format. But, what if we merge with another company, how do we integrate their data? The same time consuming way? What if the format's insufficient? Normally any format in use becomes insufficient and assuming the opposite is a dangerous path to take.

Helioid's business intelligence tools offer a different solution. We build a proxy between proprietary internal company data and our tools, we integrate data in a transparent manner. That is, a company can keep its data in whatever format it likes and the data will be seamlessly integrated into our products and services. Furthermore, we use common open source formats so that other companies or organizations can integrate your data with their services and formats.

Another interesting quote in the article comes from James McGeever, "I believe that if the same piece of data exists in two places then one will be wrong." Hmm... what our algorithms do is evaluate both pieces of data and based on how they fit into the larger structure of an organization's information architecture they are assigned different probabilities of being correct (or wrong, whatever measure is most useful in the situation at hand). More importantly, it is assumed that both pieces of data, being information, will be useful in making the company more efficient and more intelligently run. Perhaps the duplication shows what was in the past an inefficient collection process, converting the data into one format will ignore this former problem, a problem that could be investigated and learned from.

Improved business intelligence offers the opportunity to improve concrete results, like sales or revenue, however, the path to these improvements must enable companies to be self reflective with respect to their own business processes. A new format and a dashboard that lets you see trends in sales data are the step being taken in current business intelligence products. This will allow for better decision making and will result in concrete results. The algorithms we develop not only improve decisions by making data available but, also address how to improve the use of this data and the dashboard. They address continually improving companies' work flow.