Wednesday, April 20, 2011

BI and Quantum Physics? What is the relation?

I got a lot of interesting comments on the Quantum Physics Blog and how it could possibly relate to Business Intelligence. So here comes a little explanation.

When we design and construct BI systems we often expose facts and information which was previously "sensed, or intuitively" obtained. In other words, there was a feeling something were not accurate or facts were presented with emotional and political filters to prevent or avert a conflict or reality check.  As BI engineers we look at the World through a methodical and data driven approach. We go to the core of the facts and validate or question every assumption and data point until the absolute lowest possible denominator is found. Sometimes, this causes a huge impact on the fundamental nature of the systems and data we scrutinize.

Additionally, in Business Intelligence as in quantum physics, the fact something starts being observed modifies it.  In retail scenario, when associates know their performance is being monitored or sales are being presented in some kind of performance evaluation dashboard their behaviors change. This is called the observer effect.

A commonly debated use of the term "Observer effect" refers to quantum mechanics, where, if the outcome of an event has not been observed, it exists in a state of 'superposition', which is akin to being in all possible states at once. In the famous thought experiment known as Schrödinger's cat the cat is supposedly neither alive nor dead until observed.

When quantum physicists exercise resolving Schrödinger's seeming paradox, they understand and take into consideration the acts of 'observation' and 'measurement' must also be defined in quantum terms before the question they are asking makes sense. From this point of view, there is no 'observer effect', only one vastly entangled quantum system. This is akin to a good Business Intelligence Cube Designer who must understand all the facts, dimensions and interactions before they design their OLAP or MOLAP solution.

It is important to note the Heisenberg uncertainty principle is confused with the "observer effect". The uncertainty principle actually describes how precisely we may measure the position and momentum of a particle at the same time - if we increase the precision in measuring one quantity; we are forced to lose precision in measuring the other. This applies well to data and information.

Thus, the uncertainty principle deals with measurement, and not observation. The idea that the Uncertainty Principle is caused by disturbance (and hence by observation) is not considered to be valid concept in Quantum Physics. However, if in Business Intelligence the fact you observe something changes its behavior, in a constructive and positively measureable way defines success of the observation metrics and justifies the merits of BI.

PS: In a way, the inclusion of a seemingly non pertinent blog was my own observational method to see whether people are reading the blogs and the people who questioned me about it shows me they are...