Are maturing technologies in Big Data offering real competitive advantage to businesses by informing senior leaders with accurate facts in the strategic decision making process?
I remember during a key management meeting asking a key question of my boss behind closed doors and feeling that awkward silence as he looked at me with a “are you for real?” look! I kept my poker face and thought, “either he is really stupid or I am, we both can’t be right!” Anyhow, data driven logic lost that day and despite my protestations, we went with his way. I redid the functional task some time later when he had the bright idea to do the task (my way) again to address the same issue! Fun times indeed!! As I sighed and moved on, my reflection on the experience left me with a lasting question! Are data driven decisions really enabling a competitive edge in the boardroom or is it just a new toy for us technologists and data nerds?
Apparently, I am not alone in my wonder as there is a growing body of evidence that data driven decisions are a real competitive edge in an ever faster changing marketplace. Too often, strategic decisions are sabotaged by intuitive bias and/or risk paralysis (i.e. not enough information ergo no decision made). This in turn offers a competitive edge to the competitor who has embraced the power of big data and its maturing technology! McKinsey & Co offered survey results that points out certain traits of management decision making where key areas are effectively compromised by the management process of decision making in all but a few management cultures who are extremely well led but somewhat risk adverse.
The survey also points out key areas suggesting that even visionary leaders who have a transformative and compelling talent for story telling can often sabotage their own ability to drive clear minded strategic decisions leading to a loss of competitive edge over time.
Historically, the open source community has driven big data’s development with a huge amount of energy, money and time spent by companies and enthusiasts alike into mainly Apache projects like Hadoop, Spark, Flink plus more. The technical aspects of big data from ingestion to cleansing to transformation to analysis are too lengthy to discuss here but needless to say, the improvements in technologies that make analysis possible over petabytes of data is a remarkable achievement, which underpins the rapidly growing industry.
PwC shared a 2014 Survey Result, which effectively concurs with the McKinsey & Co research indicating a high level of intuitive bias in management decision-making at 30%. This figure has dropped considerably in my view over the years as the march of big data continues onwards. I remember reading a McKinsey & Co article over a decade ago putting intuitive bias at 80% for strategic business decisions.
So, do senior leaders who cling to a past era of “gut calls” do so at the expense of a sustainable future for their company? I would submit that “gut calls” on good data is a good idea but “gut calls” on nothing but intuition is a depreciating strategy. After all, when near real time analytics are possible with a well-constructed data pipeline, competitive edge is best delivered in my view with the following questions answered by the senior team such as:
Do we know what our strategy is for the next 5 years plus?
Do we know the marketplace environment from a risk and business perspective?
Do we know the environmental threats and opportunities?
Do we know our competitors and how they are getting ahead in business, do they use big data?
Do we have a plan for developing our company and what factors/elements underpin that plan (a product release, new product research plans, marketplace sentiment analysis, etc)?
Do we have a risk assessment that highlights competitive, regulatory, legal, process and/or financial risk factors affecting our current plans?
Can we formulate questions that will allow us to reach and/or adjust our strategic goals over a 5 plus year period?
How current do we need our data to be in our analytics and reporting suite in order to meet our goals?
Do we need data driven analysis (looking backwards) and maybe predictive analytics (looking forwards) to fulfil our requirements?
If the senior team can thoughtfully explore and answer these questions, then the technology team can fulfil the data, information and knowledge requirements once the timelines for deliverables are realistically explored and agreed upon by all concerned in a structured manner. If that synergy is achieved, then anything is possible for the business who translates embracing big data into embracing the ever faster pace of change in today’s world as a competitive edge rather then the threat it once was. That is a great place to be in today and even more importantly, it’s a great start for tomorrow.