Planning for the Shortage of Big Data Professionals

DSS providers will need to bridge the gap between trained analysts and the demand for them

According to a recent report by The McKinsey Global Institute, “The United States alone faces a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of big data.”

To me, the study highlights two institutions that need to address the anticipated shortfall.

The first institution is higher education.  A few years ago, the only people who needed to learn how to analyze data were those number crunchers who had planned on spending their careers in the dark recesses of their offices punching away at spreadsheets. Now, Big Data technology is infiltrating courses of study that were once considered a safe haven from math by liberal arts majors. Clearly, Universities need to scramble to integrate more classes dealing with decision sciences and analytics into their existing curricula. But, because of the rigid structure of most universities, there will be some lag time before they are able to catch up to the realities of the big data job market.

The second institution is that of decision support system (DSS) providers and BI Providers.  The creation of tools and systems that allow organizations to reach their potential despite a lack of trained analytical experts on staff has to be a top priority.  Over the years, we’ve seen the evolution of these tools – from static SQL based reports to interactive web-based development environments – has led to increased agility and self-service for non-technical end-users.

Simple tools that allow for analysis, exploration, and monitoring of real-time data are instrumental in communicating a narrative that can be aligned with enterprise objectives. They are especially invaluable to professionals who possess common sense but lack proper analytic training, as well as the organizations they work for. So, while it might be too late for higher education to prevent the shortfall of analytical experts in the labor market, it is clear that DSS providers will play a crucial role in helping to bridge the gap between trained analysts and the demand for them.