Solving the Big Data Talent Shortage

You don’t have to circulate very far in a room full of data analytics executives to understand that one of their top concerns is where to find the talent necessary to staff their constantly evolving ventures.

The statistics alone are enough to induce a personnel headache: the McKinsey Global Institute has projected that by 2018, the United States alone could face a shortage of as many as 190,000 people with deep analytical skills. This study also found a looming need for over 1.5 million managers and analysts who understand big data and how to apply it to business operations.

It’s no simpler on the job-hunter’s end. Take these “5 Facts from the Field:”

1. Data analytics team members are mostly hired from the inside, only 20 percent from the outside according to leaders at EMC, the storage computer maker.
2. Forget merely studying software frameworks like Hadoop – Big Data hiring managers are looking for masters.
3. While you’re at it, you’d better be a master communicator, too.
4. And don’t simply rely on your academic chops, some hiring managers look to musicians, biologists and other experts to bring diverse experience to the science of data analysis
5. Do you speak marketing? It better be your second language.

So we know what the problem is. The question is, what’s to be done about it?

As Cardinal Path has discussed in a previous blog post, “Calling All Data Scientists Please Come To Work For Us,” several big data companies – most notably IBM – are taking the lead in developing the next generation of data scientists, data analysts and data marketers.

This is a great start, but the key to scaling the availability of talented data professionals will be finding creative ways to feed the full length of the talent pipeline.

One way will be for more industry leaders to partner with higher education initiatives designed to create these specially trained professionals. Up until very recently you couldn’t actually go and get a formal education in this field. Today, a couple of universities are putting together task forces to develop curriculum to address topics in the analytics space. But even still there are very, very few institutions specifically training students for this.

One of the issues is that it’s very difficult to scale up a field of study that inevitably requires specialization in various different disciplines.

Indeed, as Big Data methodology is embraced across varying industries and organizations, specializations now reflect increasing interest in applying data science to not only marketing but operations, human resources and finance.

Unfortunately, there is really no academic discipline that has been looking at this issue of applying Big Data holistically, it’s been a more exact approach and so even though there’s some really strong training, it doesn’t do as much to make up for the skills gap.

Technological innovation is exploding and there are so many new tools but, ultimately, people must be able to understand how various things work together and complement one another and can be channeled more effectively in order to bring the full value of data analysis to bear on any undertaking.

We’re at the very beginning of companies, universities and regions coming together to explore Big Data scholarship as a method to accelerate the filling of the pipeline while creating regional economic impact and the development and maturation of such incubators can’t come quickly enough.

“There is no question that this issue is on the radar of both mid-size to large companies that are looking to compete on their analytics and the schools that serve their region,” said Dave Dimas, the director of the University of California Irvine Extension Campus’ engineering, sciences and technology programs. Dimas, whose programs focus mainly on continuing education for technology professionals told me that the needs are expanding in almost every industry in every part of the country.

“But there’s no undergraduate degree in Big Data,” Dimas said, “And there’s a misconception that you need to hire someone with a PhD in math to staff these positions. The reality is that the professionals who are most successful in data analytics are experts in their industry or discipline and can, with specialized training that certification programs or other high-level individual professional development courses in business intelligence, data warehousing or predictive analytics, make a big impact on their organizations even if they don’t have an undergraduate degree in computer science.”

Companies need to invest in training internally and externally, and to push for our professional and trade associations to partner with colleges to provide training and make sure the programs are accessible and affordable.

Even more so than simply talking about how important these skill sets are to our organizations, we need to educate ourselves about how we can help employees gain this knowledge and extend it to potential employees. There is a tremendous amount of opportunity in this field of study and work, and now is the time to lay the foundation to capitalize on the economic potential for data-driven business.