In a presentation given at Altair’s Future Industry event podcast, Ingo Mierswa, founder of RapidMiner, now part of Altair, gave a ‘requiem for a data scientist’. Here are some excerpts.
It’s 2022. We should have autonomous robot workers and flying cars, the reality is that you have invested in AI & ML and all you’ve got is a lame pie chart with moldy data. Organizations have hired a bunch of nerds who can out-math Archimedes, but they get sweaty palms when it’s time to solve real business problems. Others have advanced data scientists who can do amazing things with deep fakes but end up creating million-dollar solutions to ten cent problems. The AI honeymoon is over—and it SHOULD BE. You may be a data scientist, chief data officer or whatever, you need to be strong right now because it’s time to say goodbye to the profession as we know it. We were told, hire more data scientists, train them, empower citizen data scientists. Now while some use ML/AI, many are waiting to reap the rewards. Mierswa finds this ‘troubling’. After our last ML model has been deployed, how should we be remembered? As people who had true impact on the organization or as a bunch of eggheads toying with the latest algorithms for our own amusement.
86% of C-level respondents to a PwC survey thought AI mainstream. So the problem is not lack of investment, everyone is investing in AI. But 80% of these initiatives are not in production and have had no impact. The usual riposte to this is bring out more data scientists. ‘I do not believe this for a second!’ The problem is that there are two types of data scientist, overwhelmed and overwhelming. The former, fresh from college have never talked to a stakeholder and get passed around from department to department. They have no deep knowledge of the problems they are supposed to solve. The other type is a bunch of truly brilliant people who won’t do the simple stuff like analyzing the problem. So you can chose between a 10 cent solution for a million dollar problem or a million dollar solution for a 10 cent problem, like AlphaGo, an ‘overwhelming’ data science solution to a problem that nobody cares about.
The root case of data science’s failure is not a skills gap but rather the huge wall between business stakeholders and data scientists. Such a separation also exists in coding and is solved by scrum teams that collaborate. This is how very complicated things get delivered but it is not how most data science is organized. We need to embed data scientists, citizens or others, in the business. Upskilling is better than outsourcing. And of course, you need the right tools which is ‘why I am pleased that we are now part of Altair’.
More on RapidMiner in oil and gas.
Comment: It seems to us that Mierswa does a better job of explaining the failure of data science than he does recommending a different approach. Most large companies we have looked at to date already deploy embedded/collaborative teams. Perhaps the AI problem is more fundamental than Mierswa suggests.
© Oil IT Journal - all rights reserved.