Data Scientist vs. Business Analyst: Confusion Reigns!
In this new era of Artificial Intelligence, I’m sure you’ve heard the term “Data Scientist” a million times (some exaggeration is allowed in a world of hype). In fact, some vendors would lead you to believe that everybody can become a Data Scientist if only they learn their tools/software! Puns aside, what about the poor “business analyst”; the financial analyst, the market research analyst, the sales analyst; are they all data scientists too? What’s the difference between a Data Scientist and a Business Analyst — or is there a difference?
At a high level Data Science over the last decade can be considered an intersection of four fields: Computer Science, AI/Machine Learning, Statistics/Math, and Data Engineering.
In the good old days, these folks were called “quants” or research analysts and they typically used statistical and mathematical methods (including optimization) to build causal, predictive, and prescriptive models. These days, data science has become synonymous with AI/ML and data scientists have become much sought after rock stars or better yet rocket scientists — magicians who create powerful black boxes for a range of applications.
A Business Analyst (BA) is a functionally oriented position (e.g. finance, procurement, logistics, etc.) and typically has business, statistics/math, and data engineering requirements. These individuals are normally not in the business of developing new models and systems, but are responsible for monitoring key performance indicators (KPIs) and insight discovery. They unfortunately are the ones that bear the burden of explaining (to their bosses) “what” is happening and “why” something is happening — e.g. sales are down -3.0% — why? — “because our competitor introduced a new brand and also reduced prices.” To a large extent business analysts are focused on diagnostic analysis (along with soft causality) though some of the more experienced individuals could venture into building statistical and ML models.
So what then is the key difference between a Data Scientist and a Business Analysts? There’s scope for confusion for sure — but aside from the simplistic assessment that unlike BAs, Data Scientists have Computer Science and strong AI/ML skills, there are a few salient differences as enumerated in the diagram below:
Business analysts tend to be functionally or departmentally focused while data scientists can work “horizontally” across an organization (though domain knowledge is a key input to building sound ML models). The business analyst uses validated and published data science models, soft information, and market knowledge to answer real-time “what” and “why” questions related to Key Performance Indicators. The BA’s job requires almost continuous analysis of available information to address the “what” and “why” questions that are increasing in frequency in this new world of proliferating dashboards and data democratization. A data science project normally take time to design, build, and validate and is a very discrete process — from a time perspective. You wouldn’t expect to build a price elasticity model, or a customer churn model, or a forecasting model every day, right? Once built, models have a shelf-life before they’re updated, though tweaking could occur more frequently.
A small digression; so, if you’re a business analyst, does your boss keep asking you “why” a certain KPI (say market share, revenue, etc.) has gone down this week or month? The sky is falling and we need to know why there’s a 0.25% drop in market share right now — ASAP. In most cases this isn’t a “Data Science” question; nobody is going to build a model right away to answer this question, it’d take too long in any case. As a BA you may know intuitively that it’s not a significant change based on historical variability, but you still have to run dozens of reports and analyses to include in a report to present to your boss. What if instead, a smart system can do all of that analysis automatically for you to review and edit/modify? A key aspect of automating business analytics is to provide the capability to answer “why” questions in real-time. This has been our focus for the past few years; to provide tools to automate the “what” and “why” analytics for business analysts whose life is getting more hectic with — more data — more KPIs — and more work to identify the drivers of KPI changes. Automating this pain point is critical as we get inundated with more and more data. There’s plenty of AI hype and new products in the Data Science world, but let’s not forget the poor business analysts!
To summarize, the functional role of a Business Analyst is different versus a Data Scientist. The BA monitors corporate defined KPIs on a near continuous basis, uses internal (data) knowledge and external knowledge to determine the drivers impacting their KPIs in their drive for “execution excellence”. In the process they use models developed by their data science colleagues and collaborate with them too — but they’re not expected to build “deep learning ML” models every day — nay that’s impossible in any case. Data scientist are the new age rock stars leveraging AI/ML for decision support. If you’re an Elon Musk follower, pretty soon we won’t need CEOs or CFOs — we’ll have virtual omnipotent computers running companies (if not the world), and we can all sit back and take the rays on the beach. May not be too far from the truth in a few centuries, if we humans manage to not kill ourselves — either way the computers win😊.
