When a business collects figures on transactions, practices, and profits, it is amassing a store of information that may not, on the surface, seem to give much immediate insight. With modern analytics technology, however, the power to analyze this data can give you the tools to make huge improvements.
The Advantage of Data-Driven Decision Making
Being able to interpret massive sets of data generated by your company, and combining with information pertaining to the market, allows you to make informed decisions that deliver both operational optimizations, and the confidence and knowledge to make bolder bets on future activities.
Even though data may seem little more than a cluster of numbers, the proper synthesis of information and interpretation of patterns can be astonishingly powerful when applied as creative solutions to your most challenging problems.
For example, high-level data analysis can minimize unnecessary energy consumption in an office building or a manufacturing plant, automate traffic flow for efficiency to reduce journey times, pollution, and quality of life for millions of people. Interpretation of patterns within data sets can also wield great power, such as predictions of machine breakdowns – for jet engines, oil pumps, or even your dishwasher, and also in forecasting stock markets, crowd behavior, or customer churn.
Who does the work?
Data scientists or analysts are usually the people doing the actual interpretation, efficiently translating data into some sort of actionable insight for decision-makers at your company. They analyze your company’s data to generate useful reports or visualizations that are approachable even for those who may not have the technical knowledge to fully understand the mathematical complexities, e.g. Executives ;)
Skilled data scientists have the required high-level understanding of statistics and mathematics, as well as proficiency in computer science, so as to be able to correctly process and extract meaning from the computerized data. Ideally, they will also be versed in the domain to which the data pertains so they can gain a full understanding of the intricacies and subtleties of the problem.
But … getting the “right” data can be a nightmare.
For any of this to be possible, these analysts must be given comprehensive data sets that contain sufficient information to ultimately yield meaningful insight. This is the responsibility of your company to some extent; good and thorough data collection must be done at frequent, regular intervals to ensure that the info is complete and exhaustive, giving an accurate, full picture of business practices and the resultant implications of those practices. Data should also be collected on as many different variables as possible to ensure that a “full picture” is attained. Combining and blending this data can be a highly complex task in its own right, often requiring specialized data experts and tools, in addition to the data scientists.
The complexities of deriving actionable insights from data
Most analytics programs require a level of coding knowledge in order to specify the specific tests and tasks that are to be carried out, which include the fitting of data to a model, the ability to predict future values given past trends, tests for significance of particular variables in the model, the analysis of data clusters, and more. Because they are user-specified, programs such as these can also be used to generate meaningful graphics and reports for the company in question. The goal is to process and interpret data sets and generate meaningful models and visualizations of the processed data. For purposes of user-guided analysis, programs can be used to read organized data sets and perform specific forms of statistical analysis on the variables. To take an oil example, firms in the extraction industry use software that can predict the breakdown of an oil pump before it happens and can furthermore “learn” this pattern as it operates to continually better its own predictive ability.
In this process, it is essential that the data analysis ultimately yields results that are personally meaningful to the business in question. As such, it is imperative that the end result of the analysis be given in an accessible form. Thus, the model created should be informative without being esoteric. It should give sufficient enough information that the administrators making decisions can glean the same insights as the data analysts and at the same time not be so simple that important information is lost for the sake of accessibility. This ultimately proves to be a challenge unto itself, as all the analysis in the world is useless unless it can be explained to the person making the ultimate decisions. So in the end, in order to yield actionable insights, the model itself must be distilled down into a presentable format that can be read and understood at a reasonably non-technical level.
How fast can you really make a “data-driven” decision?
The timeline for all this varies, of course, depending on the size and ambition of the project. A first challenge is to get the analysts and business executives on the same page with regards to the goals and the task. After all, miscommunication of expectations can be an expensive failure if the data model is useless to the company. After this, data must be collected and passed into the hands of the analysts, who must organize it and run critical evaluations, create a good model, and in the case of “smart” technology apply an appropriate algorithm such that the model learns and develops itself. Legal procedures may also be required if an outside analytics company is to be involved. While it should not take too long to generate a model to present to the executives, it’s also important not to overlook potentially important information. A data scientist will be expected to perform their task in days or weeks rather than months, since important business decisions are rarely allowed to wait for that long. Finally, it’s on the shoulders of the business to integrate the results of the report into their business model, and this timeline can vary greatly. Major changes can take years to integrate into the day-to-day operations of a company, whereas minor alterations can be made as policy updates almost instantaneously.
For a company, the ultimate goal of all this effort is, of course, to gain a competitive advantage over less-informed companies, which translates into additional profit, market share, or achievement of strategic goals. By the same logic, it is also in the best interest of a data analytics firm to streamline its own process, creating software that optimizes functions pertaining to report generation, the finding of appropriate models and methods, and the tailoring of the end report such that it is useful to the business. Faster time to insight is a key goal for both the data science team, and the business.
Automating data science gives super powers to business users.
In theory, we could teach a computer by intelligent “learning” methods what the best features of data science should look like. In the long run, this would lower the computer’s dependency on a human operator for analysis and ultimately quicken the speed and efficiency at which this analysis can be performed. This translates into increased profitability for a data analytics company, or department within a large organization. The amount of time saved by this smart process will ideally counterbalance the amount of time spent developing the intelligent algorithm, ultimately creating a business that is vastly more profitable in the long term. However, most data analytics firms and departments are so overwhelmed with immediate requests for specific project support that they don’t have the time to even consider building such a sophisticated suite of algorithms. Yet if such a program existed and could operate effectively, learning quickly and generating appropriate results completely on its own, then one could imagine that the resulting benefits would be a massive source of profit for the analytics company or department.
Guess what! I found a tool like this…
It is easy to imagine the ways in which fully-automated data science could transform the way that businesses analyze their data, and that’s the kind of super powers that I discovered UBIX is putting in the hands of business users. I spoke to Mark McNally, CEO of UBIX, who use the tagline Powering the Insight Economy.
“UBIX’s Auto-Curious engine can take on the whole process of automating data science, although you still need those moments of human brilliance to nudge it in the right direction. We accept queries using natural language, and learn how to read the data and find appropriate aspects of that data for consideration. By iterative machine-learning, the engine generates appropriate models based on detected features and past experience to produce visuals and reports that are of most value to businesses.”
I asked Mark how they put the insights into the hands of business users, whether they were linked to Siri or Cortana, like a virtual data scientist.
“We publish our results in the UBIX Advanced Insight Portal – which is really an intuitive, natural way to ask questions for a specific domain – we have portals for different subject areas. Over time, we anticipate that as the number of portals grows, and we learn about the common patterns between industries, UBIX will allow you to ask more complex questions across broader range of knowledge. If you imagine the Social Graph that allows people to look at relationships, we are building a Question Graph that lets everyone address problems traditionally tackled with manual data science. Right now, the interface is natural language – you can type in a question just like you would in a search engine, and then make selections with point and click. There’s no reason why this type of automated data science shouldn’t be plugged in beneath Cortana, Siri, Viv, or any other chatbot system.”
UBIX have only recently emerged from stealth mode, but already have some remarkable use cases where they have been working on areas as diverse as rapidly identifying cancer treatments (cutting a key workflow from 18 weeks to 2 days), and managing complex analytics on trading and shipping of commodities. Ultimately, as tools like UBIX become commonplace, they will evolve into a critical part of our technology infrastructure. The internet has sometimes been referred to as a global brain, but so far it has been largely a collection of raw data and the connections between them. We might be at the stage where neurons are beginning to fire, and that raw data is put to use, providing a leap forwards in terms of our ability to comprehend and confront some of the largest issues that face us.
[Guest blog by Matt Oberdorfer, CTO at Frost Data Capital]