Signs You’re Ready for Automated Intelligence
July 9, 2020
Three Things to Look For in Your Data and Reports
At Nousot, one way that we describe automated intelligence is a state where your business automatically trains and refreshes predictive models, automatically generates reports, and automatically creates visualizations.
It sounds great, because it is. Automated intelligence means continually updated, predictive answers with much less manual effort.
But artificial intelligence has made many promises, so we’re here to keep it real:
- Automated intelligence is not magic. It’s not a wand you can wave, but a customized system of inputs, processes, and technologies you can build. You’re still in charge of all the components in the system (because humans still rule the day), but you can offload work to this system and gain speed to insights.
- You can’t automate everything at once (just ask Elon). In fact, automated intelligence delivers the most value when you apply it incrementally, to one use case at a time.
- There are prerequisites.
The prerequisites are what we want to focus on today, and specifically those regarding data and reporting in your business. So let’s dive in.
Following are three signs in your current data and reports that you’re ready to get started with automated intelligence – and that you’ll have clear, actionable results:
1. Homogenous reports are common in your organization.
If you’re like most businesses, you rely on specific data that you capture weekly, monthly, quarterly, and yearly in order to analyze KPIs and make decisions.
The following may also describe your business:
- You dedicate resources to compiling the data and building the reports, which require multiple steps and take hours, days, even significant chunks of each month to complete.
- You spend time and effort identifying trends, patterns, and their drivers within and across the reports, and making projections to ready your business for whatever is coming your way.
- You routinely create presentations or executive dashboards to articulate your analyses for decision makers.
If you and your team find yourselves doing this kind of work over and over – wrangling, interpreting, and delivering data, and never quite fast enough – there’s good news. The repetitive steps you’ve been taking to produce regularly updated answers to a stable set of questions is ideally suited to two things:
- An automated data ingestion pipeline that updates the data for you.
- Automated models that generate and maintain results for you.
This isn’t about automating tasks “away” from your team and underutilizing them. Quite the opposite: it’s about giving them the time to actually apply the insight they’ve been working so hard to discern. It’s about putting their talent to the higher-order tasks of informing all your initiatives to streamline processes, serve customers, and compete.
You and your team don’t have to be in the business of data, unless you want to be. You can be in the business of your business.
2. You’ve governed at least part of your data (or you’re willing to start).
No technology can learn or automate anything it hasn’t been taught. And often, data is the primary teacher.
For this reason, you want to build automation with the training data that you’ve governed. And the governance is on you, not on a machine. That’s a good thing, because you want to rigorously define all your business entities like customers, households, and employees. You also want to maintain control and oversight of your governed data with as diverse a team of data owners and stewards as possible.
Data governance is important because data quality is important. In the nearly 70 years since the phrase was coined, we still haven’t found a way around “garbage in, garbage out.” So automated intelligence will deliver value only where the data pipeline contains elements with well-documented and well-understood definitions, lineage, and stewardship. Otherwise you can’t explain model results, diagnose anomalies, fix breaks, or achieve compliance without intensive model forensics.
If you work in an industry that’s highly regulated, you likely have governance in place that will readily support efficient, transparent, and successful automated intelligence (which is sort of ironic: regulated industries are understandably cautious of automation, yet well positioned for it from a governance standpoint).
If you’ve completed little to no data governance, it’s not an automation showstopper. This is where the incremental value of automated intelligence becomes apparent: you can start small with both governance and automated intelligence in tandem and see fast results, rather than keep your automation efforts on ice until you audit, verify, define, organize, and protect data.
A great proof-of-concept for automated intelligence if you haven’t governed your data – or even if you have – is a use case where you can identify five or so data elements that would add value to your predictions. Typical examples of such use cases are determining the probability for audience members to convert via a particular marketing campaign, and predicting employee or customer attrition.
Once you decide on your use case, you can identify and configure the fields that correspond to your five (or so) data elements, create a dictionary for them in a Word document, put a monitoring system for them in place (more on that in a minute), and assign data owners for each data element – and you’re on your way with governance (as well as with automated intelligence).
3. You have (or are ready to put) a monitoring system in place.
An automated model can be like a drone that people want to shoot out of the sky. On its face it engenders suspicion, mistrust, and mythical characterizations of a cold observer or ruthless enforcer.
So when an automated model makes an error or underperforms, those with misgivings can take that opportunity to sound the alarm, declare the effort too risky, and shut it down. A recent example of this is Google’s diabetic retinopathy detection model, a black-box AI that uses eye scans to identify the condition at greater than 90% accuracy in less than 10 minutes.
In Thailand where the Google model was deployed, it worked well – except when it didn’t. It rejected more than a fifth of scans outright, deeming them low-quality, and poor internet connections often delayed scan uploads. Patients and providers lost time, grew frustrated, and stopped using the model. Google is now working to tweak it and its accompanying workflows.
So the best defense of an automated model is a good offense: a monitoring system. It’s your own proactive alarm system that runs alongside the model. It is the real cold observer and ruthless enforcer, only it’s on everybody’s side, against the model.
Imagine if the Google retinopathy model were accompanied by real-time reports that monitored the eye scan rejection rate. Even simple line charts would enable modelers to recognize an issue with the image quality threshold and address it. Instead, a model that helps patients get a diagnosis in 10 minutes, in a country where they often have to wait 10 weeks, is currently offline.
The same energy you apply to automated intelligence, then, must also be applied to monitoring. It means extra reports, but they will be among your most valuable ones. A monitoring system helps provide the transparency and safeguards that make automated intelligence successful:
- It protects the model from garbage-in, garbage-out, so it reinforces your data governance.
- It sees anomalies in data, and resulting spikes in model results, as they’re happening, so you can fix breaks before they become misguided decisions, costly misses, or a shelved model altogether.
- It detects when source data is changing so that you can recognize and investigate whether source systems are changing, or whether external forces are impacting model results for real or spurious reasons, or whether your internal data pipeline needs your attention.
At Nousot, we know what it’s like to be neck-deep in data. We enjoy it; it’s what we do. But what you do should be the business you run. So, homogenous reports, governance (even a little goes a long way), and monitoring: if your data is accompanied by these three things, it’s primed for automated intelligence, and you’ll start feeling the data flood recede.
This is Part 1 of our series Signs You’re Ready for Automated Intelligence. Look for Part 2, “What to Look For in Your Organization and Teams,” coming soon.