What are the criteria for selecting a good AIOps solution? I have the top 5 – do you have more?
As you might have already seen, I have written several articles on AIOps, Observability, and What AIOps means for Networking. In a few follow-up advisory conversations with CxO-level executives, I’ve been asked, “What are the criteria for selecting a good AIOps solution? How do you compare and measure the solutions one against another? Especially when there are so many solutions out there all claiming to solve the problem better than the others!”
After thinking about this for a few days, I developed an initial criterion to present to them, which I thought would be helpful to share. Here are my top criteria that all buyers should keep in mind when considering an AIOps solution:
Before starting down this path, make sure that the solution you are considering solves your specific use cases seamlessly. If possible, ask them for references where they solved a specific problem similar to yours. Don’t be afraid to ask for CxO references and speak to them directly.
Criteria #1: Are they eating their own dog food?
As cheesy as this phrase can be, this is a major criterion to assess when starting the process of evaluating a tool. In fact, it’s applicable to almost any software vendor—does the provider have the confidence to rely on their own solution? In the case of AIOps, if the provider’s support teams are not using the tool for the use case that you are looking for them to solve, why should you use them? I’ve heard a variety of reasons and excuses for why not, but the reality is that when vendors use their own solution, they’re brought closer to the customer (i.e., you). Only by having to use their own product will they be able to empathize with you. On the other hand, double-clicking on the real reason why they hesitate to use their own tool for a certain use case may expose not only their product limitations but also shortcomings of organizational culture in not fully understanding the solution that they are selling to others.
Criteria #2: Is the solution truly vendor-agnostic?
Imagine after purchasing a solution you find out that it only works with a small set of products—you’re locked in. Or you’re stuck with a partial solution that’s only somewhat helpful and you are expected to buy new (or more) products with that vendor only. To avoid this situation, always ensure from the start that your AIOps solution is vendor agnostic, meaning it seamlessly integrates with most of your existing monitoring, observability, and IT automation solutions. Many vendors claim they are vendor agnostic and “work with any solution,” but gloss over the ease of integration. The reality is that integrations with new solutions can quickly snowball into a mammoth undertaking, putting you on a road requiring many human hours or a costly integration project. Not to mention that if you end up on this route, maintenance and upkeep will be extremely difficult when a version change or upgrade is necessary.
Criteria #3: How much of the solution is automated?
This item can be surprising to some as a factor to consider—don’t we expect automation in every AIOps solution? However, you would be surprised at the number of solutions I’ve seen that require login to multiple tools and systems for full visibility and insights. It’s pretty common with most solutions to have to access between three to five tools to get the root cause of an issue, when an incident happens, and another one or two systems to find the fix. Then, in most cases, you’ll need to manually use a separate automation solution to fix the issue. The ideal AIOps solution should, at the very least, be able to figure out if there is an unplanned outage, find the root cause, identify the fix, and either propose the fix or better yet remediate the fix on its own. In other words, AIOps tools should be able to automate end-to-end the process of incident identification, root cause analysis, and resolution on par with human domain experts. Otherwise, you’ll be stuck looking at yet another dashboard.
Criteria #4: Does the AIOps solution have a solid cloud foundation?
If you are looking to build a hybrid solution, or eventually move to a pure cloud-native solution, you must make sure that the AIOps solution is built for the cloud. When I say built for the cloud, dropping the software package in a container and calling it “cloud-enabled” is not enough. First, it should be built cloud-native in the specific cloud that you will be using and second, it should be able to exploit the cloud-native capabilities such as cloud elasticity, pay-as-you-go, hyper scalability, pay for only what you use, and work with most cloud-native monitoring/logging/tracing options out of the box. The solution should also support the real-time processing of data using ML data pipelines (such as with Apache Storm, Flink, Spark, etc.) and integrate with most major observability systems such as Spunk, SumoLogic, Datadog, New Relic, Dynatrace, AppDynamics, etc. Working out of the box with data lakes like Snowflake, Amazon, Azure or Teradata, can also be a bonus in the long run, although this may not be a huge limiting factor in my view. I also recommend you look into how many cloud regions the solution is capable of running. I wrote an article on this subject a few months back: if the selected solution runs only on the region where your software will be running, everything will go down and you will be shooting blind if the cloud region takes a hit. These are all important considerations to evaluate how “cloud-friendly” your solution will be for your long-term strategy.
Criteria #5: Do the organization teams align well?
This can be a touchy point for some organizations: where does the solution vendor get their feature requests, roadmap, change requests, and vision? Looking at market trends and talking to analysts or strategists is a good start, but vendors should also have a strong Customer Advisory Board (CAB) of CxO level executives who live through the problems from a customer perspective every day. In the day-to-day of product building, the data scientists who are working on the experimentation of the product should be paired with the domain experts such as customer support teams who are familiar with the daily challenges of operations. Product changes shouldn’t happen because one of your strategists or a company executive thought of an idea—instead, they should stem from real-life use cases and frustrations. How often does the CAB meet? What information do they exchange? How much of that is absorbed? Answers to these questions will also give you an indication of how your future feature or change requests of the solution will be received if you choose to proceed with that vendor.
With the AIOps marketplace growing by the day, it can feel overwhelming to choose a vendor to entrust your operations to. But that’s exactly why it’s so important to be thoughtful and thorough with your research. If not, you may have to put in more work than the solution is supposed to be saving! It’s tempting to evaluate vendors with more objective factors, such as cost or their placement in a specific analyst report; however, these should not be the ultimate deal makers or breakers in your decision process. They can be utilized as supplemental proof points, but ultimately, you will have to review the criteria above to decide the best fit for an AIOps solution in your environment.
I am also working on a so-called “AIOps RFP boilerplate” to select the best AIOps solution to suit your needs, which I will publish soon. If you’re struggling to make the right choice, reach out to me. I am more than happy to provide my two cents as you go through your assessment. I know it is a tough process and wish you the best of luck with your AIOps journey.
This post is sponsored by Juniper Networks.
