We have 3 years of success and failure delivering LLMs on the back of 20+ years of delivering analytics. Our team is very bullish on AI because it’s grounded on years of success, but that comes with real AI BI Challenges. Understanding your organization’s dynamics are important to avoid them.
Enterprise dynamics that can impact your AI BI success
Uniqueness
LLMs are trained on a corpus of knowledge that is wide reaching. For example, an LLM understands all facets of a general retail store operation. However, it does not understand your inventory and supply chain management, and customer buying patterns. These are nuanced problems that have required some form of AI. Your business is unique… Maybe it’s part of your secret sauce to success or maybe your uniqueness is holding you back. Like the team members that manage your business, AI requires direction from anything that breaks for “norms.”
Ambiguity
Ambiguity causes human confusion requiring “alignment.” We call bad output from an LLM a hallucination which ambiguity can easily trigger. In the workplace, tribal knowledge typically reduces ambiguity and fills in gaps. To be in the business of controlling the quality of AI / BI is removing ambiguity from data driven decisions. A process that is well defined documented and followed is easy to explain to people and a system. The “gray” area or human reasoned connections are both an incredible use case for using AI reasoning models but a very painful way to experience AI aided decision support. Data influenced decisions should be clear and consistent to be trust worthy.
Business Semantics Disconnect
Two team members show up to a meeting with 2 versions of revenue… This problem is painful, but often overblown to sell software and services. We understanding how decisions happen, semantics break down, and how to design systems that surface these disconnects early and often. All paths lead back to “governance” of some form, but we believe governing your semantics is just as important as the data itself!
Inconsistency
Consistency over time wins. This is especially true when your enterprise does not operate at high data volumes. A process that is well defined, documented, and followed is easy to explain to people and to a system. The gray areas and human-reasoned connections are both a powerful use case for AI reasoning and a painful way to experience AI-aided decision support. Data-influenced decisions need to be clear and consistent to be trustworthy.
Want to avoid AI BI challenges?
The businesses that win are not the ones with the most data or the most tokens burned on AI services . They are the ones who understand how to focus their teams on the right problems, AI BI concepts, and make consistent improvement to effectively use data for continuous improvement. DataTools Pro is here to help!
