Semantic and Contextual Debugging for Vertical-Specific AI
Understanding the opportunity for Vertical-Specific AI as AI shifts from data processing to semantic processing and how it parallels human to human interactions.
The single largest shift in AI that is happening at the moment is the shift from data processing to semantic processing. As LLMs create a layer of abstraction, turning language into machine-readable code, interacting with AI systems will increasingly be about semantic and contextual optimization and debugging. Semantic and contextual optimization will unlock vertical AI, as AI systems get empowered to move past generalistic vague answers, to well-reasoned specific replies.
But what does that mean? How is it that a system which can understand words, construct arguments, and somewhat reason and digest new documents also require “semantic and contextual optimization”?
One way to think about working with LLMs is a bit like hiring an intern from Harvard. An intern from Harvard is likely to have a large amount of processing power, and very likely has a few areas of knowledge in which they are incredibly deep. However, like all other people, when they are thrown into a new contextual setting, they need some guidance to succeed.
For example, asking an economics intern to develop a legal document will be a challenge. Although the intern can be provided with templates, dictionaries, and access to the internet, it cannot be guaranteed that the intern will be able to adequately track down every concept and reason through the implications of multiple concepts within an argument. The intern will probably produce a piece of work that does not account for conceptual nuances which change the direction of the analysis.
This is not just about domain expertise. Let us say that we are a hospital in the UK and we hire the best neurosurgeon in the US to come work for us. The American neurosurgeon can certainly treat any patient in front of them, but consider a very ordinary day-to-day request: “please go to the OR, check the patient’s records and see if Surgery is ready for them.” The newly arrived American neurosurgeon is going to need context — How do you find the OR, where do you normally keep the patient’s records, and how can I find out who is the right person in the Surgery department to contact?
The neurosurgeon and the intern will likely receive lots of feedback and context from their colleagues about how and why specific pieces of information are crucial in certain circumstances. It is important to note that the information is likely within reach of all participants so this is not a question of data access, but rather an issue of context — being able to identify how, when and why particular information should be found and applied.
In other words, these semantic and contextual issues are the same ones that non-technical people face on a day-to-day basis when performing information retrieval from other humans and workflows. Just as LLMs will as they enter the realm of language,interface more directly with humans and unstructured information, and gain greater access to tools, the issues LLMs will face in extracting relevant information are the same ones that humans may recognize as pertaining to their own experience.
How does this translate practically? In RAG, there are contextual problems that occur around:
Aligning to a company’s internal world-view and vocabulary
Accounting for domain expertise to short-cut generalized search processes
Reasoning through information requests
1. Aligning to a company’s internal world-view and vocabulary
The easiest example of context optimization is vocabulary. A company may associate unique acronyms with terms or definitions in order to imply meaning and create reliable output to questions containing those abbreviations.
A more complicated example would be translating world-views on certain words. For example, in the context of a fund administration context, a ‘management fee’ is the fee that a fund charges its investors. However, the underlying LLM (GPT-4) thought that the management was the amount that an investor receives as revenue.
It is hard to classify this as a hallucination, but rather a misinterpretation of the term and the recipient. This type of issue can be solved with a dynamic prompt rule that tells the LLM that the word ‘fee’, refers to the fund as opposed to the investor. Now, imagine serving an entire list of prompt rules that reflect the company’s every world-view each time a general query is used. This is where on-demand prompt/query augmentation, driven by a knowledge base of rules can come into play.
2. Accounting for domain expertise to short-cut generalized search processes
This is essentially query augmentation that injects just-in-time context. For example, when performing retrieval on a research report about treatment of Alzheimer’s Disease, a doctor understands that you always need to mention Drug A and Drug B in the context of Alzheimer’s. Instead of writing hundreds of prompts that inject this particular context into a variety of situations, you can instead write and store these contexts to be pulled to augment the prompt only when needed, for example, when “Alzheimer’s” is mentioned.
Being able to build a system that allows many people to contribute domain expertise that is stored and pulled in on a just-in-time basis will be an incredibly powerful contextual debugging process.
3. Reasoning through information requests
Let us go back to the example with the neurosurgeon: giving instructions as to where information is so they don’t have to perform a search process across every single document in the hospital. Using a document hierarchy and a contextual dictionary, we are able to guide the LLM to retrieve relevant and contextually necessary chunks of info that are required to adequately answer the question. Thus, instead of having to search every document and then having the LLM figure out if the documents are relevant to answering pieces of the question, we are able to rely on a structured meta-information data store to reach the right parts of the answer directly
In conclusion, the evolution from data processing to semantic processing in the realm of LLMs mark a significant shift in AI, underscoring the critical role of contextual understanding and domain expertise to empower vertical-specific AI. This analogy of needing to guide a Harvard intern or an American neurosurgeon in unfamiliar settings parallels the necessity for LLMs to receive targeted inputs and structured information to ensure accuracy and relevance in their outputs. The challenges of aligning with an organization’s unique vocabulary, leveraging domain knowledge effectively, and adeptly navigating through complex information requests underscore the importance of advanced semantic and contextual optimization. Platforms and infrastructure that excel in these areas are not just contributing to the advancement of AI but are positioning themselves at the forefront of the need for consistent, accurate context optimization for enterprise AI.
WhyHow.AI is focused on contextual optimization and semantic debugging of probabilistic LLM systems with deterministic relationship and semantic mapping through knowledge graph technology. If you’re a developer early in thinking about, in the process of, or have already incorporated knowledge graphs in RAG, we’d love to chat at team@whyhow.ai.