When adding RAG or rolling own, extend the Qos 'CounterMeasures' interface for butler to let devs essentially use secondary LLM to fact check the any returned RAG from a source for like did it help.
Theoretical example:
RAG:
500 page policy doc.
User:
What's the policy on sick time when I'm new and call out?
AI:
RAG [returns policy on sick leave, PTO] did not return anything about new employee
QOS:
Hey do an extended search for theses [probation, time off, new, 60 days, ect....]
AI: Runs rag again.
AI Final output example: If your within your 30 dayss, while you earn PTO, it's unsable. You may want to check with manager.
When adding RAG or rolling own, extend the Qos 'CounterMeasures' interface for butler to let devs essentially use secondary LLM to fact check the any returned RAG from a source for like did it help.
Theoretical example:
RAG:
500 page policy doc.
User:
What's the policy on sick time when I'm new and call out?
AI:
RAG [returns policy on sick leave, PTO] did not return anything about new employee
QOS:
Hey do an extended search for theses [probation, time off, new, 60 days, ect....]
AI: Runs rag again.
AI Final output example: If your within your 30 dayss, while you earn PTO, it's unsable. You may want to check with manager.