Issues

• JSON._check_json_obj coerces a judge's quoted-string "false" verdict to True, inverting ContextCoverage's score

  relari-ai/continuous-eval#86 · Filter by author shaurya416 opened on Sep 25, 2026

• SQLASTSimilarity raises ParseError on malformed SQL instead of returning the documented -1.0 score

  relari-ai/continuous-eval#85 · Filter by author graysoncooper opened on Aug 19, 2026

• Feature request: standalone EvalPort adapter for continuous-eval metrics

  relari-ai/continuous-eval#84 · Filter by author adhabnr-ux opened on Aug 15, 2026

• LLMBasedFaithfulness fails when response cannot be split into score and reasoning

  relari-ai/continuous-eval#71 · Filter by author kelvinchanwh opened on Jun 11, 2024

• LLMBasedFaithfulness does not work when using LLMs other than OpenAI

  relari-ai/continuous-eval#69 · Filter by author kelvinchanwh opened on Jun 11, 2024
  2

• Integration with DSPy

  enhancementgood first issue
  relari-ai/continuous-eval#64 · Filter by author pantonante opened on May 21, 2024
  4

• Metadata comparison for retrieval metrics

  enhancement
  relari-ai/continuous-eval#54 · Filter by author yisz opened on Apr 1, 2024
  1

• Enhancement: Integration with LiteLLM

  enhancement
  relari-ai/continuous-eval#53 · Filter by author rachittshah opened on Mar 28, 2024
  4

• Highlight LLM Based Custom Metric in ReadMe and Docs

  relari-ai/continuous-eval#51 · Filter by author yisz opened on Mar 18, 2024
  

• Handling context limit

  relari-ai/continuous-eval#50 · Filter by author yisz opened on Mar 18, 2024

• Error handling for generation LLM metrics

  bug
  relari-ai/continuous-eval#19 · Filter by author yisz opened on Jan 31, 2024

• Add BERT variations for faithfulness

  enhancement
  relari-ai/continuous-eval#5 · Filter by author yisz opened on Jan 3, 2024