All pull requests

Pull requests

• fix(metrics): return tool correctness score in default async mode

  #3490·Ricardo-M-L opened Oct 19, 2026·

  ·
  1

• feat(models): bring your own System One decision provider

  #3489·aigauravsingh-star opened Oct 19, 2026·

  ·
  1

• fix(metrics): preserve System One backend configuration when batching

  #3488·aigauravsingh-star opened Oct 19, 2026·

  ·
  1

• fix(metrics): reject empty required conversational strings

  #3486·original4422 opened Oct 18, 2026·

  ·
  1

  1

• feat(metrics): add community CalibrationMetric (ECE + Brier)

  #3485·ghubuser-dnt opened Oct 18, 2026·

  ·
  2

• feat(metrics): add ThoughtActionConsistencyMetric to community

  #3484·wjl8636 opened Oct 18, 2026·

  ·
  1

• fix(scorer): report missing BERTScore dependencies

  #3483·devgprime opened Oct 18, 2026·

  ·
  2

• fix: preserve async turn faithfulness window order

  #3481·kausthubhk opened Oct 18, 2026·

  ·
  1

  1

• fix(agent_loop_detection): hash full inputs to avoid false cycle detection

  #3479·Aling33 opened Oct 17, 2026·

  ·
  1

• fix(dataset): preserve voice fields when exporting conversation turns

  #3478·G1lollipop opened Oct 17, 2026·

  ·
  1

• fix(test_case): never read local files from unregistered image placeholders

  #3477·ybai08 opened Oct 17, 2026·

  ·
  1

  1

• test: cover public TaskCompletionMetric measurement paths

  #3476·KyPython opened Oct 17, 2026·

  ·
  1

• fix(dataset): treat null tools as empty in add_test_cases_from_json_file

  #3470·nancy6547 opened Oct 16, 2026·

  ·
  1

  2

• fix(dataset): synchronize mode when replacing test cases

  #3468·yunyv opened Oct 16, 2026·

  ·
  1

• docs: Change referenceless to referenceBased for Tool Correctness metric

  #3466·andreabosisio opened Oct 15, 2026·

  ·
  1

  1

• fix(cache): preserve ordered test-case inputs

  #3464·March-7 opened Oct 15, 2026·

  ·
  1

• fix(dataset): preserve token usage when exporting goldens

  #3463·March-7 opened Oct 15, 2026·

  ·
  1

• feat(metrics): add FilterCorrectnessMetric for retrieval metadata filters

  #3458·Christian-Sidak opened Oct 15, 2026·15 tasks done·

  ·
  1

  2

• feat: add group_by to DisplayConfig for tag/metadata aggregate slicing

  #3457·Christian-Sidak opened Oct 15, 2026·2 tasks done·

  ·
  1

  2

• fix(dataset): preserve test case names and comments in exports

  #3456·laulauraluo opened Oct 15, 2026·

  ·
  1

• docs: fix typos in STT integration and non-advice metric pages

  #3454·Mrxuba opened Oct 14, 2026·

  ·
  1

• fix(voice): point LiveKitConnector install hint at livekit packages

  #3451·li12345678990 opened Oct 14, 2026·

  ·
  2

• feat(metrics): add require_citations option to CitationFaithfulnessMetric

  #3450·pruthvishetty opened Oct 13, 2026·

  ·
  2

• feat(litellm): add litellm optional extra

  #3449·misbahsy opened Oct 13, 2026·

  ·
  1

• feat(turns): add support for metadata in turn api

  #3448·tanayvaswani opened Oct 13, 2026Contributor·

  ·
  1