Skip to content

ID-165: Error handling and cleanup for must support response selection - #84

Merged
karlnaden merged 3 commits into
inferno-framework:mainfrom
abhinandan2012:ID-165-response-selection-cleanup
Sep 4, 2026
Merged

karlnaden merged 3 commits into
inferno-framework:mainfrom
abhinandan2012:ID-165-response-selection-cleanup

Conversation

@abhinandan2012

Copy link
Copy Markdown
Contributor

Summary

Follow-up cleanup for the criteria-based response selection introduced in #79, addressing code review feedback. The selection semantics are unchanged; these changes harden error handling so that tester input or environment problems produce a logged default response instead of an HTTP 500 to the client under test, and clean up a few internals:

  • An invalid or inverted requestRange (for example "1-2,a" or "3-2") no longer raises. It is logged and treated as unmet, so the entry is skipped and later entries are still considered.
  • FHIRPath service failures (non-2xx responses, unreachable service, unparseable response bodies) during criteria evaluation or token replacement are rescued in the endpoint. Inferno logs a warning and generates a default response.
  • The operation is now parsed with the query string stripped, so requests to .../Claim/$submit?param=value are tagged, counted, and matched correctly.
  • When a response input is present but unusable (invalid JSON, or an entry that is neither a Bundle nor a wrapper with a bundle key), the reason and offending entry number are logged instead of silently falling back to default responses.
  • The request number is computed once per request instead of once per candidate with a requestRange criterion, removing redundant repository queries.
  • Removed a dead rescue in token replacement and skipped the FHIR round trip when the selected bundle contains no tokens.
  • The ms_submit_responses / ms_inquire_responses input descriptions are now generated from a single shared helper in client_input_descriptions.rb so the documentation cannot drift between the two inputs.
  • FHIRPath service error messages no longer interpolate the full serialized resource.

How to test

Run the suites against each other following docs/Running-Suites-Against-Each-Other.md:
client suite session (Da Vinci PAS Client Suite v2.2.1, Other Authentication, preset
"Run Against the PAS Server Suite") in one tab, server suite session (preset
"Run Against the PAS Client Suite") in another.

Regression: existing behavior unchanged

  1. In the client session, run group 10 Must Support Elements
    (pas_client_v221_must_support_no_auth) and leave test 10.1.01
    (pas_client_v221_gather_must_support) waiting.
  2. In the server session, run groups 3.1 $submit Element Support and
    3.2 $inquire Element Support.
  3. Back in the client session, click the confirmation link in the 10.1.01 wait dialog
    and complete the attestations.

Expected: the group passes as before, including 10.2.01
(pas_client_v221_request_bundle_validation_test) and 10.2.02
(pas_client_v221_response_bundle_validation_test).

Error handling scenarios

Run group 10 Must Support Elements with the Must Support $submit Response Bundles
input set to a list whose first entry is a wrapper with an invalid range and whose
second entry is a plain Bundle:

[
  { "criteria": { "requestRange": "3-2" }, "bundle": { ...bundle A... } },
  { ...bundle B... }
]

Send a $submit request to the endpoint shown in the 10.1.01 wait dialog. Expected:
HTTP 200 with bundle B returned (the invalid entry is skipped), and the server log
contains "Ignoring unmatchable requestRange criteria".

With the FHIRPath service stopped (docker stop <fhirpath container>), provide an
entry with a fhirpath criterion and send a request. Expected: HTTP 200 with a
generated default response and a logged warning, rather than an HTTP 500.

Send a $submit request with a query string appended to the endpoint URL
(for example ...$submit?param=value). Expected: the tester-provided bundle is still
returned and the request is counted normally.

Provide input that is not valid JSON, or a wrapper entry missing its bundle key.
Expected: default responses, with a log line naming the input and the offending entry.

Where to verify results

  • Test 10.2.01: the REQUESTS tab lists the received $submit requests; open DETAILS
    on a request and expand Response Body to see which bundle was returned.
  • The Inferno server log shows one line per request: the selection decisions
    ("Selected tester-provided response bundle N of M...") and the new warnings for
    invalid ranges, unusable inputs, and FHIRPath service failures.
  • The run dialog for group 10 Must Support Elements shows the (now shared) input
    documentation on both response inputs.

@abhinandan2012
abhinandan2012 marked this pull request as draft August 31, 2026 14:07
@abhinandan2012
abhinandan2012 marked this pull request as ready for review September 3, 2026 09:26
@abhinandan2012
abhinandan2012 force-pushed the ID-165-response-selection-cleanup branch from 1d7765a to c73cb19 Compare September 3, 2026 09:31

@karlnaden karlnaden left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to report errors in the test if possible, not to the console log which the tester won't have access to. Since this is a tester issue in a specific session, it won't be helpful to Inferno maintainers with access to these logs.

Comment thread lib/davinci_pas_test_kit/client/user_input_response.rb Outdated
Comment thread lib/davinci_pas_test_kit/cross_suite/response_selection_utils.rb Outdated
Comment thread lib/davinci_pas_test_kit/client/endpoints/claim_endpoint.rb Outdated
Comment thread lib/davinci_pas_test_kit/client/endpoints/claim_endpoint.rb
Comment thread lib/davinci_pas_test_kit/cross_suite/fhirpath_utils.rb
… response selection inputs and FHIRPath failures

@karlnaden karlnaden left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

several minor language tweaks

Comment thread lib/davinci_pas_test_kit/client/user_input_response.rb
Comment thread lib/davinci_pas_test_kit/client/user_input_response.rb Outdated
Comment thread lib/davinci_pas_test_kit/cross_suite/response_selection_utils.rb Outdated
@karlnaden
karlnaden merged commit 22b46d0 into inferno-framework:main Sep 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants