Conformance Testing, Full Matrix Test vs. test harness
Minutes
Test harness (Geant fed lab): now being developed for OpenID/Connect. SMAL not available yet in the second version that separates backend and frontend. A new version of the test harness was developed for OpenId/Connect. The SAML-engine needs to be integrated into it.
Test harness should be configurable for various deployment profiles (US, CA, FI, AT, ..)
John: Scenario and deployment testing is more useful than testing of isolated functions, because the latter does not tell you if actual features really work.
To use the fedlab test harness one has to instrument the proxy to get the feedback. Keith noted that he failed to do that.
John: Can we get the resources together to put together the test cases needed for the Full Matrix test?
Keith: Despite interest in the resulting test cases, there has been no volunteers to do the work in developing test cases for the Full Matrix Test to test eGov 2.0 Interop profile.
It seems to be reasonable to focus on the development of test cases for the test harness
Future Full Matrix Tests could be conducted based on test harness infrastructure if there is demand; Uninett could be contracted for that.
John says that a number of weeks would have to be invested to develop the eGov 2.0 test cases.
Rainer says that the allocation of some modest budget to implement the tests for the Austrian eGov profile is likely to be awarded.
Leif said he will establish a contact between Hedberg Roland working for Géant on the test harness and Rainer to see how resources could be joined.
Action item
WG-Wiki: link final version from front page. Currently the draft version is shown.