...
Section
...
numbers
...
correspond
...
to
...
the
...
eGov
...
2.0
...
draft
...
profile.
...
Table of Contents |
---|
...
|
...
|
2.2.1
...
Metadata
...
Profiles
...
(IOP,
...
Scott
...
Cantor
...
-
...
cantor.2@osu.edu;
...
PKIX,
...
Terry
...
McBride)
...
Test
...
the
...
ability
...
of
...
an
...
implementation
...
to
...
produce
...
and
...
consume
...
metadata
...
in
...
accordance
...
with
...
the
...
two
...
mandatory
...
profiles
...
identified,
...
and
...
to
...
support
...
the
...
"Metadata
...
Extension
...
for
...
Entity
...
Attributes"
...
profile.
...
Production
...
of
...
IOP-compliant
...
Metadata
...
Scope
- Verify the ability to produce metadata conformant to the Metadata IOP.
IOP-conformant metadata has a different meaning from metadata intended to be evaluated in a PKIX environment, but syntactically should be identical based on the eGov profile language, so this should be sufficient to test production of metadata for both profiles.
Preconditions
- Implementation configured sufficiently to produce metadata identifying its signing, TLS, and encryption keys.
- Details of expected md:KeyDescriptor content available to tester
Test Sequence
1. Access published metadata
The metadata produced by the implementation is obtained.
CONFIRM: The content(s) of the md:KeyDescriptor element(s) matches the expected output.
Consumption of IOP-compliant Metadata
Testing is best accomplished with a fixed implementation to test against, as the purpose is not to test actual protocol correctness, but use of the metadata itself.
Scope
- Verify the ability to process metadata conformant to the Metadata IOP.
- Verify support for bare keys, self-signed, arbitrary-CA, and expired certificates.
- Check for bypass of CRL or OCSP extensions in certificates.
- Verify ability to match keys between different certificates
- Verify recognition of a removed/revoked key.
- Verify enforcement of validUntil.
Preconditions
- Keys and certificates generated with various characteristics:
- Self-signed, no revocation-related extensions
- Signed by a CA, with revocation-related extensions
- Expired
- Multiple certificates with the same public key
Test Sequence
1. Test Bare Keys
a. Metadata is prepared for the fixed implementation containing a pair of md:KeyDescriptors both containing a ds:KeyValue containing a ds:RSAKeyValue. The fixed implementation is configured to use one of the keys for signing and/or TLS purposes. The metadata is supplied to the candidate system, and one or more SSO operations is attempted. Bindings should be chosen to exercise both signature verification and TLS authentication if possible.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
b. The fixed implementation is switched to use the second of the pair of keys in its metadata.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
c. The fixed implementation is switched to use a key that is NOT in its metadata.
CONFIRM: The SSO operations are unsuccessful.
2. Test Self-Signed Certificates
a. Metadata is prepared for the fixed implementation containing a pair of md:KeyDescriptors both containing a ds:X509Certificate, with the certificates self-signed. One of the certificates should be expired. The fixed implementation is configured to use one of the certificates for signing and/or TLS purposes. The metadata is supplied to the candidate system, and one or more SSO operations is attempted. Bindings should be chosen to exercise both signature verification and TLS authentication if possible.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
b. The fixed implementation is switched to use the second of the pair of certificates in its metadata.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
c. The fixed implementation is switched to use a different certificate that is not in the metadata but contains the same key as one that is.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
d. The fixed implementation is switched to use a certificate and key that is NOT in its metadata.
CONFIRM: The SSO operations are unsuccessful.
2. Test CA-Issued Certificates
a. Metadata is prepared for the fixed implementation containing a pair of md:KeyDescriptors both containing a ds:X509Certificate, with the certificates signed by one or more arbitrary CAs. The certificates should, if possible, include a CRL distribution point or OCSP location extension. These endpoints should not exist or be unavailable to the testing environment. The fixed implementation is configured to use one of the certificates for signing and/or TLS purposes. The metadata is supplied to the candidate system, and one or more SSO operations is attempted. Bindings should be chosen to exercise both signature verification and TLS authentication if possible.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata. No delays in processing occur as a result of failed attempts to contact non-existent or unavailable CRL or OCSP endpoints.
b. The fixed implementation is switched to use the second of the pair of certificates in its metadata.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
c. The fixed implementation is switched to use a different certificate that is not in the metadata but contains the same key as one that is.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
d. The fixed implementation is switched to use a key that is NOT in its metadata.
CONFIRM: The SSO operations are unsuccessful.
3. Verify Revocation and validUntil
a. Metadata is prepared for the fixed implementation containing a md:KeyDescriptor containing a key in one of the supported formats. The fixed implementation is configured to the key for signing and/or TLS purposes. The metadata is supplied to the candidate system, and one or more SSO operations is attempted. Bindings should be chosen to exercise both signature verification and TLS authentication if possible.
CONFIRM: The SSO operations are successful based on the use of the key(s) that appear in the metadata.
b. The metadata is altered to remove the md:KeyDescriptor and the candidate implementation is configured to refresh the appropriate source of metadata.
CONFIRM: The SSO operations are unsuccessful.
c. The metadata is altered to add back the md:KeyDescriptor, but a validUntil attribute is added such that the metadata is expired, and the candidate implementation is configured to refresh the appropriate source of metadata.
CONFIRM: The SSO operations are unsuccessful.
Support for "Metadata Extension for Entity Attributes" Profile
Scope
- Test SP acceptance of SSO based on IdP metadata extension content
The proposed tag would be an attribute named "urn:oasis:names:tc:SAML:attribute:assurance-certification"
...
and
...
would
...
appear
...
as
...
follows:
...
Code Block | ||||
---|---|---|---|---|
| ||||
<EntityDescriptor entityID="https://idp.example.org/SAML" ... >
<Extensions>
<attr:EntityAttributes xmlns:attr="urn:oasis:names:tc:SAML:metadata:attribute">
<saml:Attribute
NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
Name="urn:oasis:names:tc:SAML:attribute:assurance-certification">
<saml:AttributeValue>
http://foo.example.com/assurance/loa1
</saml:AttributeValue>
</saml:Attribute>
</attr:EntityAttributes>
</Extensions>
<IDPSSODescriptor...>
...
</IDPSSODescriptor>
</EntityDescriptor>
{code}
h5. Preconditions
* SP configured with metadata for candidate IdP containing acceptable LOA |
Preconditions
- SP configured with metadata for candidate IdP containing acceptable LOA "tag".
...
- SP
...
- configured
...
- with
...
- metadata
...
- for
...
- candidate
...
- IdP
...
- not
...
- containing
...
- acceptable
...
- LOA
...
- "tag".
...
- SP
...
- configured
...
- to
...
- require
...
- presence
...
- of
...
- "tag"
...
- in
...
- metadata
...
- for
...
- IdPs
...
- before
...
- it
...
- will
...
- accept
...
- SSO
...
- from
...
- them.
...
Test
...
Sequence
...
1.
...
Verify
...
use
...
of
...
acceptable
...
IdP
...
SP-initiated
...
or
...
IdP-initiated
...
SSO
...
is
...
used
...
to
...
produce
...
an
...
assertion
...
response
...
from
...
the
...
candidate
...
IdP.
...
CONFIRM
...
:
...
SSO
...
is
...
successful.
...
2.
...
Verify
...
non-use
...
of
...
unacceptable
...
IdP
...
SP-initiated
...
or
...
IdP-initiated
...
SSO
...
is
...
used
...
to
...
produce
...
an
...
assertion
...
response
...
from
...
the
...
candidate
...
IdP.
...
CONFIRM
...
:
...
SSO
...
is
...
unsuccessful
...
based
...
on
...
the
...
policy
...
requiring
...
the
...
tag.
...
2.2.2
...
Metadata
...
Exchange
...
(Scott
...
Cantor
...
-
...
cantor.2@osu.edu)
...
Test
...
the
...
ability
...
of
...
an
...
implementation
...
to
...
publish
...
and
...
consume
...
metadata
...
documents,
...
and
...
maintain
...
the
...
information
...
in
...
real
...
time.
...
Testing
...
import
...
and
...
verification
...
is
...
best
...
accomplished
...
with
...
a
...
fixed
...
set
...
of
...
"sources"
...
to
...
import
...
and
...
test
...
against,
...
as
...
the
...
purpose
...
is
...
not
...
to
...
test
...
actual
...
protocol
...
correctness,
...
but
...
use
...
of
...
the
...
metadata
...
itself.
...
It
...
may
...
be
...
necessary
...
to
...
bypass,
...
or
...
incorporate
...
a
...
limited
...
amount
...
of,
...
metadata
...
verification
...
functionality
...
in
...
order
...
for
...
metadata
...
import
...
to
...
be
...
tested.
...
Publication
Scope
- Publication (and maintenance) of metadata via Well-Known-Location
...
- resolution
...
- profile.
...
Preconditions
- An http/https
...
- entityID
...
- defined
...
- that
...
- is
...
- suitable
...
- for
...
- dereferencing
- Appropriate configuration of that entityID is completed
- Multiple details of configuration are available to tester (location of a profile endpoint, a key descriptor, etc.)
- Any pre-publishing step required is completed
Test Sequence
1. Access published metadata
The entityID is dereferenced to obtain the metadata document.
CONFIRM: The metadata is available, and correctly reflects the entityID accessed, and is returned with the correct MIME type (application/xml+samlmetadata).
...
The
...
configuration
...
details
...
expected
...
are
...
found
...
in
...
the
...
metadata.
...
2.
...
Alter
...
metadata
...
and
...
republish
...
Alter
...
the
...
configuration
...
(changing
...
an
...
endpoint,
...
a
...
key
...
descriptor,
...
etc.)
...
and
...
republish,
...
then
...
repeat
...
the
...
first
...
test.
...
CONFIRM:
...
As
...
in
...
(1),
...
but
...
also
...
that
...
the
...
implementation
...
did
...
not
...
require
...
a
...
restart
...
or
...
disruption
...
of
...
service.
...
Import
...
from
...
File
Scope
- Metadata consumption via local file
- Ability to detect and ignore invalid metadata
- Support for batches (md:EntitiesDescriptor)
...
- Ability
...
- to
...
- update
...
- from
...
- a
...
- changed
...
- source
...
- without
...
- disruption
...
- Maintenance
...
- of
...
- valid
...
- operation
...
- after
...
- a
...
- change
...
- that
...
- renders
...
- a
...
- source
...
- invalid.
...
Preconditions
- Valid metadata is available to the implementation via a local filesystem path
- The valid metadata contains at least two md:EntityDescriptor elements inside an md:EntitiesDescriptor element
- Invalid metadata is available to the implementation via a (different) local filesystem path
- Appropriate configuration for the use of those paths is applied
- No configuration of the information supplied via metadata is in place prior to import
Test Sequence
1. Import valid metadata
The implementation is directed in whatever manner is required to import or make use of the valid metadata. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Operation of a defined set of SAML interactions with the metadata subject is successful based on the content of the metadata (correct endpoints used, keys used in accordance with one of the supported metadata profiles, etc.).
2. Import invalid metadata
The implementation is directed in whatever manner is required to import or make use of the invalid metadata. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Import and/or interaction with the metadata subject is unsuccessful.
3. Update valid metadata
The valid metadata is modified in some manner that is detectable via the interactions used to confirm successful import (changing an endpoint, a key descriptor, etc.), but remains valid. If the implementation requires manual intervention to recognize the change, this is done. The SAML interactions are repeated.
CONFIRM: The interactions remain successful but cognizant of the change(s). No restart or other service interruption was required to accomodate the change.
4. Update valid metadata with invalid change.
The valid metadata is modified in some manner that renders it invalid. If the implementation requires manual intervention to recognize the change, this is done. The SAML interactions are repeated.
CONFIRM: The interactions remain successful in accordance with the metadata that existed prior to the change. No restart or other service interruption was required to accomodate the change.
Import from URL
Scope
- Metadata consumption via multiple http and https sources
- Ability to detect and ignore invalid or unavailable metadata
- Support for caching
- Ability to update via a changed source without disruption
- Maintenance of valid operation after a change that renders a source invalid.
Preconditions
- Valid metadata is available to the implementation via at least two URLs (one http, one https)
- Invalid metadata is available to the implementation via a different, possibly unavailable, URL
- Appropriate configuration for the use of those URLs is applied
- No configuration of the information supplied via metadata is in place prior to import
Test Sequence
1. Import valid metadata
The implementation is directed in whatever manner is required to import or make use of the valid metadata. A set of SAML interactions is then attempted between the implementation and the metadata subjects (at least two, one for each source of metadata). A basic test of SP-initiated SSO is sufficient.
CONFIRM: Operation of a defined set of SAML interactions with the metadata subjects is successful based on the content of the metadata (correct endpoints used, keys used in accordance with one of the supported metadata profiles, etc.).
2. Import invalid metadata
The implementation is directed in whatever manner is required to import or make use of the invalid metadata source. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Import and/or interaction with the metadata subject is unsuccessful.
3. Re-import unchanged metadata
The implementation is directed in whatever manner is required to re-import/refresh the valid source of metadata, with the source maintaining the same caching indicator(s). The SAML interactions are repeated.
CONFIRM: The import resulted in no exchange of the metadata document across the network, and the interactions remain successful in accordance with the metadata that existed prior to the re-import.
4. Re-import changed metadata
The valid metadata is modified in some manner that is detectable via the interactions used to confirm successful import (changing an endpoint, a key descriptor, etc.), but remains valid. The implementation is directed in whatever manner is required to re-import/refresh the valid source of metadata. The SAML interactions are repeated.
CONFIRM: The interactions remain successful but cognizant of the change(s). No restart or other service interruption was required to accomodate the change.
5. Update valid metadata with invalid change.
The valid metadata is modified in some manner that renders it invalid. The implementation is directed in whatever manner is required to re-import/refresh the now-invalid source of metadata. The SAML interactions are repeated.
CONFIRM: The interactions remain successful in accordance with the metadata that existed prior to the change. No restart or other service interruption was required to accomodate the change.
2.2.2.1 Metadata Verification (Scott Cantor - cantor.2@osu.edu)
Test the ability of an implementation to verify metadata before importing/accepting it for use. The focus is on import of remote sources, since local file sources can naturally undergo checking outside of the import process before being made available.
Verification by Known Key
Scope
- Test verification of root level signature via a known key.
Preconditions
- Any MTI signature algorithm may be used.
- Valid metadata signed by a known key is available at an http or https URL.
- Valid metadata with an invalid signature is available via a different URL.
- The key should not be present inside the signature of the metadata document.
- Appropriate configuration for the use of the URLs and verification with the key is applied.
- No configuration of the information supplied via metadata is in place prior to import
Test Sequence
1. Import and verify valid metadata
The implementation is directed in whatever manner is required to import or make use of the valid metadata. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Operation of a defined set of SAML interactions with the metadata subject is successful based on the content of the metadata (correct endpoints used, keys used in accordance with one of the supported metadata profiles, etc.).
2. Import and (fail to) verify invalid signature
The implementation is directed in whatever manner is required to import or make use of the metadata with the invalid signature. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Import and/or interaction with the metadata subject is unsuccessful.
Verification by Certificate Validation
Scope
- Test verificiation of root level signature via path validation of a signing certificate.
Preconditions
- Any MTI signature algorithm may be used.
- Two certificates issued by a sample certificate authority are created, one valid, one expired.
- The certificate must be present inside the signature of the metadata document.
- Valid metadata signed by the key in the valid certificate is available at an http or https URL.
- Valid metadata signed by the key in the invalid certificate is available via a different URL.
- Appropriate configuration for the use of the URLs and verification with the issuing CA is applied.
- No configuration of the information supplied via metadata is in place prior to import
Test Sequence
1. Import and verify valid metadata
The implementation is directed in whatever manner is required to import or make use of the valid metadata. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Operation of a defined set of SAML interactions with the metadata subject is successful based on the content of the metadata (correct endpoints used, keys used in accordance with one of the supported metadata profiles, etc.).
2. Import and (fail to) verify invalid signature
The implementation is directed in whatever manner is required to import or make use of the metadata signed with the invalid certificate. A set of SAML interactions is then attempted between the implementation and the metadata subject. A basic test of SP-initiated SSO is sufficient.
CONFIRM: Import and/or interaction with the metadata subject is unsuccessful.
2.3 Name Identifiers (Paul Madsen - paulmadsen@rogers.com)
The tests will confirm that IdP and SP properly implement support for the Name Identifier formats of
- urn:oasis:names:tc:SAML:2.0:nameid-format:transient
...
- urn:oasis:names:tc:SAML:2.0:nameid-format:persistent
...
Such
...
support
...
will
...
consiste
...
of
...
proper
...
handling
...
of
...
the
...
- AuthnRequest's
...
- NameIDPolicy
...
- NameID
...
- in
...
- the
...
- returned
...
- Response.
...
PaulM: The existing SAML test plan 3.2.2
...
would
...
appear
...
to
...
cover
...
the
...
necessary
...
tests
...
for
...
'persistent',
...
but
...
testing
...
for
...
'transient'
...
is
...
far
...
more
...
limited,
...
ie
...
only
...
in
...
Test
...
Case
...
J
...
and
...
only
...
in
...
the
...
case
...
of
...
an
...
unsolicited
...
Response.
...
Should
...
this
...
be
...
more
...
symmetrical?
...
NB the SAML eGov 1.5
...
profile,
...
in
...
addition
...
to
...
defining
...
name
...
identifier
...
formats
...
of
...
transient
...
and
...
persistent
...
as
...
MTI,
...
also
...
defined
...
'unspecified'
...
as
...
MTI.
...
The
...
corresponding
...
tests
...
in
...
the
...
SAML
...
test
...
plan
...
3.2.2
...
need
...
be
...
removed/softened.
...
2.4 Attributes (Denny Prvu - Denny.Prvu@ca.com)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
2.5.1
...
Identity
...
Provider
...
Discovery
...
(Andrew
...
Lindsay-Stewart
...
-
...
alindsay-stewart@fugensolutions.com)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
2.5.2.1
...
Authentication
...
Requests
...
Binding
...
and
...
Security
...
Requirements
...
(Jonathan
...
Scudder
...
-
...
Jonathan.Scudder@forgerock.com)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
2.5.2.2
...
Authentication
...
Requests
...
Message
...
Content
...
(Jonathan
...
Scudder
...
-
...
Jonathan.Scudder@forgerock.com)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
2.5.3.1
...
Responses
...
Binding
...
and
...
Security
...
Requirements
...
(Diego
...
Lopez
...
-
...
diego.lopez@rediris.es)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
*
...
See
...
Bob
...
Sunday's
...
example
...
test
...
case
...
below
...
2.5.3.2
...
Responses
...
Message
...
Content
...
(Diego
...
Lopez
...
-
...
diego.lopez@rediris.es)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
2.5.4.1
...
Artifact
...
Resolution
...
Requests
...
(Andrew
...
Lindsay-Stewart
...
-
...
alindsay-stewart@fugensolutions.com)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
...
2.5.4.2
...
Artifact
...
Resolution
...
Responses(Andrew
...
Lindsay-Stewart
...
-
...
alindsay-stewart@fugensolutions.com)
...
[What
...
should
...
be
...
CONFIRMED
...
through
...
testing
...
from
...
this
...
section?
...
]
2.7 SAML 2.0 Proxying
The objective is to test the ability of an implementation to be configured to provide the mapping options required by the eGov2.0 profile. The eGov requirements for 2.7.1 Proxying Authentication Requests and 2.7.2 Proxying Responses can be combined into a single set of test cases.
The software under test would need to be configured 2 different ways as a Proxying IdP:
- to provide