Sender Policy Framework

Release

The current RFC 4408 test suite release is 2008.08 (changelog, license). The test cases are specificied in YAML syntax using this schema.

Older releases of the test suite are also available for reference.

Development

The test suite's development version is a work in progress and thus is not to be considered authoritative.

The following describes the procedures and work environment used for developing the official SPFv1 implementation test suite.

Discussion

Theoretical discussion about the test suite and the aspects of RFC 4408 that it tests should happen on the spf-discuss mailing list, whereas discussion about the test suite's technical development (YAML issues, etc.) should take place on the spf-devel mailing list.

Goals

The test suite should...

• be pure, implementation-independent data that can be used offline without access to the DNS.
• be available online in the DNS, too.
• cover all the RFC 2119 keywords (MUST, SHOULD, RECOMMENDED, MAY) in RFC 4408 that are relevant to SPF implementations (some are not); there should be at least one positive and one negative test for each keyword.
• cover other important aspects (syntax, tables, common mistakes, tricky corner cases, etc.) of the specification that are relevant to SPF implementations.
• optionally cover non-definitive best practices (e.g. strict vs. lax interpretation) that are relevant to SPF implementations.
• cover known errata when they are relevant for implementations.

Development Roadmap

1. Gather all the old test data that is available from:
   1. Wayne's test suite
   2. the libspf2 Subversion repository
   3. the spf1-test.mailzone.com. and spftest.org. DNS zones
2. Create an annotatable version of the SPF specification so tests can be added inline.
3. Evaluate test cases from the old test data with regard to suitability for testing the relevant parts of the specification, or invent new test cases.
4. Define a suitable format for a text database of test cases with comments.
5. Create the text database of test cases.
6. Check that known errata are covered when they are finished.
7. Create a formal Relax NG schema for the test suite.
8. Create a logo authorized for use only by implementations that pass the test suite.
9. Create a script to load test case data into an online DNS server suitable for testing implementations that cannot hook DNS lookups.
10. Write a script to merge test cases into the annotatable specification, either statically or on demand (CGI).
11. Apply the test suite to existing implementations to find any remaining deviations of the test suite from the specification and to determine the implementations' standards compliance.

Revision Control Repository

The actual test suite is maintained in a Subversion repository: