The following describes the procedures and work environment used for developing the official SPFv1 implementation test suite.
Discussion
Discussion about the test suite and its development should happen on the spf-devel mailing list.
Goals
The test suite should...
- be pure, implementation-independent data that can be used offline without access to the DNS.
- be available online in the DNS, too.
- cover all the RFC 2119 keywords (MUST, SHOULD, RECOMMENDED, MAY) in RFC 4408 that are relevant to SPF implementations (some are not); there should be at least one positive and one negative test for each keyword.
- cover other important aspects (syntax, tables, common mistakes, tricky corner cases, etc.) of the specification that are relevant to SPF implementations.
- optionally cover non-definitive best practices (e.g. strict vs. lax interpretation) that are relevant to SPF implementations.
Development Roadmap
- Gather all the old test data that is available from:
Wayne's test suite ✔- the libspf2 Subversion repository (Julian Mehnle)
the spf1-test.mailzone.com. and spftest.org. DNS zones ✔
Create an annotatable version of the SPF specification so tests can be added inline. ✔- Evaluate test cases from the old test data with regard to suitability for testing the relevant parts of the specification, or invent new test cases.
- Define a suitable format for a text database of test cases with comments. Appropriate meta formats might be XML, YAML, JSON, RFC 2822 header blocks (sort of YAML), or TSV (tab separated values).
- Create the text database of test cases.
- Apply the test suite to existing implementations to find any remaining deviations of the test suite from the specification and to determine the implementations' standards compliance.
Development
Please add/edit tests in the annotatable SPF specification. See the existing test cases there for how to format things.
Revision Control Repository
The actual test suite is maintained in a Subversion repository:
