ACATS Test Modification List

Version 4.0L
March 31, 2016

What is this?

This is the ACATS modification list for ACATS 4.0. This document includes test corrections, test removals, and new tests for ACATS 4.0. ACATS 4.0 is the current test suite for Ada, covering the third edition of Ada, ISO/IEC 8652:2012 (usually known as "Ada 2012").

ACATS Modification List version 3.1L for ACATS 3.1 is the current list for that test suite. ACATS 3.1 should be used for testing implementations that support Amendment 1 (ISO/IEC 8652:1995/AMD 1:2007, often known as "Ada 2005"), and will continue to be maintained as necessary for the foreseeable future.

ACATS Modification List version 2.6D for ACATS 2.6 is the current list for that test suite. ACATS 2.6 should be used for testing implementations of the second edition of Ada, ISO/IEC 8652:1995 (usually known as "Ada 95"); it also will continue to be maintained as necessary for the foreseeable future.

Changes from the last list:

Test B611006 was corrected that operations Proc3 and Proc4 are abstract (the precondition would be illegal for a null procedure). Tests B480003, B611001, B611002, B611003, B611004, B611005, B750A07, BB42001, BB42002, and BB42003 were updated with error location codes; these tell the future grading tools (see discussion below) exactly where an error should be expected. (All future new tests will also have these codes.)

New tests B611007, B611008, B611009, B611010, B611011, B611012, B611013, C611A01, C611A02, C611A03, C611B01, and C611B02 check additional objectives for preconditions, postconditions, and related attributes.

Test CD11001 had an incorrect value (which caused an unexpected Assertion_Error).

The support files REPSPEC.ADA and REPBODY.ADA have been replaced by file REPORT.A. (More discussion below.)

New tests posted between January 1, 2016 and March 31, 2016 will become required for conformity assessments on October 1, 2016. New tests posted between October 1, 2015 and December 31, 2015 will become required for conformity assessments on July 1, 2016. New tests posted between July 1, 2015 and September 30, 2015 will become required for conformity assessments on April 1, 2016. New tests posted before June 30, 2015 are now required for conformity assessments. All tests posted to modification lists for other versions of the ACATS are included in ACATS 4.0 if appropriate and thus are required for conformity assessment using ACATS 4.0 The effective dates of tests are marked in the lists.

Questions about this document? Contact the ACAA Technical Agent at agent@ada-auth.org.

Technical Corrigendum 1:

Beginning with ACATS Modification List 4.0B, we have included tests for corrections (including new features) in Technical Corrigendum 1, ISO/IEC 8652:2012/Cor 1:2016. All of these changes are corrections to Ada 2012; as such we will assume all such corrections are implemented in Ada 2012 compilers.

ACATS 4.1 Announced

The ACAA has announced its intention to create a new version of the ACATS under the baselining policy. Such a new ACATS version will include new and modified test files that are effective by the official use date of the new ACATS (in this case, July 1, 2016).

ACATS 4.1 will include optional Test Grading tools

ACATS 4.1 will introduce new test grading tools. The new version of Report included in this ACATS Modification List is the first piece of those tools.

When the ACATS was designed (as the ACVC in the early 1980s), the intention always was that running it would give a simple and clear Pass or Fail result. However, grading of tests (particularly of B and L Tests) is somewhat subjective and very time-consuming. (Test grading typically involved poring over compiler listings of the entire ACATS with a large highlighter.) The subjectivity was acceptable in the days of common formal testing, as the formal testers would enforce a common grading standard (even if no one could describe what that was), but in an age of self-testing, it reduces the value of the ACATS.

As such, we're introducing a long-overdue set of automated grading tools. These tools are intended to: