Home » SSTDocumentation » SST Element Maturity Level ("EML")

SST Element Maturity Level (“EML”)

Elements in SST are categorized by a maturity level which describes their development state and level of testing. It is important to note that these levels communicate the confidence that the SST project and developers have in individual element and models whilst at the same time reflecting the leading edge nature of the elements being added to the toolkit. It is the responsibility of the element developer to provide and maintain the associated documentation and tests for each level.

There are five levels of readiness used for elements in the SST project:

Level 0 - Sandbox - this element is a unique “one-off” or specialized implementation which is not intended to be part of a longer term SST element or part of on-going research. Each of these elements exists on a branch of the SST trunk and will not be merged into future releases, test procedures or documentation. There are no requirements on these elements other than they must not commit source or modifications back into the main SST code base, elements at Level 1 or above may also not depend on code or elements at Level 0.
Level 1 - Development - the element is being actively developed. APIs and behavior of the elements is very likely to change and should therefore not be considered stable. The element will have very limited testing and is likely to produce unreliable or unreproducible results. At this stage very limited if any verification or validation activities will have taken place and the element is unlikely to provide accurate behavior. Only limited, if any, documentation will be available for the element and users may need to consult the element source for comments or descriptions of the behavior. Development-grade elements are not required to provide testing facilities although they are encouraged to do so, specifically, these elements are not required to be included in SST project overnight builds. All elements at Level 1 and above are required to be compliant with the [wiki:DeveloperCodingStandards SST Coding Standards].
Level 2 - Prototype - the major structuring and development around the element has been completed and minor changes are now being made. Whilst the element’s API and interaction with other elements is mostly fixed by this stage, additional tweaks and modifications may be made to improve performance or help improve element verification or validation results. elements at the Prototype stage will pass initial functionality and correct compile/build tests included in the overnight project builds. At prototype stage limited documentation for the element will be available but will include a statement of the element’s behavior and a list of the parameters which can be used for SDL configuration.
Level 3 - Beta - the development of the element is now almost complete. Element functionality and integration with external SST elements has been defined and is considered stable. During the Beta-phase SST developers will typically be improving the verification of the elements behavior and, where appropriate, may be conducting validation studies to show confidence in the simulation results. Beta-grade elements always compile and are expected to have functionality and performance tests integrated into the overnight project build. Documentation for Beta elements will include a description of how to ensure the element is built during the SST configure, compile and build phases, references to any external dependencies and how these are integrated into the build, a full list of element SDL parameters and an example SDL file for inclusion in SST releases.
Level 4 - Production - elements at the production grade include extensive functionality and performance tests. Users can expect a high degree of confidence in simulation results using production elements with reference studies typically appearing in academic publications or laboratory technical reports. These elements have extensive functionality and performance tests integrated into the SST overnight build systems and, where applicable, the element may also have verification or correctness tests. Each production element will have fully documented SDL parameter information, a description of the element behavior, full documentation of any build procedures including external dependencies and documentation of interactions with other SST elements.
Level 5 - End of Life - elements which reach this level are having only minor bug fixes applied to continue their use. There will be no new features added to the element and existing failures in verification or validation tests are not expected to be addressed. Overnight testing and release check procedures continue to be applied meaning the element is still routinely tested.

After elements have been in the Level 5 state they are subject to be retired and are not expected to be included in any future SST releases or overnight build processes.