H ow to measure and articulate the impact that voluntary consensus standards can have on public health and safety is the focus of a new white paper from the American National Standards Institute (ANSI) and UL Standards & Engagement (ULSE). Principles for Measuring the Impact of Voluntary Consensus Standards on Human Health and Safety was developed by a working group that included representatives of U.S. federal government agencies, standards developing organizations, and others.
Voluntary consensus standards for health and safety are diverse in their scope, technical approach, industry, and user base. This variation between standards imposes limits on the ability to specify one single approach to measuring the impact of standards on health and safety. The working group focused on a set of concepts, frameworks, and considerations that may be used to develop a specific approach based on the standard being assessed and other variables such as the availability of data. The resulting document provides a starting point for an impact assessment.
“Much has been written about the economic benefits of standards,” said Joe Bhatia, president and CEO, ANSI. “What sets this effort apart is its focus on calculating the health and safety benefits. Bringing together the community of interested stakeholders to tackle such issues is something that ANSI can always be counted on to do.”
Voluntary Standards: Effectiveness And Conformance
A key finding of the paper is that impact is the combination of two critical elements: conformance and effectiveness. Effectiveness represents how successfully the requirements in the standard, as written, would address or mitigate risks to human health and safety such as illness and injury. Conformance is a composite factor characterizing both the degree to which the standard is adopted or how broadly the standard is used and the degree of adherence to the standard or how precisely the standard is followed.
The impact of a standard will change over time. Conformance occurs over time with the diffusion of the standard across the industries or sectors it affects. Products, systems, components, processes, and people subject to the scope of the standard may pre-date the requirements and thus reduce overall conformance. Assessing the impact of a standard requires a structured framework including stakeholder engagement, planning, measurement, and analysis.
Data gathering or measurement can be particularly challenging in the impact assessment process. Direct measurement of injury or illness associated with the specific requirements of a standard is extremely difficult due to the limitations of existing data sources. The use of incident rates is preferred to incident counts, but data for normalization can also be difficult to obtain. Confounding factors must be addressed to establish the relationship between the change in baseline with the measured outcome, or false conclusions may be drawn. Because direct outcome data are difficult to obtain, proxy data such as the economic costs of injury may be useful to estimate the impact of a standard. Additionally, leading indicators of impact may be helpful in the qualitative assessment of impact. Leading indicators may include data on the adoption of the standard.
Topics covered in the white paper include:
- Existing efforts by the standardization community to measure the impact of standards, and the methodologies used
- The goals of different stakeholders for measuring impact
- Frameworks for conducting an impact assessment, including the different types of standards, and the importance of the voluntary standards adoption process
- The components of measuring impact, including effectiveness and conformance models
Building Performance Standards: A Key To Reducing Emissions
ACEEE looks at the expanding role of building performance standards as a key strategy to reduce emissions from existing buildings. Read more…
- The relationship between the effectiveness of a standard and conformance to a standard in achieving impact
- Approaches to measuring the impact of standards, including challenges and quantifiers
- Collecting and selecting data for measuring outcomes, with a discussion of assessment challenges, including confounding factors
“The benefits of measuring the impact of voluntary consensus standards on human health and safety are significant,” said David Wroth, director of technology and systems, ULSE. “This tool allows us to take a holistic approach to standards including necessary resources and investments, portfolio management and effective communication among stakeholders.”