Benchmarking Safety

New Brunswick is taking a new approach to evaluating workplace safety across the province with WorkSafeNB’s adoption of the Organizational Performance Metric (OPM) questionnaire.

The eight-question survey was developed in 2009 by Toronto’s Institute for Work and Health (IWH) and other stakeholders to measure leading indicators of occupational health and safety performance. The OPM questionnaire was one of six surveys that WorkSafeNB came across when studying similar surveys across the country in an effort to develop a new workplace safety-measurement tool.

Since 2001, the province had been using its own survey to assess health and safety practices. With 22 basic questions covering five fundamentals to be administered to all employees, supervisors and managers in an organization, “that survey was too complex and cumbersome,” says Anne Lise Albert, assistant director of program development and evaluation at WorkSafeNB in Saint John.

As a result, WorkSafeNB came up with an 18-item questionnaire called the Internal Responsibility System Questionnaire (IRSQ), which WorkSafeNB asked the IWH to evaluate. The two organizations collaborated on a validation study that saw employers complete both the IRSQ and the OPM.

Based on the results of nearly 600 surveys completed by 330 firms, it turned out that among all sectors, results of the eight-question OPM correlated very closely with the respective firms’ historical claim rates — meaning that higher scores coincided with lower compensation claim rates and lower scores with higher claim rates, including both lost-time and no-lost-time claims — whereas the IRSQ results generally did not. “It takes barely any time at all to complete the survey,” Albert says of the OPM’s brevity. “Eight questions is beautiful.”

Dr. Benjamin C. Amick III, the IWH’s scientific director who helped develop the survey, says it is not clear why only the OPM tracked well with compensation claims. “The fact that we see that in two provinces is, of course, encouraging. It has good reliability.” He adds the OPM was intended not as an audit tool, but to aid decision makers within safety groups. “It’s a thermometer,” Dr. Amick says. “It’s not going to tell you what to do, but what you need to be thinking about doing.”

HEADS UP

New Brunswick is not the only province where safety authorities see the OPM as an applicable safety measurement tool. Dr. Amick says British Columbia is considering adopting it, while Manitoba, Nova Scotia, Alberta and even the United States have expressed interest.

While no province in Canada has yet established the OPM as a universal tool, WorkSafeNB plans to use it in its Focus Firm Initiatives, which assign consultants to work with firms struggling with safety issues. The organization has also added the OPM questions to its annual client survey, which evaluates safety culture in workplaces.

Albert says the OPM can help WorkSafeNB assess the effectiveness of its programs or services when it intervenes with a company in the future. Apart from indicating whether certain interventions result in safety improvements, “a lack of improvement might lead us to modify a program or strategy.”

The IWH is conducting further research on the OPM. “We’re doing that science now. And we also hope that what people say and how they score actually reflects what’s going on in their workplace,” Dr. Amick says.

Chris McKean, manager of research, education and specialty consulting with the Infrastructure Health and Safety Association in Mississauga, Ontario and a member of the team that created the OPM, points out a few patterns that the team observed during the survey’s testing phase, in which data was collected in three ways: in person one-on-one, over the phone or in group meetings.

“The ones over the phone tended to score the highest, whereas if you had people in a public meeting, they actually scored a little lower,” McKean reveals. “It’s hard to explain that bias. Maybe in a group meeting, people feel that because they’re not being singled out, they can be a little more honest about it.”

McKean adds that the highest scorers tended to have more senior health and safety professionals answering the questions. “If I’m a health and safety manager or coordinator, I may answer higher, as opposed to a supervisor or worker who has different roles from health and safety. They don’t see it on a daily basis, so they may score lower.” But overall, he says the team found the survey responses authentic.

One of the employers who completed the OPM survey during its initial testing phase was David Hannon, health, safety and environment manager with electrical-utility contractor The K-Line Group of Companies in Stouffville, Ontario. Hannon thinks that the questionnaire would work well as a tool for auditing or evaluating workplace safety.

But he cautions the survey’s usefulness may be predicated on the people filling it out and how much time they are willing to spare. “I know sometimes people put them on the back burner, and they’ll start them and not finish them, because they don’t allot enough time in the day to do it.”

Dr. Amick acknowledges that involving as many people as possible when measuring an organization’s safety climate is ideal if time and resources permit. “But the advantage to the OPM might be that it’s short, and you may be able to give it to six or seven people if you know who the right people are and to get to triangulate to a pretty reasonable approximation of what’s going on in the organization.”

The IWH is currently using the OPM in another research project. “I think that several health and safety associations have considered it for different types of uses,” Dr. Amick says, “but they respect the need to move the science forward just a teeny bit more before they go ahead.”

Although WorkSafeNB initially preferred the IRSQ, Albert is not losing any sleep wondering why it did not turn out to be as reliable as the OPM. “We
wanted a survey that works. The OPM works in New Brunswick, so we’re going to move forward with it.”

Follow us on Twitter @OHSCanada

Jeff Cottrill is editorial assistant of OHS Canada.