Fire Safety Competency Testing - Industry Improvement

One of the most satisfying outcomes for QPS staff is when we witness industry wide improvement in our client database. One area where we have noticed such improvement is in the area of Fire and Emergency Competency Test results. Although the test has been modified from time to time the Competency Test content has remained essentially the same and requires staff to be knowledgeable on the same processes since 2000.

QPS clients have increased the industry mean score from 77% in 2000 to 91% in 2008. This indicates that more and more aged care staff have retained important emergency procedure knowledge that might one day save lives.

The graph shown below plots the average industry wide scores achieved by QPS clients since 2000. The above graph shows the average % score for all aged care facilities (high and low care combined). It shows that the average compliance score has increased each year since it was first used in 53 facilities in 2000. The reasons for the steady improvement are:

• Facilities doing the test repeatedly almost always improve. Staff members learn from doing the test and do not tend to make the same mistakes in the second and subsequent tests. Facilities are usually able to implement easy and effective improvement opportunities to assist staff that may struggle with certain questions e.g. poster information, information on ID badges, personal tuition of staff with literacy problems.
• The results support the theory that the process of testing staff knowledge supports and enhances the learning process. Clients comment that staff members tend to pay more attention at education if they know they will be tested on the content at some later stage.
• As the number of facilities doing the test increases the percentage of facilities doing the test for the first time decreases. Facilities doing the test for the first time score, on average, lower results.

It is noted that trainers and organisations providing training are aligning their training to the outcomes required by the tests. This reflects the acceptance of the test as meeting the required standards, being up to date, practical and useful. Acknowledgement that the content of the test was originally developed by both the NSW Fire Brigades and Metropolitan Fire Brigade in Victoria and the last version formally reviewed by the NSW Fire Brigades is important.

Discussion with many clients indicates that most facilities are setting themselves achievement targets of 90%+.

Increased Participation = Improvement in Fire Safety at John Paul Village, Heathcote

Whilst the overall compliance result that plots your position on the graph is important the denominators in some indicators can also shed light on achieved improvements. Take for example the results being achieved at John Paul Village located at Heathcote in the very south of Sydney. Over the past three years this facility has made steady improvement in respect of its overall compliance score. However one of the things the benchmarking graph does not show is the improvement being made in getting increased staff participation. This trend is reflected in the denominator scores.

Both the overall benchmarked score and participation rates have increased at JPV over the past three years.

Not satisfied with the overall results or their participation rates John Paul Village embarked on a campaign this year to increase the number of participants.

The above graph shows that the number of participants this year has doubled on previous years and this in itself represents significant improvement. It is also noted that increased participation occurred at the same time that the overall average score improved. Quality Manager Elaine Hall explains that the increased participation rate was due to:

• Changing / improving the delivery of Fire and Safety education through a different style of lecture - a more interactive style seems to have improved retention of information and general fire awareness.
• Improving the way staff are notified of compulsory education sessions - setting dates and times more convenient to staff doing shift work. Staff also have an increased understanding of the need for Fire Education through follow up of those not attending.
• Following the test, staff are given feedback. The QPS scorecards are used to inform staff of performance in each question. Areas which have the poorest results are reinforced.
• The educator has been actively involved in facilitating small groups to work on the questionnaires - this has greatly improved the participation rates as the questionnaires are completed on site.

John Paul Village is setting itself a target of a 90%+ average result next year and to maintain or even increase participation rates.