An Education Program to Increase the Utilisation of Australian NSW Health Paediatric Clinical Practice Guidelines in Rural Emergency Departments

Clinical practice guidelines (CPGs) are evidence-based publications designed to provide direction and recommendations in the assessment and management of patients. They aim to reduce the level of inappropriate practice and improve safety for patients. However, CPG development does not guarantee their utilisation. A study directly investigated the barriers to utilisation in a population group within rural Australia. They found the most common reason for underutilisation were: a lack of awareness of the guideline, a lack of access to the guideline and the belief that there was inadequate education and training in the guidelines. This study evaluates the effectiveness of an evidence-based education program (EP) to increase the utilisation of New South Wales (NSW) Health paediatric CPGs in rural NSW within Australia. This study investigates the effectiveness of an evidence-based multifaceted EP, targeting pre-determined barriers to utilisation, which was subsequently delivered to ten rural emergency departments within the Murrumbidgee Local Health District. Following the intervention, a review of patient records over a three-month period was conducted to determine its effectiveness in change of utilisation. Compliance rates of CPG utilisation was found to significantly improve from 52.1% to 75.8% over 3 months. Furthermore, the evidence-based EP significantly increased CPG compliance, potentially improving the care of sick children in rural NSW Australia.


Introduction
Clinical practice guidelines (CPGs) are evidence-based documents designed to provide direction in the assessment and management of patients and reduce levels of inappropriate practice. [1][2] Although CPGs aim to ensure a universally high level of care, the existence of published guidelines does not guarantee their utilisation. [3] At the time of the study the Australian NSW Department of Health had published 12 paediatric CPGs that aim to standardise care for the most common acute paediatric conditions presenting to emergency departments. [4][5][6][7][8][9][10][11][12][13][14][15] A recent study explored barriers to the use of these CPGs. [16] Their findings indicated that the three main barriers to their utilisation were: 1. A lack of awareness that the guidelines existed, 2. A lack of access to the guidelines, 3. There was inadequate education and training in use of the guidelines. These findings are consistent with several published studies that show poor compliance with CPGs in other settings. [17][18] Grol and Grimshaw highlight that a common findings in research of health services, is the gap that exists between evidence and practice. [19] The impact of this is important where it is possible that patients do not receive the optimal care that could be achieved had guideline adherence been in place. Therefore, it has been suggested that improving compliance has the potential to influence clinical decision-making, and result in a change to clinical practice and better health outcomes. [3] Cohn, Gautam and Preddy identified barriers to compliance with the NSW paediatric CPGs in rural NSW. [16] There have been no studies published that investigated whether an evidence-based education program could improve compliance with multiple CPGs in regional Australia.
Hysong, Best and Pugh suggest that to increase compliance, guidelines must be implemented using a multifaceted, targeted approach. They emphasised the importance of resource allocation for education and adopting an EP to best meet local needs. They further stated that the simultaneous implementation of multiple CPG's is likely to require a range of carefully chosen techniques, including grand rounds, reminders, academic detailing, educational outreach visits and barrier-oriented education programs. [20] The aim of this study was to determine if an evidencebased EP could improve compliance with paediatric CPGs in rural emergency departments.

Ethics Approval
Ethics approval was obtained from the Greater Western Human Research Ethics Committee (Project number LNR/11/GWAHS/53) and the University of New South Wales (Reference number 2011-7-51).

Pre-education Program Audit
An audit was performed of 300 randomly selected medical records from 10 hospitals in the NSW Murrumbidgee Local Health District, including two base hospitals and eight district hospitals. Records were reviewed for children aged less than 16 years who presented between the 1st of January 2009 and 31st of December 2009. Records were selected and included in the audit if the initial assessment of the child described the presentation as consistent with one of 11 clinical conditions addressed by the NSW Health CPGs. The CPG entitled 'Recognition of a Sick Baby or Child in the Emergency Department' was excluded, because it was not designed for the management of a defined diagnosis. [15] Records were excluded if patients were seen only by nursing staff, if there was no clear provisional diagnosis or if they left hospital against medical advice.
A single investigator assessed each case record to determine if medical management was compliant with the relevant CPG. All records were assessed on the basis of 4 criteria: 1) Vital sign documentation.
2) Investigation and initial management.
3) Treatment at time of discharge. 4) Follow-up plan.
A case was considered compliant if at least 3 of these categories were consistent with the CPG.

Delivery of Education Program
An EP was delivered to the same 10 hospitals between May and June 2013. Reminder emails and phone calls were made to each hospital at two and four week intervals. The EP consisted of: 1. CPGs in printed books (summary and full version) including Quick Response codes for each guideline. 2. CPGs in electronic form. 3. CIAP gateway to CPGs as a desktop icon. 4. Desktop stickers reminding users of the CPGs. 5. A4 posters reminding viewers of which specific conditions have a CPG. 6. Information sheet provided to all medical officers working in the emergency department.

Post-education Program Audit
Following the EP, records of all paediatric presentations that were relevant to a CPG were reviewed over a three-month period. This was conducted by a single investigator using the same audit tool as those used in the pre-EP audit.
A comparison was made between compliance rates for each of the 11 clinical CPGs, and between base and district hospitals, for pre-and post-EP audits. Statistical analysis was performed using chi-square tests.

Result
Pre-education program Two hundred and forty records were included in the audit. Sixty records were excluded (30 patients were seen by a triage nurse or nurse practitioner only, 5 left hospital against medical advice and 25 had no clear provisional diagnosis). The frequency of CPG use varied widely according to diagnosis (Table 1). The percentage of cases where management was consistent with the CPGs was highest for head injury (73%) and lowest for a sore ear (38%). There were fewer cases of head injury and seizures, whilst no cases of meningitis were identified. Overall 52.1% of presentations were compliant with the CPGs. One hundred and fifty-four cases were managed at the eight district hospitals, and 86 cases were managed at the two base hospitals. The CPG compliance rate at base hospitals was significantly higher than at the district hospitals (62% vs. 47% respectively, p<0.05) ( Table 2).

Post-education program
Following the EP a total of 1533 cases were considered relevant to one of the CPGs. There were 198 cases excluded (7 left against medical advice and 191 were seen by a triage nurse or nurse practitioner only). 1335 cases were included in the audit. Of these cases, 378 were seen at district hospitals and 957 at base hospitals.
The most common presentations post-EP were fever and abdominal pain (235 and 195 cases respectively) ( Table 1).
The least common presentation was bacterial meningitis with only one case identified, followed by seizures and gastroenteritis (47 and 73 cases respectively). Figure 1 shows the comparison in compliance rates between each of the CPGs pre-and post-EP. There was a statistically significant increase in compliance with NSW CPG's from 52.1% to 75.8% (p <0.01). Of the 11 guidelines, 6 showed a statistically significant increase in compliance post-EP with no significant change for the remaining 5 (Table 1). Following the EP, there was a statistically significant increase in compliance rates for base and district hospitals (p<0.01 and p<0.01, respectively) ( Table 2). Post-EP compliance was significantly higher at base hospitals (82%) compared to district hospitals (60%), p<0.05.

Discussion
CPGs have been developed to assist medical officers to make evidence-based decisions in regard to the most appropriate and resource-efficient health care for specific clinical conditions. It is becoming increasingly evident that there is often a mismatch between guideline development and guideline utilisation. [22][23] This study demonstrates that an evidence-based EP targeted at pre-identified barriers significantly increases CPG compliance. Our results also demonstrate that the increase in multiple CPG's compliance occurred at both base and district hospitals. Overall, the base hospitals showed a significantly higher compliance rate than district hospitals (p < 0.01). Therefore, possibly education surrounding the use of the CPGs should be particularly targeted to district hospitals. Furthermore, our findings are consistent with Hysong, Best and Pugh's hypothesis that a targeted and multifaceted intervention would result in a simultaneous increase in compliance of multiple CPG's.
Overall our study found a statistically significant increase in compliance pre and post-EP (p<0.01). The EP therefore successfully produced short-term beneficial changes in clinical practice, though it cannot be concluded if these changes will remain in the long term.
This work has several limitations. There is a reasonable chance that the audit tool used could over or under estimate the change in compliance. However, by using an equivalent tool for pre and post audits, it demonstrates the specific change within our population group. Under estimation could have occurred where the investigations, management and treatment were consistent with the guideline, however the vital signs and follow up plan were not documented. Vital sign documentation required 3/5 specific vital signs to be documented (temperature, respiratory rate, oxygen saturation, capillary refill and heart rate, however they may not have been specific for each different CPGs. For example, the head injury CPG does not include capillary refill as a required observation, whereas it is recommended to be used to test for the severity of dehydration in the gastroenteritis CPG. [10][11] No standardised audit tool has been produced to assess compliance with the NSW health CPGs. It is also important to consider that a retrospective study performed by looking at medical notes only may be difficult to determine if algorithms are followed correctly or not.
Currently there is limited research demonstrating whether improved compliance with guidelines leads to an improvement in practice and patient outcomes. Recent oncological research has demonstrated that adherence to national cancer network guidelines for treatment of ovarian cancer correlated with improved survival. [24] A further study in 2013 demonstrated a survival benefit for patients with stage III and high risk stage II colon cancer who received treatment that adhered to the guidelines. [25] Whilst this research is not specific for paediatric population groups, it highlights that an improvement with compliance correlates with improved survival.
Additionally, it is generally accepted that there is a high turnover of staff within emergency departments. If the increase in compliance with CPG's is to be maintained, we believe strategies are required for continuing education. It may further be necessary to incorporate the CPG's into the electronic medical record programs, such that when one of the clinical conditions relevant to a NSW Health CPG is entered as the diagnosis, an auto popup tool highlighting the correct management approach is presented. Further research should be conducted to identify the most effective education program that will maintain optimal CPG compliance.

Conclusion
This study shows that an evidence-based education program, directly targeting pre-identified barriers, is able to significantly increase compliance with CPGs. Such education programs have the potential to improve clinical outcomes with more efficient use of limited resources.