Performance of International External Quality Assessment Scheme Participant Laboratories in Ethiopia (2016-2018)

The objective of this study is to evaluate international external quality assessment scheme (IEQAS) program participant laboratories in Ethiopia. Acceptable performance was defined as any proficiency testing (PT) score 80% and above. The retrospective PT database was reviewed and analyzed to assess the testing performance of the participating laboratories and the impact of the program over time. A total of 173, 275 and 334 laboratories participated in the EQA program in 2016, 2017 and 2018 respectively; enrollment increased from 173 laboratories in 2016 to 334 in 2018. As a result, the overall participation rate is 41% whereas failure to participate accounts 52.2% and the rest are due to reagent stock out, equipment down and test suspended during the test event; among the total PT challenges distributed and performed, the rate of scoring ACC were 66.00%, 67.61%, 67.22%, 67.35%, 67.48% and 70.87% in event 2, 3 of 2015, event 1, 2, 3 of 2016 and event 1 of 2017 respectively. Using 95% confidence interval with the hypothesized proportion of ≥ 80% to be acceptable by the ISO standard. One sample proportion test calculation showed the result generated from the laboratory is not significantly meet the standard requirement P value <0.05. The present evaluation conclude that IEQAS program in Ethiopia is not utilized for improvement in the quality of laboratory diagnostic services over time across the six PT events. The laboratory result generated from the Health facilities in Ethiopia is significantly out of the standard.


Introduction
Successful performance in an external quality assessment (EQA) program reflects the effects the effectiveness of laboratory`s quality management and allows for recognition of laboratory quality by external body [1]. External quality assessment (EQA) is a program and it involves running blind patient-like samples, comparing your results to peer results, in order to retrospectively monitor the accuracy of reporting [2]. The objective of EQA is to ensure that test results are reliable and comparable no matter where they are performed. This involves standardization of procedures and establishment of national level diagnostics centers responsible for developing training modules, training, identifying the materials needed to support PT and improving the performance and maintaining the competence of EQA is essential [3].
The main purpose of External Quality Assessment (EQA) and proficiency testing (PT) preparation is to distribute a stable and consistent sample for participant laboratories to analyze the sample. Results are back to the PT program organizer who then evaluates participant laboratory result against the predefined value. The target value may be determined by peer group, a reference laboratory, or may be a consensus value for quality improvement process [4].
A non-commutable PT/EQA sample does not give meaningful information about the relationship of results for patients' samples between different measurement procedures. Reports must be understood and interpreted correctly so as prompt corrective actions and guide quality improvement [5]. Participant Laboratories in Ethiopia (2016Ethiopia ( -2018 In current years the EQA/PT enrollments by public health facility laboratories in Ethiopia is increasing but not evaluated for national participation rate and success rate accordingly so far. So the aim of the study was to review 6 PT events for International external quality assessment scheme (IEQAS) participant laboratories in Ethiopia, to identify major analytical problems identified during the PT participation, over all participation rate and success rate.

Material & Methods
PT panel description-from 2016 to 2018, panels of six PT events distributed nationally by EPHI from OASY. These panels were for 331 public health facility laboratories in different range with minimum of 1 and maximum of 28 PT programs based on their PT program enrollments. The programs were bacterial identification, basic Cardiac markers, basic transfusion medicine, Blood Gas/Electrolytes, Blood Parasites, Cardiac Markers, Cell Morphology, Chemistry/Immunoassay, Genital Culture, Gram Stain, Hematology, Hepatitis Serology, HIV, Human Chorionic Gonadotropin, KOH slides, Lymphocyte Immuno Phenotyping, Mold/Yeast Culture, Mycobacterium Acid Fast Stain, Neisseria Gonorrhoeae Culture, Special Chemistry, Special Immunoassay, Syphilis Serology, Throat Culture, Tumor Markers, Urine Culture, Wet Mount, Yeast Culture. The PT panel was imported from One World Accuracy; Canada to Ethiopian public health institute subsequently dispatch to participant health facilities laboratory through the postal system the notifications, reminders, result submission and feedback access were through online system.

Study Design
Retrospective study was used using data collection excel tool from IEQAS database from Oct 5 to Nov 30/2018 at Ethiopian public health institute Addis Ababa, Ethiopia.

Sampling Method
Systematic sampling was used and a total of 334 health facilities laboratories included; those IEQAS/PT participants during 2016 event 2 through 2018 event 1. Data from IEQAS database was used as and analyzed for the participation rate and success rate and identification analytical problems.

Data Collection Procedure
Retrospective data of six PT events (event 2/2016 to cycle 1 2018) was collected for 28 PT programs from IEQAS database of One world accuracy using pre tested separate excel sheet for qualitative test and quantitative test parameters and the information from feedback was analyzed for PT participation rate, Success rate and major analytical problems identified during PT participation.

Data Quality
Data were collected online from one world accuracy database. Pilot study was conducted using data available online for event 1 of 2016 from same database, and appropriate modification has been made. The quality of data was crosschecked by supervisors against the Original PT feedback for the PT event of each facility accessed online in the web based database system of one world accuracy.

Data Processing and Analysis
Excel summary data of PT feedback were entered on Microsoft Excel spreadsheet and analyzed using IBM SPSS statistics 23. For quantitative analyze graded as ACC (Acceptable) result when the analyzed analytical value reported by the participated laboratory meet the established target value of the PT provider or UNACC (unacceptable) result when the analyzed analytical value reported by the participated laboratory doesn't meet the established target value of the PT provider this is true for both quantitative and qualitative test parameters. For all cycles (i.e. one cycle or an entire six cycles), the proportion of acceptable responses were calculated and the participation rate as well as the identified analytical problem during PT participation were analyzed accordingly

Result
The number of international EQA/PT participant facilities were 173,275 and 331 during the 2 nd and 3 rd PT event of 2016, 1 st , 2 nd and 3 rd PT event of 2017 and 1 st PT event of 2018 respectively; the total of PT program was 28 and the number of PT challenges distributed during 6 PT events for the 28 PT program was 298417.
From the total of 298417 PT challenges distributed to all international EQA/PT program enrolled health facility laboratories throughout the country 208353 (69.8%) were distributed to participant hospitals while 58541 (19.6%), 16569 (5.6%), 9726 (3.3%), 3449 (1.2%) and 1779 (0.6%) were distributed to Health centers, Regional laboratories, Diagnostic Laboratories, National Laboratory and blood bank respectively. (Table 1) The number of health facilities participated in each PT program showed that the relatively large participant for Lymphocytes immunoassay, HIV, Chemistry, wet mount while culture tests are generally small number of participants compared to other test programs.  able to meet 80% and above score among the total of 27 PT programs. The regional distribution of EQA/PT challenges in the country showed that the majority is in Addis Ababa which cover 25.6% Oromia, Amhara, SNNP, Tigri regions were 25.2%, 12.3%, 10.4% and 5.7% respectively and the least is Gambella region which is 0.8%Trend of PT performance score by Region health facilities which is proportional compared to the number of health facilities available in the respective regions.
The participation rate throughout the six cycles among totally distributed 298417 (41%) PT challenges 121975 PT challenges were performed by participant laboratories while 176442 (59%) PT challenges were not performed due to different problems. From the total of 298417 PT challenges distributed to participant laboratories throughout of 6 PT events 176442 (51%) of PT challenges were not tested due to different reasons; out of the identified problems the majority of the challenge was failure to participate which account 156935 (52.6%) of the total PT challenges and 3.4%, 2.3% and0.6% of PT challenges were not performed due to Reagent unavailability, Analyzer out of services and Testing suspended during test event respectively. The figure below showed that the identified problem for PT participation across each PT programs and there are only 3 PT programs (Hepatitis serology, Mould/yeast culture and Syphilis serology) were performed for 80% and above PT challenges without identified analytical problem. The figure below showed that the analytical problem rate is almost similar throughout the participant regions which showed us all regions participation among the total PT challenges were less than 60% due to different identified analytical problems. The diagram below showed that the ACC and UNACC rate among different peers by instruments type for the different PT program of those participated without identified analytical problem. The number of instrument type peer group for clinical chemistry was 38 while 31 for hematology and CD4, Blood Gas, Tumor marker has 7 instrument peers for each and 4 for cardiac marker.

Performance Rate
In this study the average PT performance the 6 PT events for 28 PT programs was 67.75% success rate even though significant improvement is not observed through the trend of PT event which among participated facilities those graded as ACC or UNACC which account 97507 (79.9%) among the total PT challenges distributed and the success rate or the rate of scoring ACC were 66.00%, 67.61%, 67.22%, 67.35%, 67.48% and 70.87% which showed relatively good performance compared to the study conducted among 12 government hospitals laboratories in Addis Ababa, who had been participated in proficiency tasting schemes for 20 clinical laboratories test parameters in 6 cycles from 2012 to 2013 G. C. in over 6 cycle showed that those meet the analytical performance score of 80 and above is 40.42%, 26.67%, 26.67%, 51.67%, 37.09%, 38.37% this indicates no improvement in quality through time [9]. This study was comparable with the study conducted in Nigeria between 2009 and 2011 [7]. Four rounds of pre-characterized plasma panels for HIV, sputum films for tuberculosis and blood films for malaria were distributed quarterly scores increased from 74% to 95% from the first round to the third round, but decreased in the fourth round for HIV PT and average scores increased from 42% in the first round to 78% in the second round; but a decrease to 34% was observed in the fourth round for Tuberculosis PT while Malaria PT performance was 2% at first, but average scores increased between the second and fourth rounds, culminating in a fourth-round score of 39% which showed low performance compared to the study conducted on quality assurance for point of care testing among different heath facilities [11]. Many participants requested training and mentoring. Another study conducted in Haiti that evaluate the EQA program for HIV testing between 2006 & 2011 showed the significant improvement (P < 0.05) over the trend of PT participation which 70% success in 2006 and become 97.5% success in 2011 [6]. Another study conducted in Taiwan and the Asian Pacific region four years' experience on Hb A1C PT showed that 83% of the participant laboratory was passed in 2005 and the success increased to 93% in 2008 [8]. Another comparative study conducted by WHO-Afro between 2002-2010 within nine survey period indicate that 98.9% of average score of HIV serology PT over 30 countries among 49 laboratories frequency of accurate detection were ranging between 93% to 100% [10].

Participation Rate
According to this study the overall participation rate was 121975 (40.9%) throughout the 6 PT events while 176442 (59.1%) of PT challenges distributed were remaining with different challenges among identified analytical problems during PT participation 52%, 3.4%, 2.3% and 0.6% of PT challenges were failure to participate, test not performed due to reagent unavailability, analyzer out of services and testing suspended during test event respectively. This study was comparative with the study conducted in Addis Ababa showed the PT feedback result indicated that the major problems were failure to participate; test suspended during the test event, equipment failure and below linear limit of the analyte were 54.4%, 2.5%, 1.03% and 0.04% respectively. This study is incomparable with Responses from Pima CD4 sites participating in One World Accuracy test in 2015 event 2 in Ethiopia which was 63 respondent among 84 participant facilities. Another incomparable study finding conducted in Nigerian clinical laboratories for HIV, Malaria and Tuberculosis PT along four rounds also showed 10% failed to return results on one or more PT samples.

Conclusion
Proficiency test score of 66.00%, 67.61%, 67.22%, 67.35%, 67.48% and 70.87% in event 2, 3 of 2015, event 1, 2, 3 of 2016 and event 1 of 2017 was performed by the participant laboratories respectively, since laboratories on average scored a PT score of 67.75% which is below acceptable 80%. From the total of 298417 PT challenges distributed to participant laboratories throughout of 6 PT events 176442 (51%) of PT challenges were not tested. The laboratory result generated from the Health facilities in Ethiopia is significantly far away from the standard requirement and the service is highly interrupted due to different reasons. The substandard laboratory services may affect the patient care.

Recommendation
The laboratory quality should be strongly monitored with identified indicators and the laboratory should work on quality improvement based on feedback from mentors or PT providers. Science the major problem identified were failure to participate which is concluded as service interruption but not reasoned out further study is needed to identify the root cause for service interruption. The National laboratory should strongly follow and provide facility level assistance in case of major PT failure.