Point of Care Blood Gas Analyser Verification at the Largest Academic Hospital in Southern Hemisphere Revived by Coronavirus Calls for Tests to Be Verified

Introduction: At no point in the history of medicine has the importance of point-of-care testing (POCT) devices, tests and methods verification and utilisation been recognised and accepted than during the coronavirus pandemic. Blood gas analysers offer the best form of POCT and this has been demonstrated in the management of coronavirus patients admitted to hospital intensive care units with respiratory challenges. The new analysers were evaluated in compliance with the Clinical Laboratory Improvement Amendments (CLIA) regulation. The new method must be evaluated by comparison to the central laboratory or the outgoing analyser. The study was undertaken to verify if the new instrument, OMNI S b221 matches the wellestablished analysers currently in use at the hospital. Objectives: To evaluate method performance of pH, the partial pressure of carbon dioxide (pCO2), the partial pressure of oxygen (pO2), glucose, lactate, and electrolytes on OMNI S b221 blood gas analyser. Materials and Methods: To analyse the method performance of the pH, pO2, pCO2, glucose, lactate and electrolytes (Na + , K + Ca ++ ) on OMNI S b221 blood gas analyser 40 samples run on OMNI S b221 blood gas analyser were compared with the results obtained from ABL700, GEM PREMIER for all the parameters and the Roche Hitachi 917 analyser to compare two electrolytes (Na + and K + ). The correlation coefficient was calculated for the results obtained from each paired set of instruments (ABL 700 vs GEM PREMIER and ABL 700 vs OMNI S b221). Comparability between analyser methods performance was determined using linear regression analysis. Results: The correlation coefficient between the paired analytical platforms were all between 0.81 and 0.99, demonstrating a strong linear relationship. The coefficient of variation for all the parameters were less than 2 also demonstrating good precision. Conclusion: OMNI S b221 blood gas analyser correlated well with the outgoing ABL 700, GEM PREMIER blood gas analyser and the central laboratory analyser, Roche Hitachi 917 and therefore, verified as a good platform to run arterial blood gas samples.


Introduction
This is one of the earliest if not the first POCT verification study conducted in Africa which was not accorded its rightful place in medical research history. The importance of POCT and the need for verification of POCT devices have been highlighted by the coronavirus pandemic. The establishment of The National Health Laboratory Service (NHLS) by the government of South Africa to provide medical laboratory service to all public health institutions was aimed at standardising laboratory medicine across the whole country. At Chris Hani Baragwanath Academic Hospital (CHBAH) Chemical Pathology Laboratory took over the management of blood gas point of care services which differed between wards. Challenges were encountered when patients were transferred from one ward to another with different platforms. Therefore, a standardised and centrally managed system would be beneficial. This resulted in the replacement and addition of more blood gas analysers. A verification study was undertaken to ensure that the newly acquired blood gas analysers performed according to the manufacturer specifications.
Verification is defined as a process to determine the performance characteristics of a test before running patient samples. This exercise is carried out to demonstrate that precision, accuracy, linearity, reportable range, reference range and other analytical parameters fulfil the manufacturer specifications when applied by the laboratory staff under local conditions [1]. Verification is typically making sure that laboratory has objective evidence that tested the manufacturer's claims for their method performance characteristics, under the testing site environmental conditions like temperature, humidity, water, electricity, operator skills.
It is often incorrectly termed as 'validation', a process that is carried out by the manufacturer to establish the performance criteria and limits. The introduction of a new method or new instrument to replace existing instruments should be done following specified protocols before its routine use in the laboratory [2].
Blood gas analysers incorporating electrolytes and cooximetry are now a common feature in critical and intensive care units [3]. An error in medicine can have adverse effects on patient management and can be fatal for critically ill patients. It is mandatory to verify the performance of POCT devices not only because they are located in critical care units but as a requirement for all medical devices. This is a requirement by ISO 15189 [4] guideline which is also stipulated by the South African Health Products Regularity Authority [5].
POCT has been defined as "medical diagnostic testing at or near the point of care to facilitate immediate delivery of results and alteration in patient care" [6]. The pH and blood gases have the greatest impact on immediate patient management, therefore verifications must be done to ensure the quality of reports they generate [7].
As part of an effort to standardise maintenance and quality management of blood gas analysis throughout the hospital, Chemical Pathology initially placed 13 in 2007, increasing the number to 17 in 2018 including 2 at a district hospital inside South Western Township (SOWETO). Before this project, different wards and units in the hospital used different blood gas analysers from different suppliers, maintained and managed by different individuals with different protocols.
The verification process for the 13 analysers required the participation of all stakeholders. These stakeholders are The National Health Laboratory Service (NHLS), Department of Health (DOH), the hospital management and analyser supplier. The aim was to verify the methods employed by the OMNI S b221 blood gas series. The ABL 700 series was the existing blood gas reference analyser in the hospital wards and units GEM PREMIER 3000 in the Emergency laboratory was the additional out-going analyser. The OMNI S b221results were compared with these two analysers ABL 700 and GEM PREMIER 3000. POCT results especially the electrolytes were confirmed by the central laboratory. Aim The aim was to verify the manufacturer's analytical performance of the OMNI S b221 blood gas analyser in terms of accuracy, precision and linearity.
Objective The objective of this project was to assess the performance of the new blood gas analyser OMNI S b221 in the measurements of parameters: pH, pCO 2 , pO 2 , sodium (Na + ), potassium (K + ), calcium (Ca ++ ), glucose and lactate by: i) Comparing the ABL 700 to the GEM PREMIER 3000 results ii) Comparing the ABL 700 to the OMNI S b221 results ii) Comparing the Central laboratory Roche Hitachi 917 to the OMNI S b221 electrolytes (Na + and K + ) results.

Design and Methods
Blood Gas Analyser evaluation and a comparative study conducted between 2006 and 2007 at CHBAH.
Inclusion criteria Samples that were adequate to be run on each pair of analysers, ABL 700 versus GEM PREMIER 3000 and ABL 700 versus Cobas OMNI S b221. Samples without air bubbles were included in the study.
Exclusion criteria Samples with the opposite criteria for inclusion such as insufficient for paired analysis, samples with air bubbles and clotted samples were excluded from the study.
Study site Chris Hani Baragwanath Academic Hospital (CHBAH) is officially documented as the 3rd largest hospital in the world, with 429 buildings and ground around 173 acres with 3200 beds. [8]. Sample collection All the samples were anonymised. Arterial blood samples in heparinised syringes were used in this study. For blood gas analysis, the same sample was run on the paired blood gas analysers. Electrolytes comparison was between plasma on a central laboratory analyser Hitachi 917 and whole blood on a blood gas analyser. The only way of ensuring that the same sample was run twice was to transfer the sample from the syringe after running the blood gas to a tube for centrifugation. Vacutainer barrel transfer with the new tube in the barrel was tried but the K + results were higher than those from the blood gas analyser. This method was abandoned and tried gently pulling out the plunger and transferred the blood through the wider end of the syringe. After noticing minimal differences in K + results, this method, unconventional as it was became the prefered method of whole blood sample transfer from the syringe to tubes for immediate centrifugation in the central laboratory. Therefore for electrolyte comparison between ABG the analyser and Hitachi 917, samples were sent to the central laboratory for centrifugation within 10 minutes.
Analytical protocol The main difference between the analysers was the solution containers and to highlight this difference, the terms 'cartridge and solution' based system were coined. The cartridge-based analysers were those made up of a measuring sensor cassette contained in a multi-use disposable sealed cassette assembly and a sealed solution pack containing foil pouches each filled with calibration solution used for calibration and quality control of all analytes, sample flushing solution and a waste receptacle pouch. Therefore, both the solution pack and sensor cassette can neither be reused, refilled nor transferrable from one instrument to another.
The solution-based system has separate dedicated individual reusable and refillable solution containers and does not require proper shut down during electricity downtimes. All the solutions are reusable regardless of the length of the downtimes before the expiry date. The solutions can be refilled and use the volume adjustment function to register the new volume. This feature is important when using buffer stock while awaiting stock delivery, especially for high volume units.
The decision to use a solution-based system was informed by the intermittent power supply to the hospital due to several factors. During such downtimes, the nursing staff focus on the patients and forget to shut down the blood gas analysers and on the other hand, the POCT Technician on call is unable to institute a proper standby or shutdown mode because of the long distance between the wards and units. Remote monitoring with aspects of troubleshooting was considered as vital for a hospital with wards spread over such a vast area. The OMNI S b221 was chosen because it provided remote control through the OMNI LINK. The analyser performs system calibration every 12 hours or 24 hours for 11 minutes, one point calibrations every 30 minutes or 1 hour for 1.6 minutes and two (2) calibrations every 6 hours or 12 hours for 6.2 minutes.
The ABL 700 was initially compared with the GEM PREMIER 3000 and then the ABL 700 was compared with the new OMNI S b221. For the comparative study between each (ABL700 vs GEM PREMIER 3000) and ABL700 vs OMNI S b221), samples in heparinised syringes from the wards with arterial blood gas (ABG) analysers were used. A steering CHBAH POCT Team was formed with some modifications based on the 2007 Johns Hopkins Bayview Medical Center [9]. The principle of analysis on the blood gas analysers for the selected analytes is the same for all the three analysers ABL 700, GEM PREMIER 3000 and OMNI S b221. However, the principle of analysis for electrolytes Na + and K + between the blood gas analysers and the central laboratory analyser Hitachi 917 are different. Blood gas analysers use utilise the direct ion-selective compared to the indirect ion-selective electrode on the Hitachi 917 [10]. These two methods are often used in our laboratory as part of a protocol when investigating suspected pseudohyponatraemia.
POCT Team members (Medical Laboratory Technicians) were allocated sites/units/wards and tasked with running the samples that were brought by the clinical staff. They were also responsible for coordinating with the internal NHLS foot couriers (messengers) supervisor for immediate collection and delivery of samples to the laboratory. The routine laboratory supervisor and departmental quality control officer coordinated the ABG and central laboratory electrolytes (Na + and K + ) analyses, the overall quality management of the project, the final compilation of the results and analysis. The Emergency (STAT) laboratory supervisor coordinated all the sites and liaised with the analyser suppliers. Samples were run on the ABL 700 as the outgoing analyser and as soon as the sample inlet flap was closed (less than 5 seconds) the sample was run on the second analyser. The time interval between the two any paired analysers (ABL 700 vs GEM PREMIER 3000 or ABL 700 vs OMNI Sb221) was negligible because they were installed side by side.
In an evaluation exercise, sample transfer from the heparinised syringe to plasma separator tube (PST) served the purpose to avoid venesection of the patient twice. Transfer from syringes to tubes through the wider part of the syringe (back part after pulling out the plunger and allow the sample to flow from the syringe into the tube to reduces the level of haemolysis compared to transfer by pushing the sample through the syringe nozzle).
Quality Control The precision of evaluation of the OMNI S b221 was performed using three control levels. The controls were programmed for measurements at regular time intervals and plotted on a Levey-Jennings chart during each of the four blood gas rounds. POCT in Critical Care Units cannot afford to accept any violation and therefore, full troubleshooting after 1 2 s warning rule was conducted. The OMNI S b221 analysers were enrolled on the NHLS external quality assessment (EQA) programme.
Statistical analysis All data was retrieved/exported using a flash drive and downloaded onto a spreadsheet for analysis. EP Evaluator was used in performing the statistical analysis, which included a correlation coefficient, mean bias (MB), percentage bias [11], and Miniwebtool was used for the coefficient of variation [12]. Correlation between the analysers was done using the Scatter plot. Deming regression was used because it accounts for errors in observations on both the x-and the y-axis. MB was used to assess the overall bias error or systematic error. Percentage bias was used to assess the difference between the paired analysers ABL 700 vs OMNI S b221 and ABL 700 vs GEM PREMIER 3000.

Results
The results showing correlation coefficient, bias and coefficient of variation for each set of paired instruments are given in Table 1 and Table 2 below. The correlation coefficient between the paired analytical platforms was all between 0.81 and 0.99, demonstrating a strong correlation. Figures 1A to 10 are showing the regression analysis and Scatter plots analysis for comparison of different analysers (pH, pO 2 , pCO 2 , Na + , K + , Ca ++ , Glucose and Lactate).
The coefficient of variation (CV) for all the parameters was less than 2 also demonstrating a good precision. Quantitative method comparison results represented in Scatter plot, bias and percentage bias are shown (for ABL 700 vs GEM P 3000 and ABL 700 vs OMNI S b221) in Figures 1A and 1B Figure 9 and Figures 10. pH Regression analyses for each pair, the ABL and the GEM P 3000 and ABL 700 and the OMNI S b221 were 0.9323 and 0.9216, respectively. There was a positive bias (0.008) between ABL 700 and GEM PREMIER 3000 and evident positive trend as shown in figure 1A. However, when ABL 700 and OMNI S b221 are compared, the bias is negative (-0.002). There was no significant bias between the 3 analysers. The repeatability assessment was within the acceptable CV at 0.786 and 0.7519, respectively. All the analyser performances measured values are shown in Table  1.
PO 2 Regression analyses for each pair, the ABL 700 and the GEM P 3000 and ABL 700 and the OMNI S b221 were 0.9021 and 0.9644, respectively. The analysis showed a negative bias (-11.38) between ABL 700 and GEM PREMIER 3000 and also a negative bias (-3.34) for the ABL 700 and OMNI S b221; with the negative trend, as shown in both figures 2A and 2B. However, precisions of 0.7231 and 0.6944 were within the acceptable limit. All the analyser performances measured values are shown in Table 1.

PCO 2
Regression analyses for each pair, ABL 700 vs GEM P 3000 and ABL 700 vs OMNI S b221 were 0.9839 and 0.9747, respectively. There was no significant bias on the paired analysers; ABL 700 vs GEM P 3000 (-1.43) and ABL 700 vs OMNI S b221 (0.50). The coefficient of variation for each pair was also within the acceptable limits at 0.8199 and 0.8077. All the analyser performances measured values are shown in Table 1.
Na + Regression analyses for each pair, the ABL 700 vs GEM P 3000 and ABL 700 vs OMNI S b221 were 0.8515 and 0.9164, respectively. There was no significant bias on the paired analysers; ABL 700 vs GEM P 3000 (-0.60) and ABL 700 vs OMNI S b221 (1.05). The coefficients of variation were also within the acceptable limits at 0.0596 and 0.1726, respectively. All the analyser performances measured values are shown in Table 1.  K + Regression analyses for each pair, ABL 700 vs GEM P 3000 and ABL 700 vs OMNI S b221 were 0.8343 and 0.9134, respectively. There was no significant bias on the paired analysers; the ABL 700 vs GEM P 3000 (-0.45) and the ABL 700 vs OMNI S b221 (0.01). The coefficients of variation were also within the acceptable limits at 0.5324 and 0.4792. All the analyser performances measured values are shown in Table 1.
Ca ++ Regression analyses for each pair, ABL 700 vs GEM P 3000 and ABL 700 vs OMNI S b221 were 0.9414 and 0.9573, respectively. There was a negative bias but not significant on the paired analysers; the ABL 700 vs GEM P 3000 (-0.069) and the ABL 700 vs OMNI S b221 (-0.027). The coefficients of variation were within the acceptable limit at 1.946 and 1.692. All the analyser performances measured values are shown in Table 1

Glucose
Regression analyses for the ABL 700 vs GEM P 3000 and the ABL 700 vs OMNI S b221 were 0.9930 and 0.9944, respectively. There was an insignificant positive bias on each of the paired analysers; the ABL 700 vs GEM P 3000 (0.34) and the ABL 700 vs OMNI S b221 (0.13). The coefficients of variation were also within the acceptable limits at 0.6121 and 0.7412 for each pair. All the analyser performances measured values are shown in Table 1.

Lactate
Regression analyses for each pair, the ABL 700 vs GEM P 3000 and the ABL 700 vs OMNI S b221 were 0.881 and 0.8361, respectively. There was no significant bias on the paired analysers; the ABL 700 vs GEM P 3000 (-0.594) and the ABL 700 vs OMNI S b221 (0.546). The coefficients of variation were also within the acceptable limits at 1.751 and 1.461 for each pair. All the analyser performances measured values are shown in Table 1. Na + Regression analysis for each pair Roche Hitachi 917 vs OMNI S b221 was 0.8144. There was no significant bias on the paired analysers; Roche Hitachi 917vs Roche Cobas OMNI S (-4.0) The coefficients of variation was within the acceptable limits at 0.0599. The analyser performances measured values are shown in Table 2.

Discussion
The development of blood gas analysis by Poul Astrup during devastating polio the 1952 Copenhagen poliomyelitis epidemic brought about a revolution and demonstrated the role of Chemical Pathology processes and participation in epidemics [14]. The ABG analyser brought huge benefits in clinical decision-making by reducing laboratory diagnostic turnaround time. This is currently being applied in coronavirus (COVID-19) positive patients admitted to ICU and those requiring ventilation. Laboratory method performance verification processes are important and therefore, accrediting and standards monitoring organisations expect medical diagnostic laboratories to verify the manufacturer published method performance. Therefore, laboratories are responsible for conducting the verification of test methods used in their laboratories [15].
POCT offers improved turnaround time for test results and therapeutic action. The immediate availability of results might not translate to optimum patient management if the POCT instrument is not fully verified because instrument and method evaluation, is the available scientific and objective way to deem a method/analyser as fit for purpose [16].
These results compare very well with similar studies conducted much later in other countries [16,17]. The good comparison between the results in this project and those conducted in other countries confirms the application of good laboratory practise at the hospital. The correlation of Na+ and K+ between OMNI S (whole blood) and Roche Hitachi 917 (plasma) was extended after encountering delays between analysis as a result of the need to transfer the samples from the syringe to tubes for centrifugation. The strong correlation between the central laboratory and POCT ABG results was achieved after excluding the first set of electrolytes results that were obtained before the separation and testing times were optimised. The correlation coefficient (r) for the electrolytes between the Hitachi 917 and OMNI S b221 was Na+=0.917 and K+=0.979.
POCT and arterial blood gas verification exercises similar to this exercise were conducted at Pondicherry Institute of Medical Sciences in India [15], Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo in Brazil [16] and Clermont-Ferrand University Teaching Hospital in France [17].
Therefore, three important reasons for reporting this project are i) it was conducted much earlier than other published reports; ii) it was conducted at the third largest hospital in the world and produced similar results to those conducted later iii) it appears to be the only documented activity from the African region. This demonstrates good laboratory practices in Africa and other developing countries.

Conclusion
The verification of blood gas analysers at a large hospital demonstrated that the differences between results obtained from the three instruments were not statistically and clinically significant. Therefore, it can be concluded that the new ABG analyser, OMNI S b221 correlated well with the reference ABG analysers GEM PREMIER and ABL700 in the wards and therefore, suitable to run patient samples. The study further demonstrated an insignificant difference between the direct and indirect ion-selective electrode results for sodium and potassium.

Recommendations and Lessons Learned
Laboratory medicine providers must remember that most of the exercises they undertake constitute research and therefore must aim to get the work published as soon as possible. This important contribution to laboratory medicine was left unpublished for many years. POCT study in the diabetic clinic at the same institution conducted by clinicians was published many years ago. Lack of publication of such contribution may be viewed as a lack of contribution from Africa. Since this study was conducted several generations of improved instruments were introduced. For example, three generations of instruments have replaced the Hitachi 917 in the same laboratory, ABL 700 was replaced by ABL 800, GEM PREMIER 3000 replaced by GEM PREMIER 4000. I-STATs have become a common feature. However, the original study needs to be published so that this important scientific exercise takes its place in history.
The second recommendation is that medical laboratory scientists must be involved in the selection, placement, verification, monitoring and management of all POCT instruments.
Thirdly decongestion of academic and tertiary hospitals is lagging in Africa and other developing countries because laboratory medicine is effectively practised at higher health institutions. Effective primary health care system begins with effective laboratory medicine and the current crop of POCT instruments managed by laboratorians will provide that solution.

Strength and Limitations
This was the first POCT verification study in Africa and it was carried at one of the largest hospitals in the world. Secondly, the study was conducted with the support of the major suppliers of blood gas analysers at the time. The only limitation is that the researchers were caught up in high volume diagnostic work and did not have the time to write up the study for publication.