Accuracy of a Novel “Factory-Calibrated” Continuous Glucose Monitoring Device in Normal Glucose Levels: A Pilot Study

Free Style Libre Pro, a novel continuous glucose monitoring (CGM) device for retrospective assessment of glycemic excursion, does not require calibration during the measurement period. In this study, the accuracy of FreeStyle Libre Pro in normal glucose levels was investigated. Two meal tests, on the 7th day and 9th day of CGM usage, were performed in five non-diabetic adults. Venous blood samples were obtained at baseline, and at 2 hours, 4 hours and 6 hours after the meal. The plasma glucose levels and the glucose levels measured by FreeStyle Libre Pro were compared. Subjects were aged 41.4±7.6 years, 40% male, with body mass index 21.7±2.6, HbA1c 5.6±0.3% and a fasting plasma glucose level of 95.4±2.1 mg/dl. The plasma glucose levels and the glucose levels measured by FreeStyle Libre Pro were significantly different; 94.4±1.8 vs. 111.2±8.5 mg/dl (P <0.01) at baseline, 95.6±7.0 vs. 122.6±8.8 mg/dl (P <0.01) at 2 hours, 90.1±4.3 vs. 111.2±13.0 mg/dl (P <0.01) at 4 hours, and 88.7±3.2 vs. 105.5±7.1 mg/dl (P <0.01) at 6 hours. The absolute difference between the two methods was 17.7±7.5 (minimum 5 maximum 27) mg/dl at baseline, 27.0±9.4 (minimum 8 maximum 40) mg/dl at 2 hours, 21.1±11.3 (minimum 7 maximum 42) mg/dl at 4 hours, 16.8±5.4 (minimum 7 maximum 23) mg/dl at 6 hours, and 20.7±9.3 (minimum 5 maximum 42) mg/dl for all time points, respectively. The absolute relative difference between them was 18.7±7.8 (minimum 5.3 maximum 28.7)% at baseline, 28.7±10.7 (minimum 7.7 maximum 45.5)% at 2 hours, 23.3±12.2 (minimum 7.9 maximum 45.2)% at 4 hours, 18.9±6.0 (minimum 8.0 maximum 26.2)% at 6 hours, and 22.4±10.0 (minimum 5.3 maximum 43.5)% for all time points, respectively. The ratio of values measured by FreeStyle Libre Pro within the interval of ±15 mg/dl in less than 100 mg/dl or within the interval of ±15% in not less than 100 mg/dl plasma glucose levels was 30.0% at baseline, 10.0% at 2 hours, 40.0% at 4 hours, 50.0% at 6 hours and 32.5% for all time points, respectively. These study results suggest FreeStyle Libre Pro may not be sufficiently accurate in normal glucose levels.


Introduction
Continuous glucose monitoring (CGM) is a technology to monitor blood glucose levels continuously in humans, especially in patients with diabetes mellitus [1][2][3]. The early prototype of CGM was introduced in 1977 as a part of a device called "Biostator" intended to be used as an artificial pancreas [4]. This original method required taking small volume blood samples from the patient continuously, making the device unsuitable for long time usage in daily life.
As diabetes mellitus is a chronic disease with a potential duration of several decades, daily management to keep glucose levels within a near normal range with minimal adverse events, mainly hypoglycemia, is extremely important to prevent various diabetic complications [5,6]. To achieve this, self-monitoring of blood glucose (SMBG) has been widely Monitoring Device in Normal Glucose Levels: A Pilot Study used. The Diabetes Control and Complication Trial (DCCT) proved that intensive therapy including SMBG prevented or delayed diabetic complications in type 1 diabetes mellitus [5]. In the DCCT, SMBG was utilized for daily adjustment of insulin dosage. Currently, most of the commercially available SMBG devices satisfy the ISO 15197:2013 standard [7]. In brief, the ISO 15197:2013 standard demands at least 95% of the values obtained by SMBG devices are within the interval of ±15 mg/dl in less than 100 mg/dl or within the interval of ±15% in not less than 100 mg/dl plasma glucose levels. For patients with type 1 diabetes mellitus, ISO 15197:2013 standard also demands at least 99% of the values obtained by SMBG devices are in Zone A and Zone B of the Consensus Error Grid [8]. SMBG has been used not only for adjusting insulin dosage, but also for modifying lifestyle by providing feedback to patients regarding the effect of diet and exercise on blood glucose levels.
The limitation of SMBG is that it provides glycemic values only at the time the blood samples are obtained. To overcome this limitation, the usage of CGM, which provides more detailed information, is becoming common [1][2][3]. Most currently available CGM devices utilize a small enzyme electrode implanted subcutaneously to measure the glucose levels of interstitial fluid by detecting the electrons generated by the enzyme that reacts with glucose [9]. The signals are sent to a receiver and processed by an internal algorithm, in which a calculated approximate blood glucose level is displayed or recorded. As the change in glucose levels in the interstitial fluid is delayed for 10 to 15 minutes to capillary blood glucose levels, caution is needed when interpreting the measurement [10]. Some CGM devices use the ratio of change in glucose levels to compensate for the delay and to predict the real-time values [11]. Two types of CGM are currently used in clinical practice [3]. One type of CGM displays the real time values so that patients can use the information to help optimize their insulin therapy; this is called real-time CGM or personal CGM. Real-time CGM usually continuously transmits glucose levels from the sensor to the monitor, and has an alarm function for harmful hypoglycemia or hyperglycemia [12]. Real-time CGM is also used to control the insulin pump automatically, a portable infusion device that delivers insulin continuously to subcutaneous tissue to mimic physiological insulin secretion [13]. There is a variant of personal CGM that lacks the alarm function due to intermittent transmission of the glucose levels, which the manufacturer calls "Flash Glucose Monitoring" commercially [14]. Another type of CGM is called retrospective CGM or professional CGM, which is intended for the retrospective analysis of glycemic values [3]. Retrospective CGM is used for various purposes, such as finding nocturnal hypoglycemia, an early morning surge of blood glucose levels known as the "dawn phenomenon", postprandial hyperglycemia, and adjusting insulin pump programming. Both real-time CGM and retrospective CGM are covered by health insurance in Japan and in many other countries.
In this article, the accuracy of a novel retrospective CGM device, FreeStyle Libre Pro (Abbott Diabetes Care, Alameda, U. S. A.), in non-diabetic adults was investigated with the aim of evaluating its accuracy in normal glucose levels. FreeStyle Libre Pro does not require calibration during the measurement period, and uses a technology called "factory-calibration" by the manufacturer [15]. FreeStyle Libre Pro looks like FreeStyle Libre, but is different, as FreeStyle Libre is designed to display the approximate real-time glucose levels intermittently. FreeStyle Libre Pro and FreeStyle Libre are based on a common technology to measure the glucose concentration of subcutaneous interstitial fluid by an enzyme electrode and calculate approximate blood glucose levels using an algorithm. However, the mean absolute relative difference (MARD), a parameter to express the accuracy of CGM compared to the standard method to measure glucose, is slightly different between them [16,17]. According to the performance sheet attached to the product, the manufacturer reported the MARD of FreeStyle Libre Pro as 11.1%, whereas the MARD of FreeStyle Libre was 11.4%. The algorithm used in FreeStyle Libre Pro and FreeStyle Libre is slightly different; however, the manufacturer has not disclosed the details yet. According to a study sponsored by the manufacturer, the delay in the values displayed in FreeStyle Libre to blood glucose levels is approximately 5 minutes, which was shorter than the physiological time lag, and compatible with the understanding that this device is using the algorithm to compensate for the delay in the change in the glucose levels of interstitial fluid compared to actual blood glucose levels [10,14]. Although there are a few reports investigating the accuracy of FreeStyle Libre, to date there is no peer-reviewed report investigating that of FreeStyle Libre Pro [14,18,19]. Accuracy in normal glucose levels (70 mg/dl to 110 mg/dl) is very important, as it is quite close to the hypoglycemic level (less than 70 mg/dl). Therefore, a pilot study to evaluate the accuracy of FreeStyle Libre Pro in normal glucose levels was conducted by using it in non-diabetic adults.

Study Subjects
Five non-diabetic adults were recruited. Inclusion criteria were no younger than 20 years and no older than 65 years, and male or female. Exclusion criteria were, subjects with a present complication or past history of severe hepatic disease, severe kidney disease, severe heart disease, severe lung disease, severe gastrointestinal disease, severe metabolic disease, smoking habit, alcohol drinking habit no less than 60g of ethanol per day, night-shift workers, participants in other clinical studies, and any other reasons that made subjects unsuitable for participation in this study.

Ethics
The study was approved by the Ethical Committee of National Hospital Organization Kyoto Medical Center. Written informed consent was obtained from each participant. The study was registered on the University Hospital Network Clinical Trial Registry (UMIN000026413).

Data Collection and Measurements
Two meal tests on the 7th day and 9th day of CGM usage were performed. Subjects were provided meal A (energy 632 kcal, carbohydrate 79.8 g, protein 18.5 g, fat 27.9 g) or meal B (energy 660 kcal, carbohydrate 75.7 g, protein 19.8 g, fat 31.6 g) at 9:30 AM after crossover randomization. CGM data were downloaded from FreeStyle Libre Pro using FreeStyle Libre Pro Software (Abbott Diabetes Care, Alameda, U. S. A.) and saved to log files as text data. Glucose levels at baseline, and at 2 hours, 4 hours and 6 hours after the meal were identified from the log file. Venous blood samples were obtained at baseline, and at 2 hours, 4 hours and 6 hours after the meal using sample tubes containing sodium fluorine and sodium ethylenediaminetetraacetic acid. Plasma glucose levels were measured by JCA-BM9030 (JEOL Ltd, Tokyo, Japan) and GLU-HL (Serotec Co., Ltd, Chitose, Japan) at the laboratory of Health Science Research Institute West Japan CO., LTD (Kyoto, Japan). JCA-BM9030 was calibrated according to the instructions provided by the manufacturer.

Statistical Analysis
Absolute difference (AD) was determined as the absolute values of the difference between the readings by FreeStyle Libre Pro and plasma glucose levels. Absolute relative difference (ARD) was determined as the ratio of AD to plasma glucose levels in percentage terms. The ratio of the readings by FreeStyle Libre Pro within the interval of ±15 mg/dl in less than 100 mg/dl or within the interval of ±15% in not less than 100 mg/dl plasma glucose levels was calculated to evaluate the accuracy of FreeStyle Libre Pro, as a criteria equivalent to the ISO 15197:2013 standard for SMBG devices. The plasma glucose levels and the glucose levels measured by FreeStyle Libre Pro were compared by paired Student's t-test. Data were analyzed using the IBM SPSS Statistics for Windows software version 20.0 (IBM Corp., Armonk, NY, U. S. A.). P-values of <0.05 were considered significant.

Results
The non-diabetic participants were aged 41.4±7.6 years old, 40% of the subjects were male, with body mass index 21.7±2.6, HbA1c 5.6±0.3% and fasting plasma glucose level 95.4±2.1 mg/dl.  The plasma glucose levels and the glucose levels measured by FreeStyle Libre Pro in each subject were compared and AD was calculated ( Table 1). The glucose levels measured by FreeStyle Libre Pro were significantly higher than plasma glucose levels at all measured timings (Table 2). AD and ARD was largest at 2 hours. The ratio of values measured by FreeStyle Libre Pro within the interval of ±15 mg/dl in less than 100 mg/dl or within the interval of ±15% in not less than 100 mg/dl plasma glucose levels was as low as 30.0% at baseline, 10.0% at 2 hours, 40.0% at 4 hours, 50.0% at 6 hours and 32.5% for all time points, respectively. Compared to the change in plasma glucose levels between baseline and 2 hours, the change in the glucose levels measured by the FreeStyle Libre Pro was significantly larger (1.2±6.6 mg/dl vs. 10.5±10.3 mg/dl, P = 0.04), but there was no significant difference between baseline and 4 hours (-4.3±3.9 mg/dl vs. -0.9±5.6 mg/dl, P = 0.10), and baseline and 6 hours (-5.7±2.5 mg/dl vs. -6.6±6.1 mg/dl, P = 0.62) ( Table 3).

Discussion
In this study, the accuracy of FreeStyle Libre Pro in normal glucose levels was investigated by using it in non-diabetic adults. Despite the small sample size, the glucose levels measured by FreeStyle Libre Pro were significantly higher than plasma glucose levels, and the ratio of the measurement within the interval of ±15 mg/dl in less than 100 mg/dl or within the interval of ±15% in not less than 100 mg/dl plasma glucose levels, one of the well-accepted criteria of accuracy equivalent to ISO 15197:2013 standard used for the evaluation of conventional systems for SMBG, was far below 95%. The discrepancy was largest at 2 hours after meal intake, suggesting inaccurate measurement by FreeStyle Libre Pro during the postprandial period in non-diabetic adults. Interestingly, in a document filed by the U. S. Food and Drug Administration (FDA), there is a description that "The device may inaccurately indicate hypoglycemia. The results of the clinical study conducted for this device showed that 40% of the time when the device indicated that user sensor glucose values were at or below 60 mg/dL, user glucose values were actually in the range of 81-160 mg/dL." [20] This inaccuracy observed by the manufacturer indicates possible error in the opposite direction compared to the present study, i.e. glucose levels measured by FreeStyle Libre Pro could be lower than true values in normal glucose levels.
There is no doubt that the "factory-calibration" of FreeStyle Libre Pro is convenient for users. However, the present study suggests the possibility that the lack of the calibration during the measurement period may be one cause of inaccuracy in normal glucose levels. For real-time CGM, it was observed in a report that more frequent calibration during usage resulted in higher accuracy of CGM measurement, with a MARD less than 10%, supporting the rationale for calibration during the measurement period [21].
It is important to note that FreeStyle Libre Pro is contraindicated for use in pregnant women in the Japanese version of the user instructions. Our observation also supports that FreeStyle Libre Pro should not be used in pregnant women with diabetes, as the post-prandial blood glucose control target is less than 120 mg/dl according to the Japan Diabetes Society (JDS) guideline. At the same time, FreeStyle Libre Pro appears to be unsuitable for the evaluation of mild impaired glucose tolerance, as the excursion of the post-prandial glucose levels may be exaggerated compared to the actual excursion of the plasma glucose levels.
The reason why the error was maximal at 2 hours after the meal remains unclear, as detailed information on the algorithm used in the FreeStyle Libre Pro has not been disclosed by the manufacturer. The data from the present study showed that the increase between the baseline and 2 hours was significantly higher in FreeStyle Libre Pro than in plasma glucose levels, and we speculate that a prediction algorithm to compensate for the delay in the interstitial blood glucose levels may be involved in this observation. Considering the purpose of retrospective CGM in reviewing glycemic excursion in the past, a prediction algorithm appears to be unnecessary. However, further technical disclosure by the manufacturer is necessary to address this issue.
The strength of the current study is that this is the first study independent from the manufacturer to evaluate the accuracy of FreeStyle Libre Pro using non-diabetic adults to examine the accuracy in normal glucose levels, and using plasma glucose levels as a control rather than SMBG. The limitations of the study are the small sample size, lack of an oral glucose tolerance test, although all the measured plasma glucose levels were compatible with those of non-diabetic adults, and not using a glucose meter manufactured by YSI, Inc. (Yellow Springs, OH, U. S. A.) that is used as a standard device to measure plasma glucose levels in the U.S.A.

Conclusion
The present study results suggest that FreeStyle Libre Pro may not be sufficiently accurate in normal glucose levels. The convenience of "factory-calibration" may be related to the compromised accuracy of the measurement in this novel device. Users of FreeStyle Libre Pro should be aware of the potential risk of obtaining inaccurate data in normal glucose levels either in a higher or lower direction. Although this device is intended for use to analyze glycemic excursion retrospectively, and therefore the risk of providing the wrong treatment to patients is theoretically low, the study results obtained by this device should be confirmed by conventional methods such as measuring plasma glucose levels or SMBG prior to interpretation by physicians. Further clinical studies independent from the manufacturer are essential to conclude how accurate FreeStyle Libre Pro is, and whether the MARD provided by the manufacturer is reproducible in the real-world clinical setting.