There are data capture devices that attach to the FreeStyle Libre sensor and convert its communication from NFC (Near-field communication) to Bluetooth technology, generating real-time continuous glucose monitoring. The accuracy of hypoglycemia measurements displayed by smartphone apps using this device has not been established.
MethodsStudy of diagnostic tests. Numerical accuracy was evaluated, utilizing the absolute difference with respect to capillary glucometry (ISO 15197:2015 standard) and clinical accuracy, using the Clarke and Parkes (Consensus) error grids, for glucose measurements less than 70mg/dL performed with the FreeStyle Libre system and with the digital estimation xDrip+ app, in diabetic patients managed with insulin therapy.
ResultsTwenty-seven patients were included (TIR 73.4%, TBR70 5.6%), who contributed 83 hypoglycemic events. Numerical accuracy was adequate in similar proportions with the FreeStyle Libre system compared to the xDrip+ app (81.92% vs. 68.67%, p=0.0630). The clinical accuracy evaluation showed that 92.8% of the measurements for xDrip+ and 98.8% for FreeStyle libre met the criteria according to the Parkes (Consensus) grid (p=0.0535); and 79.5% and 91.6% of the measurements met the criteria according to the Clarke grid (p=0.0273), being higher with FreeStyle libre.
ConclusionsThe use of the NFC-Bluetooth transmitter (Miao-Miao) associated with the xDrip+ app does not improve numerical or clinical accuracy for detecting hypoglycemic events in diabetic patients managed with insulin therapy, compared to the FreeStyle Libre device.
Existen dispositivos de captura de datos que se acoplan al sensor del FreeStyle Libre y convierten su comunicación de Near-field communication (NFC) a tecnología bluetooth para lograr una monitorización continua de glucosa en tiempo real. No se ha establecido la precisión de las mediciones en hipoglucemia mostradas por aplicaciones para smartphones que utiliza este dispositivo.
MétodosEstudio de pruebas diagnósticas. Se evaluó la exactitud numérica mediante la diferencia absoluta en los valores con respecto a la glucometría capilar (norma ISO 15197:2015) y la exactitud clínica, mediante las gradillas de error de Clarke y Parkes (Consensus), para mediciones de glucosa inferiores a 70mg/dl realizadas con el sistema FreeStyle Libre y con la aplicación xDrip+ de estimación digital, en personas con diabetes con insulinoterapia.
ResultadosSe incluyó a 27 pacientes (TIR 73,4%, TBR70 5,6%), quienes aportaron 83 eventos de hipoglucemia. La exactitud numérica fue adecuada en proporciones similares con el sistema FreeStyle Libre en comparación con la aplicación xDrip+ (81,92% vs. 68,67%, p=0,0630). La evaluación de la precisión clínica mostró que el 92,8% de las mediciones para xDrip+ y el 98,8% para FreeStyle libre cumplieron el criterio según la gradilla de Parkes (Consensus) (p=0,0535), y el 79,5 y el 91,6% de las mediciones cumplieron el criterio según la gradilla de Clarke (p=0,0273) siendo superior con Libre.
ConclusionesEl uso del transmisor NFC-Bluetooth (Miao-Miao) asociado a la aplicación xDrip+ no mejora la precisión numérica ni clínica para la detección de los eventos de hipoglucemia en los personas con diabetes con insulinoterapia en comparación con el dispositivo FreeStyle Libre.
Intensification of therapy in diabetic patients by starting insulin enables better glycaemic control, which is reflected in a lower incidence of microvascular and macrovascular complications1,2. The cost of this therapy is an increased risk of hypoglycaemia, which sometimes becomes a limiting factor in achieving glycaemic control goals. Given the possibility of hypoglycaemia, more strict follow-up is required, with capillary blood glucose monitoring being the most commonly used strategy since it facilitates adjustment of insulin doses and confirmation of hypoglycaemic events. However, it is not always possible to achieve this strict follow-up due to the discomfort caused by frequent blood glucose monitoring3. That is why devices for continuous glucose monitoring that alert to events of hypo- and hyperglycaemia, thereby decreasing extreme glycaemic excursions4, have been implemented in clinical practice.
The FreeStyle Libre device (Abbott Diabetes Care, Alameda, CA) consists of a sensor of similar size to a coin, and a reader that is similar to a conventional glucometer. The sensor is attached to the skin with a special applicator and worn for 14 days, requiring scans every eight hours to continuously chart glucose levels. However, it does not warn of glycaemic excursions, which would only be detected at the time of the scan5. In addition, there is evidence suggesting lower precision in episodes of hypoglycaemia and during physical activity, so in these scenarios confirmation with capillary blood glucose monitoring is required. These variations may be explained by delays in glucose diffusion from plasma into the interstitial space, which may be reflected in discordant results, especially when there are rapid changes in plasma glucose concentrations6–8.
In Latin America, continuous real-time monitoring is currently not available. In view of this limitation, there are transmission devices for data capture that can be paired to the FreeStyle Libre sensor, converting its communication from near-field communication (NFC) to Bluetooth technology to achieve real-time glucose monitoring. One of these devices, known as Miao-Miao, automatically sends the measurements obtained by the FreeStyle Libre sensor to different applications, such as xDrip+ (Android), Tomato (Android & iOS) or Glimp (Android), which are installed on a smartphone that receives the data. In particular, the xDrip+ application is compatible with several continuous monitoring systems and is used to view glucose levels and share this information remotely with other users, as well as enabling the implementation of alarms, calibration processes and glucose prediction algorithms9. To date there are no studies that evaluate the precision and usefulness of this application, which has motivated us to objectively evaluate its performance.
The objective of this study is to establish whether there are differences in the numerical and clinical precision of the measurements below 70mg/dl provided by the FreeStyle Libre system, when compared with the measurements provided by the xDrip+ application, in diabetic patients managed with insulin therapy who experience hypoglycaemic events confirmed by capillary blood glucose monitoring.
MethodsA diagnostic test study was conducted at the Hospital Universitario San Ignacio in Bogota (Colombia) from May 2021 to May 2022. Men and women over 18 years of age who attended the Endocrinology Department as outpatients with a diagnosis of type 1 or type 2 diabetes mellitus on insulin therapy, who were users of FreeStyle Libre intermittently-scanned glucose monitoring, who had a previous history of hypoglycaemic events and who had adherence to sensor use greater than 70% of the time, were included. Pregnant patients, those with a history of hypoglycaemia unawareness or infection at the sensor insertion site that limited its use, and those patients with cognitive or visual abnormalities that did not allow for adequate use of the sensor or for taking blood glucose measurements as necessary, were excluded. The study was approved by the institution's ethics committee (approval number MI 065-2020).
Each patient had a FreeStyle Libre sensor (version 1) implanted, with which they were required to perform the interstitial glucose recording in accordance with the manufacturer's recommendations for use. Subsequently, an NFC-Bluetooth transmitter (Miao-Miao, version 2) was connected using the adhesive tape supplied by the manufacturer. The transmitter was paired via Bluetooth with the xDrip+ application (version of 8 March 2021, obtained from https://github.com/NightscoutFoundation/xDrip/releases) on each patient’s mobile phone, allowing the information to be captured and then processed by the aforementioned application. The information displayed in the application was blinded for patient safety. In the event of showing scanned measurements lower than 70mg/dl or exhibiting symptoms of hypoglycaemia, the patient had to measure capillary blood glucose levels with the FreeStyle Optium Neo Glucose Meter. The blood glucose monitoring performed with this device was our gold standard for accuracy testing. During each sensor change, the respective information from the different devices involved in the study was downloaded.
Of all the records obtained by each patient, the measurements obtained by capillary blood glucose monitoring that were 70mg/dl or lower were taken to evaluate clinical and numerical precision in comparison with the FreeStyle Libre and the xDrip+ application records, corresponding to the exact minute at which blood glucose was measured. If the measurements from the FreeStyle Libre system or the xDrip+ did not match the exact minute of the blood glucose measurements, a linear interpolation process was performed between the immediately preceding and subsequent measurements to estimate the corresponding value of each system.
To assess numerical precision, ISO standard 15197:2015 for accuracy was used for glucose values less than 100mg/dl, defined as an absolute difference less than or equal to 15mg/dl (the standard requires that 95% of readings less than 100mg/dl meet that criterion)10.
To determine clinical precision, the Clarke error grid and the Parkes (or consensus) error grid were used, where the values recorded by each device were compared with the values from capillary blood glucose monitoring. The proportion of the results that were located in zones A, B C, D and E was determined. These zones are structured to inform the clinician of the severity of the diagnostic test error by evaluating whether the clinical action taken would be appropriate. Zone A shows the measurements closest to the reference method and which, therefore, would lead to correct clinical action. Zone B contains measurements with a greater difference than zone A, but not to such a degree that it affects clinical decisions or could lead to serious adverse events. In contrast, the measurements that are in zones C, E and, to a greater extent, D, represent such a degree of error that they may involve treatment decisions that could incur serious adverse effects11,12. To confirm adequate clinical precision, the sum of the proportions of the measurements located in zones A and B was calculated, requiring a minimum of 95% for the Clarke grid and at least 99% for the Parkes (consensus) grid.
A quantitative statistical analysis was performed, evaluating the proportion of patients who met the criterion of adequate numerical precision according to ISO 15197:2015 for glucose values less than 100mg/dl, and the proportion of patients in zones A and B of the error grids. A p-value <0.05 was considered significant. Statistical analysis was performed using the STATA 17.0 software (Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC).
ResultsWe obtained the records of 27 patients, who provided 83 hypoglycaemic events confirmed by blood glucose monitoring. The mean age of our population was 48.25 years and 62.9% were male. Body mass index was 25.49kg/m2, close to the normal range. Table 1 lists the demographic characteristics of the population.
Demographic characteristics of the population.
| Variable | n=27 |
|---|---|
| Male, n (%) | 17 (62.9) |
| Age in years, mean (SD) | 48.25 (11.39) |
| Height in cm, mean (SD) | 162.88 (7.35) |
| Weight in kg, mean (SD) | 68.09 (9.19) |
| BMI in kg/m2, mean (SD) | 25.49 (1.70) |
| Abdominal circumference, mean (SD) | 88.37 (10.41) |
| Duration of disease in years, mean (SD) | 21.5 (7.10) |
| % HbA1c, mean (SD) | 7.54 (1.22) |
| GFR by CKD-EPI in ml/min/1.73 m2, mean (SD) | 92.05 (16.87) |
| History of coronary disease, n (%) | 1 (3.71) |
| History of cerebrovascular accident, n (%) | 0 |
| History of peripheral arterial disease, n (%) | 0 |
| Kidney disease, n (%) | 2 (7.41) |
| Neuropathy, n (%) | 0 |
| Gastroparesis, n (%) | 1 (3.71) |
| Retinopathy, n (%) | 3 (11.1) |
| History of hypoglycaemia, n (%) | 27 (100) |
| History of severe hypoglycaemia, n (%) | 1 (3.71) |
| Basal-bolus insulin therapy, n (%) | 26 (96.29) |
| Basal insulin therapy, n (%) | 1 (3.71) |
| Total daily dose of insulin in IU/kg/day, mean (SD) | 0.63 (0.19) |
| % TIR, mean (SD) | 73.48 (14.90) |
| % TAR 180, mean (SD) | 20.88 (13.93) |
| % TBR 70, mean (SD) | 5.62 (2.85) |
| % CV, mean (SD) | 35.21 (7.52) |
| GMI, mean (SD) | 6.68 (0.53) |
BMI: body mass index; CKD-EPI: Chronic Kidney Disease Epidemiology Collaboration; CV: coefficient of variation; GFR: glomerular filtration rate; GMI: glucose management indicator; HbA1c: glycosylated haemoglobin; SD: standard deviation; TAR: time above range; TBR: Time below range; TIR: time in range.
The vast majority were users of basal-bolus insulin therapy (96.3%), with a history of hypoglycaemia found in all patients, and with only one patient with a history of severe hypoglycaemia. The daily dose of insulin was approximately 0.63 IU/kg/day. The most prevalent complications were retinopathy and kidney disease.
Regarding the glycaemic control variables, the patients achieved a time in range (TIR) of 73.48% and a time above range (TAR 180) of 20.88%, which was within the goals established by different consensuses (TIR greater than 70% and TAR 180 less than 25%), reflected in glucose management indicator goals. However, they presented a time below range (TBR 70) of 5.6% and a TBR 54 of 1.1%, higher than the proposed goals (less than 4% and less than 1%), compatible with a population with greater susceptibility to hypoglycaemia. The average coefficient of variation was 35.32%, confirming this to be a population without criteria of high glycaemic variability (coefficient of variation less than 36%).
Regarding numerical precision, the proportion of measurements that met ISO standard 15197:2015 was 68.67% for the xDrip+ application and 81.92% for Libre, with an absolute difference of 13.25% (95% CI, −28.13%–1.62%) in favour of Libre, which was not statistically significant (p=0.0630).
The mean absolute relative difference (MARD) was estimated for both records at 23.72 for xDrip+ and 18.21 for Libre, with an absolute difference of 5.51 (95% CI, –0.40−11.42) in favour of Libre. However, it was not statistically significant (p=0.0673).
Figs. 1 and 2 show the Clarke and Parkes (consensus) error grids for the measurements obtained by the xDrip+ application. Figs. 3 and 4 show the measurements obtained by the FreeStyle Libre.
Overall, 92.8% and 98.8% of the measurements were located in zones A and B of the Parkes grid (consensus) for xDrip+ and Libre, respectively, a difference that was not statistically significant (p=0.0535). In contrast, the proportion of measurements in zones A and B of the Clarke grid was significantly higher for FreeStyle Libre (79.5% vs 91.6%, p=0.0273).
DiscussionIt was suggested that the use of an NFC-Bluetooth transmitter (Miao-Miao) connected to the xDrip+ digital estimation application could work as an option to more accurately document hypoglycaemic events. However, our study found no differences between the devices in terms of numerical precision, but rather higher clinical precision for the FreeStyle Libre.
To date, there have been no comparative studies evaluating the numerical and clinical precision obtained with this application, and our study was the first to evaluate these aspects. Being an application that does not have strong scientific support, it still does not have the endorsement of the Colombian regulatory entities for its widespread use. The results of our study do not demonstrate a benefit from using Miao-Miao with xDrip+, which, added to the additional costs for the use of the FreeStyle Libre sensor, does not support its use in Colombia.
The MARD is a precision parameter that evaluates the mean of the absolute differences of the device with its active comparator (in this case capillary blood glucose monitoring), proposing a better analytical performance the lower this value is. A cut-off point of 10% has been proposed as the value that denotes good performance by the evaluated device13.
When comparing the precision of xDrip+ in hypoglycaemia with the MARD compared to other available real-time monitoring devices, we found that it is higher than that reported by Freckmann et al.14 for Dexcom G5 (Dexcom Inc, San Diego, CA, USA) (10.1%), the Guardian Connect System (Medtronic MiniMed Inc, Northridge, CA, USA) (11.2%) and the Roche CGM System (Roche Diabetes Care GmbH, Mannheim, Germany) (11.3%). Additionally, new devices for real-time continuous glucose monitoring (CGM) have improved precision in hypoglycaemia, showing that diagnostic precision (defined as the proportion of CGM system values that were within ±15%, or 15mg/dl of matched venous values) for Dexcom G6 ranges from 79.2 to 100%, for FreeStyle Libre 2 it is 94.3% and for Eversense it is greater than 85%. However, these devices are not available for use in Latin America15. If we take into account the clinical precision using the consensus grid, the three devices in the previously-mentioned study achieved values greater than 99% for zones A and B, which supports adequate clinical precision, but this was not the case for the xDrip+ algorithm as it did not achieve the criterion of adequate precision with only 92.8% of measurements in the required zones. Similar findings were found with numerical precision according to ISO standard 15197:2015 for glucose values under 100mg/dl, as the xDrip+ algorithm did not achieve the minimum measurements for adequate reliability of the system. At least 95% is required and the algorithm achieved only 68.67%. Therefore, our study suggests that the xDrip+ application connected to the FreeStyle Libre 1 through devices such as the Miao-Miao should not be used for clinical decision-making due to the risk of error in hypoglycaemia.
Although the values shown by the xDrip+ application differ from the FreeStyle Libre system, the fact that they shared a similar source of information (FreeStyle Libre sensor) could be the reason why it was not possible to document differences in favour of the algorithm. This supports the importance of variations in interstitial glucose levels when compared to serum levels, an aspect that significantly impacts the variation of devices that measure interstitial glucose due to the diffusion mechanisms involved7. Moreover, the xDrip+ application implements a calibration algorithm that uses glucose measurements entered by the user to adjust the estimate of the measurements displayed (https://xdrip.readthedocs.io/en/latest/calibrate/calibrate/). This algorithm could also have adversely influenced the precision of the application in hypoglycaemia despite the fact that patients were asked to follow the calibration instructions stated by the authors.
Regarding the hypoglycaemia precision of the FreeStyle Libre, contrary to what was reported in the publication by Moser et al.6, where the MARD in hypoglycaemia was 31.6% and the clinical precision according to Clarke was 91% for any measurement, our data suggest better precision considering the MARD obtained of 18.21%, and clinical precision with the Clarke grid of 91% for hypoglycaemic events only. These data are similar to those obtained by Aberer et al., who documented a MARD of 14.6% for the FreeStyle Libre system16. The precision of this device in real life is an aspect to take into account for future research.
A limitation of our study is its relatively small sample size in terms of the number of patients evaluated, but with a greater number of hypoglycaemic episodes, which improves the precision of our estimates.
In conclusion, the use of the NFC-Bluetooth transmitter (Miao-Miao) connected to the xDrip+ digital estimation application does not improve numerical or clinical precision for the detection of hypoglycaemic events in diabetic patients managed with insulin therapy, compared to the FreeStyle Libre device. Further, in terms of clinical precision, the use of xDrip+ does not meet ISO standard 15197:2015 in hypoglycaemia either in the Clarke or Parkes error grids, for which reason xDrip+ connected to the FreeStyle Libre 1 should not be used to make clinical decisions.
Ethical declarationsThe authors declare that the procedures followed complied with the regulations of the responsible independent ethics committee, the World Medical Association and the Declaration of Helsinki. The authors declare that this article does not contain patient data.
FundingThe project was funded by the Universidad EAN (Grant # TO_P01_2022) and by Universidad Antonio Nariño (Grant # 2022033).
Conflicts of interestThe authors declare that they have no conflicts of interest.