Repeatability of the evaluation of systemic microvascular endothelial function using laser doppler perfusion monitoring: clinical and statistical implications

OBJECTIVE:

An awareness of the repeatability of biological measures is required to properly design and calculate sample sizes for longitudinal interventional studies. We investigated the day-to-day repeatability of measures of systemic microvascular reactivity using laser Doppler perfusion monitoring.

METHODS:

We performed laser Doppler perfusion monitoring in combination with skin iontophoresis using acetylcholine and sodium nitroprusside as well as post-occlusive reactive and thermal hyperemia twice within two weeks. The repeatability was assessed by calculating the within-subject standard deviations, limits of agreement, typical errors and intra-class correlation coefficients between days 1 and 2. The ratio of the within-subject standard deviation to the mean values obtained on days 1 and 2 (within-subject standard deviation/GM) was used to determine the condition with the best repeatability.

RESULTS:

Twenty-four healthy subjects, aged 24.6±3.8 years, were recruited. The area under the curve of the vasodilatory response to post-occlusive reactivity showed marked variability (within-subject standard deviation/GM  = 0.83), while the area under the curve for acetylcholine exhibited less variability (within-subject standard deviation/GM  = 0.52) and was comparable to the responses to sodium nitroprusside and thermal treatment (within-subject standard deviations/GM of 0.67 and 0.56, respectively). The area under the blood flow/time curve for vasodilation during acetylcholine administration required the smallest sample sizes, the area under the blood flow/time curve during post-occlusive reactivity required the largest sample sizes, and the area under the blood flow/time curves of vasodilation induced by sodium nitroprusside and thermal treatment required intermediate sizes.

CONCLUSIONS:

In view of the importance of random error related to the day-to-day repeatability of laser Doppler perfusion monitoring, we propose an original and robust statistical methodology for use in designing prospective clinical studies.

INTRODUCTION

During the last two decades, it has become apparent that endothelial dysfunction is a central phenomenon in the pathophysiology of cardiovascular and metabolic diseases.1,2 Endothelial dysfunction is also considered to be a specific marker of atherosclerotic risk^3–5 and is now widely recognized as the earliest clinically detectable indication of subclinical atherosclerosis.6 Moreover, due to the systemic nature of endothelial dysfunction, which simultaneously affects the coronary circulation and peripheral vascular beds,7 it has become clear that endothelial dysfunction in conduit arteries, small-resistance vessels and the microcirculation can be used as a surrogate marker for coronary endothelial damage.8,9 Assessing peripheral endothelial function can also be useful for establishing a correlation between improvements in endothelial function after therapeutic interventions and improvements in cardiovascular outcomes.^10–13

Due to the clinical relevance for the prediction of long-term cardiovascular risk, therefore, there is growing interest in assessing endothelial-dependent vasodilatation using non-invasive techniques.9,14 Skin laser Doppler perfusion monitoring (LDPM) coupled with the delivery of vasoactive drugs across the skin using iontophoresis is increasingly being used for the clinical evaluation of microvascular endothelial function.15 Nevertheless, the use of single-point LDPM has some pitfalls that can interfere with its usefulness in the clinical setting and possibly preclude comparisons of results obtained by different investigators. The main concern is the poor reproducibility of the methodology, which is mainly due to the spatial heterogeneity of skin blood flow (for LDPM)16,17 and differences in skin resistance (for iontophoresis).18 The use of standard, well-defined experimental protocols and the proper expression of cutaneous blood flow changes in response to different stimuli may improve the reproducibility of LDPM.19 One major concern is the repeatability of the methodology, given that the sequential assessment of endothelial function is mandatory for evaluating medical interventions in cardiometabolic diseases. Finally, most studies in the literature that investigate endothelial function using LDPM are cross-sectional and do not consider the variability of the methodology when calculating the statistical power and the sample size.

The present study was designed to investigate the day-to-day repeatability of skin LDPM measurements coupled with physiological and pharmacological local vasodilatory stimuli in young, healthy volunteers. In view of our results, we propose the use of a statistical methodology that takes the repeatability of LDPM into consideration when calculating the sample size for longitudinal clinical and drug-testing studies.

METHODS

Study design

This study was performed on 24 healthy volunteers without any underlying diseases (hospital employees and medical students). No subjects were taking any medications, and they were all judged to be healthy on the basis of a physical examination and routine laboratory screening, which revealed normal lipid and glycemic profiles. Each subject was studied twice in two weeks (all of the subjects were tested at 7-day intervals). The testing occurred in the morning after a 20-minute rest period; subjects were instructed to assume the supine position in a metabolic unit with a controlled temperature (23±1°C) after an overnight fast. The subjects were instructed to refrain from drinking alcohol- or caffeine-containing beverages and from exercising for at least 12 hours before the study. The study was approved by the local ethics committee, and the participants gave written informed consent.

Evaluation of microvascular reactivity to pharmacological stimulation

Endothelium-dependent and -independent perfusion changes resulting from vasodilatation of the skin microcirculation (measured in perfusion units, PU = 10 mV) were noninvasively and continuously evaluated using an LDPM system in combination with iontophoresis (Periflux 5001 and PeriIont, Perimed, Järfälla, Sweden) of acetylcholine (ACh) and sodium nitroprusside (SNP). Drug-delivery electrodes (PF 383, Perimed) were incorporated into the head of the laser probe (PF 481-1, Perimed), and the probe temperature was standardized to 32°C to avoid variations in skin temperature and, consequently, in the measurements of microvascular flow. The drug-delivery electrodes were filled with 200 μl of 0.1 g/L ACh (Sigma Chemical CO, USA) dissolved in distilled water and were attached with the laser probe to a standardized site on the forearm (2–3 cm from the wrist) that had no visible veins. The dispersive electrode was attached approximately 15 cm away from the electrophoresis chamber, according to the manufacturer's instructions. After measuring the resting flow for 5 minutes, 4 doses of ACh were delivered using an anodal current (0.1 mA for 10, 20, 40 and 80 seconds, with total charges of 1, 2, 4 and 8 millicoulombs) at 120-second intervals. The mean values of the resting flow were considered to be the basal flow values for each patient. Using a new delivery electrode applied to a different location on the same forearm, four doses of a solution of 0.1 g/L SNP (Sigma Chemical CO, USA) dissolved in distilled water were delivered using a cathodal current (with the same charges and intervals as for ACh). The curves of the microvascular flow increases induced by ACh and SNP always had a total recording time of 10 minutes: 2 minutes for each dose and 2 minutes to reach the plateau response after the last dose. Previous data from our group showed that the maximum responses to both ACh and SNP occur during this time interval (unpublished observations).

Physiological stimulation

After measuring the resting flow for 5 minutes using another laser probe (PF 457, Perimed) that had been positioned at the beginning of the recordings, the post-occlusive reactive hyperemia (PORH) test was performed by placing the cuff of a sphygmomanometer on the distal portion of the subjects' arms and achieving arterial occlusion by increasing the pressure to 50 mmHg above the systolic blood pressure over a 3-minutes interval (biological zero). Following the release of the pressure, the maximum flow, time to maximum flow (TM), time to half recovery after hyperemia (TH2) and the area under the PORH curve were measured using a heating probe (PF 457, Perimed). Using PeriSoft for Windows 2.5 (Perimed, Järfälla, Sweden), the curve of the PORH was calculated from the release of the cuff until the flow value had returned to the basal value. The mean value of the resting flow was considered to be the basal flow value.

When the microvascular flow had returned to the basal value after the PORH (typically 5–10 minutes), we investigated the maximal skin microvascular vasodilatation using prolonged (20 minutes) local heating of the laser probe to 44°C. The baseline microvascular flow value was calculated as described above. The use of the area under the blood flow/time curve (AUC) for assessing skin microvascular reactivity using LDPM is well validated because it represents the global flow response to different physiological and pharmacological stimuli.^20–22 The AUC was calculated using PeriSoft for Windows 2.5 (Perimed, Järfälla, Sweden).

The four different stimulations described above (ACh, SNP, PORH and heating) were performed consecutively.

Data and statistical analysis

The values obtained for the different parameters are presented as the mean and standard deviation for each study day. An analysis of the day-to-day repeatability was carried out using the limits of agreement (LA), within-subject standard deviation (WSSD), typical error (TE) and intraclass correlation coefficient (ICC).

The WSSD was calculated as the standard error of each subject's measurements between day 1 and 2 and is given by

, where n is the sample size and d is the number of measurements obtained for each subject. The WSSD was used in the calculations of the sample sizes.

The typical error was calculated as described previously by Hopkins23 and is given by the formula

. The standard deviation of the differences between measurements (the day-to-day repeatability) is divided by the square root of the number of measurements (two in this case).

The ICC was calculated using the formula described by McGraw and Wong (24),

, where MS is the square root of the mean between the subjects (R) or the error (E) from a two-way ANOVA table, and k is the number of values per group. The significance of the mean error between days 1 and 2 was evaluated using the t-test for dependent samples (the paired t-test), with an α of 0.0125 (α of 0.05 adjusted for multiple comparisons by the Bonferroni method).

Considering that there was a correlation between the error (the day-to-day variability) and the magnitude of the measurements, the data were log-transformed to calculate the limits of agreement, in keeping with the approach of Bland and Altman.25 In this case, the limits of agreement were found to be a multiplicative function instead of an additive one.

The sample sizes were calculated according to the formula of Arango (26) for longitudinal studies. The size of each group is given by

, where WSSD is the within-subject standard deviation, and (as described above) MDE indicates the minimum difference expected between the groups. Using α = 0.05 (z_α/2 = 1.96) and β = 0.20 (z_1-β = 0.84), we can simplify the formula to

. We also calculated the minimum statistically significant differences for different levels of microvascular reactivity according to the sample size of the hypothetical study populations. To compare the measurement techniques under consideration and to determine which is the most stable and adequate, we calculated a global mean (the mean of all of the individual values for the same technique at days 1 and 2) and divided the WSSD by this global mean. The technique with the smallest ratio between the WSSD and the global mean was considered to be the most suitable measurement for clinical studies.

RESULTS

Study sample

This cross-sectional study included 24 healthy subjects (11 males) aged 24.6±3.8 years, with an average body mass index of 23.0±3.4 kg/m². The systolic and diastolic arterial pressures on day 1 were 114.8±2.1 and 69.4±5.0 mmHg, respectively. On day 2, the systolic and diastolic arterial pressures were 114.3±2.3 and 68±0.9 mmHg, respectively. There were no significant differences in the values of the arterial pressure between days 1 and 2. All of the subjects had normal lipid and glycemic profiles, and all of the females were taking oral contraceptives.

Microvascular reactivity and systematic error

There were no significant intra-subject differences between the baseline measurements on each of the study days (Table 1). The mean baseline values for skin microvascular flow were 4.0±2.1 PU (day 1) and 4.0±1.9 PU (day 2) (P = 0.74). The AUC values were not significantly different between days 1 and 2 for the ACh (10937.37±7010.15 and 9079.15±5462.00 PU, P = 0.22) and SNP (12808.42±11819.00 and 1083192±6481.20 PU, P = 0.40) administrations. The AUC absolute values were also not significantly different between days 1 and 2 for the PORH (706.97±418.15 and 875.06±796.36 PU, P = 0.39) and TH (86287.69±54045.63 and 103987.36±60900.94 PU, P = 0.26) groups. These results indicate that there was no systematic error between the two days.

Table 1.

The mean values for microvascular reactivity measured one week apart in healthy subjects using laser Doppler perfusion monitoring.

Parameter	Day 1	Day 2	P value
AUC-ACh (PU)	10,937.37 (7,010.15)	9,079.15 (5,462.00)	0.22
AUC-SNP (PU)	12,808.42 (11,819.00)	10,831.92 (6,481.20)	0.40
AUC-TH (PU)	86,287.69 (54,045.63)	103,987.36 (60,900.94)	0.26
AUC-PORH (PU)	706.97 (418.15)	875.06 (796.36)	0.39

The values are presented as the mean (standard deviation).

Ach  =  acetylcholine, AUC  =  area under the blood flow/time curve, SNP  =  sodium nitroprusside, PORH  =  post-occlusive reactive hyperemia, PU  =  arbitrary perfusion units, and TH  =  thermal hyperemia.

Random errors

An assessment of the repeatability of the responses between the two different days using the Bland-Altman analysis is shown in Figure 1. The difference in the responses between days 1 and 2 is plotted as a function of the average of the responses of both days for the AUCs of the vasodilator response induced by ACh, SNP, PORH and TH.

Figure 1.

Bland-Altman plots for the assessment of the day-to-day repeatability of microvascular reactivity measurements obtained one week apart in healthy subjects using laser Doppler perfusion monitoring. The difference in the responses measured on the two days (Day 1– Day 2) is plotted against the average of the responses [½ * (Day 1 + Day 2)]. The solid lines correspond to the mean of the differences, and the dashed lines indicate the 95% limits of agreement for the differences between the days. The values were log-transformed. Ach  =  acetylcholine, AUC  =  area under the blood flow/time curve, SNP  =  sodium nitroprusside, PORH  =  post-occlusive reactive hyperemia, and TH  =  thermal hyperemia.

Table 2 shows the limits of agreement in the day-to-day measurements of skin microvascular reactivity using LDPM for each parameter. A preliminary analysis of the data showed that the variability was not significantly different between males and females (data not shown). Repeatability is mainly expressed by the ratios of the within-subject standard deviations to the global means of the variables (WSSD/GM). The vasodilatory response AUC for PORH showed marked within-subject variability (WSSD/GM  = 0.83). By contrast, the AUC for the skin vasodilatation induced by ACh exhibited less variability (WSSD/GM  = 0.52) than the AUC for PORH and was comparable to the AUCs for SNP and TH (WSSD/GM of 0.67 and 0.56, respectively).

Table 2.

An assessment of the limits of agreement in the day-to-day measurements of skin microvascular reactivity using laser Doppler perfusion monitoring.

Parameter	ME (SL/IL)	WSSD	WSSD/GM Ratio	TE	ICC
AUC ACh (PU)	1,858.22 (10,015.76/344.76)	5,173.05	0.52	5,111.01	0.338
AUC SNP (PU)	1,976.5 (9,834.75/397.22)	7,863.02	0.67	7,904.24	0.312
AUC TH (PU)	−17,699.67 (−97,193.05/−3,223.26)	53,159.32	0.56	52,776.22	0.160
AUC PORH (PU)	−168.08 (−1,024.57/−27.57)	659.88	0.83	663.05	0.087

Ach  =  acetylcholine, AUC  =  area under the blood flow/time curve, ICC  =  intraclass correlation coefficient, ME  =  mean error (SL  =  superior limit and IL  =  inferior limit), SNP  =  sodium nitroprusside, PORH  =  post-occlusive reactive hyperemia, PU  =  arbitrary perfusion units, TH  =  thermal hyperemia, Ratio WSSD/GM  =  ratio between within-subject standard deviation and global mean values, TE  =  typical error, and WSSD  =  within-subject standard deviation.

Intra-class correlation coefficients

The ICC was determined for each variable for all subjects, and the data are presented in Table 2. All of the variables studied showed poor ICCs (<0.35). Similar to the other repeatability indices presented above, the ICC values were much lower for the AUC for PORH than for the AUC for ACh. The AUCs for SNP and TH presented intermediate values.

Sample size calculations

Sample size estimates using the minimum differences that were expected to be statistically significant for the different microvascular reactivity measurement techniques in longitudinal studies using LDPM are presented in Table 3. The AUC for PORH required a much larger sample size than did the other techniques. The AUC for ACh administration required the smallest sample size, and the AUCs of SNP and TH required intermediate values.

Table 3.

Estimates of the minimum differences expected to be statistically significant according to the sample size under different conditions of microvascular reactivity in longitudinal studies using laser Doppler perfusion monitoring.

	Sample size
Parameter	50	25	20	10
AUC-ACh (PU)	2,927	4,139	4,627	6,544
AUC-SNP (PU)	4,448	6,291	7,033	9,947
AUC-TH (PU)	30,072	42,528	47,548	67,242
AUC-PORH (PU)	374	528	591	835

Ach  =  acetylcholine, AUC  =  area under the blood flow/time curve, SNP  =  sodium nitroprusside, PORH  =  post-occlusive reactive hyperemia, PU  =  arbitrary perfusion units, and TH  =  thermal hyperemia.

DISCUSSION

The main finding of the present study is that there is important within-subject variability that is not related to systematic error in the evaluation of skin microvascular reactivity using single-point LDPM. Although it is non-invasive, operator-independent and easy to perform, LDPM is hampered by its poor repeatability, particularly in measures obtained using physiological stimuli, such as thermal and post-occlusive hyperemia. Thus, the random error inherent in LDPM repeatability should be thoroughly evaluated when calculating the statistical power and sample sizes in longitudinal studies evaluating endothelial function as a surrogate marker for cardiovascular disease.

LDPM coupled to skin iontophoresis of endothelial-dependent vasodilators, essentially acetylcholine, has already been used to demonstrate the existence of endothelial dysfunction in several cardiometabolic disorders, including arterial hypertension27,28 , type 1^29–31 and type 2 diabetes,32 insulin resistance33 and the metabolic syndrome itself.34 However, all of the studies mentioned above were cross-sectional and observational, and they included only a small number of subjects (typically fewer than 50 patients/healthy volunteers).

Although most investigators limit their statistical analysis to the calculation of correlation coefficients, it has been well established that the simple use of correlation coefficients to evaluate the repeatability of methods is not adequate.25 In fact, this approach evaluates the existence of an association between values measured on different days but not the agreement (repeatability) of the measures.25 An evaluation of the day-to-day repeatability of LDPM ACh responses using just the coefficients of variation yielded values of up to 42%.15

When calculating the repeatability of methods, it is important to differentiate intervention effects from measurement errors. Measurement errors can be divided into two main components: random and systematic errors. Random error is always present in a measurement and is not related to the clinical intervention because it is caused by inherently unpredictable fluctuations in the readings of the measuring equipment. By contrast, systematic errors are predictable biases in measurement, also not related to the clinical intervention, that can be attributed to the experimental protocol. Therefore, they can be observed in all of the subjects in question, and they interfere directly with the accuracy of the measurement. In this situation, the effect of a clinical intervention must be greater than the systematic error for it to be detected and considered a real effect regardless of the sample size. By contrast, random error directly affects the sample size required to achieve sufficient power to detect the effects of a clinical intervention.

The results of this study show that there is important random error in the measurements of microvascular reactivity performed one week apart in healthy subjects. Moreover, we did not detect a systematic error in the measurements. In fact, the values of the AUC of cutaneous vasodilatation for all of the techniques in question were not significantly different between days 1 and 2, indicating that there was no systematic error between days. Moreover, the minimum differences expected to be statistically significant according to the sample size for the different techniques showed that the endothelium-dependent responses induced by ACh were much more adequate than the responses induced by PORH because their variability was significantly lower. This result was confirmed by the analysis of the ratio between the within-subject standard deviations and the global means of the values, in which the smallest AUC value was obtained from the ACh administration. In addition, we found that sample sizes of about 50 and 25 subjects are necessary to detect changes of about 30 and 40%, respectively, in the AUC following ACh administration. Our results also showed that the variability observed in the responses to SNP and TH was comparable to that observed in ACh administration. These results imply that the endothelium-dependent response of the cutaneous microcirculation can be better evaluated using pharmacological rather than physiological provocation. These findings confirm our previous results showing that the skin microvascular vasodilatory response to iontophoresis with ACh is significantly reduced in patients with type 1 diabetes compared to healthy subjects, while the responses to PORH and skin heating are preserved.29 In that study, a significant difference of ∼37% between diabetic patients and healthy controls in the AUC following ACh administration was detected using a sample of 50 diabetic patients and 46 control subjects. The absence of significant differences in the responses observed using physiological stimulation in our previous study29 may be explained by the higher variability compared to pharmacological (ACh) stimulation.

In the present study, the analysis of the measurements obtained on the two days using the method of Bland and Altman25 indicated that most of the differences were within the 95% confidence intervals. The AUC values obtained using ACh administration exhibited a more uniform distribution than those obtained from endothelial-dependent vasodilatation induced by physiological stimulation (PORH and TH).

The intraclass correlation coefficient (ICC), a statistical measure of between-subject repeatability, was also lower for the microvascular reactivity resulting from PORH compared to that resulting from ACh. Our results are in close agreement with those of Roustit et al.,16 who demonstrated that the low ICC values obtained from skin LDPM in the forearm using PORH are related to the high anatomical variability of capillary density. Even if we had used a standardized forearm recording site in the present study, which is rather difficult to maintain in longitudinal studies, the low repeatability may also have been explained by the anatomical variation in capillary density at this recording site. Moreover, in contrast to the vasodilatation curves induced by ACh, SNP and TH, which always had identical time spans, the PORH curves were recorded until the microvascular flow returned to the baseline values, thus generating recordings of different durations. This characteristic could have contributed to the poor repeatability of the microvascular flow responses to PORH. Finally, ICC is by definition population-based and thus cannot be used for study design and sample size calculations.

Some limitations of our study population should be discussed. About 50% of the study volunteers were healthy young women who were taking oral contraceptives. Measurement of the day-to-day variability of LDPM using measurements obtained one week apart implies different hormonal states at days 1 and 2. Considering that the aim of the study was to investigate the variability of methodologies for evaluating microvascular function under real-life conditions, it was important to include volunteers of both sexes. Other studies testing the variability of laser-Doppler techniques used similar study populations. Roustit et al.16 and Cracowski et al.,35 for example, also investigated single-point LDPM using a population that included 50% females, all of whom were taking oral contraceptives. Other studies that have included up to 50% females do not report on their use of contraceptives or the phases of their menstrual cycles.18,36–41 Finally, the day-to-day variability was not different between the males and females in our study.

In conclusion, our results confirm and extend those of Roustit et al.16 by showing that single-point laser Doppler monitoring has low reproducibility in the human forearm. Newer full-field imaging methods, such as laser Doppler imaging and laser speckle contrast imaging, appear to have a much better inter-day reproducibility42 than LDPM, and thus their use should be encouraged. Nevertheless, a recent study comparing LDPM skin blood-flow responses elicited by acetylcholine to flow-mediated dilation in the brachial artery43 showed a high correlation between the endothelial responses in the large arteries and skin microcirculation. Finally, considering the important random error in the day-to-day variability of evaluating microvascular endothelial function using single-point LDPM, we propose the use of an original and robust statistical methodology for designing prospective clinical studies.

Financial support: This investigation was supported by grants from FAPERJ (Fundação de Amparo à Pesquisa, Rio de Janeiro, Brazil) and CNPq (Conselho Nacional de Desenvolvimento Tecnológico, Brasília, Brazil). Sandro Sperandei has a doctoral fellowship from CNPq (Conselho Nacional de Desenvolvimento Tecnológico, Brasília, Brazil).