External validation of the current scoring systems and derivation of a novel scoring system to predict stone free rates after retrograde intrarenal surgery in patients with cumulative stone diameter of 2–4 cm

Abstract

Objective

Our primary aim is to perform the external validation of the current scoring systems in predicting stone-free status (SFS) after retrograde intrarenal surgery (RIRS) for renal stones 2–4 cm and develop a novel scoring system by re-examining possible predictive factors related to SFS.

Methods

Patients who underwent RIRS due to renal stones with a cumulative stone diameter of 2–4 cm between January 2017 and March 2021 were retrospectively screened. Residual stones ≤2 mm were defined as clinically insignificant, and these cases were considered to have SFS. Possible predictive factors related to SFS were examined using the multivariate logistic regression analysis. A nomogram and a scoring system were developed using independent predictive variables. The prediction ability of the previous and the new scoring system were evaluated with the ROC analysis.

Results

The existing scoring systems were found to be insufficient in predicting SFS (AUC < 0.660 for all). The independent predictors of SFS were identified as stone surface area (OR: 0.991, p < 0.001), stone density (OR: 0.998, p < 0.001), number of stones (OR: 0.365, p = 0.033), and stone localization (p = 0.037). Using these predictive markers, a new scoring system with a score ranging between 4 and 15 was developed. The AUC value for this scoring system was 0.802 (0.734−0.870).

Conclusion

The RUSS, S-ReSC and R.I.R.S. scoring systems and Ito’s nomogram failed to predict SFS in stones >2 cm. The SFS predictive ability of our new scoring system was higher in >2 cm stones compared to the other scoring systems.

Resumen

Objetivo

Nuestro objetivo principal es realizar la validación externa de los sistemas de puntuación actuales para predecir el estado libre de cálculos (ELC) después de la cirugía intrarrenal retrógrada (CRIR) para cálculos renales de 2−4 cm y desarrollar un nuevo sistema de puntuación reexaminando los posibles factores predictivos relacionados con el ELC.

Métodos

Se evaluaron retrospectivamente los pacientes que recibieron CRIR para el tratamiento de cálculos renales con diámetro acumulado de 2−4 cm, entre enero de 2017 y marzo de 2021. Los cálculos residuales ≤2 mm se definieron como clínicamente insignificantes, y estos casos se consideraron como ELC. Se examinaron los posibles factores predictivos relacionados con el ELC mediante el análisis de regresión logística multivariante. Se elaboró un nomograma y se creó un sistema de puntuación utilizando variables predictivas independientes. Mediante el análisis ROC se evaluó la capacidad de predicción de los sistemas de puntuación actuales y del recién desarrollado.

Resultados

Los sistemas de puntuación existentes resultaron insuficientes para predecir el ELC (AUC < 0,660 en todos los casos). Se identificaron como predictores independientes del ELC el área de superficie (OR: 0,991, p < 0,001), la densidad (OR: 0,998, p < 0,001), el número (OR: 0,365, p = 0,033) y la localización de los cálculos (p = 0,037). Utilizando estos marcadores predictivos, se desarrolló un nuevo sistema de puntuación cuyos resultados oscilan entre 4 y 15. El valor AUC de este sistema de puntuación fue de 0,802 (0,734–0,870).

Conclusión

Los sistemas de puntuación RUSS, S-ReSC y R.I.R.S. y el nomograma de Ito no lograron predecir el ELC en cálculos de >2 cm. Nuestro nuevo sistema de puntuación tuvo una capacidad predictiva del ELC mayor en cálculos de >2 cm, en comparación con los otros sistemas de puntuación.

Introduction

Percutaneous nephrolithotomy (PCNL) is recommended in current guidelines for the treatment of renal stones larger than 2 cm.¹ Despite its high stone-free rate (SFR), PCNL causes severe complications. PCNL is also limited or contraindicated for patient-related reasons, including morbid obesity, anticoagulant therapy, severe obstructive pulmonary disease, and extremity contractures/severe kyphoscoliosis.² With the recent advances in flexible ureterorenoscopy (f-URS), as well as increased experience, retrograde intrarenal surgery (RIRS) now has similar SFR and lower complication rates compared to PCNL in large renal stones.^3–5 As a result of the increased use of fURS and its promising results, treatment selection becomes challenging for stones 2−4 cm. In these patients, the selection of an appropriate surgical method by preoperatively predicting stone-free status (SFS) is critical in preventing both PCNL complications and anesthesia burden created by multi-session RIRS. Scoring systems are used to predict operation success in urinary system stones, inform patients, and standardize reporting of academic studies. To date, the Resorlu-Unsal stone score (RUSS),⁶ modified Seoul National University Renal Stone Complexity (S-ReSC),⁷ and R.I.R.S.⁸ score and Ito’s nomogram-based scoring system⁹ have been defined to predict SFS after RIRS. The original studies of these scoring systems reported that they could predict SFS.^6–10 However, it is seen that a specific grading of >2 cm is not undertaken, and all stones larger than 2 cm are graded as the worst group in all scoring systems except RUSS. To our knowledge, there is no study in the literature evaluating the efficacy of the existing scoring systems in stones larger than 2−4 cm. Therefore, the current study aimed to perform the external validation of the existing scoring systems for patients with renal stones of 2−4 cm and to develop a scoring system by re-examining the factors related to SFS.

Material and methods

Patients who underwent RIRS due to renal stones with a cumulative stone diameter (CSD) of 2−4 cm in the centers participated in the study between January 2017 and March 2021 were retrospectively screened. Patients younger than 18 years, those with staghorn or ureteral stones, and those with lacking data to be analyzed were excluded from the study. The study was conducted in accordance with the Helsinki Declaration revised in 2013. Ethical approval was obtained from the Local Ethics Committee (Decision No: 2021-11-09).

In the centers participated in the study, the general approach to urinary system stones is as follows: PCNL is recommended for stones larger than 2 cm; however, in patients using anti-coagulant and anti-aggregant drugs, those with residual stones after a failed PCNL procedure, or those that are concerned about PCNL complications, f-URS is indicated as the first or second option depending on the preference of the patient and/or surgeon.

All operations were performed by two experienced surgens at the centers participating in the study. Ureteral access sheath (10.7 or 12 F, Cook Medical, or 11/13 or 13/15 F, Boston Scientific) was placed in all patients. All procedures were performed by different makes. of f-URS (7.5 F; Karl Storz Flex-X2, Tuttlingen, Germany and Olympus P-5TM, Olympus, Tokyo, Japan) and 270 μm Holmium laser. Preoperative D-J stent placement was applied in cases with treatment-resistant renal colic, pyelonephritis, and a narrow ureter that could prevent access to the stone. A postoperative D-J stent or ureteral catheter was placed according to the surgeon’s preference and clinical necessity. Uretral catheter was removed at POD 1, D-J stent was removed 2−4 weeks after the procedure.

SFS was evaluated with non-contrast computed tomography (NCCT) in the first month. Residual stones of 2 mm were defined as clinically insignificant, and SFS was considered to be achieved in these patients.

Clinical characteristics of patients, stone characteristics, and perioperative findings were obtained from the medical records and radiological images. Complications were graded according to the Clavien–Dindo classification. The degree of hydronephrosis was measured according to the Society For Fetal Urology Hydronephrosis Grading System.¹⁰ CSD was measured as the longest diameter in the axial or reconstructed coronal section of NCCT. Stone surface area (SA) was calculated using the formula, length x width x π x 0.25.¹¹ Stone volume (SV), and lower pole infundibulopelvic angle (IPA) were measured and calculated as described in previous studies.^8,9 Mean Hounsfield unit (HU) measurement was performed by taking the middle of the stone as the center and including its edges. If multiple stones were present, each stone CSD, SA, and SV were calculated independently and added to determine a total size. The HU value of multiple stones was calculated by averaging the individual HU average of each stone.

Current scoring systems

RUSS

This is a scoring system in which each of the parameters (i.e., stone size >20 mm, lower pole stone localization and IPA < 45 degrees, number of stones in different calyces >1, and abnormal renal anatomy in radiological imaging) is scored as 1 point. The authors also provided an additional score for each 10 mm for stones over 20 mm. The total score ranges from 0 to 5, varying according to stone size.⁶

Modified S-ReSC

In this scoring system, stones localization in the pelvicalyceal system are marked and scored individually. An additional score is given for stones located in the inferior pole. Stone localizations and their scoring are as follows: renal pelvis (#1), superior (#2) and inferior (#3) major calyceal groups, and the anterior and posterior minor calyceal groups of the superior (#4−5), middle (#6−7), and inferior calyx (#8−9).⁷

Ito’s nomogram

This is a nomogram-based scoring system that provides a total score between 0 and 25 and consists of the parameters of stone volume, lower pole stone localization, operator experience, hydronephrosis, and stone number. A high score indicates an increased probability of SFS.⁹

R.I.R.S

This scoring system comprises stone density (≤1,000 or >1,000), inferior pole localization (non-inferior or inferior with IPA >30 ° and IPA ≤30 °), renal infundibular length (≤25 or >25 mm), stone surface area (≤10 or >10 and ≤20 or >20 mm). The total score range is 4–10 points. High scores are associated with more complex stones.⁸

Statistical analysis

Categorical data were presented as numbers and column percentages. The conformance of continuous data to normal distribution was analyzed with the Shapiro–Wilk test. Data conforming to normal distribution were given as mean ± standard deviation, and those that were not normally distributed were given as median and interquartile range (IQR) values. In the comparison of continuous data, the independent-samples t-test or the Mann–Whitney U test was used as appropriate. Pearson’s chi-square test was conducted to compare categorical data. The ability of CSD, SA, SV, and scoring systems to predict SFS was evaluated using receiver operating characteristic (ROC) analysis. SA and SV were found to have similar AUC values in predicting SFS. SA was used in further analyses since it can be easily obtained and calculated from NCCT or kidney, ureter and bladder radiography in clinical practices.

The parameters are found to be significantly correlated with SFS, also those that were not statistically significant but considered to be associated with SFS were included in multivariate logistic regression analysis. A new nomogram was constructed using the regression coefficients of independent predictive variables. The predictive ability of the nomogram was evaluated with ROC analysis, and it was validated using the bootstrap (n = 1000) method. Then, a new scoring system was defined based on the score weights of variables in the nomogram. The scoring system was evaluated with the ROC analysis. A p-value of <0.05 was considered statistically significant. SPSS software package (version 23.0; IBM Corporation, Armonk, NY, USA) was used for statistical analyses and R-project software for the construction of nomogram and bootstrap validation.

Results

A total of 185 patients were included in the study. The median age of patients was 48 years (35–59). Median CSD, SA and SV were 28 (25–32) mm, 368 (280–462) mm² and 2,293 (1,933−2,732) mm³, respectively. The most common stone localization was pelvis (28.6%), multi-calyces (28.6%), and lower calyx (27.6%). Total SFR were 65.9% and the complication rates were 13.5%. Major complications occurred in 3 (1.6%) patients (urosepsis in 2 patients and ureteral stricture in one patient). The comparison of the characteristics and clinical features of patients according to SFS is tabulated in Table 1.

Table 1.

Comparison of the patient characteristics according to postoperative SFS.

Variables	Non-SFS (n = 63)	SFS (n = 122)	P value
Age, years	47 (30−56)	51 (36−61)	0.197
Gender, n (%)			0.489
Female	26 (41.3)	44 (36.1)
Male	37 (58.7)	78 (63.9)
BMI, kg/m^2	26 (24−27.6)	26.6 (24−29)	0.102
Side			0.399
Left	31 (49.2)	68 (55.7)
Right	32 (50.8)	54 (44.3)
ASA category, n (%)			0.183
ASA I	26 (41.3)	57 (46.7)
ASA II	35 (55.6)	54 (44.3)
ASA ≥ III	2 (3.2)	11 (9.0)
Previous surgery			0.678
Failed PNL	8 (12.6)	13 (10.6)
None	55 (87.4)	109 (89.4)
Preoperative stent, n (%)	12 (19.0)	27 (22.1)	0.626
Preoperative hydronephrosis, n(%)			0.129
Normal/mild (Grade 0−2)	42 (66.7)	25 (77.0)
Moderate/severe (Grade 3−4)	21 (33.3)	28 (23.0)
Stone location, n (%)*			0.076
Lower pole	25 (39.7)	26 (21.3)
Middle pole	4 (6.3)	15 (12.3)
Upper pole	2 (3.2)	7 (5.7)
Pelvis	14 (22.2)	39 (32.0)
Multi-calyceal	18 (28.6)	35 (28.7)
Number of stones			0.051
Single	32 (50.8)	80 (65.6)
Multiple	31 (49.2)	42 (34.4)
Stone density, HU	1130 (716−1340)	726 (491−1236)	0.033
CSD, mm	30 (25−35)	25 (23.75−30)	<0.001
Stone area, mm^2	460 (368−567)	341 (264.5−413.5)	<0.001
Stone volume, mm^3	2700 (2250−3446)	2156 (1771−2453)	<0.001
Operation time, min	110 (90−130)	100 (80−115)	0.008
Length of hospital stay, days	2 (1−3)	1 (1−2)	0.002
Complication rates, n(%)	19 (30.2)	6 (4.9)	<0.001
Complication grades, n(%)*			0.001**
Clavien 1	14 (22.2) ^a	4 (3.3) ^b
Clavien 2	3 (4.8) ^a	1 (0.8) ^a
Clavien 3	1 (1.6) ^a	0 ^a
Clavien 4	1 (1.6) ^a	1 (0.8) ^a
RIRS score	7 (7−8)	7 (6−7)	0.004
Ito’s nomogram score	8 (5−10)	10 (7−10)	0.016
RUSS	2 (1−3)	1.5 (1−2)	0.001
Modified S-ReSC	2 (1−3)	1.5 (1−2)	0.051

SFS, stone free status; BMI, body mass index; ASA, American Society of Anesthesiologists; HU, Hounsfield unit; CSD, cumulative stone diameter, R.I.R.S.

The bold entries shows a p value of < 0.05 which is considered statistically significant.

*

No significant difference between the same superscripts.

**

Fisher–Freeman–Halton test.

It was determined that stone density, CSD, SA, SV, operation time, length of hospital stay, and complication rate were statistically lower in patients with SFS. While the R.I.R.S and RUSS scores and Ito’s nomogram were found to be associated with SFS, the modified S-ReSC score was not related. The ROC curves of the scoring systems, CSD, SA and SV for SFS prediction are shown in Fig. 1, and their cut-off, sensitivity, specificity and AUC values are tabulated in Table 2. The results show that the modified S-ReSC scoring system was not able to predict SFS. Although the remaining scoring systems and CSD predicted SFS, it was seen that their AUC values were off from the optimal value. SA and SV were found to provide good predictive values with AUC values of 0.756 and 0.753, respectively. Table 3 shows the results of multivariate logistic regression analysis of possible predictive factors. Only stone density and SA were found to be independent predictive factors for SFS. In the model created using the Backward elimination method, stone density, SA, multiple stones, and stone localization were found to be independent markers of SFS.

Figure 1.

Receiver operating characteristics (ROC) curve analysis of the scoring systems, cumulative stone diameter (CSD), stone volume (SV) and stone surface area (SA) in predicting stone-free status.

Table 2.

Cut-off, sensitivity, specificity and AUC values of the scoring systems, CSD, SA, and SV for predicting of stone-free status.

Stone >20 mm	Cut-off value	Sensitivity (%)	Specificity (%)	AUC value (95% CI)	P value
R.I.R.S. score	7.5	87.7	27.0	0.615 (0.530−0.701)	0.010
RUSS	1.5	50.0	71.4	0.637 (0.551−0.722)	0.002
Modified S-ReSC	–	–	–	0.582 (0.496−0.668)	0.067
Ito's nomogram	7.5	73.8	50.8	0.658 (0.578−0.738)	<0.001
CSD, mm	25.5	51.6	74.6	0.668 (0.586−0.751)	<0.001
SA, mm^2	382	68.0	73.0	0.756 (0.680−0.832)	<0.001
SV, mm^3	2393	72.1	68.3	0.753 (0.677−0.828)	<0.001

AUC, area under the curve; CSD, cumulative stone diameter; SA, stone area; SV, stone volume.

The bold entries shows a p value of < 0.05 which is considered statistically significant.

Table 3.

Multivariate logistic regression analysis of the independent predictors of post-operative stone-free status.

	Multivariate		Reduced (Backward)
Variables	OR (95% CI)	P value	OR (95% CI)	P value
Stone location
Lower calyx	Ref.	0.169	Ref.	0.037
Middle calyx	9.600 (1.489−61.893)	0.017	7.379 (1.579−34.483)	0.011
Upper calyx	3.877 (0.454−33.144)	0.216	3.336 (0.501−22.222)	0.213
Pelvis	4.901 (1.102−21.804)	0.037	3.920 (1.410−10.894)	0.009
Multi-calyceal	1.699 (0.444−6.504)	0.439	1.687 (0.582−4.887)	0.336
Stone density	0.998 (0.996−0.999)	0.001	0.998 (0.997−0.999)	<0.001
SA	0.989 (0.985−0.993)	<0.001	0.991 (0.988−0.994)	<0.001
Number of stones
Single stone	Ref.		Ref.
Multiple stone	0.328 (0.091−1.181)	0.088	0.365 (0.145−0.921)	0.033
Moderate/severe hydronephrosis	0.776 (0.311−1.935)	0.586
R.I.R.S. score	2.202 (0.777−6.238)	0.137
Ito's nomogram	1.059 (0.880−1.276)	0.543
RUSS	1.695 (0.792−3.632)	0.174
Modified S-ReSC	0.614 (0.282−1.339)	0.220

OR, odds ratio; CI, confidence interval; SA, stone area.

The bold entries shows a p value of < 0.05 which is considered statistically significant.

Nomogram and simple scoring system construction

A nomogram was created using the parameters determined as predictive factors for SFS in multivariate analysis. The AUC value of the nomogram was determined as 0.818 (0.755−0.882). In internal validation using the bootstrap method, the optimized mean AUC value of the nomogram was 0.795 (Figs. 2 and 3). In order to be easily applicable in the clinical practice, a simple scoring system was developed (Fig. 2). SA was graded over 5 points, stone density over 3 points, and stone number and localization over 2 points each. The total score ranged from 4 to 12. A high score indicated an increased probability of SFS. The new scoring system was applied to patients in our sample one by one, and their internal validation was undertaken. The mean score was 9.2 ± 1.13 in the group with SFS and 7.6 ± 1.46 in the group that did not achieve SFS (p < 0.001). The AUC value of the scoring system was 0.802 (0.734−0.870) (Figs. 2 and 3). The cut-off value of the scoring system for SFS prediction was 8.5, at which it had 71.3% sensitivity and 73.0% specificity.

Figure 2.

Nomogram and scoring system developed for practical use based on the results of the multivariate regression analysis.

Figure 3.

Receiver operating characteristic curve analysis of the newly developed nomogram (left) and scoring system (right) in the prediction of stone-free status.

Discussion

RIRS is a less invasive treatment modality than PCNL in the treatment of renal stones. The advantages of f-URS and technological developments have led f-URS to become an increasingly preferred method in the treatment of large renal stones.^3,12 In prospective studies comparing f-URS with standard PCNL (stones >20 mm) and mini-PCNL (stones between 20 and 30 mm), f-URS was reported to have a final SFR comparable to PCNL and indicated as an appropriate treatment option for large renal stones.^13,14 In studies, mini-PNL has a similar operation time and higher initial SFR compared to fURS in 2−3 cm stones. However, fURS has lower hospitalization time, lower complication rates, and a final SFR similar to miniPNL with an average of 1.5 procedures.¹⁴ According to a recent meta-analysis, the initial SFR of f-URS was 71.2%, and the final SFR was 89.4% with a mean number of 1.4 procedures.¹⁵ In another study, the initial SFR was 73.3% and the final SFR was 93.0% with a mean number of 1.2 procedures for 2–2.9 cm stones and the final SFR was 84.5% with a mean number of 1.8 procedures for 3–3.9 cm stones. In the same study, overall SFR was found to be 61.9% in a single session.³ In our study, overall SFR after a single session 65.9%, and our complication rate of 13.5% was found to be similar to the literature.^16,17

An accurate prediction of SFR is important when considering surgical modalities and necessity of any ancillary procedures. In this context, many scoring systems have been developed for RIRS. RUSS, S-ReSC, Ito’s nomogram, and R.I.R.S. are the most widely known scoring systems.^6–9 Single and multiple validation studies of these scoring systems have been conducted and sufficiently discussed.^16,18–21 However, all studies on scoring systems and their validation except Ito’s nomogram consisted of patients with stones <2 cm. Ito et al. developed a nomogram in a series with a mean stone size of 22 mm and reported the AUC value was 0.870. In the validation studies, the AUC value was reported to range from 0.697 to 0.735 in stones a mean size of 10.9–15.6 mm.^9,16,18 Although Ito’s nomogram included patients with stones >2 cm, its usability in large renal stones remains unclear since both mean stone size being close to 2 cm and simultaneous presence of <2 cm stones would cause a positive bias.

In our study, the external validation of the existing scoring systems in renal stones 2−4 cm was performed, and it was observed that they did not have sufficient ability to predict SFS. CSD, SA, and SV were each found to be superior to scoring systems when evaluated individually. It was determined that the scoring system with the poorest performance was modified S-ReSC (AUC: 0.582). This outcome can be attributed to fact the that this scoring system does not contain the characteristic features of stones. According to this scoring system, a 1 cm stone and a 3 cm stone can be assigned the same score if they have the same localization. Also, the authors recommended PCNL for larger stones, therefore they excluded these stones from their analysis.^7,10 It was also observed that Ito’s nomogram (AUC: 0.658) and RUSS (AUC: 0.637) performed better than other scoring systems. The most important feature that distinguishes these scoring systems from the others is the grading of large stones.^6,9 Although Ito’s nomogram graded SV and RUSS graded stones over 20 mm, they did not fully predict SFS. This can be caused by the use of stone diameter to indicate stone burden in the RUSS scoring system. CSD does not fully reflect the stone burden in large renal stones.²² Ito’s nomogram and RUSS do not include stone density whose correlation with SFS are shown in many studies.²³ Although it is known that laser lithotripsy can fragment any stone, there is more indirect fragmentation and shorter operation time in stones with low density.^23,24 In addition, Ito’s nomogram detected the presence of hydronephrosis as an independent marker, while the presence of moderate/severe hydronephrosis was not detected as a marker in our data set.⁹ All of the issues indicated above may explain the insufficiency of RUSS and Ito’s nomogram in predicting SFS.

The studies which evaluated predictive factors for SFS after RIRS in large renal stones, reported stone burden, lower calyceal localization, multicalyceal localization, and moderate/severe hydronephrosis as predictive factors.^25,26 In our study, SA, stone density, the number of stones, and stone localization were determined as independent predictive markers for SFS. The relation between IPA and SFR has been shown in the literature.²⁷ However, in our study, although the SFR was low in patients with a low IPA, it was not statistically significant. The low number of patients with low IPA and patients with other stone localizations besides the lower calyx may have caused this. The lower calyx and multi-calyceal stones had the lowest SFRs at 51.0% and 66.0%, respectively. Huang et al. reported similar results for lower calyceal and multi-calyceal stones.³ In our study, although the history of failed PCNL and the presence of hydronephrosis was high among patients with residual stones, this was not statistically significant. We think that the low SFR in patients with failed PCNL may be related to the difficulty of accessing the stone due to anatomical changes and the decrease in the spontaneous stone passage. Similar results were obtained in studies by Guzel et al.²⁸ In our study, preoperative stenting was not found to be associated with SFS. In the literature, it is stated that preoperative stenting increases the success of ureteral access sheath application and which is not associated with SFS.²⁹

Our study is the first in the literature to evaluate the efficacy of the existing scoring systems in predicting the success of RIRS in stones 2−4 cm and presents a new scoring system developed based on predictive markers, which was confirmed to be suitable for use in large stones (AUC value 0.802). The main limitations of our study include its retrospective nature and the limited number of patients. Other limitations are the exclusion of complex cases such as musculoskeletal anomaly and staghorn stones, and the inability to evaluate stone compositions. On the other hand, our study has certain strengths, such as standardization of SFS by performing a control NCCT at the first month in all patients, multi-centered design, and the reproducibility of the results. However, there is still a need for prospective studies with a larger series focusing particularly on complications and need validation in smaller stones.

Conclusion

The developed scoring system based on SA, stone density, number of stones, and stone localization was able to predict SFS much better compared to the existing scoring systems. Our new scoring system can reliably predict SFS in patients with stones 2−4 cm, guiding the surgeon’s decision concerning optimal treatment.

Funding

No funding was received for this work.

Conflict of interest

None of the authors have conficts or competing interests.