Skip to main content
SpringerLink
Log in

Robustness versus disease differentiation when varying parameter settings in radiomics features: application to nasopharyngeal PET/CT

  • Molecular Imaging
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To investigate the impact of parameter settings as used for the generation of radiomics features on their robustness and disease differentiation (nasopharyngeal carcinoma (NPC) versus chronic nasopharyngitis (CN) in FDG PET/CT imaging).

Methods

We studied 106 patients (69/37 NPC/CN, pathology confirmed), and extracted 57 radiomics features under different parameter settings. Robustness was assessed by the intra-class correlation coefficient (ICC). Logistic regression with leave-one-out cross validation was used to generate classification probabilities, and diagnostic performance was assessed by the area under the receiver operating characteristic curve (AUC).

Results

Varying averaging strategies and symmetry, 4/26 GLCM features showed poor range of pairwise ICCs of 0.02–0.98, while depicting good AUCs of 0.82–0.91. Varying distances, 5/26 GLCM features showed ICCs of 0.82–0.99 while corresponding AUCs were 0.52–0.91. 6/13 GLRLM features showed both high AUC (0.81–0.89) and high ICC (0.85–0.99) regarding to averaging strategies. 7/13 GLSZM features showed AUCs of 0.81–0.90 while having ICCs of 0.01–0.99 under different neighbourhoods. 2/5 NGTDM features showed AUCs of 0.81–0.85 while having ICCs of 0.19–0.89 for different window sizes. Differentiating a subset of NPC (stages I–II) form CN, both SumEntropy and SZLGE achieved significantly higher AUCs than metabolically active tumour volume (AUC: 0.91 vs. 0.72, p<0.01).

Conclusions

Radiomics features depicting poor absolute-scale robustness regarding to parameter settings can still lead to good diagnostic performance. As such, robustness of radiomics features should not be overemphasized for removal of features towards assessment of clinical tasks. For differentiating NPC from CN, some radiomics features (e.g. SumEntropy, SZLGE, LGZE) outperformed conventional metrics.

Key Points

• Poor robustness did not necessarily translate into poor differentiation performance.

• Absolute-scale robustness of radiomics features should not be overemphasized.

• Radiomics features SumEntropy, SZLGE and LGZE outperformed conventional metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

AUC:

Area under the ROC curve

CN:

Chronic nasopharyngitis

18F-FDG:

2-[18F]-fluoro-2-deoxy-D-glucose

GLCM:

Grey level co-occurrence matrix

GLRLM:

Grey level run length matrix

GLSZM:

Grey level size zone matrix

ICC:

Intra-class correlation coefficient

LOOCV:

Leave-one-out cross validation

MATV:

Metabolically active tumour volume

NGTDM:

Neighbourhood grey tone difference matrix

NPC:

Nasopharyngeal carcinoma

ROC:

Receiver operating characteristic

SUV:

Standardized uptake value

TLG:

Total lesion glycolysis

References

  1. Liu FY, Lin CY, Chang JT et al (2007) 18F-FDG PET can replace conventional work-up in primary M staging of nonkeratinizing nasopharyngeal carcinoma. J Nucl Med 48:1614–1619

    Article  PubMed  Google Scholar 

  2. O'Donnell HE, Plowman PN, Khaira MK, Alusi G (2008) PET scanning and Gamma Knife radiosurgery in the early diagnosis and salvage "cure" of locally recurrent nasopharyngeal carcinoma. Br J Radiol 81:e26–e30

    Article  PubMed  CAS  Google Scholar 

  3. Ng SH, Chan SC, Yen TC et al (2009) Staging of untreated nasopharyngeal carcinoma with PET/CT: comparison with conventional imaging work-up. Eur J Nucl Med Mol Imaging 36:12–22

    Article  PubMed  Google Scholar 

  4. Wu H, Wang Q, Wang M, Zhen X, Zhou W, Li H (2011) Preliminary study of 11C-choline PET/CT for T staging of locally advanced nasopharyngeal carcinoma: comparison with 18F-FDG PET/CT. J Nucl Med 52:341–346

    Article  PubMed  CAS  Google Scholar 

  5. King AD, Ma BB, Yau YY et al (2008) The impact of 18F-FDG PET/CT on assessment of nasopharyngeal carcinoma at diagnosis. Br J Radiol 81:291–298

    Article  PubMed  CAS  Google Scholar 

  6. Strauss LG (1996) Fluorine-18 deoxyglucose and false-positive results: a major problem in the diagnostics of oncological patients. Eur J Nucl Med 23:1409–1415

    Article  PubMed  CAS  Google Scholar 

  7. van Waarde A, Cobben DC, Suurmeijer AJ et al (2004) Selectivity of 18F-FLT and 18F-FDG for differentiating tumor from inflammation in a rodent model. J Nucl Med 45:695–700

    PubMed  Google Scholar 

  8. Hustinx R, Smith RJ, Benard F et al (1999) Dual time point fluorine-18 fluorodeoxyglucose positron emission tomography: a potential method to differentiate malignancy from inflammation and normal tissue in the head and neck. Eur J Nucl Med 26:1345–1348

    Article  PubMed  CAS  Google Scholar 

  9. Wahl RL (2008) Principles and practice of PET and PET/CT. Lippincott Williams & Wilkins, Philadelphia

    Google Scholar 

  10. Gerlinger M, Rowan AJ, Horswell S et al (2012) Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. New Engl J Med 366:883–892

    Article  PubMed  CAS  Google Scholar 

  11. Parekh V, Jacobs MA (2016) Radiomics: a new application from established techniques. Expert Rev Precis Med Drug Dev 1:207–226

    Article  PubMed  PubMed Central  Google Scholar 

  12. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: Extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446

    Article  PubMed  PubMed Central  Google Scholar 

  13. Aerts HJ, Velazquez ER, Leijenaar RT et al (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006

    Article  PubMed  CAS  Google Scholar 

  14. Mu W, Chen Z, Liang Y et al (2015) Staging of cervical cancer based on tumor heterogeneity characterized by texture features on 18F-FDG PET images. Phys Med Biol 60:5123–5139

    Article  PubMed  Google Scholar 

  15. Yip SS, Coroller TP, Sanford NN, Mamon H, Aerts HJ, Berbeco RI (2016) Relationship between the temporal changes in positron-emission-tomography-imaging-based textural features and pathologic response and survival in esophageal cancer patients. Front Oncol 6:72

    Article  PubMed  PubMed Central  Google Scholar 

  16. Coroller TP, Agrawal V, Narayan V et al (2016) Radiomic phenotype features predict pathological response in non-small cell lung cancer. Radiother Oncol 119:480–486

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wu W, Parmar C, Grossmann P et al (2016) Exploratory study to identify radiomics classifiers for lung cancer histology. Front Oncol 6:71

    PubMed  PubMed Central  Google Scholar 

  18. Soussan M, Orlhac F, Boubaya M et al (2014) Relationship between tumor heterogeneity measured on FDG-PET/CT and pathological prognostic factors in invasive breast cancer. PLoS One 9:e94017

    Article  PubMed  PubMed Central  Google Scholar 

  19. Lovinfosse P, Janvary ZL, Coucke P et al (2016) FDG PET/CT texture analysis for predicting the outcome of lung cancer treated by stereotactic body radiation therapy. Eur J Nucl Med Mol Imaging 43:1453–1460

    Article  PubMed  CAS  Google Scholar 

  20. Tixier F, Hatt M, Valla C et al (2014) Visual versus quantitative assessment of intratumor 18F-FDG PET uptake heterogeneity: prognostic value in non-small cell lung cancer. J Nucl Med 55:1235–1241

    Article  PubMed  CAS  Google Scholar 

  21. El NI, Grigsby P, Apte A et al (2009) Exploring feature-based approaches in PET images for predicting cancer treatment outcomes. Pattern Recognit 42:1162–1171

    Article  Google Scholar 

  22. Win T, Miles KA, Janes SM et al (2013) Tumor heterogeneity and permeability as measured on the CT component of PET/CT predict survival in patients with non-small cell lung cancer. Clin Cancer Res 19:3591–3599

    Article  PubMed  CAS  Google Scholar 

  23. Cheng NM, Fang YH, Lee LY et al (2015) Zone-size nonuniformity of 18F-FDG PET regional textural features predicts survival in patients with oropharyngeal cancer. Eur J Nucl Med Mol Imaging 42:419–428

    Article  PubMed  CAS  Google Scholar 

  24. Tixier F, Groves AM, Goh V et al (2014) Correlation of intra-tumor 18F-FDG uptake heterogeneity indices with perfusion CT derived parameters in colorectal cancer. PLoS One 9:e99567

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  25. Soufi M, Kamali-Asl A, Geramifar P, Rahmim A (2017) A Novel Framework for Automated Segmentation and Labeling of Homogeneous Versus Heterogeneous Lung Tumors in [F-18]FDG-PET Imaging. Mol Imaging Biol 19:456–468

    Article  PubMed  CAS  Google Scholar 

  26. Vallieres M, Freeman CR, Skamene SR, El Naqa I (2015) A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys Med Biol 60:5471–5496

    Article  PubMed  CAS  Google Scholar 

  27. Lambin P, Leijenaar R, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol14:749-762

  28. Aerts HJ (2016) The Potential of Radiomic-Based Phenotyping in Precision Medicine: A Review. JAMA Oncol 2:1636–1642

    Article  PubMed  Google Scholar 

  29. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: Images Are More than Pictures, They Are Data. Radiology 278:563–577

    Article  PubMed  Google Scholar 

  30. Yip SS, Aerts HJ (2016) Applications and limitations of radiomics. Phys Med Biol 61:R150–R166

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  31. Galavis PE, Hollensen C, Jallow N, Paliwal B, Jeraj R (2010) Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters. Acta Oncol 49:1012–1016

    Article  PubMed  PubMed Central  Google Scholar 

  32. van Velden FHP, Kramer GM, Frings V et al (2016) Repeatability of radiomic features in non-small-cell lung cancer [18F]FDG-PET/CT studies: Impact of reconstruction and delineation. Mol Imaging Biol 18:788–795

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  33. Doumou G, Siddique M, Tsoumpas C, Goh V, Cook GJ (2015) The precision of textural analysis in 18F-FDG-PET scans of oesophageal cancer. Eur Radiol 25:2805–2812

    Article  PubMed  Google Scholar 

  34. Hatt M, Tixier F, Cheze LRC, Pradier O, Visvikis D (2013) Robustness of intratumour 18F-FDG PET uptake heterogeneity quantification for therapy response prediction in oesophageal carcinoma. Eur J Nucl Med Mol Imaging 40:1662–1671

    Article  PubMed  Google Scholar 

  35. Leijenaar RT, Nalbantov G, Carvalho S et al (2015) The effect of SUV discretization in quantitative FDG-PET Radiomics: the need for standardized methodology in tumor texture analysis. Sci Rep 5:11075

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  36. Lu L, Lv W, Jiang J et al (2016) Robustness of radiomic features in [11C]choline and [18F]FDG PET/CT imaging of nasopharyngeal carcinoma: impact of segmentation and discretization. Mol Imaging Biol 18:935–945

    Article  PubMed  CAS  Google Scholar 

  37. Hatt M, Tixier F, Pierce L, Kinahan PE, Le Rest CC, Visvikis D (2017) Characterization of PET/CT images using texture analysis: the past, the present... any future? Eur J Nucl Med Mol Imaging 44:151–165

    Article  PubMed  Google Scholar 

  38. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cyb. SMC-3:610–621

    Article  Google Scholar 

  39. Soh L, Tsatsoulis C (1999) Texture Analysis of SAR Sea Ice Imagery Using Gray Level Co-Occurrence Matrices. IEEE T Geosci Remote 37:780–795

    Article  Google Scholar 

  40. Metser U, Jhaveri KS, Murphy G, Halankar J (2015) Multiparameteric PET-MR assessment of response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: PET, MR, PET-MR and tumor texture analysis: A pilot study. Adv Mol Imaging 5:49–60

    Article  Google Scholar 

  41. Roy A, Warbey V, Ferner R, O’Doherty M, Marsden P (2012) Feature based differentiation of benign, malignant and atypical neurofibroma in FDG-PET scans. J Nucl Med 53:2256

    Google Scholar 

  42. Rahmim A, Salimpour Y, Jain S et al (2016) Application of texture analysis to DAT SPECT imaging: Relationship to clinical assessments. Neuroimage Clin 12:e1–e9

    Article  PubMed  PubMed Central  Google Scholar 

  43. Gelzinis A, Verikas A, Bacauskiene M (2007) Increasing the discrimination power of the co-occurrence matrix-based features. Pattern Recogn 40:2367–2372

    Article  Google Scholar 

  44. Rahmim A, Salimpour Y, Blinder S, Klyuzhin I, Sossi V (2016) Optimized haralick texture quantification to track Parkinson’s disease progression from DAT SPECT images. J Nucl Med 57:428

    Google Scholar 

  45. Nanni L, Brahnam S, Ghidoni S, Menegatti E, Barrier T (2013) Different approaches for extracting information from the co-occurrence matrix. PLoS One 8:e83554

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  46. Hatt M, Majdoub M, Vallieres M et al (2015) 18F-FDG PET uptake characterization through texture analysis: investigating the complementary nature of heterogeneity and functional tumor volume in a multi-cancer site patient cohort. J Nucl Med 56:38–44

    Article  PubMed  CAS  Google Scholar 

  47. Yu H, Caldwell C, Mah K, Mozeg D (2009) Coregistered FDG PET/CT-based textural characterization of head and neck cancer for radiation treatment planning. IEEE Trans Med Imaging 28:374–383

    Article  PubMed  Google Scholar 

  48. Delbeke D, Coleman RE, Guiberteau MJ et al (2006) Procedure guideline for tumor imaging with 18F-FDG PET/CT 1.0. J Nucl Med 47:885–895

    PubMed  Google Scholar 

  49. Jiang J, Wu H, Huang M et al (2015) Variability of Gross Tumor Volume in Nasopharyngeal Carcinoma Using 11C-Choline and 18F-FDG PET/CT. PLoS One 10:e131801

    Google Scholar 

  50. Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc. 36:111–147

    Google Scholar 

  51. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    Article  PubMed  CAS  Google Scholar 

  52. Shiri I, Rahmim A, Ghaffarian P, Geramifar P, Abdollahi H, Bitarafan-Rajabi A (2017) The impact of image reconstruction settings on 18F-FDG PET radiomic features: multi-scanner phantom and patient studies. Eur Radiol 27:4498–4509

    Article  PubMed  Google Scholar 

  53. Bailly C, Bodet-Milin C, Couespel S et al (2016) Revisiting the robustness of PET-based textural features in the context of multi-centric trials. PLoS One 11:e159984

    Google Scholar 

  54. Orlhac F, Boughdad S, Nioche C, Alberini JL, Soussan M, Buvat I (2017) An original approach to deal with multi-center variability of PET textural features. J Nucl Med 58:506

    Article  Google Scholar 

  55. Lin C, Bradshaw T, Perk T, Harmon S, Liu G, Jeraj R (2015) Repeatability of [18F]-NaF PET imaging biomarkers for bone lesions: A multicenter study. Med Phys 42:3587

    Article  Google Scholar 

  56. Busson P (2013) Nasopharyngeal carcinoma keys for translational medicine and biology. Landes Bioscience and Springer Science+Business Media, Austin

    Book  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under grants 61628105, 81501541, U1708261, 61471188, 81271641, the National key research and development program under grant 2016YFC0104003, the Natural Science Foundation of Guangdong Province under grants 2016A030313577, and the Program of Pearl River Young Talents of Science and Technology in Guangzhou under grant 201610010011.

Author information

Authors and Affiliations

  1. School of Biomedical Engineering and Guangdong Provincal Key Laboratory of Medical Image Processing, Southern Medical University, 1023 Shatai Road, Baiyun District, Guangzhou, Guangdong, 510515, China

    Wenbing Lv, Jianhua Ma, Jun Jiang, Wei Yang, Qianjin Feng, Wufan Chen & Lijun Lu

  2. Nanfang PET Center, Nanfang Hospital, Southern Medical University, 1023 Shatai Road, Baiyun District, Guangzhou, Guangdong, 510515, China

    Qingyu Yuan & Quanshi Wang

  3. Department of Radiology, Johns Hopkins University, 601 N. Caroline St, Baltimore, MD, 21287, USA

    Arman Rahmim

  4. Department of Electrical and Computer Engineering, Johns Hopkins University, 3101 Wyman Park Drive, Baltimore, MD, 21218, USA

    Arman Rahmim

Authors
  1. Wenbing Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingyu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Quanshi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianhua Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qianjin Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wufan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Arman Rahmim
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Lijun Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jianhua Ma or Lijun Lu.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Dr. Lijun Lu.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• diagnostic study

• performed at one institution

Electronic supplementary material

ESM 1

(DOC 554 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lv, W., Yuan, Q., Wang, Q. et al. Robustness versus disease differentiation when varying parameter settings in radiomics features: application to nasopharyngeal PET/CT. Eur Radiol 28, 3245–3254 (2018). https://doi.org/10.1007/s00330-018-5343-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-018-5343-0

Keywords

Search

Navigation