검색
검색 팝업 닫기

Advanced search

Quick links

Most Cited Archives For Authors For Reviewers

Article

Split Viewer

Korean J Thorac Cardiovasc Surg 2015; 48(2): 112-119

Published online April 5, 2015 https://doi.org/10.5090/kjtcs.2015.48.2.112

Copyright © Journal of Chest Surgery.

Validity and Reliability of the Turkish Version of the Nuss Questionnaire Modified for Adults

A. Tugba Bahadir1, Pinar Kuru2, Ceyda Afacan3, Nezih Onur Ermerak4, Korkut Bostanci4, and Mustafa Yuksel4

1Department of Child & Adolescent Psychiatry, Northwestern University Feinberg School of Medicine, 2Department of Neurology, Northwestern University Feinberg School of Medicine, 3Statistical Expert, Marmara University School of Medicine, 4Department of Thoracic Surgery, Marmara University School of Medicine

Correspondence to:Corresponding author: A. Tugba Bahadir, Department of Child & Adolescent Psychiatry, Marmara University School of Medicine, Istanbul, Fevzi Cakmak Mah., Mimar Sinan Cad., No: 41, 34899, Ust Kaynarca, Pendik, Istanbul, Turkey, (Tel) 90-216-625-4545/1635 (Fax) 90-533-425-5888 (E-mail) tugba77@gmail.com

Received: July 21, 2014; Revised: November 3, 2014; Accepted: November 5, 2014

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Go to

Background

Pectus excavatum (PE) is the most common chest wall deformity. The Nuss Questionnaire modified for Adults (NQmA) is a disease-specific health-related quality of life assessment tool for patients with pectus deformities. The aim of this study is to adapt the NQmA into Turkish.

Methods

Two hundred and sixty-five patients with PE were participated, with an age range of 14 to 29 years. All patients underwent a physical examination and had not undergone corrective surgery. The Turkish version of the NQmA was completed by patients and their parents.

Results

The content validity index based on expert opinions was 91% for the patient questionnaire and 96% for the parent questionnaire. The Cronbach’s alpha value for the NQmA was found to be 0.805 for the patient questionnaire and 0.800 for the parent questionnaire. Exploratory factor analysis was used to assess construct validity. Two factors explained 51.1% of the total variance in the patient questionnaire (psychosocial: 31.145%, Cronbach’s alpha=0.818; physical: 19.955%, Cronbach’s alpha=0.862). In the parent questionnaire, two factors explained 51.422% of the total variance (psychosocial: 26.097%, Cronbach’s alpha=0.743; physical: 25.325%, Cronbach’s alpha=0.827). Construct validity was confirmed by confirmatory factor analysis.

Conclusion

The Turkish version of the NQmA was found to be valid and reliable for the assessment of quality of life in patients with PE.

Keywords: Funnel chest, Quality of life, Psychometrics, Validation studies

INTRODUCTION

Go to

Pectus excavatum (PE), also known as funnel chest, is the most common chest wall deformity, characterized by inward displacement of the sternum and the costal cartilages. PE occurs in males three to five times more often than in females, with an incidence of one case in every 400 to 1,000 births [1,2]. Physical limitations and/or psychosocial distress can be observed in patients with PE and may result in a decreased quality of life [3]. The physical limitations caused by PE are mostly cardiopulmonary restrictions resulting in fatigue, exercise intolerance, shortness of breath, palpitations, and chest discomfort or pain [4]. The psychosocial problems associated with PE are mostly characterized by low self-esteem and avoidance of social and physical activities due to the appearance of the chest [5].

Minimally invasive repair of PE, developed by Donald Nuss in 1987, is the current surgical approach for the correction of PE. Nuss et al. [6] published their 10-year experiences in 1998. Since that time, minimally invasive repair has mostly replaced the Ravitch procedure, because minimally invasive repair involves shorter operations and minimal surgical scarring [7]. However, Kang et al. [8] reported in 2012 that long-term satisfaction was not related to the operation type, but instead was related to the completeness of the correction and the absence of recurrence.

The primary aim of medical care is to increase the patient’s quality of life, which includes physical and psychosocial well-being. The Nuss Questionnaire was first published and validated by Lawson et al. [9] in 2003 as a disease-specific tool for the evaluation of quality of life in pediatric patients with PE. Krasopoulos et al. [10] modified the questionnaire for adults. Improvements in physical and psychosocial well-being, with a better quality of life, have been reported after the minimally invasive repair of PE [5,9?13].

In the present study we aimed to evaluate the validity and the reliability of the Turkish version of the Nuss Questionnaire modified for Adults (NQmA).

METHODS

Go to

Patients with PE (age range, 14 to 29 years) were recruited from the Thoracic Surgery Clinic of Marmara University Hospital. Patients with any previous thoracic surgery history or complex deformities were excluded. This study was approved by the Clinical Research Ethics Committee of the Marmara University School of Medicine. Informed consent was obtained from each participant. All patients underwent a physical examination, and posteroanterior and lateral chest radiographs were obtained. The Turkish version of the NQmA was used to assess the patients’ quality of life. Patients and their parents completed the questionnaires.

The two-step Nuss Questionnaire for pediatric patients and their parents is used to assess the disease-specific quality of life of patients with pectus deformities. Child and parent versions of the questionnaire were first published and validated by Lawson et al. [9]. The Nuss Questionnaire was modified for adult patients by Krasopoulos et al. [10], and the scoring of the first three questions was reversed.

The patient and parent questionnaires were translated into Turkish. Fourteen experts working with chest wall deformities and/or quality of life reviewed the parent and patient versions of the questionnaire for the suitability and comprehensibility of the items. The patient form of the NQmA included 12 items and the parent form included 13 items. Each item received a score between 1 and 4, with higher scores indicating a better quality of life (Tables 1, 2).

1) Statistical analysis

The AMOS ver. 6.0 (AMOS Development Co., SPSS Inc., Chicago, IL, USA) and SPSS ver. 15.0 (SPSS Inc., Chicago, IL, USA) software packages were used for statistical analysis. The content validity ratio and content validity index were used to examine the validity of the content based on the assessment of experts. Content validity means that the items in the test appropriately sample the content domain. The content validity ratio is used to determine whether specific test items should be retained or deleted. The content validity index is the mean of the content validity ratios of the remaining items in the test [14]. Cronbach’s alpha was calculated to assess the reliability of the overall questionnaire, as well as to assess its physical and psychosocial components individually. Cronbach’s alpha is a number between 0 and 1 [15] that measures internal consistency and reliability [16]. Internal consistency indicates the degree to which test items all measure the same concept [15]. In order to evaluate the contribution of each item to the questionnaire, item-total correlation coefficients and Cronbach’s alpha values if item deleted were also calculated. Exploratory factor analysis (EFA) was used to assess construct validity. EFA is used to reduce the amount of data by grouping related items together [16]. The Kaiser-Meyer-Olkin measure of sampling adequacy is a number between 0 and 1 that measures the adequacy of the data in terms of the sample size. Bartlett’s test of sphericity, which is a prerequisite for EFA, demonstrates whether the data have a multivariate distribution. Eigenvalues are used to decide the number of factors included in EFA [17], and each eigenvalue accounts for a certain percentage of the variance [18]. Confirmatory factor analysis (CFA) was used to confirm the construct validity. CFA is employed to determine whether the factor structure indicated by EFA is represented in the data or not [18], and fit indices are used to confirm the CFA model [17]. The fit indices that we evaluated were the root mean square error of approximation, the goodness of fit index, the comparative fit index, the standardized root mean square error, and the chi-square/degrees of freedom test. Spearman correlation analysis was carried out to compare the physical and psychosocial component scores of the patient and the parent questionnaires. For all analyses, p<0.05 was considered to indicate significance.

RESULTS

Go to

A total of 265 patients with PE participated to the study, of whom 233 (87.9%) were male and 32 (12.1%) were female. The mean age of the patients was 18.83±3.84 years (range, 14 to 29 years). Eighty-six (32.5%) of the patients had at least one associated disease or anomaly, and 21 patients (7.9%) had scoliosis. Symmetric deformities were observed in 176 (69%) of the patients, and the others had asymmetric deformities. A total of 265 patient and parent questionnaires were completed.

1) Expert opinions and content validity

We asked 14 experts for their opinions. The expected minimum content validity ratio for 14 experts was 51% [14]. Content validity ratios were provided for all items, and were between 57% and 100% in both of the questionnaires. The content validity index of the expert opinions was 91% for the patient questionnaire and 96% for the parent questionnaire. The content validity indices were 79% or higher for all of the items in both of the patient and the parent questionnaires [19].

2) Reliability and construct validity

(1) Patient questionnaire

The Cronbach’s alpha value of the 12 items in the patient questionnaire was 0.805, indicating that it was reliable. The item-total correlation coefficients of items in the patient questionnaire were between 0.28 and 0.63 (Table 3). No increase in the Cronbach’s alpha value was observed if any item was deleted.

(2) Parent questionnaire

Items were excluded from the questionnaire if they met the following criteria: (1) an item-total correlation coefficient <0.40, (2) a factor loading <0.40 [16], (3) if deleting the factor increased the Cronbach’s alpha value, and (4) if an item did not load on the expected factor. The sixth and thirteenth items were deleted based on all the above criteria. These question stems were ‘made’ fun of him/her’ and ‘have problems gaining weight.’ After two items deleted, the Cronbach’s alpha value and item-total correlation coefficients were calculated and the factor structure was examined. The Cronbach’s alpha value of the remaining 11 items in the parent questionnaire was 0.800. The item-total correlation coefficients of the questionnaire were between 0.32 and 0.62 (Table 4).

3) Exploratory factor analysis

(1) Patient questionnaire

EFA was conducted to assess the construct validity of the questionnaire, using the Varimax rotation method. In the EFA procedure, the parameters were grouped into two factors with eigenvalues >1. Two factors explained 51.1% of the total variance (Table 5). According to the Kaiser-Meyer-Olkin measure of sample adequacy, which was 0.812 (>0.80), the sample size was adequate. According to the Bartlett’s test, the hypothesis was accepted that the correlation matrix was not an identity matrix (Bartlett’s test, p=0.000 [<0.05]) [16]. Items 1?9 were included in the psychosocial component and items 10?12 were included in the physical component; factor loadings ranged from 0.341 to 0.888.

(2) Parent questionnaire

In the parent questionnaire, the parameters were similarly grouped into two factors with eigenvalues >1. Two factors explained 51.422% of the total variance (Kaiser-Meyer-Olkin measure=0.815 [>0.80], Bartlett’s test p=0.000 [<0.05]) (Table 6). According to the EFA, items 1?8 were included in the psychosocial component and items 9?12 were included in the physical component; the factor loadings ranged from 0.436 to 0.856.

4) Confirmatory factor analysis

(1) Patient questionnaire

CFA was used to confirm the two-factor structure revealed in the EFA. According to CFA, the standardized regression weights ranged from 0.32 to 0.84 (Fig. 1), and the t-tests indicated statistical significance.

(2) Parent questionnaire

According to CFA, the standardized regression weights ranged from 0.46 to 0.86 (Fig. 2), and the t-tests indicated statistical significance. According to the fit indices for both the patient and the parent questionnaires, the two-factor (psychosocial and physical) model demonstrated good fit (Table 7). Modification indices between covariance errors >20 were allowed to correlate (Figs. 1, 2) [20]. As a result, the fit of the model improved for both questionnaires. CFA confirmed the construct validity of the factors that were obtained for both the patient and the parent questionnaires.

The physical and psychosocial components were correlated between the patient and the parent questionnaires (Table 8). The two components were also correlated within the patient and the parent questionnaires, with a higher correlation between the patient and parent questionnaires observed for the physical component.

DISCUSSION

Go to

Recent studies have shown that the minimally invasive repair of PE generally results in improved self-esteem and quality of life [5,9?13]. In order to evaluate the effect of this surgical technique on the quality of life of our patients, we translated the questionnaire into Turkish and adapted it.

In the patient questionnaire, four of the question stems dealt with concerns about physical appearance: ‘looks in general,’ ‘how chest looks without shirt,’ ‘spending rest of life as chest looks now,’ and ‘bothered because of the way chest looks.’ These question stems had the lowest median scores, indicating unhappiness and anxiety related with appearance.

The correlation between the patient and the parent questionnaires was higher for the physical component. Physical complaints may be more observable than psychosocial problems. The question stems in the psychosocial component deal with internalizing symptoms, such as anxiety and depressive symptoms, and parents may not be fully aware of the internal states of adolescent or young adult patients with PE.

With regard to content validity, expert opinions demonstrated that the Turkish version of the NQmA was comprehensible. Our sample size was adequate. Cronbach’s alpha values demonstrated good level of reliability for the Turkish version of the patient and the parent questionnaires [15,18]. As a result of the reliability analyses and EFA, there was no need to exclude any item from the patient questionnaire, although two items were excluded from the parent questionnaire. Reliability and construct validity were demonstrated for the remaining 11 items. EFA indicated that both the patient and the parent questionnaires were divided into physical and psychosocial components, which was confirmed by CFA results.

In conclusion, the adaptation of the NQmA into Turkish was successful, and it was found to be valid and reliable for the assessment of quality of life in Turkish patients with PE. The NQmA is easy for participants to complete and was found to be helpful for clinicians performing follow-up on their patients’ quality of life [3,9,10].

Go to Figures
Fig. 1.Factor structure for the patient form.
Fig. 2.Factor structure for the parent form.

Tables

Go to

Patient form of the Nuss Questionnaire modified for Adults and the median and interquartile range results for each question stem

Question stemScoringMedian (interquartile range)
1. Looks in general1: Very unhappy, 2: mostly unhappy, 3: mostly happy, 4: very happy2 (2?3)
2. How chest looks without shirt1: Very unhappy, 2: mostly unhappy, 3: mostly happy, 4: very happy1 (1?2)
3. Spending rest of life as chest looks now1: Very unhappy, 2: mostly unhappy, 3: mostly happy, 4: very happy1 (1?2)
4. Others make fun of him/her because of chest1: Very often, 2: often, 3: sometimes, 4: never4 (3?4)
5. Avoids doing things1: Very often, 2: often, 3: sometimes, 4: never3 (3?3)
6. Hides chest1: Very often, 2: often, 3: sometimes, 4: never3 (1?3)
7. Bothered because of the way chest looks1: Very often, 2: often, 3: sometimes, 4: never2 (1?3)
8. Feels shy/self-conscious because of chest1: Very often, 2: often, 3: sometimes, 4: never3 (2?4)
9. Feels bad about self1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)
10. Has trouble exercising1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)
11. Chest causes shortness of breath1: Very often, 2: often, 3: sometimes, 4: never3 (2?4)
12. Chest causes him/her to be tired1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)

Parent form of the Nuss Questionnaire modified for Adults and the median and interquartile range results for each question stem

Question stemsScoringMedian (interquartile range)
1. Irritable1: Very often, 2: often, 3: sometimes, 4: never3 (3?3)
2. Frustrated1: Very often, 2: often, 3: sometimes, 4: never3 (2.5?3)
3. Sad/depressed1: Very often, 2: often, 3: sometimes, 4: never3 (3?3)
4. Restless1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
5. Isolated1: Very often, 2: often, 3: sometimes, 4: never4 (3?4)
7. Reluctant to be in public while wearing bathing clothes that would show the chest1: Very often, 2: often, 3: sometimes, 4: never3 (1?4)
8. How often parent is concerned about effects of the deformity on patient’s life1: Very often, 2: often, 3: sometimes, 4: never3 (1?3)
9. Have trouble exercising1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
10. Have chest pain1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
11. Have shortness of breath1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
12. Feel tired1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)

Internal consistency of the patient questionnaire

Question stemScale mean if item deletedScale variance if item deletedCorrected item-total correlationCronbach’s alpha if item deleted
1. Looks in general27.9731.3210.4130.794
2. How chest looks without shirt28.7431.8220.4970.789
3. Spending rest of life as chest looks now28.8832.9930.3980.796
4. Others make fun of him/her because of chest26.8133.1920.2780.804
5. Avoids doing things27.3531.3490.4630.790
6. Hides chest27.7528.6370.5500.781
7. Bothered because of the way chest looks27.9628.6840.6280.773
8. Feels shy/self-conscious because of chest27.3629.3820.5160.784
9. Feels bad about self27.4930.6450.5130.785
10. Has trouble exercising27.5630.6340.3900.798
11. Chest causes shortness of breath27.3831.2590.3820.798
12. Chest causes him/her to be tired27.4230.9410.4130.795

Internal consistency of the parent questionnaire

Question stemScale mean if item deletedScale variance if item deletedCorrected item-total correlationCronbach’s alpha if item deleted
1. Irritable29.2225.9970.4570.784
2. Frustrated29.3726.7180.4660.785
3. Sad/depressed29.1725.3560.6200.770
4. Restless29.1125.8510.5070.780
5. Isolated28.8926.0860.3880.792
7. Reluctant to be in public while wearing bathing clothes that would show the chest29.4625.2650.3220.807
8. How often parent is concerned about effects of the deformity on patient’s life29.8025.2230.4130.790
9. Have trouble exercising29.1924.9510.5410.775
10. Have chest pain28.7826.6920.4710.784
11. Have shortness of breath28.9826.2120.4740.783
12. Feel tired29.3824.5090.5370.775

Exploratory and confirmatory factor analysis for the patient questionnaire

FactorsQuestion stemsEFA factor loadingsCFA factor loadingsCronbach’s alpha
Psychosociala)0.818
1. Looks in general0.5320.423
2. How chest looks without shirt0.6810.553
3. Spending rest of life as chest looks now0.5560.407
4. Others make fun of him/her because of chest0.3410.321
5. Avoids doing things0.5410.506
6. Hides chest0.7800.765
7. Bothered because of the way chest looks0.8000.803
8. Feels shy/self-conscious because of chest0.7140.691
9. Feels bad about self0.6970.618
Physicalb)0.862
10. Has trouble exercising0.8870.843
11. Chest causes shortness of breath0.8880.839
12. Chest causes him/her to be tired0.8580.788

EFA, exploratory factor analysis; CFA, confirmatory factor analysis; AVE, average variance extracted.

a)Eigenvalues=3.962; EFA variance explained=31.145%; AVE for CFA=88.6%.

b)Eigenvalues=2.170; EFA variance explained=19.955%; AVE for CFA=94.3%.


Exploratory and confirmatory factor analysis for the parent questionnaire

FactorsQuestion stemsEFA factor loadingsCFA factor loadingsCronbach’s alpha
Psychosociala)0.743
1. Irritable0.6560.469
2. Frustrated0.6810.492
3. Sad/depressed0.7060.788
4. Restless0.6980.650
5. Isolated0.5820.514
7. Reluctant to be in public while wearing bathing clothes that would show the chest0.6090.458
8. How often parent is concerned about effects of the deformity on patient’s life0.4360.458
Physicalb)0.827
9. Have trouble exercising0.8560.856
10. Have chest pain0.6650.552
11. Have shortness of breath0.8270.721
12. Feel tired0.8300.827

EFA, exploratory factor analysis; CFA, confirmatory factor analysis; AVE, average variance extracted.

a)Eigenvalues=3.908; EFA variance explained=26.097%; AVE for CFA=88.5%.

b)Eigenvalues=1.749; EFA variance explained=25.325%; AVE for CFA=93.6%.


Goodness-of-fit results for the patient and the parent questionnaires

Goodness-of-fit indicesValue of each index indicative of good fitGoodness-of-fit results for the patient questionnaireGoodness-of-fit results for the parent questionnaire
Chi-square/degrees of freedom<31.8201.525
Root mean square error of approximation≤0.0800.0560.045
Standardized root mean square error≤0.0800.0530.053
Comparative fit index>0.900.9600.975
Goodness of fit index>0.900.9450.958

The correlation of the components in the patient and the parent questionnaires

Patient psychosocialPatient physicalParent psychosocialParent physical
Patient psychosocial
?r-value0.1950.4390.114
?p-value0.0010.0000.065
Patient physical
?r-value0.1580.665
?p-value0.0100.000
Parent psychosocial
?r-value0.340
?p-value0.000

References

Go to
  1. Fokin, AA, Steuerwald, NM, Ahrens, WA, and Allen, KE (2009). Anatomical, histologic, and genetic characteristics of congenital chest wall deformities. Semin Thorac Cardiovasc Surg. 21, 44-57.
    Pubmed CrossRef
  2. Jaroszewski, D, Notrica, D, McMahon, L, Steidley, DE, and Deschamps, C (2010). Current management of pectus excavatum: a review and update of therapy and treatment recommendations. J Am Board Fam Med. 23, 230-9.
    Pubmed CrossRef
  3. Steinmann, C, Krille, S, Mueller, A, Weber, P, Reingruber, B, and Martin, A (2011). Pectus excavatum and pectus carinatum patients suffer from lower quality of life and impaired body image: a control group comparison of psychological characteristics prior to surgical correction. Eur J Cardiothorac Surg. 40, 1138-45.
    Pubmed
  4. Kelly, RE, Shamberger, RC, Mellins, RB, Mitchell, KK, Lawson, ML, and Oldham, K (2007). Prospective multicenter study of surgical correction of pectus excavatum: design, perioperative complications, pain, and baseline pulmonary function facilitated by internet-based data collection. J Am Coll Surg. 205, 205-16.
    Pubmed CrossRef
  5. Roberts, J, Hayashi, A, Anderson, JO, Martin, JM, and Maxwell, LL (2003). Quality of life of patients who have undergone the Nuss procedure for pectus excavatum: preliminary findings. J Pediatr Surg. 38, 779-83.
    Pubmed CrossRef
  6. Nuss, D, Kelly, RE, Croitoru, DP, and Katz, ME (1998). A 10-year review of a minimally invasive technique for the correction of pectus excavatum. J Pediatr Surg. 33, 545-52.
    Pubmed CrossRef
  7. Kim do, H, Hwang, JJ, Lee, MK, Lee, DY, and Paik, HC (2005). Analysis of the Nuss procedure for pectus excavatum in different age groups. Ann Thorac Surg. 80, 1073-7.
    Pubmed CrossRef
  8. Kang, CH, Park, S, Park, IK, Kim, YT, and Kim, JH (2012). Long-term surveillance comparing satisfaction between the early experience of Nuss procedure vs. Ravitch procedure. Korean J Thorac Cardiovasc Surg. 45, 308-15.
    Pubmed KoreaMed CrossRef
  9. Lawson, ML, Cash, TF, and Akers, R (2003). A pilot study of the impact of surgical repair on disease-specific quality of life among patients with pectus excavatum. J Pediatr Surg. 38, 916-8.
    Pubmed CrossRef
  10. Krasopoulos, G, Dusmet, M, Ladas, G, and Goldstraw, P (2006). Nuss procedure improves the quality of life in young male adults with pectus excavatum deformity. Eur J Cardiothorac Surg. 29, 1-5.
    CrossRef
  11. Kim, HK, Shim, JH, Choi, KS, and Choi, YH (2011). The quality of life after bar removal in patients after the nuss procedure for pectus excavatum. World J Surg. 35, 1656-61.
    Pubmed CrossRef
  12. Metzelder, ML, Kuebler, JF, Leonhardt, J, Ure, BM, and Petersen, C (2007). Self and parental assessment after minimally invasive repair of pectus excavatum: lasting satisfaction after bar removal. Ann Thorac Surg. 83, 1844-9.
    Pubmed CrossRef
  13. Onen, A, Sanli, A, Eyuboglu, GM, Gokcen, KM, and Karacam, V (2008). Minimal invaziv teknik uygulanan pektus ekskavatumlu olgularda erken donem memnuniyet bildirimi. Turk Gogus Kalp Damar Cer Derg. 16, 113-7.
  14. Lawshe, CH (1975). A quantitative approach to content validity 1. Pers Psychol. 28, 563-75.
    CrossRef
  15. Tavakol, M, and Dennick, R (2011). Making sense of Cronbach’s alpha. Int J Med Educ. 2, 53-5.
    CrossRef
  16. Munro, BH (2005). . Statistical methods for health care research, , ed. (Philadelphia:Lippincott Williams & Wilkins), pp. .
  17. Cokluk, O, Sekercioglu, G, and Buyukozturk, S (2010). . Sosyal bilimler icin cok degiskenli istatistik: SPSS ve LISREL uygulamalari, , ed. (Ankara:Pegem Akademi), pp. .
  18. Ozdamar, K (2004). . Paket programlar ile istatistiksel veri analizi, , ed. (Eskisehir:Kaan Kitabevi), pp. .
  19. Lynn, MR (1986). Determination and quantification of content validity. Nurs Res. 35, 382-5.
    Pubmed CrossRef
  20. Aluja, A, Blanch, A, and Garcia, LF (2005). Reanalyzing the 16pf-5 second order structure: exploratory versus confirmatory factorial analysis. Eur J Psychol Educ. 20, 343-53.
    CrossRef

Article

Clinical Research

Korean J Thorac Cardiovasc Surg 2015; 48(2): 112-119

Published online April 5, 2015 https://doi.org/10.5090/kjtcs.2015.48.2.112

Copyright © Journal of Chest Surgery.

Validity and Reliability of the Turkish Version of the Nuss Questionnaire Modified for Adults

A. Tugba Bahadir1, Pinar Kuru2, Ceyda Afacan3, Nezih Onur Ermerak4, Korkut Bostanci4, and Mustafa Yuksel4

1Department of Child & Adolescent Psychiatry, Northwestern University Feinberg School of Medicine, 2Department of Neurology, Northwestern University Feinberg School of Medicine, 3Statistical Expert, Marmara University School of Medicine, 4Department of Thoracic Surgery, Marmara University School of Medicine

Correspondence to:Corresponding author: A. Tugba Bahadir, Department of Child & Adolescent Psychiatry, Marmara University School of Medicine, Istanbul, Fevzi Cakmak Mah., Mimar Sinan Cad., No: 41, 34899, Ust Kaynarca, Pendik, Istanbul, Turkey, (Tel) 90-216-625-4545/1635 (Fax) 90-533-425-5888 (E-mail) tugba77@gmail.com

Received: July 21, 2014; Revised: November 3, 2014; Accepted: November 5, 2014

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Pectus excavatum (PE) is the most common chest wall deformity. The Nuss Questionnaire modified for Adults (NQmA) is a disease-specific health-related quality of life assessment tool for patients with pectus deformities. The aim of this study is to adapt the NQmA into Turkish.

Methods

Two hundred and sixty-five patients with PE were participated, with an age range of 14 to 29 years. All patients underwent a physical examination and had not undergone corrective surgery. The Turkish version of the NQmA was completed by patients and their parents.

Results

The content validity index based on expert opinions was 91% for the patient questionnaire and 96% for the parent questionnaire. The Cronbach’s alpha value for the NQmA was found to be 0.805 for the patient questionnaire and 0.800 for the parent questionnaire. Exploratory factor analysis was used to assess construct validity. Two factors explained 51.1% of the total variance in the patient questionnaire (psychosocial: 31.145%, Cronbach’s alpha=0.818; physical: 19.955%, Cronbach’s alpha=0.862). In the parent questionnaire, two factors explained 51.422% of the total variance (psychosocial: 26.097%, Cronbach’s alpha=0.743; physical: 25.325%, Cronbach’s alpha=0.827). Construct validity was confirmed by confirmatory factor analysis.

Conclusion

The Turkish version of the NQmA was found to be valid and reliable for the assessment of quality of life in patients with PE.

Keywords: Funnel chest, Quality of life, Psychometrics, Validation studies

INTRODUCTION

Pectus excavatum (PE), also known as funnel chest, is the most common chest wall deformity, characterized by inward displacement of the sternum and the costal cartilages. PE occurs in males three to five times more often than in females, with an incidence of one case in every 400 to 1,000 births [1,2]. Physical limitations and/or psychosocial distress can be observed in patients with PE and may result in a decreased quality of life [3]. The physical limitations caused by PE are mostly cardiopulmonary restrictions resulting in fatigue, exercise intolerance, shortness of breath, palpitations, and chest discomfort or pain [4]. The psychosocial problems associated with PE are mostly characterized by low self-esteem and avoidance of social and physical activities due to the appearance of the chest [5].

Minimally invasive repair of PE, developed by Donald Nuss in 1987, is the current surgical approach for the correction of PE. Nuss et al. [6] published their 10-year experiences in 1998. Since that time, minimally invasive repair has mostly replaced the Ravitch procedure, because minimally invasive repair involves shorter operations and minimal surgical scarring [7]. However, Kang et al. [8] reported in 2012 that long-term satisfaction was not related to the operation type, but instead was related to the completeness of the correction and the absence of recurrence.

The primary aim of medical care is to increase the patient’s quality of life, which includes physical and psychosocial well-being. The Nuss Questionnaire was first published and validated by Lawson et al. [9] in 2003 as a disease-specific tool for the evaluation of quality of life in pediatric patients with PE. Krasopoulos et al. [10] modified the questionnaire for adults. Improvements in physical and psychosocial well-being, with a better quality of life, have been reported after the minimally invasive repair of PE [5,9?13].

In the present study we aimed to evaluate the validity and the reliability of the Turkish version of the Nuss Questionnaire modified for Adults (NQmA).

METHODS

Patients with PE (age range, 14 to 29 years) were recruited from the Thoracic Surgery Clinic of Marmara University Hospital. Patients with any previous thoracic surgery history or complex deformities were excluded. This study was approved by the Clinical Research Ethics Committee of the Marmara University School of Medicine. Informed consent was obtained from each participant. All patients underwent a physical examination, and posteroanterior and lateral chest radiographs were obtained. The Turkish version of the NQmA was used to assess the patients’ quality of life. Patients and their parents completed the questionnaires.

The two-step Nuss Questionnaire for pediatric patients and their parents is used to assess the disease-specific quality of life of patients with pectus deformities. Child and parent versions of the questionnaire were first published and validated by Lawson et al. [9]. The Nuss Questionnaire was modified for adult patients by Krasopoulos et al. [10], and the scoring of the first three questions was reversed.

The patient and parent questionnaires were translated into Turkish. Fourteen experts working with chest wall deformities and/or quality of life reviewed the parent and patient versions of the questionnaire for the suitability and comprehensibility of the items. The patient form of the NQmA included 12 items and the parent form included 13 items. Each item received a score between 1 and 4, with higher scores indicating a better quality of life (Tables 1, 2).

1) Statistical analysis

The AMOS ver. 6.0 (AMOS Development Co., SPSS Inc., Chicago, IL, USA) and SPSS ver. 15.0 (SPSS Inc., Chicago, IL, USA) software packages were used for statistical analysis. The content validity ratio and content validity index were used to examine the validity of the content based on the assessment of experts. Content validity means that the items in the test appropriately sample the content domain. The content validity ratio is used to determine whether specific test items should be retained or deleted. The content validity index is the mean of the content validity ratios of the remaining items in the test [14]. Cronbach’s alpha was calculated to assess the reliability of the overall questionnaire, as well as to assess its physical and psychosocial components individually. Cronbach’s alpha is a number between 0 and 1 [15] that measures internal consistency and reliability [16]. Internal consistency indicates the degree to which test items all measure the same concept [15]. In order to evaluate the contribution of each item to the questionnaire, item-total correlation coefficients and Cronbach’s alpha values if item deleted were also calculated. Exploratory factor analysis (EFA) was used to assess construct validity. EFA is used to reduce the amount of data by grouping related items together [16]. The Kaiser-Meyer-Olkin measure of sampling adequacy is a number between 0 and 1 that measures the adequacy of the data in terms of the sample size. Bartlett’s test of sphericity, which is a prerequisite for EFA, demonstrates whether the data have a multivariate distribution. Eigenvalues are used to decide the number of factors included in EFA [17], and each eigenvalue accounts for a certain percentage of the variance [18]. Confirmatory factor analysis (CFA) was used to confirm the construct validity. CFA is employed to determine whether the factor structure indicated by EFA is represented in the data or not [18], and fit indices are used to confirm the CFA model [17]. The fit indices that we evaluated were the root mean square error of approximation, the goodness of fit index, the comparative fit index, the standardized root mean square error, and the chi-square/degrees of freedom test. Spearman correlation analysis was carried out to compare the physical and psychosocial component scores of the patient and the parent questionnaires. For all analyses, p<0.05 was considered to indicate significance.

RESULTS

A total of 265 patients with PE participated to the study, of whom 233 (87.9%) were male and 32 (12.1%) were female. The mean age of the patients was 18.83±3.84 years (range, 14 to 29 years). Eighty-six (32.5%) of the patients had at least one associated disease or anomaly, and 21 patients (7.9%) had scoliosis. Symmetric deformities were observed in 176 (69%) of the patients, and the others had asymmetric deformities. A total of 265 patient and parent questionnaires were completed.

1) Expert opinions and content validity

We asked 14 experts for their opinions. The expected minimum content validity ratio for 14 experts was 51% [14]. Content validity ratios were provided for all items, and were between 57% and 100% in both of the questionnaires. The content validity index of the expert opinions was 91% for the patient questionnaire and 96% for the parent questionnaire. The content validity indices were 79% or higher for all of the items in both of the patient and the parent questionnaires [19].

2) Reliability and construct validity

(1) Patient questionnaire

The Cronbach’s alpha value of the 12 items in the patient questionnaire was 0.805, indicating that it was reliable. The item-total correlation coefficients of items in the patient questionnaire were between 0.28 and 0.63 (Table 3). No increase in the Cronbach’s alpha value was observed if any item was deleted.

(2) Parent questionnaire

Items were excluded from the questionnaire if they met the following criteria: (1) an item-total correlation coefficient <0.40, (2) a factor loading <0.40 [16], (3) if deleting the factor increased the Cronbach’s alpha value, and (4) if an item did not load on the expected factor. The sixth and thirteenth items were deleted based on all the above criteria. These question stems were ‘made’ fun of him/her’ and ‘have problems gaining weight.’ After two items deleted, the Cronbach’s alpha value and item-total correlation coefficients were calculated and the factor structure was examined. The Cronbach’s alpha value of the remaining 11 items in the parent questionnaire was 0.800. The item-total correlation coefficients of the questionnaire were between 0.32 and 0.62 (Table 4).

3) Exploratory factor analysis

(1) Patient questionnaire

EFA was conducted to assess the construct validity of the questionnaire, using the Varimax rotation method. In the EFA procedure, the parameters were grouped into two factors with eigenvalues >1. Two factors explained 51.1% of the total variance (Table 5). According to the Kaiser-Meyer-Olkin measure of sample adequacy, which was 0.812 (>0.80), the sample size was adequate. According to the Bartlett’s test, the hypothesis was accepted that the correlation matrix was not an identity matrix (Bartlett’s test, p=0.000 [<0.05]) [16]. Items 1?9 were included in the psychosocial component and items 10?12 were included in the physical component; factor loadings ranged from 0.341 to 0.888.

(2) Parent questionnaire

In the parent questionnaire, the parameters were similarly grouped into two factors with eigenvalues >1. Two factors explained 51.422% of the total variance (Kaiser-Meyer-Olkin measure=0.815 [>0.80], Bartlett’s test p=0.000 [<0.05]) (Table 6). According to the EFA, items 1?8 were included in the psychosocial component and items 9?12 were included in the physical component; the factor loadings ranged from 0.436 to 0.856.

4) Confirmatory factor analysis

(1) Patient questionnaire

CFA was used to confirm the two-factor structure revealed in the EFA. According to CFA, the standardized regression weights ranged from 0.32 to 0.84 (Fig. 1), and the t-tests indicated statistical significance.

(2) Parent questionnaire

According to CFA, the standardized regression weights ranged from 0.46 to 0.86 (Fig. 2), and the t-tests indicated statistical significance. According to the fit indices for both the patient and the parent questionnaires, the two-factor (psychosocial and physical) model demonstrated good fit (Table 7). Modification indices between covariance errors >20 were allowed to correlate (Figs. 1, 2) [20]. As a result, the fit of the model improved for both questionnaires. CFA confirmed the construct validity of the factors that were obtained for both the patient and the parent questionnaires.

The physical and psychosocial components were correlated between the patient and the parent questionnaires (Table 8). The two components were also correlated within the patient and the parent questionnaires, with a higher correlation between the patient and parent questionnaires observed for the physical component.

DISCUSSION

Recent studies have shown that the minimally invasive repair of PE generally results in improved self-esteem and quality of life [5,9?13]. In order to evaluate the effect of this surgical technique on the quality of life of our patients, we translated the questionnaire into Turkish and adapted it.

In the patient questionnaire, four of the question stems dealt with concerns about physical appearance: ‘looks in general,’ ‘how chest looks without shirt,’ ‘spending rest of life as chest looks now,’ and ‘bothered because of the way chest looks.’ These question stems had the lowest median scores, indicating unhappiness and anxiety related with appearance.

The correlation between the patient and the parent questionnaires was higher for the physical component. Physical complaints may be more observable than psychosocial problems. The question stems in the psychosocial component deal with internalizing symptoms, such as anxiety and depressive symptoms, and parents may not be fully aware of the internal states of adolescent or young adult patients with PE.

With regard to content validity, expert opinions demonstrated that the Turkish version of the NQmA was comprehensible. Our sample size was adequate. Cronbach’s alpha values demonstrated good level of reliability for the Turkish version of the patient and the parent questionnaires [15,18]. As a result of the reliability analyses and EFA, there was no need to exclude any item from the patient questionnaire, although two items were excluded from the parent questionnaire. Reliability and construct validity were demonstrated for the remaining 11 items. EFA indicated that both the patient and the parent questionnaires were divided into physical and psychosocial components, which was confirmed by CFA results.

In conclusion, the adaptation of the NQmA into Turkish was successful, and it was found to be valid and reliable for the assessment of quality of life in Turkish patients with PE. The NQmA is easy for participants to complete and was found to be helpful for clinicians performing follow-up on their patients’ quality of life [3,9,10].

Fig 1.

Download
Figure 1.Factor structure for the patient form.
Journal of Chest Surgery 2015; 48: 112-119https://doi.org/10.5090/kjtcs.2015.48.2.112

Fig 2.

Download
Figure 2.Factor structure for the parent form.
Journal of Chest Surgery 2015; 48: 112-119https://doi.org/10.5090/kjtcs.2015.48.2.112

Table 1 . Patient form of the Nuss Questionnaire modified for Adults and the median and interquartile range results for each question stem.

Question stemScoringMedian (interquartile range)
1. Looks in general1: Very unhappy, 2: mostly unhappy, 3: mostly happy, 4: very happy2 (2?3)
2. How chest looks without shirt1: Very unhappy, 2: mostly unhappy, 3: mostly happy, 4: very happy1 (1?2)
3. Spending rest of life as chest looks now1: Very unhappy, 2: mostly unhappy, 3: mostly happy, 4: very happy1 (1?2)
4. Others make fun of him/her because of chest1: Very often, 2: often, 3: sometimes, 4: never4 (3?4)
5. Avoids doing things1: Very often, 2: often, 3: sometimes, 4: never3 (3?3)
6. Hides chest1: Very often, 2: often, 3: sometimes, 4: never3 (1?3)
7. Bothered because of the way chest looks1: Very often, 2: often, 3: sometimes, 4: never2 (1?3)
8. Feels shy/self-conscious because of chest1: Very often, 2: often, 3: sometimes, 4: never3 (2?4)
9. Feels bad about self1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)
10. Has trouble exercising1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)
11. Chest causes shortness of breath1: Very often, 2: often, 3: sometimes, 4: never3 (2?4)
12. Chest causes him/her to be tired1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)

Table 2 . Parent form of the Nuss Questionnaire modified for Adults and the median and interquartile range results for each question stem.

Question stemsScoringMedian (interquartile range)
1. Irritable1: Very often, 2: often, 3: sometimes, 4: never3 (3?3)
2. Frustrated1: Very often, 2: often, 3: sometimes, 4: never3 (2.5?3)
3. Sad/depressed1: Very often, 2: often, 3: sometimes, 4: never3 (3?3)
4. Restless1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
5. Isolated1: Very often, 2: often, 3: sometimes, 4: never4 (3?4)
7. Reluctant to be in public while wearing bathing clothes that would show the chest1: Very often, 2: often, 3: sometimes, 4: never3 (1?4)
8. How often parent is concerned about effects of the deformity on patient’s life1: Very often, 2: often, 3: sometimes, 4: never3 (1?3)
9. Have trouble exercising1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
10. Have chest pain1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
11. Have shortness of breath1: Very often, 2: often, 3: sometimes, 4: never3 (3?4)
12. Feel tired1: Very often, 2: often, 3: sometimes, 4: never3 (2?3)

Table 3 . Internal consistency of the patient questionnaire.

Question stemScale mean if item deletedScale variance if item deletedCorrected item-total correlationCronbach’s alpha if item deleted
1. Looks in general27.9731.3210.4130.794
2. How chest looks without shirt28.7431.8220.4970.789
3. Spending rest of life as chest looks now28.8832.9930.3980.796
4. Others make fun of him/her because of chest26.8133.1920.2780.804
5. Avoids doing things27.3531.3490.4630.790
6. Hides chest27.7528.6370.5500.781
7. Bothered because of the way chest looks27.9628.6840.6280.773
8. Feels shy/self-conscious because of chest27.3629.3820.5160.784
9. Feels bad about self27.4930.6450.5130.785
10. Has trouble exercising27.5630.6340.3900.798
11. Chest causes shortness of breath27.3831.2590.3820.798
12. Chest causes him/her to be tired27.4230.9410.4130.795

Table 4 . Internal consistency of the parent questionnaire.

Question stemScale mean if item deletedScale variance if item deletedCorrected item-total correlationCronbach’s alpha if item deleted
1. Irritable29.2225.9970.4570.784
2. Frustrated29.3726.7180.4660.785
3. Sad/depressed29.1725.3560.6200.770
4. Restless29.1125.8510.5070.780
5. Isolated28.8926.0860.3880.792
7. Reluctant to be in public while wearing bathing clothes that would show the chest29.4625.2650.3220.807
8. How often parent is concerned about effects of the deformity on patient’s life29.8025.2230.4130.790
9. Have trouble exercising29.1924.9510.5410.775
10. Have chest pain28.7826.6920.4710.784
11. Have shortness of breath28.9826.2120.4740.783
12. Feel tired29.3824.5090.5370.775

Table 5 . Exploratory and confirmatory factor analysis for the patient questionnaire.

FactorsQuestion stemsEFA factor loadingsCFA factor loadingsCronbach’s alpha
Psychosociala)0.818
1. Looks in general0.5320.423
2. How chest looks without shirt0.6810.553
3. Spending rest of life as chest looks now0.5560.407
4. Others make fun of him/her because of chest0.3410.321
5. Avoids doing things0.5410.506
6. Hides chest0.7800.765
7. Bothered because of the way chest looks0.8000.803
8. Feels shy/self-conscious because of chest0.7140.691
9. Feels bad about self0.6970.618
Physicalb)0.862
10. Has trouble exercising0.8870.843
11. Chest causes shortness of breath0.8880.839
12. Chest causes him/her to be tired0.8580.788

EFA, exploratory factor analysis; CFA, confirmatory factor analysis; AVE, average variance extracted..

a)Eigenvalues=3.962; EFA variance explained=31.145%; AVE for CFA=88.6%.

b)Eigenvalues=2.170; EFA variance explained=19.955%; AVE for CFA=94.3%.

Table 6 . Exploratory and confirmatory factor analysis for the parent questionnaire.

FactorsQuestion stemsEFA factor loadingsCFA factor loadingsCronbach’s alpha
Psychosociala)0.743
1. Irritable0.6560.469
2. Frustrated0.6810.492
3. Sad/depressed0.7060.788
4. Restless0.6980.650
5. Isolated0.5820.514
7. Reluctant to be in public while wearing bathing clothes that would show the chest0.6090.458
8. How often parent is concerned about effects of the deformity on patient’s life0.4360.458
Physicalb)0.827
9. Have trouble exercising0.8560.856
10. Have chest pain0.6650.552
11. Have shortness of breath0.8270.721
12. Feel tired0.8300.827

EFA, exploratory factor analysis; CFA, confirmatory factor analysis; AVE, average variance extracted..

a)Eigenvalues=3.908; EFA variance explained=26.097%; AVE for CFA=88.5%.

b)Eigenvalues=1.749; EFA variance explained=25.325%; AVE for CFA=93.6%.

Table 7 . Goodness-of-fit results for the patient and the parent questionnaires.

Goodness-of-fit indicesValue of each index indicative of good fitGoodness-of-fit results for the patient questionnaireGoodness-of-fit results for the parent questionnaire
Chi-square/degrees of freedom<31.8201.525
Root mean square error of approximation≤0.0800.0560.045
Standardized root mean square error≤0.0800.0530.053
Comparative fit index>0.900.9600.975
Goodness of fit index>0.900.9450.958

Table 8 . The correlation of the components in the patient and the parent questionnaires.

Patient psychosocialPatient physicalParent psychosocialParent physical
Patient psychosocial
?r-value0.1950.4390.114
?p-value0.0010.0000.065
Patient physical
?r-value0.1580.665
?p-value0.0100.000
Parent psychosocial
?r-value0.340
?p-value0.000

References

  1. Fokin, AA, Steuerwald, NM, Ahrens, WA, and Allen, KE (2009). Anatomical, histologic, and genetic characteristics of congenital chest wall deformities. Semin Thorac Cardiovasc Surg. 21, 44-57.
    Pubmed CrossRef
  2. Jaroszewski, D, Notrica, D, McMahon, L, Steidley, DE, and Deschamps, C (2010). Current management of pectus excavatum: a review and update of therapy and treatment recommendations. J Am Board Fam Med. 23, 230-9.
    Pubmed CrossRef
  3. Steinmann, C, Krille, S, Mueller, A, Weber, P, Reingruber, B, and Martin, A (2011). Pectus excavatum and pectus carinatum patients suffer from lower quality of life and impaired body image: a control group comparison of psychological characteristics prior to surgical correction. Eur J Cardiothorac Surg. 40, 1138-45.
    Pubmed
  4. Kelly, RE, Shamberger, RC, Mellins, RB, Mitchell, KK, Lawson, ML, and Oldham, K (2007). Prospective multicenter study of surgical correction of pectus excavatum: design, perioperative complications, pain, and baseline pulmonary function facilitated by internet-based data collection. J Am Coll Surg. 205, 205-16.
    Pubmed CrossRef
  5. Roberts, J, Hayashi, A, Anderson, JO, Martin, JM, and Maxwell, LL (2003). Quality of life of patients who have undergone the Nuss procedure for pectus excavatum: preliminary findings. J Pediatr Surg. 38, 779-83.
    Pubmed CrossRef
  6. Nuss, D, Kelly, RE, Croitoru, DP, and Katz, ME (1998). A 10-year review of a minimally invasive technique for the correction of pectus excavatum. J Pediatr Surg. 33, 545-52.
    Pubmed CrossRef
  7. Kim do, H, Hwang, JJ, Lee, MK, Lee, DY, and Paik, HC (2005). Analysis of the Nuss procedure for pectus excavatum in different age groups. Ann Thorac Surg. 80, 1073-7.
    Pubmed CrossRef
  8. Kang, CH, Park, S, Park, IK, Kim, YT, and Kim, JH (2012). Long-term surveillance comparing satisfaction between the early experience of Nuss procedure vs. Ravitch procedure. Korean J Thorac Cardiovasc Surg. 45, 308-15.
    Pubmed KoreaMed CrossRef
  9. Lawson, ML, Cash, TF, and Akers, R (2003). A pilot study of the impact of surgical repair on disease-specific quality of life among patients with pectus excavatum. J Pediatr Surg. 38, 916-8.
    Pubmed CrossRef
  10. Krasopoulos, G, Dusmet, M, Ladas, G, and Goldstraw, P (2006). Nuss procedure improves the quality of life in young male adults with pectus excavatum deformity. Eur J Cardiothorac Surg. 29, 1-5.
    CrossRef
  11. Kim, HK, Shim, JH, Choi, KS, and Choi, YH (2011). The quality of life after bar removal in patients after the nuss procedure for pectus excavatum. World J Surg. 35, 1656-61.
    Pubmed CrossRef
  12. Metzelder, ML, Kuebler, JF, Leonhardt, J, Ure, BM, and Petersen, C (2007). Self and parental assessment after minimally invasive repair of pectus excavatum: lasting satisfaction after bar removal. Ann Thorac Surg. 83, 1844-9.
    Pubmed CrossRef
  13. Onen, A, Sanli, A, Eyuboglu, GM, Gokcen, KM, and Karacam, V (2008). Minimal invaziv teknik uygulanan pektus ekskavatumlu olgularda erken donem memnuniyet bildirimi. Turk Gogus Kalp Damar Cer Derg. 16, 113-7.
  14. Lawshe, CH (1975). A quantitative approach to content validity 1. Pers Psychol. 28, 563-75.
    CrossRef
  15. Tavakol, M, and Dennick, R (2011). Making sense of Cronbach’s alpha. Int J Med Educ. 2, 53-5.
    CrossRef
  16. Munro, BH (2005). . Statistical methods for health care research, , ed. (Philadelphia:Lippincott Williams & Wilkins), pp. .
  17. Cokluk, O, Sekercioglu, G, and Buyukozturk, S (2010). . Sosyal bilimler icin cok degiskenli istatistik: SPSS ve LISREL uygulamalari, , ed. (Ankara:Pegem Akademi), pp. .
  18. Ozdamar, K (2004). . Paket programlar ile istatistiksel veri analizi, , ed. (Eskisehir:Kaan Kitabevi), pp. .
  19. Lynn, MR (1986). Determination and quantification of content validity. Nurs Res. 35, 382-5.
    Pubmed CrossRef
  20. Aluja, A, Blanch, A, and Garcia, LF (2005). Reanalyzing the 16pf-5 second order structure: exploratory versus confirmatory factorial analysis. Eur J Psychol Educ. 20, 343-53.
    CrossRef

Article Tools

Stats or Metrics

Share this article on :

  • line

Related articles in JCS

more +