Translation and validation study of the Indonesian version of the practice environment scale of the nursing work index

I Gd Juanamasta ,Yupin Aungsuroch ,Mary L.Fishr ,Siluh Nyoman Alit Nuryani ,Ni Nyoman Ayuningsih

a Faculty of Nursing, Chulalongkorn University Bangkok, Thailand

b Nursing Program, STIKES Wira Medika Bali, Indonesia

c Indiana University School of Nursing, Indianapolis, USA

d College of Nursing, University of Florida, Gainesville, USA

e Nursing Committee, Prof IGNG Ngoerah General Hospital, Bali, Indonesia

f Quality Assurance Committee, Prof IGNG Ngoerah General Hospital, Bali, Indonesia

Keywords:Indonesia Nurses Psychometrics Working conditions

ABSTRACT Objective: This study aimed to translate and validate the Practice Environment Scale -Nursing Work Index (PES-NWI) among nurses in Indonesia.Methods: A scale translation and cross-sectional validation study was conducted.The English version was translated into Indonesian,which involved five steps: forward translation,compare the translation,backward translation,compare the translation,and pilot testing with a dichotomous scale (clear or unclear).Thirty inpatient department nurses were involved in checking readability and understandability.A cross-sectional study was conducted from August to October 2022 at 17 hospitals across Indonesia,involving 350 nursing professionals.The validity test included structural validity and convergent validity.The internal consistency reliability was tested by Cronbach’s α coefficient,item-total correlation,and composite reliability.Results: Confirmatory factor analysis (CFA) showed an acceptable fit.The correlation of all dimensions was between 0.70 and 0.88,and all items had item loading higher than 0.6.Convergent validity of each dimension ranged from 0.61 to 0.74,internal consistencies with Cronbach’s α coefficient was 0.97,corrected item-to-total correlation ranged from 0.62 to 0.85,and composite reliability of each dimension was higher than 0.89.Conclusions: Good homogeneity and construct validity have been demonstrated for the Indonesian version of the PES-NWI,nursing management can use it to measure the work environment.

What is known?

· A positive work environment could support nurses’performance.

· The Practice Environment Scale-Nursing Work Index(PES-NWI)is a well-known,valid,reliable,and feasible tool for assessing nurses’ working conditions.The scale has been adapted into numerous languages.

What is new?

· The Indonesian version of PES-NWI demonstrates that each question in the evaluation holds significant value in examining the work environment.

· The nursing director,chief nursing officer,and first-line nurse manager can use this instrument to assess the inpatient departments’ work environment.

1.Introduction

The work environment has a significant impact on the safety climate[1],quality[2,3],and productivity[4].It is“a set of concrete or abstract psychological features,such as autonomy and advancement opportunities perceived by an incumbent who compares these opinions against a set of standards,values,or needs”[5].A workplace’s institutional elements that either support or hinder the provision of high-quality nursing care are referred to as the “work environment” [6].Prior studies found that a good work environment could provide nurses with a good atmosphere and motivate nurses to work well [7-9].Healthcare organizations need to prioritize creating a supportive and empowering work environment to promote the well-being of their nursing staff and ultimately improve patient outcomes.

An unfavorable work environment adversely affects nursing staff and patient care [10,11].It leads to increased stress,burnout,and nurse dissatisfaction,compromising their ability to provide quality care[7].High nurse turnover rates result from this negative environment,causing a shortage of experienced professionals and straining the healthcare system [2].Additionally,it hampers communication and collaboration among healthcare team members [12],leading to misunderstandings and care errors due to nurses feeling unsupported or undervalued [1].This lack of cooperation affects patient safety and outcomes.Furthermore,a hostile work environment fosters a culture of blame and secrecy,discouraging issue reporting [13].

One way to create a supportive work environment is by fostering strong relationships among the nursing staff [14].Building a sense of camaraderie and teamwork can enhance communication and collaboration,leading to better patient care.Additionally,providing professional development and growth opportunities can contribute to a positive work environment [15].When nurses feel supported in their career advancement and have access to ongoing education and training,they are more likely to feel valued and engaged in their work.

To measure nurse work environment,several studies have established the instruments.In the field of nursing research,two instruments known as the Revised Nursing Work Environment Index (NWI-R) [16] and the Practice Environment Scale of the Nursing Work Index (PES-NWI) [6] have been very common.The others [17] are the Ward Organizational Features Scales (WOFS)[18],Brisbane Practice Environment Measure [19],Essentials of Magnetism(EOM II)Scale[20],Practice Environment Index-Single Factor Model [21],Job Content Questionnaire (JCQ) [22],and the Registered Nurse Working Conditions Barometry Index form (RNWCBI) [23].The other instruments were rarely used,and they might be specific to the hospital situation in their country.

Lake [6] revised the original NWI and created the Practice Environment Scale (PES) to develop a more concise,psychometrically sound scale with empirically derived subscales.She defined the nursing practice environment as the organizational characteristics of a workplace that facilitate or impede professional nursing practice.Thirty-one items and five dimensions indicating key domains in the hospital environment that support professional nursing: 1) nurse participation in hospital affairs (9 items),2)nursing foundations for quality care (10 items),3) nurse manager ability,leadership,and support of nurses (5 items),4) staffing and resource adequacy (4 items),and 5) collegial nurse-physician relations (3 items).All dimensions loadings were at least 0.40,and internal consistency coefficients for subscales ranged from 0.71 to 0.84,with an overall Cronbach’s α coefficient reported as 0.82[6].It used four points on the Likert scale.The range of possible scores was from 1 (strongly disagree)to 4 (strongly disagree).

PES-NWI’s content is comparable to other nursing practice environment instruments,and its length exceeds that of other instruments[24].In addition,the PES-NWI factor structure has been validated with 1998 data from 8,597 nurses in Ontario and Alberta[25],1999 data from staff nurses throughout Pennsylvania (Lake,2002),2001 data from 243 nurses in a Quebec hospital [26],and 2004 data from 2,900 nurses in 14 hospitals in Texas[27].In 2006,the National Database of Nursing Quality Indicators introduced the PES-NWI as an option for the annual nurse survey [24].The PESNWI is a valid and widely utilized instrument for evaluating the nursing practice environment in Western healthcare systems[28,29].In addition,the dimensions of this instrument are suitable for use with Indonesian nurses due to their cultural relevance[28].Despite the availability of an Indonesian translation and face validity was done by the earlier study,the prior studies[28,29]did not demonstrate a transparent procedure for translation,and the validity score was derived from a first-order confirmatory factor analysis.

The purpose of this scale is to gain insight into the working conditions of nurses from the perspective of the nurses themselves.The utility of the scale must be consistent across the board,which includes the culture from which it was derived,as well as all other cultures and languages[30].Therefore,its translation,validity,and reliability must be confirmed in Indonesian nursing culture for its use.

2.Methods

2.1.Study design

The study utilized 7-step translation,adaptation,and validation of instruments by Sousa and Rojjanasrirat [31].The 7-steps included 1) forward translation,2) compare the translation,3)backward translation,4) compare the translation,5) pilot testing with a dichotomous scale (clear or unclear),6) preliminary psychometric testing(the study did not use this step),7)psychometric testing.First,the PES-NWI underwent a process of translation into Indonesian,ensuring that semantic and idiomatic equivalence were maintained.Second,the Indonesian scale underwent cultural and conceptual adaptations.Third,the validation of its application for applying PES-NWI in nurses was conducted.

2.2.Ethical approval

The ethical standards outlined in the Declaration of Helsinki were maintained throughout this inquiry.The National Research and Innovation Agency of the Republic of Indonesia (BRIN)reviewed and approved ethical clearance(Ref.No:176/KE.01/SK/8/2022).In addition,the hospital director granted permission to carry out the study.Before enrolling in the research study,each participant provided written informed consent.Participants were free to deny or withdraw from this research.

2.3.Translation procedure

The translation steps included 1) forward translation,2)compare the translation,3) backward translation,4) compare the translation,and 5) pilot testing with dichotomous scale (clear or unclear).First,a translator and a nursing professor worked independently to translate the items.These individuals were selected based on their linguistic proficiency in English and Indonesian and their familiarity with the cultural context in which the instrument would be utilized.Second,one official translator and three registered nurses with advanced degrees and extensive clinical experience reviewed the items after they were translated.Reviewing the translated materials for relevance and conceptual ambiguities was the reason for this analysis.An expert panel reached an agreement to modify the Likert scale.Initially,the instrument featured a fourrank scale,whereas the Indonesian version uses a five-rank scale.Hospitals in Indonesia prompted consideration from the expert panel.They introduced a midpoint into the instrument to ensure fair scoring,a decision supported by previous research suggesting the inclusion of a midpoint in such situations[32].Two translators,both possessing backgrounds in nursing and health sciences,conducted a process known as backward translation.Initial testing was done on a draft of the instrument in the target language.A backward version was compared and reviewed by the expert panel.The Indonesian version used “chief nursing officer” rather than“nursing administrations.”The nurse manager was also changed to ward manager/head nurse because “nurse manager” is unfamiliar in Indonesia.There is no content validity in this study.Translation validity was done based on the consensus of the expert panel.All the experts discussed each item’s agreement after comparing forward and backward translations.

After acquiring the Indonesian version,a pilot study was conducted to test the questionnaire on 30 nurses who shared similar characteristics with the target population.This pilot study assessed the nurses’comprehension of the instrument’s instructions,items,and response format.Every individual involved in the study was requested to evaluate the items using a dichotomous scale,which consisted of clear or unclear options.No items in the questionnaire posed difficulties in terms of comprehension.Consequently,it was decided that no modifications were required after completing the pilot test.The instrument demonstrated satisfactory validity and successfully conveyed the intended meaning of each item.

2.4.Scale evaluation and validation procedure

2.4.1.Study settings

A convenient sample of nursing professionals employed in the inpatient department (IPD) provided the information gathered from August to October 2022.A total of 17 hospitals from all regions in Indonesia (East,Middle,and West Region),type A (tertiary),B(secondary),and C (primary),were gathered to generalize the results of the instrument.

2.4.2.Participants

Information was gathered from 350 nursing professionals currently employed at hospitals in Indonesia.The following were the criteria for inclusion: 1) have earned a nursing diploma or baccalaureate certificate from an approved program,and 2) have experience working in an IPD setting for at least one year.The criteria for excluding participants were professionals not currently engaged in paid work when the data were collected.

The selection of an appropriate sample size was an important decision that must be made.There is currently no universally accepted standard for determining sample size in validation research,specifically in the context of factor analysis,whether it is exploratory or confirmatory [33].Most of them are constructed based on rules of thumb,which typically range in size from three to twenty participants for each item [34] or ≥200 [35].The PES-NWI scale contains 31 items,so the 350 participants in this study fall within the parameters of the previous range.

2.4.3.Instruments

The socio-demographic questionnaire consisted of age,gender,ward (intensive or non),education,and marital status.

PES-NWI was developed based on the NWI.PES-NWI is comprised of 31 items that are organized into five dimensions.The scoring used a 4-point Likert scale ranging from 1 to 4 (strongly disagree-strongly agree).A higher mean score of more than 2.5 is favorable;otherwise,it is unfavorable [6,36].According to expert translation consensus,there are five levels of the Indonesian version of PES-NWI scores,with 1 being the lowest and 5 the highest.The average score was divided into five categories using the class interval formula x=(xmax-xmix)/k.Each successive lower limit was also padded with 0.01 to avoid overlap between intervals[37].The mean scores of the PES-NWI can be categorized into five levels of interpretation.The first level,1.00 to 1.80,represents a poor work environment.The second level,ranging from 1.81 to 2.60,indicates a poor work environment.The third level,ranging from 2.61 to 3.40,signifies a fair work environment.The fourth level,ranging from 3.41 to 4.20,denotes a good work environment.Lastly,the fifth level,ranging from 4.21 to 5.00,represents a good work environment.

After obtaining the formal authorization letter from the hospital’s director,we contacted either the director of nursing or the chief nursing officer.The ward coordinator or unit coordinator ensured the research was thoroughly explained to all individuals in attendance.The ward or unit coordinator carried out the distribution of questionnaires to the nurses.Following a thorough elucidation of the study by a research team member,all participants were promptly directed to complete a questionnaire.The participants collected the completed questionnaires and deposited them in a secure container at their assigned units’ nursing stations.The container mentioned above was solely accessible to the individuals tasked with supervising the management of the research inquiry.

2.4.4.Data analysis

The IBM SPSS Statistics 22 program,which is part of the Statistical Package for the Social Sciences,was utilized to conduct statistical analysis.An examination of the socio-demographic variables was subjected to descriptive analysis.It is critical to determine whether or not the absence of data is due to some sociodemographic bias.There was no missing data in this study.

We are examining the structural validity of a confirmatory factor analysis(CFA)with the help of the program LISREL 8.72.The most frequently utilized statistical method is known as factorial analysis when investigating the dimensions or subscales of a test based on the scores obtained.Factorial analysis can generally be broken down into two categories: exploratory and confirmatory factorial analysis.Since the PES-NWI is an evaluation based on an earlier theory and has a predetermined format [5],it is recommended to conduct a confirmatory analysis to determine whether or not it maintains the initial structure described in the data collection instrument subsection.

The criteria applied by Hair was used to evaluate the measurement model fit with research data,including significant Chisquare-test (χ2),Chi-square-test/degree of freedom (χ2/df) ＜3.00,comparative fit index (CFI) ＞0.90,root-mean-square error of approximation (RMSEA) ＜0.07,standardized root-mean-square residual (SRMR) ＜0.08,goodness-of-fit index (GFI) ＞0.80,and adjusted goodness-of-fit index (AGFI) ＞0.80 [38].

To evaluate the convergent validity of the data,the average variance extracted (AVE) was utilized,as recommended by Hair,Black,Babin et al.[38].High AVE accurately represents the underlying latent construct.For a construct to be considered valid,it is recommended that the AVE ＞0.50 [38].If AVE ＜0.50 and the construct reliability ＞0.6,the convergent validity of the construct is sufficient [39].

Internal consistency reliability was assessed at composite reliability and Cronbach’s α coefficient.There was a high degree of reliability across the board,with all construct reliabilities exceeding the critical value of 0.70[38].The most widely employed reliability coefficient for this function is Cronbach’s α coefficient.In addition,the average correlations between the items themselves and between the items and the total were reported.It is fine if the former falls between 0.30 and 0.70,while the latter needs to be higher than 0.3 to be considered acceptable [40].

3.Results

3.1.Demographic characteristics of participants

Most of the nurses who participated in the research were 30 years old.The age range was 22-57,with an average of 33.There were approximately 73.4 %of female nurses.Most nurses working in non-intensive care settings comprised 74.3 % of the total.The percentage of nurses holding a diploma was 57.2 %,followed by those holding a bachelor’s degree in nursing (42.3 %),a master’s degree (0.3 %),and a specialist (0.3 %).According to the statistics,81.1 % of the nurses were married.

3.2.Validity

3.2.1.Structural validity

The results of the second-order CFA showed that there were five dimensions to the PES-NWI,each containing 31 items.The initial model showed the results of χ2/df=4.11,RMSEA=0.09,GFI=0.76,and AGFI=0.71 were not acceptable.Meanwhile,χ2=0.00,CFI=0.98,and SRMR (0.04) were acceptable.Then,a modified model was produced by setting error covariance free for some items of observed variables.After modifying the model,the Goodness of Fit(GOF)statistics indicated that PES-NWI was acceptable,as shown in Table 1,except for GFI(0.82)and AGFI(0.78).AGFI score was nearly 0.80.Findings showed the five dimensions and 31 items were significant construct validity of PES-NWI.

Table 1 PES-NWI goodness-of-fit statistics (n=350).

Five dimensions of the PES-NWI were statistically significant from the original.As shown in Fig.1,the correlation for all dimensions was between 0.70 and 0.88 at the 0.05 significance level.In particular,standardized items loading ranged from 0.67 to 0.88 for nurse participation in hospital affairs,from 0.75 to 0.89 for nursing foundations for quality care,from 0.78 to 0.91 for nurse manager ability,leadership,and support of nurses,from 0.70 to 0.87 for staffing and resource adequacy,and 0.91 for collegial nurse-physician relations.

Items loading greater than 0.6 were observed for the vast majority of items.PES-NWI was effectively built with the latent dimension variable.The highest item loading was item 23,29,30,31(B=0.91),followed by item 15 and item 24 with item loading of 0.89.Item five was the lowest item loading with 0.67,followed by item nine and item one with 0.68 and 0.69,respectively.

3.2.2.Convergent validity

The average variance of each latent variable ranged from 0.60 to 0.82,as shown in Table 2.At the level of significance(ρvc＞0.5),all dimensions have the potential to account for the variance of the latent variable [38].

Table 2 Summary of the construct reliability,average variance extracted,and internal consistency in the PES-NWI.

3.3.Reliability

Internal consistency was measured using Cronbach’s α coefficient and found to be 0.97.All five dimensions had reliability coefficients of 0.89-0.95 according to Cronbach’s α coefficient.Corrected item-to-total correlations (Table 2) for all items were greater than 0.30,indicating they all provided sufficient item discrimination.Table 2 shows that the PES-NWI latent variables have a high level of composite reliability,with values ranging from 0.89 to 0.95.The dimension of nursing foundations showed the highest composite reliability score (ρc=0.95) for quality care,while the dimensions of nurse managers’ ability,leadership,and support of nurses,as well as nurses’participation in hospital affairs,each showed a composite reliability score of 0.93.

4.Discussion

Within the scope of this investigation,an exhaustive translation,validity,and reliability check was carried out.After being translated,some of the terms appeared to have different meanings.Compared to the earlier studies[28,29],the most notable difference was that chief nursing officers,rather than nursing administrators,were referred to in the research.Another problem is that most hospitals in Indonesia,particularly public hospitals,do not have a nursing director at the top of their organization.The director of medical,nurse,and support service or the director of health service oversees the chief nursing officer.Contrarily,private hospitals typically have a nursing director responsible for overseeing all nursing services.The professionals used the term nursing director to maintain consistency with the instrument’s character.

Almost all of the nurses in the pilot study could provide very satisfactory responses to questions.The nurse can comprehend and read all of the questions.However,there were a few nurses who needed some items clarified.These items included“opportunity for advancement”and“Administration listens and responds to nurses’concerns.” It’s possible that this situation arose because different types of hospitals (private vs.public,type C vs.type A) have different levels of authority.

The CFA found that the original PES-NWI questionnaire’s five dimensions held statistical significance when translated into Indonesian after modifying the model.The initial model was insignificant in most indicators,except CFI(0.98)and SRMR(0.04).Modified indices were done through the error covariance among several items.Furthermore,the GFI score was 0.82,while the previous literature stated that ＞0.80 is acceptable [41].While the literature provides a variety of goodness-of-fit indices that vary by sample size and study design,these indices are often interpreted differently,and their thresholds for significance are usually not agreed upon [42].

According to previous studies,PES-NWI was translated and adapted by Indonesian researchers [28,29].The former study used first-order CFA and this study use second-order CFA which might impact upon the results.Meanwhile,this study found better results for the score of construct reliability,AVE,Cronbach’s α coefficient,and item-to-total correlation.The construct reliability is measured by how well individual items on the same test correlate.The score similarity between items used to measure the same construct wasexamined [43].High construct reliability in this study indicated that all the items constantly measure the same construct.In addition,the high AVE in this study showed that the convergent construct could explain the variance of the indicators well.

Correlation between a single item and all other items in the test could be expressed using the corrected item-total correlation[44].According to the results,all items had high item-to-item and itemto-test coherence levels.Additionally,the study’s internal consistency (α=0.97) was higher than previous studies that Lake conducted (α=0.82),Neves,Parreira,Graveto [45] (α=0.91),and Chiang and Lin [46] (α=0.90).

Previous research suggests that Cronbach’s α coefficient above 0.95 may imply redundant items [47].Another study contradicts Cronbach’s α coefficient assumption of identical factor loadings across indicators,known as tau-equivalence,arguing that this assumption is not valid for latent constructs [48].Reliability estimation of latent constructs using Cronbach’s α coefficient can be challenging,particularly when indicators have different component loadings.According to Cheung,Cooper-Thomas,Lau et al.[48],Cronbach’s α coefficient is unreliable because it doesn’t consider the second-order factor structure.The topic of remarkable reliability is often discussed.This study analyzed the potential duplication in meaning after translation and the limits of utilizing the overall Cronbach’s α coefficient (Table 2) as a representative measure.

The translated instruments’ validity,reliability,and quality depend on a methodological approach,openness,and strict respect for criteria for the translation and cross-cultural validation of research instruments[49].These standards conducted this research because standard operating procedures were used,meticulous records were kept,and people with clinical and research backgrounds were included.The research team had members with backgrounds in research methodology,psychology,nursing education,statistics,and project management to foster and guarantee these conditions throughout the study.

5.Limitations

Further studies can use the findings of this study as an essential value of validity and reliability,especially in Indonesian hospital settings.However,the study has several limitations.First,the utilization of convenience sampling in this study has resulted in the introduction of selection bias.However,the study’s use of a multicentre hospital (tertiary,secondary,and primary) could mitigate this issue.Second,while nurse research assistants have extensive training and experience,the level of agreement among observers remained uncertain.Third,participants focused on the inpatient and outpatient departments might give different results.Additionally,test-retest reliability might be needed in further studies.

6.Conclusions

The study results indicate that the PES-NWI has sufficient construct validity,convergent validity,and internal consistency.It is possible to use the PES-NWI to evaluate how IPD nurses feel about their participation in the hospital,the foundations for quality care,the ability of the unit manager,the staffing and resource adequacy,and the collegial nurse-physician relations in Indonesian hospitals.The hospital,chief nursing officer,or nurse first-line manager can use the PES-NWI to measure the work environment.

Funding

This study was supported by the Second Century Fund(C2F)and 90th Anniversary of Chulalongkorn University,Rachadapisek Sompote Fund.

Data availability statement

The datasets generated during and/or analyzed during the current study are not publicly available due to hospital requests but are available from the corresponding author upon reasonable request.

CRediT authorship contribution statement

I Gede Juanamasta: Conceptualization,Methodology,Validation,Formal analysis,Investigation,Data curation,Writing -original draft,Writing-review&editing,Project administration.Yupin Aungsuroch: Conceptualization,Methodology,Validation,Formal analysis,Investigation,Resources,Data curation,Writing-review&editing,Supervision,Project administration.Mary L.Fisher:Conceptualization,Methodology,Validation,Formal analysis,Investigation,Resources,Data curation,Writing-review&editing,Supervision,Project Administration.Siluh Nyoman Alit Nuryani:Methodology,Validation,Formal analysis,Investigation,Resources,Writing-review&editing.Ni Nyoman Ayuningsih:Methodology,Validation,Formal analysis,Investigation,Resources,Writing -review&editing.

Declaration of competing interest

The authors have declared no conflict of interest.

Acknowledgments

The copyright of the original instrument retains by Professor Eileen T.Lake,PhD,RN,FAAN.It is free to read,copy,and translate for non-commercial and academic purposes.The link is here https://www.qualityforum.org/QPS/MeasureDetails.aspx?standardID=1824&print=0&entityTypeID=1.

Appendix A.Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ijnss.2023.09.018.