Danish translation and cultural adaptation of the ‘ What do you think of hospital ’ patient reported experience measure for children and adolescents in outpatient settings

Purpose: The aim of this study was to translate the patient reported experience measure (PREM) questionnaire ” What do you think of the hospital? Help us to get better! ” into Danish used in outpatient clinics and to explore its face and content validity. Designandmethods: The translation processfollowedWHOrecommendations and includedforwardtranslation, expert panel evaluation,backtranslation,pre-testing and cognitive interviews with 23 childrenand adolescents. Results: Children and adolescents were positive to using PREM as a way to express their experiences. The layout of the questionnaire was important as use of colours was more appealing and the topics of the questionnaire were better visualised. The concepts in the original questionnaire related to distinguishing between different rooms for examination and conversation are not used in a Danish context. Otherwise, only minor translation adjustments were needed to match the Danish target group. Conclusion: Children and adolescents found that the Danish version of the PREM questionnaire tool was easy to read and understand, and thelayout emphasisedthat they are thetarget group. Afterpre-testingamong23chil-dren and adolescents, the questionnaire is now ready for pilottest in a larger group. Practice implications: The present study provides a tool to generate knowledge and evaluate the experiences of childrenandadolescentsinanoutpatientclinic.Usingthequestionnaire,healthcarestaffmaymonitorthequality oftheexperiencesofchildrenandadolescentsandcollectdataforresearchpurposes.Likewise,itwillbepossible to compare hospitals and organizations nationally. © 2022 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http:// creativecommons.org/licenses/by/4.0/).


Background
Surveys on patient satisfaction are increasingly used to measure quality at hospitals and identify areas of improvement.These surveys may be seen as the patients voice in healthcare and are considered useful as important quality development instruments (Ambresin et al., 2013;Hargreaves & Viner, 2012;Hopwood & Tallett, 2011;Wray et al., 2018).
In Denmark, approximately every sixth child between five and 17 years has a chronic or long-term condition (Danish Patients, 2020) and the number follows an increasing trend (Danish Health Authority, 2014).Children and adolescents constitute a patient group with special needs and challenges, and it is important that the healthcare system adapts to meet their needs.Being a child or an adolescent with a chronic condition is challenging and often highly incompatible with a normal youth life (Meinike et al., 2010).
Healthcare professionals have an important responsibility to listen to children and adolescents and try to understand their reality.Based on World Health Organization (WHO) recommendations, the Department of Health in England has prepared quality criteria for healthcare services to meet the needs of and rights of children and adolescents (Department of Health, 2011).One of these criteria regarding the participation of children and adolescents is the evaluation of their treatment.The healthcare system should be attentive to the experiences of children and adolescents as part of their service development, monitoring and evaluation.Therefore, children and adolescents should routinely report on current services, relevant new developments and be included in surveys on patient satisfaction (Department of Health, 2011).
Children and adolescents wish to participate, contribute and share their experiences to the benefit of others (Children's Welfare, 2017;Department of Health, 2011;Wray et al., 2018).Great Ormond Street Hospital in England has developed a questionnaire targeting three age groups to give the children and adolescents a voice in the evaluation of their experiences while visiting an outpatient hospital clinic.The three groups covered ages 8-11 years, 12-13 years and 14-16 years.Across age groups, children and adolescents provided feedback and were actively involved in the drafting and design of the questionnaires consisting of 30, 37 and 38 questions, respectively, for the three age groups (Wray et al., 2018).
The purpose of the present study was to translate into Danish the questionnaire"What do you think of the hospital?Help us to get better!" which is used in outpatient clinics and explore its face and content validity through cognitive interviews.

Method
The questionnaire" What do you think of the hospital?Help us to get better!" was translated into Danish following a standardized WHO protocol.The purpose of this process is to obtain different translated versions of instruments that are conceptually identical in different cultures (WHO, 2013).The translation process comprised four phases.In the present project, the translation of a questionnaire for one age group was completed before starting the translation of the questionnaire for the next age group.
Permission to translate and validate the questionnaire in Danish was provided by the authors of the original questionnaire (Wray et al., 2018).The questionnaire was developed for children and adolescents between 8 and 16 years of age treated in a specialist outpatient clinic for children and adolescents.In Denmark, adolescents aged 17 years are treated in a specialist outpatient clinic and were thus also included in this study to ensure adequate adaptation to the Danish context.

Phase 1
The original questionnaire was translated from English into Danish by an expert group consisting of four nurses specialized in child and adolescent nursing and a psychologist experienced in research and in translation procedures; all were fluent in English.Members of the expert group translated the questionnaire individually.Translation was made as close to the original as possible, though ensuring adaptability to the Danish context (Van Widenfelt et al., 2005).

Phase 2
The expert group met to compare the translated versions and reach a consensus on the best version for each question.The expert group was aware of ensuring cultural adaptation of concepts, which is an important part of the process (Van Widenfelt et al., 2005).The first author recorded notes and comments related to the expert group's choices.

Phase 3
Back-translation of the questionnaires from Danish into English was performed independently by a bilingual paediatric nurse with experience in translation of questionnaires into English but no knowledge of the original questionnaires.The original versions were compared with the back-translations and the expert group made minor adjustments.

Phase 4 -Cultural validation
The purpose of the cognitive interviews was to identify the wording, phrasing and relevance of the questions as well as design, flow and length.For this purpose, a semi-structured interview guide was developed (see Fig. 1).
Interviews were recorded and transcribed by the first author and all statements were anonymized.The analysis was conducted in accordance with Kvale and Brinkmann (Kvale & Brinkmann, 2015) with coding of meaning, meaning condensation and interpretation (see Table 1).
The questionnaire was tested among children and adolescents between 8 and 17 years of age at their follow-up visits in the outpatient clinic at the Department of Paediatric and Adolescent Medicine, Aarhus University Hospital, Denmark.Purposively sampling was used to achieve diversity in age and gender of the participating children and adolescents.Before the appointment children and adolescents were approached by a nurse from the department.If oral consent to participate was provided, AMR informed the children or adolescents and their parents about the project and obtained their written consent.AMR asked the children and adolescents to read and complete the questionnaire with a focus on their understanding of the questions.The included children and adolescents were developed age-appropriately and able to read and understand Danish but parents provided help if needed to complete the questionnaire.The cognitive interviews uncovered the children and adolescents' understanding of the phrasing, the clarity of the questions and if there were any suggestions for changes, which might improve the translation of the questionnaire (Van Widenfelt et al., 2005).The interviews were conducted in a room in the outpatient clinic and present at the interview were only AMR and the informant.The interviews lasted between 15 and 23 min and

Examples of quesƟons included in the Interview guide
What do you think of the quesƟonnaire?
Were the quesƟons easy to understand?
The quesƟonnaire uses words like: treatment room, consultaƟon room and examinaƟon room.Do you use these words or what do you call the rooms in the outpaƟent clinic?
The quesƟonnaire uses the word intravenous access.Do you know what that is?What do you call it?Do you think the quesƟonnaire is missing some important topics or quesƟons?were tape recorded and transcribed.Interviews and transcriptions were conducted by the first author.No financial compensation was awarded for participation.
Using the data from the interview and analysis, the expert group adapted the questionnaires to the final Danish versions.As specialist outpatient clinics at the Department of Paediatrics and Adolescent Medicine, Aarhus University Hospital, are organized in different ways and offer different healthcare services, the expert group found it relevant to add the question, "Which outpatient clinic did you visit today?" to focus quality improvement initiatives further.

Ethics
This project complied with the Helsinki Declaration and was conducted in line with ethical guidelines of the Nordic countries and the UN focusing on respecting the rights of children and adolescents (Northern Nurses' Federation, 2003; The National Council for Children, 1989;World Medical Association, 2001).All names and information were anonymized to comply with the ethical obligations applicable in interview studies.Parental consent was provided for interviews with children and adolescents below the age of 15 years.Adolescents above 15 years provided informed consent themselves.According to Danish law, this project did not require ethics approval (R. No. 74277).Permission to conduct the study was obtained from the hospital department management.

Findings Findings and challenges made during the translation process
Early in Phase 2, the expert group chose to delete a question in the original questionnaire involving a specific restaurant as such a facility is not linked to the outpatient clinic in the Danish setting.The response category "I went through some tests, examinations or treatments" was moved to the first response category as recommended in a previous study on hospitalized children between 8 and 11 years (Buus et al., 2021).
The expert group also found it necessary to adjust the question, "If you were awake during your test, examination or treatment, did the staff do any of these things to help you?".New coping strategies are constantly emerging, and several facilities in Denmark now offer virtual reality glasses and laughing gas.Thus, in addition to a free-text field to the specific question, virtual reality glasses and laughing gas were added to the response categories.The expert group added the word"draw" to the question in which children and adolescents were encouraged to write comments to make it possible to both write an draw.
Data from a national counselling service for children and adolescents in Denmark show that an increasing number of children and adolescents are insecure of their gender identity (Children's Welfare, 2021).Thus, the expert group chose to add the response category "neither" to questions related to the gender of the child/adolescent.Moreover, the expert group changed the list of different languages related to the question, "Which language do you primarily speak at home?" to tick a box for "Danish" and a free-text field if a language other than Danish was the primary language.

Findings in cognitive interviews
The questionnaires were pilot tested on 23 children and adolescents between 8 and 17 years of age.Participant characteristics are presented in Table 2.
Three themes (Table 3) were established with sub-themes illustrating the experiences of children and adolescents when completing the questionnaires with a focus on understanding, applicability and meaningfulness.

Comprehensibility
Design.The design of the questionnaire was made easier to comprehend by applying colours to separate the categories and to make children and adolescents perceive the questionnaire as uncomplicated.It was an advantage that responses could be ticked off as this did not require much writing.
"The questions were well-connected and logically organized.It wasn't just like a random question popping up" Girl, 12 years.
"Well … because there were many things down here (pointing at response categories), it was easy and it was easy just to tick off the answer."Boy, 8 years.
The children and adolescents were positive about key aspects of the questionnaire such as use of text boxes with fun facts and drawings because it created a break from reading and made them curious about the questionnaire.
"Yes, the smileys and texts boxes make you wake up a little when completing the questionnaire so that was positive."Girl, 17 years.
Choice of wording in questions.The children and adolescents said that the questions were generally easy to read, especially the brief questions.The questionnaire used the phrase, "Mark your answer in a column".The expert group doubted if the children would understand the word "column" and changed it to "row".All participants in the pilot test understood the word "row", and the expert group chose to stick with this word.
"Row is fine.I think it would be more difficult if it said column".Boy, 12 years.
In questions related to the hospital, the response categories for specific rooms included waiting room, treatment room and examination room.The interview clarified that all participants understood the concept of "waiting area", but they were unsure about and unable to differentiate between the concepts of "treatment room" and "examination room".Thus, these two terms were replaced by the response category "other rooms".
"..I don't know if I would call it a treatment room but that's probably because I hadn't really thought it was a word…I wouldn't call it anything special because it's kind of the same.I've tried to talk to a doctor in the same room as where I was pricked with a needle so…" Girl, 12 years.
One of the boys through that this ambiguity about the appropriate name of the room extended to parents as he noted; "Yes, but we didn't know the difference, not even my mum, so we just answered it was the same" Boy, 15 years.
Only a few treatment-related concepts were used in the questionnaire and the concept "intravenous access" was unfamiliar to all age groups, but all participants knew the synonym "drip".Therefore, "Intravenous access" was replaced by"drip".To inform children and adolescents that these are synonyms, intravenous access" was inserted in parentheses after the term "drip".
"yes, it (the word intravenous access) is definitely a word I've heard a lot of times when I've been here and it's starting to become a word that I use at home.But in the beginning, it was mostly the word "drip".But now I'm getting used to "intravenous access".Girl, 17 years.The colours and drawings make it less boring and serious but rather more exiting and fun and make them want to read more.The use of drawings related to the question makes the questionnaire easier to read

Importance of fun facts, colours and drawings
It is important that the questions use the words familiar to the child and adolescents as another girl said; "So drip is the same as intravenous access?Well, then I know what it is" Girl, 13 years.
In recent years, the term "outpatient clinic" has been replaced by the term "child and adolescent clinic" in Denmark.During the interviews, it was obvious that the children and adolescents knew the names of locations and that they used the name that fitted the specific location.
"It's a little different from what I call it; it depends on where I am" Girl, 10 years.
Another child elaborated; "I just say I'm going to the hospital because if I tell my friends why I'm not in school, they may be confused if I just say the clinic or the outpatient clinic" Girl, 12 years.
An uncommon sentence construction made a question unclear to children and adolescents and, in fact, the question did not make sense to them even though the sentence was short and employed familiar words.Several of the 8-13-year-olds found it difficult to understand the question "Was what they told you about why you were in hospital easy to understand?" "That was the question where I didn't understand a thing."Boy, 9 years."Well, first I had to read it a couple of times.I understand the words and all, but it was more the way it was writtenit confused me a little when you read it."Boy, 13 years.
The 14-17 year-olds had no problems understanding the question, but it may have been the sentence structure that caused the confusion.The sentence structure of this questions was thus changed to make it more straight forward.
"When I read the question, I started forgetting the middle part and I had to read it again and then I understood."Girl, 14 years.

Applicability Wishes and suggestions for changes
At the interviews, children and adolescents expressed wishes for additions and changes.It became clear that the questionnaire needed to clarify which visit the children and adolescents needed to evaluate.
"I thought: Should I answer in general from other hospitals also; it would have been nice if they had written that."Girl, 14 years.
Moreover, it turned out that children and adolescents in the age groups 12-13 years and 14-17 years lacked a response category for the question, "Did these places offer enough activities for adolescents your age?" Thus, the response category, "I did not need any activities" was added to the two questionnaires.
"There might have been an answer category saying that I didn't need anything."Girl, 12 years.
Likewise adolescents didn't feel like the answer categories was appropriate to the age group and they didn't feel like it was addressed to them"I thought like it was for slightly younger children needing something to play with.I don't need anything; I just sit and wait and check my phone or something."Boy, 17 years.
The choice of words was important in determining how the different age groups perceived the question.This became clear in two of the questions.The 14-17 year-olds perceived the choice of words to be inappropriate in the question, "Did you feel safe about the staff taking care of you in the children and adolescent outpatient clinic?", which indicated that specific words would affect them negatively.
"..I feel like I'm a dog or some small child."Girl, 14 years.Moreover, they pointed out that the response categories did not fit their age.The response categories "Showed you a book" and "Showed you a movie" were thus deleted."Well,this showing you a book or a movie may be for smaller children, so I think they're not that relevant for somebody in this age group."Girl,17 years.

View of the smiley questions
Various participants found that the smiley questions were appropriate and produced a more varied questionnaire.Children and adolescents did not find them disturbing but rather relaxing because they reduced the amount of text to be read.Some expressed that the smiley questions were easy to respond to because they just needed to look at the smileys.Even so, they did not wish for many more smiley questions.
"..I don't think there should be too many but it's nice to have a break from reading so muchthen there's something in betweenthat's really nice."Girl, 12 years.

Importance of fun facts, colours and drawings
It was of major importance that the questionnaire included fun facts, different colours and drawings because these elements had a positive effect on interest, focus and honesty in the responses of children and adolescents.The drawings supplemented the questions, and the children and adolescents did not perceive the questionnaire to be grave and serious.Children and adolescents expressed that fun facts and colours made the questionnaire funny, different and entertaining.We found that the design could be directly transferred to a Danish context including 17 year-olds.
"Yes, I have written that it was good with different drawings, colours and animals and it doesn't make is all grown up and serious."Girl, 12 years.
The fact that drawings related to the questions are used means that child and adolescents read and understand the questions more easily.
"..it's a really good idea with a drawing next to it because if you don't know what it means, you can just look at that."Boy, 14 years.

My opinion matters
The children and adolescents thought that completion of the questionnaire was worthwhile.They perceived that findings from the questionnaire might improve clinical pathways and have a positive influence on the future treatment of children and adolescents.Moreover, they contributed with knowledge about being a child and adolescent at an outpatient clinic in the hospital.
"Yes, I think it made sense and you can get an impression of what it's like to be young and be here."Boy, 14 years.
All children and adolescents agreed that we should use the questionnaire routinely in the outpatient clinic, but they found it difficult to answer how often it would be relevant for them.Even so, they did express that many things happen in their lives, so maybe it would make sense once or twice a year.
"I just want to say that it's a really good initiative because I've needed this a lot over the years I have come here," Girl, 17 years.
The questionnaires were handed out on paper, and the children and adolescents thought this was positive because it made it easier for them to draw, write comments and focus on the questions.
"Well, I would definitely have it on paper because then you can write extra stuff yourself or draw on it.I just think it's easier to figure out when it's on paper."Girl, 13 years.
"I think it's best on paper because you have to kind of concentrate a little more and read it all through.When using an app, you get through it quite fast."Girl, 17 years.

My voice is important
Children and adolescents have a voice in the healthcare system and they value knowing that they -rather than their parents -will be asked for their ideas and views in the future.By listening to the voices of children and adolescents, more precise responses are achieved than when just asking their parents, as they are able to voice their own ideas about their care.
"Then you get the precise opinion instead of the opinion of your parents.It's much better that you ask the person who is actually experiencing it instead of one who just observes it.This will definitely give some better answers."Boy, 17 years.
Children and adolescents found a great interest in sharing their experiences because these experiences are those hospitals should listen to and learn from.
"It's good because if they don't ask us, then they don't know what it not good."Girl, 8 years.

Discussion
This study aimed to translate into Danish the questionnaire"What do you think of the hospital?Help us to get better!" used in outpatient clinics, and explore its face and content validity through cognitive interviews.The study followed the steps for translation, adaptation and validation of a questionnaire instrument recommended by the WHO (WHO, 2013).The advantages of adapting an already existing instrument to a new cultural context is that it is less costly and timeconsuming than developing a new instrument.However, cross-cultural adaptation involves many participants and is time consuming (Epstein et al., 2015).
We found that children and adolescents considered the Danish version of the questionnaire easy to read and understand, and they expressed that the layout emphasised that they were the target group of the questionnaire.The questionnaire has the potential to give a voice to children and adolescents who are in contact with the hospital allowing them to convey their experiences.
In questionnaires for children and adolescents, a focus should be on making a clear introductory text even though the text will be longer, because the introduction prepares the target group for the questions (Bell, 2007).According to Bell (2007), the use of smileys may be beneficial.One explanation for the positive findings in our study may be that the design of the questionnaires made them interested and this motivated the children and adolescents to complete them.Another way to obtain a similar positive reception on the part of children and adolescents may also be to structure the questions in a way that will make them just as easy to respond to as smiley questions.
The present study showed that only a few elements in relation to the structure needed to be clarified, and the questionnaire was not considered overly complicated or difficult to navigate.Most uncertainties were related to the wording of questions, ambiguous meanings of concepts and grammar issues.This is in line with the findings of Kalfoss (2019), who identified that language issues often are relate to the ambiguity about the meaning of a word or where phrasing is idiomatic.When designing or translating a questionnaire it is important to identify unfamiliar words and replace these words or expressions with more family ones (Krause et al., 2022).
When translating a personality disorder instrument from English into Brazilian, Oliveira (2012) found that when a sentence is not understood it may be difficult to respond to the question.Moreover, the level of education of the children and adolescents may have influenced their understanding of specific words (Oliveira, 2012).The developmental level of the children and adolescents may have influenced how they perceived specific words.An expression like "caring for you" might be more appropriate for 9-year-olds than for 15 year-olds.
A study by Sawyer et al. (2014) showed that it is important to work with age-specific questionnaires because children and adolescents have different needs and skills both during their youth and in adulthood.The questions related to transition were not relevant to either the youngest children or to the adults (Sawyer et al., 2014).
Design, choice of words and phrasing in questionnaires for children and adolescents should be different from those used for adults.All questionnaires should, however, ensure that readability (LIX) is adapted to the target group (Taylor et al., 2019).The LIX is a scale measuring the readability of a text.Taylor et al. (2019) worked with patient-reported outcome measures (PROM) in a paediatric surgery department and found that the readability of several of the PROMs used was above the readability level of the target group, which led to a poorer understanding.The LIX of the original questionnaire is between 14 and 34, which is within the categories "easy" and "easy for all".After deleting words like treatment room and examination room, the LIX was reduced to 13-28 and a lower score reflect better readability.Together with drawings supporting the individual questions, we consider the LIX of the translated instrument to be appropriate for our target group.In addition to ensuring the readability this study also has also deliberately ensured short sen.
Several countries have developed national health policies promoting a more child-and youth-friendly hospital system (Ambresin et al., 2013;Sawyer et al., 2014).To our knowledge, only a few international surveys have been conducted about children and adolescents' experiences of the healthcare system.Although research shows that evaluation of the quality of care from the perspectives of children and adolescents deviates from that of their parents, as seen when parents have completed questionnaires as their child's proxy (Byczkowski et al., 2010;Hargreaves et al., 2018(Chappuis et al., 2010).This implies that, in general, the views of children and adolescents have been ignored (Hargreaves et al., 2018).This is in accordance with the present study.We found that children and adolescents have a strong wish to contribute with their experiences from the healthcare system.This is useful because feedback coming directly from them will be more accurate.With the questionnaire "help us get better" children and adolescents will have the opportunity to evaluate indicators that are important to them because it includes several of the indicators that adolescents in the literature finds relevant and important for an adolescent-friendly care (Ambresin et al., 2013;Daley et al., 2017;Söderbäck et al., 2011).
Practice and research Implications The translation into Danish and validation of the outpatient clinic questionnaire"What do you think of the hospital?Help us to get better!" provided a tool to generate knowledge of and evaluate children' and adolescents' experiences of outpatient clinics.The culturally adapted Danish versions of the questionnaire will be available for healthcare staff so that they may monitor the quality of the experiences of children and adolescents at outpatient clinics.Furthermore, the tool may be used for research purposes, as the tool will allow for inter-hospital and -organization comparisons in Denmark.

Limitations
One strength of this study was that it included children and adolescents from every sub special in the outpatient clinic.The study has some limitations that must be addressed.It is a single centre study and the number of participants in some of the age categories were low limiting the scope of the collected data and the generalizability of the findings.
The result of the present study is believed to be able to contribute new knowledge because child and adolescents' experiences with a questionnaire are articulated and described, which has not been done before.
However, the result should be interpreted in lights of limitations.

Conclusion
In conclusion, the translation and cultural adaption of an English PREM questionnaire for hospitalized children and adolescents to a Danish paediatric outpatient clinic was a success.Most children found the questions easy to read and understand.The children were positive about the layout including the various colours and fun facts provided in the PREM.The children and adolescents also expressed how they liked a paper-based version of the questionnaire.Furthermore, they were all very keen to express their views about their experiences in the outpatient clinic.

Fig. 1 .
Fig. 1.Examples of questions included in the Interview guide.

Table 1
Examples of coding of meaning, meaning condensation and interpretation.

Table 3
Themes and sub-themes in cognitive interviews.