Reliability and validity of the electronic version of the short questionnaire to assess health-enhancing physical activity (E-SQUASH)

The Short Questionnaire to Assess Health-enhancing Physical Activity (SQUASH) measures physical activity. This is a paper-and-pencil assessment. An electronic version of this scale, namely, the E-SQUASH, was developed to address the limitations of the paper-and-pencil version (e.g., missing data, time-consuming data entry processes, delay in sending feedback) and facilitate the handling of population data. This was a scale validation study. The test-retest reliability of the scale was examined the reproducibility of its scores. The consistency between the scores yielded by the E-SQUASH and paper-and-pencil version of the scale was examined. A total of 104 university students (response rate: 77%) participated. Regarding reproducibility, the two sets of total activity scores were strongly correlated (r = 0.68, p < 0.01), and the 95% CI (0.56–0.78) was adequate. There was a high level of consistency between the total activity scores that were yielded by the two versions of the scale (Intraclass correlation coefficient = 0.87). In this study, the reliability and validity of the E-SQUASH were established among university students. Further research is needed to convert this system into an application and translate its contents into different languages as well as across other age groups.


Introduction
Maintaining health-enhancing physical activity improves skeletal muscle function and reduces the risk of diabetes, coronary heart disease, stroke, cancer, and cognitive disorders [1]. Since it is important to assess physical activity, several measurement instruments have been developed. Although mechanical instruments (e.g., accelerometer, pedometer) are more commonly used, paper-and-pencil questionnaires are also reliable and valid measures of physical activity. They are advantageous because they are more cost effective and can be used with larger samples, when compared to mechanical instruments [2]. Nevertheless, paper-and-pencil questionnaires entail a few limitations [3] (e.g., missing values due to inattentive responding, longer time to enter the data into spreadsheets, delay in providing feedback to participants, and inefficient for use with population survey data). Furthermore, paper is neither cost effective nor environment-friendly (i.e., paper production leads to deforestation) in a highly information technology (IT)-dependent society.
The short questionnaire to assess health-enhancing physical activity (SQUASH) is a measure of engagement in daily (e.g., work/study, household) and leisure (e.g., walking, gardening, and sports) activities, and is on official national measurements in Netherlands [4]. The SQUASH is reliable, valid, and suitable for use with adults [4)]and older adults [5]. It is used in many clinical settings worldwide [6][7][8][9][10]. Paper-and-pencil assessments offer greater data security and ease of use, especially to older adults. However, electronic versions of assessments may serve as a cornerstone in future large-size surveys that aim to generate population-level data. The improvement of physical activity is a worldwide concern. For this purpose, monitoring of physical activity levels is of crucial importance. Having an electronic version of the SQUASH (E-SQUASH) available would be a favorable development.
Our diverse international research team includes specialists from the fields of health science, medicine, epidemiology, and IT. The conclusions of numerous discussions and pilot studies suggest that a web-based assessment format is appropriate because it entails a relatively lower initial cost than applications do. Accordingly, the web-based E-SQUASH was developed following the saturation of prior tests/revision of systems [11]. This study aimed to examine the reliability and validity of the web-based E-SQUASH.

Paper-and-pencil version of the SQUASH
The SQUASH is a self-report measure that takes 3 to 5 minutes to complete [4]. Using Ainsworth's Compendium of Physical Activity, activities that burn more than 2 metabolic equivalents have been included to assess engagement in health-enhancing physical activities. Activity scores (per week) are calculated for the following categories: commuting activities (walking, bicycling), light/intense activities at work/school and household activities, and leisure-time activities (e.g., walking, bicycling, gardening, playing sports). Finally, total scores can be calculated by summing the individual category scores.

Development of the E-SQUASH
First, the research team members discussed the limitations of paper-and-pencil assessments, the ideal functions/design of the E-SQUASH, and security concerns that pertain to the E-SQUASH [12]. Regarding the sports category, the paperand-pencil assessment contained an open-ended question, which required participants to indicate the specific sports that they regularly engage in with a maximum of four sports. To identify the sports that adults and older adults commonly played, the paper-and-pencil version of the SQUASH was administered to 50 adults and 50 older adults. Subsequently, the 26 most popular sports were identified. The names of these sports were incorporated into the E-SQUASH. Open ended questions style for writing the type of sports was kept in four sports of the E-SQUASH for unexpected sports.
To ensure easy access to the website on which the E-SQUASH is hosted, a Quick Response (QR) code was created. Therefore, the participants were not required to enter the entire Uniform Resource Locator (URL). To enhance the ease of responding, multiple-choice questions were used to elicit all information, except email address. For example, to indicate their gender, the participants were required to simply select one of two options (i.e., man, woman). Their email addresses were collected as individually identifiable data, and they were provided with feedback immediately upon submission of the form. Using the form validation system, we ensured that all the required fields were filled in prior to submission. Specifically, when the participant tried to submit a form with incomplete data, messages in a red font were displayed beside the blank fields. This webpage could be accessed on smartphones, tablets, and personal computers.
Feedback was immediately sent to the email address of the participants. Specifically, their physical activity scores within each category (e.g., work/school, household, and leisure activities) and their total scores were summarized as a bar graph. Their scores were displayed as a bar graph because users are likely to understand their results better when their scores are depicted as a visual image than when they are provided with only raw scores. The file format of the bar graph was Joint Photographic Experts Group (JPEG). The file size was small (i.e., 13 kigabytes), and this ensured that the maximum email size limit was not exceeded. Additionally, messages were sent to the participants. The first message informed them about the benefits of physical activity and importance of assessment. Next, when they responded to the assessment a second time, they received one of three messages. If the increase in their total activity scores was > 200, they received a message that praised them for increasing their physical activity levels and encouraged them to continue engaging in physical activities. If their total scores were identical to their first set of scores, they received a message that praised them for maintaining their physical activity levels and encouraged them to continue or increase their engagement in physical activities. If the decrease in their total activity scores was > 200, they received a message that empathetically described how difficult it is to maintain or increase one's engagement in physical activity and encouraged them to continue or increase their engagement in physical activity. Activity scores of 200 was used, because the score was approximately a quarter activity level which was recommended by World Health Organization for Adult and relatively achievable.
Security-related concerns were handled by an information technology company, which was registered as Privacy Makers (16190236(01)). They installed standard security updates and conducted computer virus checks. To ensure secure accessibility, an ID and a password were necessary to access the E-SQUASH system. The entered data were stored in Comma Separated Value (CSV) files and saved in a computer server in the IT Company. Only the initial researcher had access to the specific URL from which the data could be downloaded. The downloaded data could only be accessed using identification (ID) and a password, which were created specifically for this purpose.

Design
This was a scale validation study. The test-retest reliability of the scale was examined to assess the reproducibility of its scores. To minimize the effect of memory biases on participant responses, the second administration of the test was conducted two weeks after the first administration [4]. The validity of the E-SQUASH was examined against the paperand-pencil version of the SQUASH.

Participants and data collection
Undergraduate nursing students (years 2 and 3) were recruited from a governmental university in July 2019. The timing was chosen to avoid extremely heated season and extremely heavy snow season in the area. Indeed, these two seasons influence physical activity [4,5]. We recruited university students because young adults are likely to be familiar with using devices such as smartphones to access webpages as well as importance of physical activity in this generation [13]. To ensure that the response rate is high, we recruited nursing students because they tend to be very interested in healthrelated matters.
To examine participant characteristics, we collected demographic data, assessed lifestyle factors, and used questionnaires that measure health status. The demographic characteristics that were assessed included age, sex, and height/weight (i.e., to calculate Body Mass Index (BMI)). The following lifestyle factors were assessed: alcohol intake (no intake, occasional intake, or regular intake), smoking status (no smoking or smoking), and sleeping status (can sleep, cannot sleep, or neither). Health condition was assessed in terms of the presence of disease (yes or no) and the number of body parts in which the participant experienced pain (head/neck/trunk, arms, or/and legs). We assessed the presence of bodily pain because it can influence physical activity.

Data analysis
Descriptive statistics were computed to examine the demographic characteristics of the participants. The test-retest reliability of the scale was assessed by examining Spearman's correlation (95% CI) between the two sets of E-SQUASH scores. The validity of the scale was examined by inspecting the intraclass correlation coefficients (ICCs) between the E-SQUASH and paper-and-pencil SQUASH scores. ICC analysis differs from other correlation analyses; it analyses not only the correlations but also the consistency and agreement between two sets of scores [10]. The JMP software package (SAS Institute) was used to conduct all data analyses. The significance level was set as p < 0.05.

Ethical considerations
This study was approved by the ethics committee of the institution to which the first author is affiliated (1765). The participants were informed about the aims and methods of the study both in person as well as through written communication. Specifically, they were informed that (1) their privacy would be strictly protected, (2) refusal to participate would not adversely influence their education (some of the researchers were the participants' teachers), (3) their participation was voluntary, and (4) only the aggregate results would be published in academic journals. They were also informed that the reception of feedback about their physical activity levels and the time spent responding to the questionnaires were the benefits and disadvantages of participation, respectively. The participants were offered a clear file as an incentive to participate in this study. Table 1 summarizes the participant characteristics. A total of 104 students participated (response rate: 77%). Most of them were woman (88%), and their mean age and BMI were 20 ± 1 (SD) and 21 ± 2 (SD), respectively. Regarding lifestyle factors, most of them consumed alcohol (65%), did not smoke (99%), and did not experience trouble sleeping (84%).

Results
Most of the participants did not have a disease (92%). Among those who experienced pain in more than one site, 37%, 28%, and 11% of them experienced pain in their head/neck/trunk, arms, and legs, respectively. Table 2 presents the test-retest reliability coefficients that emerged for the E-SQUASH. The two sets of total activity scores were strongly correlated (r = 0.68, p < 0.01), and the 95% CI (0.56-0.78) was adequate. Regarding the different items, the strongest correlations emerged for sports (r = 0.81, p < 0.01), followed by bicycling to commute (r = 0.80, p < 0.01), intense household activities (r = 0.72, p < 0.01), and light (r = 0.61, p < 0.01) and intense (r = 0.62, p < 0.01) activities at work/school. Moderate correlations emerged for light household activities (r = 0.42, p < 0.01), and weak correlations emerged for bicycling for leisure (r = 0.30, p < 0.01). The correlation coefficients that emerged for walking and odd jobs performed during leisure time were not significant. Gardening during leisure time could not be examined because no responses were recorded for this category. Table 3 presents the ICCs that were computed to examine the validity of the E-SQUASH against the paper-and-pencil SQUASH. There was a high level of consistency between the two sets of total activity scores (ICC = 0.87). Regarding the items, the highest ICC emerged for light household activities (ICC = 0.   ± 1109 632 ± 1108 0.87* *ICC > 0.80 (strong consistency), a Activity score = minutes x intensity. N/A = not applicable. "Gardening" was not analysed due to no one answering.

Discussion
In this study, the reliability and validity of the E-SQUASH were examined and established among university students. The E-SQUASH is a highly secure electronic questionnaire system, and a validation system prevents the submission of incomplete forms. This system allows us to immediately import the data into Excel spreadsheets and send feedback to the participants' email addresses. The E-SQUASH can be used to overcome the limitations of the paper-and-pencil version of the SQUASH and efficiently handle large amounts of data.
The test-retest reliability coefficient of the scale, which was higher than the accepted threshold of 0.70, was satisfactory [14]. Furthermore, this value is higher than other recently reported figures [10]. This could suggest that university students possess better cognitive abilities and engage in more intense and or structured activities (e.g., sports), when compared to older populations. Conversely, weak reliability coefficients emerged for walking, gardening during leisure time, and household activities. These activities are not very common among university students and may therefore pose a challenge in recall. This consequence is comparable with other reports of lower reliability caused by lower participation [4,5].
Regarding validity, the consistency between the E-SQUASH and the paper-and-pencil version of the SQUASH was found to be satisfactory. In past studies, the validity of the paper-and-pencil version of the SQUASH was tested using accelerometer readings [4] and other measures of physical activity [5]. This time, the consistency comparing with paper-based SQUASH is focused. Some differences exist between scales. For example, in the E-SQUASH, the number of minutes for which one engages in physical activity is recorded on a scale with response options that progressively increase by 5 minutes. In contrast, in the paper-and-pencil version of the SQUASH, an open-ended item is used, and the participant is required to choose a value between 1 and 60. This difference may account for the lower consistency values that emerged for some items. One noteworthy difference pertains to the sports category. Specifically, in the E-SQUASH, the participants are required to simply choose one option from a list of commonly played sports, which were identified across a series of pilot studies. In contrast, in the paper-and-pencil version of the SQUASH, an open-ended item is used to assess this variable. Nevertheless, sufficient consistency was observed because sports are intense and structured activities. Finally, the weak consistency that emerged for unpopular activities is consistent with recent findings [4,5,10]. Physical activity levels tend to be overestimated using questionnaires as compared to objective measurement like accelerometers. The E-SQUASH has similar feature. Combining objective measurement with an online monitoring system that provides details about setting (work, home etc.) and type of activity (walking, cycling, sports) would be a greater value.
Recently, many paper-and-pencil questionnaires/scales have been developed within the field of health science. This traditional method of surveying individuals has many advantages. Specifically, it is cost effective, tolerating with moderate sample size, relatively secure, and participant-friendly, especially to older adults. Electronic questionnaire systems entail novel advantages. Most advantage is easier to combine the questionnaire with giving feedback about physical activity levels. In addition, they are environmentally friendly (i.e., because paper is not used) and more cost effective, prevent the submission of incomplete forms, and make it possible for researchers to handle big data [15][16][17]. Additionally, electronic questionnaire systems can avoid infection and continue to survey during any pandemic as COVID-19 happened [18]. University students are familiar with using electronic devices. Several studies have found that older adults can effectively benefit from information and communication technologies [19]. Nevertheless, further investigation is necessary because older adults tend to have dry fingers and using such devices and systems may be more cognitively challenging to them than to younger individuals.
The present findings have clinical implications because improving physical activity levels will facilitate the prevention of numerous health problems [20]. Parental perception also effects on children's habit on physical activity [21]. Although the measurement of engagement in physical activity is a challenging enterprise, the E-SQUASH is a keystone that allows us to ascertain population-level physical activity levels. The paper-based one is originally easy to answer, thus E-SQUASH can accelerate acceptance of it. The E-SQUASH is an evidence-based system, but its interface should be enhanced so that it is more appealing to the user. Then, application is more general, easy to use, preferred for majorities in these days, and used on different type of devices (phone, tablet, desktop) and within different systems (Apple / Microsoft). To motivate participants to use the E-SQUASH, further development of this system is necessary (e.g., an application that features on the first page of the loading screen, meaningful functions). Further, participants should be invited and encouraged to assess their physical activity levels regularly.
This study has a few limitations. We did not recruit participants during the winter season, when physical activity levels tend to be lower. We also excluded students who had participated in our pilot studies. Despite these limitations, the reliability and validity of the E-SQUASH were found to be satisfactory. These students present a relatively homogenous group with comparable ability concerning using devices and interpreting questions. Further research is needed to convert this system into an application and translate its contents into different languages. Finally, this scale should be validated across other age groups. The real challenge would be to develop a system that works well with more difficult groups dealing with health literacy and older adults.

Conclusion
In this study, the reliability and validity of the E-SQUASH was examined and established among university students. The E-SQUASH is a highly secure system, and its validation system prevents the submission of incomplete forms. This system allows us to immediately import the data into Excel spreadsheets and send feedback to participants' email addresses. The E-SQUASH can be used to overcome the limitations of the paper-and-pencil version of the SQUASH, and it makes it possible for us to handle big data. Further research is needed to convert this system into an application, translate its contents into different languages, and validate it among other age groups.

Funding source
This study was funded by the Japan Grant-in-Aid for Scientific Research (C, 26463444). The founders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.