The SPAN-ET is a standardized instrument that provides a comprehensive observational assessment of the features, practices, and policies within the school environment that might affect the physical activity and nutrition habits in elementary school children.8 As a result, the SPAN-ET identifies the critical areas within the school that are in need of improvement, specifically the environments that promote healthy nutrition and/or physical activity habits, enhance student learning outcomes, and support core competences of partnerships and learning strategies.8

The SPAN-ET evaluates the school's physical activity and nutrition contexts through 27 items, referred to as “areas of interest” (AI), which are organized within three environmental categories: physical, situational, and policy. Figure 1 visualizes the multidimensional and interactive framework of the SPAN-ET model for quantifying the quality of the school's physical activity and nutrition contexts across the three environmental categories.8

Figure 1.

Schema explaining the multidimensional and interactive framework of the SPAN-ET.

(0.18MB).

In order to evaluate each AI, the SPAN-ET provides a set of criteria that describes evidence-based best practices for each AI. If a school under evaluation fulfils at least 75% of the set criteria, we can conclude that this institution uses the best practices of this given AI. If, in contrast, it only fulfils 25% or less of the criteria, we conclude that this institution has poor practices. Table I in the online Appendix A shows the entire set of evaluations for each AI or environment.

The set of criteria for each AI might cover different types of aspects that require different types of information. Thereby, the use of the SPAN-ET implies the triangulation of different qualitative and quantitative information using direct observation, photography, interviews, document review, among others. During the school visit, two auditors observe physical and situational environments and interview all informants necessary to adequately assess each AI (Table II in the online Appendix A).

Schools are asked to provide two main sources of information: 1) institutional documentation (secondary data), and 2) interviews of key informants listed in Table II in the online Appendix A (primary data). The two auditors independently code each item. Before doing the interviews and physical visits, the auditors review in detail all the documentation, making an initial assessment of each AI. These data sources are complemented with systematic direct observations of physical and situational environments that are conducted before, during, and after the school day.

After the completion of data collection, auditors review the data, item by item, to identify the level of agreement. When a disagreement occurs between the auditors, they re-review all the available evidence as well as their perceptions to come to an agreement. Thus, the final scoring of a given AI assessment is the result of a consensus between the two auditors. Finally, a comprehensive report is delivered to each school to provide them with a set of recommendations.

Cross-cultural adaptation was conducted from September 2018 to August 2019. Based on the Guideline for Process of Cross-cultural Adaptation proposed by Beaton et al.,9 we followed a five-stage methodological process that contains the standard stages suggested by the authors as well as a content validity analysis to obtain a better adapted version for the Spanish context. Figure 2 describes each stage. As a result, we provided a new version of the SPAN-ET for the Spanish context, which we have called SPAN-ET-ES. Both the translation and cross-cultural adaptation were conducted using the English version of the SPAN-ET, provided directly by the original developers.

• 1)

  Stages I and II: forward translation and synthesis

Figure 2.

Global schema illustrating the different phases of cross-cultural adaptation and validation of the SPAN-ET questionnaire to the Spanish context. Stages proposed by Beaton et al.9 for cross-cultural adaptation are indicated as Stages I to V. The content validity analysis was performed by a panel of five experts just before the pilot testing. FT, forward translation; BT, back translation.

(0.16MB).

The first step for the cross-cultural validation was the initial translation from English into Spanish. In order to identify translation discrepancies and the absence of equivalent wording in specific questions as well as other theoretical or cultural differences, two independent translators with proficient English and different backgrounds were selected.9 Our research team compared the translations in order to identify the main agreements and disagreements. In most cases, both translators agreed on the wording and phrasing of the translation. The SPAN-ET developers clarified a few disagreements in the Spanish translation. After their feedback and discussion, the synthesis version of the SPAN-ET-ES was completed

• 2)

  Stage III: back translation

Thereafter, two independent professional English translators with no knowledge of the original SPAN-ET were asked to translate the SPAN-ET-ES back from Spanish to English. These two new versions of the SPAN-ET were again reviewed and refined. The reviewed version was sent to the original developers to analyse the accuracy and proximity of this new version with respect to the original. Based on the final feedback and discussion, a second version of the SPAN-ET-ES was concluded.

• 3)

  Stage IV: context adaptation

Starting with the new version, our research team performed a comprehensive context adaptation in two main areas. In the first area, as the original instrument was based on US legislation, we used the Spanish regulation counterpart. In the second area, we incorporated the Spanish equivalences of all references related to the US education system. All these adaptations required an extensive revision of Spanish legislation, regulations, and guides on different issues and, in some cases, their comparison with their US counterparts. All the literature consulted can be found in the Appendix B online.

After the context validation, SPAN-ET-ES was tested to assess the level of comprehensibility, applicability, and cultural appropriateness in the Spanish context. To do this, we performed a content validity analysis10 where a panel of five experts with different backgrounds were invited to evaluate each AI of the SPAN-ET-ES11,12. Using a four-point scale, each expert was asked to evaluate four dimensions of each AI: relevance, clarity, simplicity, and ambiguity (Table 1).

The panel of experts was made up of five professionals from different fields: a doctor in Physical Activity and Exercise Sciences specialized in motor learning in childhood, a doctor in Food Science and Technology specialized in Food Hygiene, a dietitian and doctor in medicine specialized in collective catering a dietitian-nutritionist responsible for the program for the revision of school menus of the Generalitat de Catalunya, and a dietitian and anthropologist specialized in nutrition in childhood and adolescence.

Based on the expert assessment, we computed the content validity of each individual item (I-CVI), which is the share of experts giving a rating of either 3 or 4 for each dimension:12

Once the I-CVI was calculated for each AI, we estimated an aggregated approach of content validity by environment and by the entire instrument.13 For doing this, we estimated the content validity of the overall scale average (S-CVI/Ave):

We developed an online tool using Open Data Kit 8 (opendatakit.org) in order to facilitate the review by the experts. Once collected, the information was analysed using Stata 14 (StataCorp LLC, Texas, USA).

4) Stage V: pilot testing

Once the new version of the SPAN-ET-ES was obtained from the content validity analysis, we performed a pilot test. This was carried out in Santa Coloma de Gramenet, a city close to Barcelona, in Catalonia, Spain. According to the latest national registry in 2019,14 Santa Coloma was estimated to have 119,215 inhabitants and have a per capita gross income of €22,539 per capita. In total, we visited five public schools (four public schools and one sponsored by a public voucher system). Two well-trained researchers performed the evaluations, following the guidelines provided by the original developers.8

After the forward and back translations were done, a revised version was discussed with the original developers who agreed on the new wording and re-rephrasing. Once our research team had the certainty that the SPAN-ET-ES preserved the main dimensions of the original instrument, the context adaptation was performed in order to make the SPAN-ET-ES compatible with the legislation and other particularities of the Spanish context. In this process we added three items and deleted one. First, in order to gather information about the participation in various physical activity programs,15–17 we included a new item in AI-12 (Table 2). Additionally, we added two items in AI 16 and AI 27 that refer to the use of new technologies and their potential effects on the physical activity (AI-16) and nutrition (AI-27) of children. Second, as the item related to the presence of a health and nutrition educator did not apply to the Spanish context, we deleted item H in AI-27.

Furthermore, the new version underwent small changes in format, phrasing, and content, in order to simplify its application in Spain. In total, we made 38 changes throughout the instrument; no changes were made to the answer scale (Table 2). A detailed report on the changes are available upon request. An example of an extract of preliminary instructions and some AI translated to Spanish can be found in the Appendix C and D online.

After the translation and cross-cultural adaptation process, as well as the feedback from and discussion with the original developers, a content validity analysis was performed. Overall, the expert panel considered that, on average, items included in the revised version of the SPAN-ET-ES were found relevant (S-CVI/Ave=0.96), clear (S-CVI/Ave=0.96), simple (S-CVI/Ave=0.98), and non-ambiguous (S-CVI/Ave=0.98). When looking at the environmental level, we found that whereas the AI contained within the physical activity environment was considered very relevant, we had to implement minor changes to four AIs (relevance - CVI <0.8) within the nutrition environment, as the experts considered them not applicable to the Spanish context. Table 3 summarizes the content analysis results for the SPAN-ET-ES.

Table 4 presents the results of the SPAN-ET-ES pilot testing phase for five Spanish schools located in Santa Coloma de Gramenet. Following the SPAN-ET-ES field work guidelines, the two trained auditors collected the documentation needed to assess the 27 AIs for each school. In all cases, they confirmed the existence of each data source as well as its accessibility. Besides small changes for the field work guidelines, the SPAN-ET-ES is based on information that is available in the Spanish context, which allows for a proper assessment of each AI. The auditors also indicated that the items included in the different AIs were easily understandable. On average, contacting schools, collecting, organizing, assessing the agreement and analyzing all the required information (primary and secondary data) for the SPAN-ET-ES took around 15hours per school.

Overall, we found a 93% of agreement in the item scores between the two auditors when evaluating each AI. We only found two main disagreements in AI 1 and 23, which were easily resolved after reviewing complementary data sources. This high level of agreement between auditors was due to previous training in the administration of SPAN-ET-ES and the homogeneity achieved in the evaluation criteria. Table 4 presents the results of the assessment after applying the SPAN-ET-ES to the pilot testing sample.