In the present study, the short-term reliability of a non-quantitative FFQ (NZAFFQ) was established by comparing two administrations of the FFQ over a two-week period while relative validity was established against a 4DFR.
The results of this study demonstrated that the NZAFFQ yielded good test-retest reliability. The median ICC of 0.69 (range 0.26-0.92) compared favorably to those reported in previous studies in adolescent populations (ICC range 0.01-0.83) [38–40]. The median SCC was 0.71, with all food groups achieving Spearman’s correlations above 0.46. This reliability fell within a range considered good for an FFQ (0.50–0.80) [27, 41] and was similar to the reliability of the CNS02 FFQ, the only previous FFQ designed for New Zealand children . The median test-retest correlation for the CNS02 FFQ for food servings was 0.73, ranging from 0.54 for mixed meat dishes to 0.89 for convenience meals in the 10–14 year age group (n = 42). As indicated by the ICC, we found that foods that were consumed regularly (e.g. milky or chocolate drinks) were recalled with more consistency than foods that were consumed occasionally or variably (e.g. red meat, processed meats and poultry), as observed in previous studies [17, 42, 43]. We acknowledge that the two-week interval between the administration of the NZAFFQs may have led to overestimation of the reliability of this FFQ.
Some variation was seen in the levels of validity between food groups. Among 34 food groups, most food groups (67%) yielded SCCs between 0.32 and 0.70 while 11 food groups produced correlations below 0.30. In particular, the NZAFFQ was less accurate in estimating the group intakes of some vegetables (cruciferous, green leafy, red or yellow vegetables and potatoes), fruits and red meat and processed meat. There are several possible explanations for this observed poor validity for these food groups.
Firstly, within-participant intakes of fruits, vegetables and meats were shown to be highly variable . It is therefore possible that some of the food items consumed occasionally or episodically were not being consumed during the four-day recording period. This is a known limitation when a reference method that covers only a limited period of time is used to validate an FFQ . Notably, 64% of vegetables, 85% of fruits and 64% of meat groups were consumed ‘once a week or less’ by more than two-thirds of the participants. These foods each had a 54% chance of not being consumed during the recording period. Although extending the number of recording days may potentially improve the correlations, this would have caused reporting fatigue and reduced the quality and completion of the food records .
Secondly, we noted that different recording methods and time frame might have attenuated the correlations between the NZAFFQ and the 4DFR. For the NZAFFQ, a particular food eaten both alone and in mixed dishes was recorded in a combined frequency. Conversely for the 4DFR, mixed dishes were recorded then segregated into their component foods and apportioned to their matching food groups. Because of this, foods often consumed as part of mixed dishes such as red or yellow vegetables (e.g. tomatoes and capsicum), red meat and processed meat (e.g. sausage) may either be forgotten (thus underestimated in the NZAFFQ) or miscoded in the food records due to insufficient information [7, 46]. In addition, following recommendations by Cade and colleagues , the FFQtime1 was administered before the 4DFR to eliminate learning effects from completion of a more onerous dietary method. As the NZAFFQ asked about ‘past seven days’ intakes for fruits and vegetables (in Section 3), it assessed diet retrospectively over a slightly different time span from the reference method. These issues of methodological difference between the two dietary methods (i.e. coding decisions and reference period) may have had a negative impact on the correlations of the food group intakes. It is also important to acknowledge that using FFQtime1 in validity analyses could potentially result in an underestimation of validity.
Comparison with other studies
Although validation studies of FFQs in adolescents have been previously reported, most studies validated their FFQs in terms of nutrients or absolute food intakes [47, 48]. Since this study focused on validating the actual responses of intake frequency, precise comparisons of this study with existing studies are not possible. One exception is a study by Vereecken and Maes , which validated a 15-item HBSC FFQ against a 7-day food diary using a similar approach to this study. These same 15 food items were included in the present NZAFFQ. A direct comparison of the two studies revealed that the NZAFFQ showed similar reliability (mean SCC = 0.71) compared to the original HBSC FFQ validation study (mean SCC = 0.67) for the 15 food items. Likewise, the validity of most food items was comparable to those of the original HBSC FFQ (mean SCC = 0.41), achieving an average SCC of 0.38. Vereecken and Maes found high correlations for milk (whole and semi-skimmed) and brown bread (SCC = 0.51-0.65) but low correlations for crisps, diet soft drinks, sweets and fruits (SCC = 0.10–0.34). Similarly, our study showed that validity was good for milk and breakfast cereals (SCC = 0.59-0.70), but less favorable for soft drinks (regular and diet), chips, crisps and sweets (SCC = 0.06-0.30). The most striking observation that emerged from the data comparison was the rather low validity for foods perceived as being ‘less healthy’. We speculate that these foods may be underreported in the NZAFFQ due to social undesirability . This is evident for regular soft drinks where 50% of participants who reported usual consumption of ‘once or less per week’ in the NZAFFQ specified intake on two or more days during their 4-day recording periods.
Strengths and limitations
This study has several strengths and limitations. The main limitation of this study was the small sample size (n = 41), which may have limited the observation of significant correlations in food group intakes. Previous authors have suggested that a sample size of at least 50 is desirable , and ideally a sample of between 100 and 200 should be used, particularly if the FFQ is designed to provide information on nutrient intakes . Although the recruitment deadlines were shifted several times, it was difficult to recruit more participants. A high percent (47%) of those recruited failed to complete the study due to the demanding task of keeping a 4DFR, even though estimated rather than weighed records were used. Our low compliance rate fell within the response range of 48% to 60% typically observed in previous validation studies of adolescents [19, 50, 51]. The sample in our study may comprise participants who were highly motivated; hence generalizability of these findings to other adolescent populations in New Zealand may be limited. On the other hand, this reinforces the clear need to develop a simple FFQ to accurately assess diet among adolescents, including those who are unlikely to provide high quality food records.
In the absence of an absolute gold standard for dietary assessment, we chose an estimated food record as the reference method. This method is advantageous in its ability to capture all food intakes without the reliance of memory and hence has the fewest correlated errors with an FFQ . Additional effort was taken to prepare the Food and Drink Diary as an easy-to-carry booklet to facilitate recording ‘in situ’. Although participants were instructed to conduct recording ‘at the time’ of food and beverage consumption, we acknowledge that this may not be entirely possible. Food underestimation may still occur due to forgetfulness and the limited food knowledge among adolescents [47, 53, 54]. In addition, as the present NZAFFQ also assesses food intakes in the past, the different time frame between the FFQs and the food records may have had an effect on the correlations. Nevertheless, we found similar correlations between this study and other studies with overlapped time frame [38, 39, 44].
The strength of this study lies in the design of a non-quantitative FFQ, which is relatively short and practical for use in time-limited surveys where detailed measures of food intakes are not feasible. We attempted to address the limited motivation and portion size estimation skills among adolescents by omitting the requirement to provide food quantities in the NZAFFQ. As a result, this FFQ was highly repeatable and could be self-completed within 15 minutes. The median SCC of 0.40 obtained from this study was comparable to other validation studies of quantitative FFQs in adolescent populations [38, 44, 50]. Encompassing a wide range of food items from different food groups, this FFQ may offer a viable approach to measure diet diversity and derive dietary patterns or diet quality indices in large studies of adolescents. Whilst the intended use of the NZAFFQ is to assess food group intakes of adolescents in New Zealand, there is a potential for the frequency data to be used alongside other more intensive dietary assessment methods such as the 24-hour diet recall to estimate usual intake . As it was adapted from previously validated questionnaires and pretested rigorously, we believe that the food list sufficiently covers the common foods consumed by New Zealand adolescents and is hence suitable to assess food group intakes in this age group.