Skip to main content
Download PDF

Comparison of the NHANES dietary screener questionnaire to the Automated Self-Administered 24-Hour Recall for Children in the Healthy Communities Study

Nutrition Journal volume 17, Article number: 111 (2018) Cite this article

Abstract

Background

A dietary screener questionnaire (DSQ) was used to assess dietary outcomes among children in the Healthy Communities Study (HCS), a study of the relationships between programs and policies to prevent child obesity and child diet, physical activity and weight outcomes.

Methods

To compare dietary intake estimates derived from the DSQ against those from the Automated Self-Administered 24-Hour Recalls for Children (ASA24-Kids) among children, a measurement error model, using structural equation modelling, was utilized to estimate slopes, deattenuated correlation coefficients, and attenuation factors by age and sex, ethnicity, and BMI status.

Participants/setting

A randomly selected sub-sample of HCS participants aged 4–15 years in 130 communities throughout the U.S. who completed the DSQ and up to two ASA24-Kids recalls (n = 656;13% of HCS participants).

Results

For most nutrient/foods examined, the DSQ yielded larger mean intake estimates than the ASA24-Kids, and agreement between the two measures varied by food/nutrient, age and sex, ethnicity, and BMI category. Deattenuated correlation coefficients of 0.4 or greater were observed for added sugars from SSBs (0.54), fruits and vegetables (0.40), and dairy foods (0.50). Lower deattenuated correlation coefficients were seen for total added sugars (0.37), whole grains (0.34), and fiber (0.34). Attenuation factors were most severe for total added sugars intake among overweight children, and for several other dietary outcomes among children aged 9–11 years.

Conclusions

The DSQ was found to be a tool with acceptable agreement with the ASA24-Kids for measuring multiple dietary outcomes of interest in the HCS, although there may be potential due to measurement error to underestimate results (bias towards the null). In future studies, measurement error modelling and regression calibration may be possible solutions to correct for bias due to measurement error in most food/nutrient intake estimates from the DSQ when used among children.

Peer Review reports

Background

Assessing dietary intakes can be done in multiple ways, but each method has advantages and disadvantages [1]. For example, the use of recovery biomarkers allows intakes to be measured without relying on self-report but limits outcomes to those for which there are known recovery biomarkers. In contrast, assessing diet through methods reliant on self-report such as 24-hour recalls, food frequency questionnaires, and dietary screeners allow for the examination of a greater number of food groups and nutrients of interest, but also contain varying degrees of measurement error due to factors such as recall error and social desirability bias. Measurement error not only varies by dietary assessment tool, but also by dietary intake and study population.

In addition to the expected amount of measurement error, other concerns such as feasibility, cost, and participant burden play a role in determining the most appropriate tool to use in a study. Researchers must make tradeoffs to obtain the most accurate measures for the study population and resources available, and accept the type and magnitude of measurement error associated with the method of choice. For example, 24-hour recalls tend to have less measurement error than food frequency questionnaires and dietary screeners, but are more resource-intensive and require more from participants to complete. Regardless of the dietary assessment method used, it is crucial for researchers to be aware of the nature, direction and magnitude of measurement error so as to interpret results appropriately.

This paper aims to describe the measurement error present in intake estimates of added sugars, added sugars from sugar-sweetened beverages (SSBs), fruits and vegetables, whole grains, fiber, and dairy derived from the NHANES dietary screener questionnaire (DSQ) among children 4–15 years of age. In this study, we have evaluated agreement with intake estimates derived from the Automated Self-Administered 24-Hour Recall for Children (ASA24-Kids), using data from the Healthy Communities Study (HCS) (see Additional file 1).

The HCS aimed to assess associations between community programs and policies to prevent childhood obesity and outcomes including nutrition and physical activity behaviors and BMI, in a large, diverse sample of children from 130 communities across the United States. To accomplish this, a measure of dietary intake that could easily be administered by a large number of field data collectors for a large sample of children across the country was required. Dietary assessments were made on the entire HCS sample of over 5000 children, using a modified version of the DSQ [2] that was developed by the National Cancer Institute (NCI) and used with children and adults in the National Health and Nutrition Examination Survey (NHANES) [3].

This short food frequency questionnaire-based method was selected for use in the HCS because it assessed dietary factors related to obesity risk and because it was feasible to administer to a large study population in a short time by field interviewers who had no special nutrition training [2, 4]. It also allowed for comparison to national data using the same method and enabled the calculation of quantitative intake estimates from reported frequencies [2]. Previously, DSQ intake estimates for foods such as fruits and vegetables have been shown to correlate well with estimates from interviewer-administered 24-hour recalls among adults [5]. However, as a brief dietary screener, the DSQ relies on complex cognitive tasks to calculate usual frequency of intake of various foods and is thus more challenging and potentially less accurate when used with children [1].

While the DSQ was the most feasible dietary assessment tool to be used in the HCS, information was lacking on the measurement error associated with intake estimates in relation to a more accurate dietary assessment method, especially among children. Dietary assessments on a subset of the HCS sample were therefore made using a 24-hour dietary recall to describe and quantify the measurement error present in DSQ intake estimates. The ASA24-Kids recall was selected as the reference method with which to compare the DSQ in the HCS. The ASA24-Kids was the most feasible, affordable comparison method because it is a standardized, automated online interview and does not require specialist trained nutrition researchers for data collection. While past studies have shown that some children at younger ages find it difficult to accurately report their own intakes using the ASA24-Kids, other studies have shown that among adults, the ASA24 yields results comparable to that of interviewer-administered 24-hour recalls [6,7,8]. Two recalls were collected from each of the sub sample respondents to adjust for in-person variation [2].

Methods

Study population

Sampling, measures, and methods used in the HCS have been described in detail elsewhere [2, 9,10,11,12,13,14]. In brief, HCS study participants (n = 5138) were recruited from a large, stratified sample of 130 high school catchments areas, or “communities”, across the U.S. [9]. By design, the HCS oversampled lower income and minority communities and was not a representative sample of US children. Within each community, up to two elementary and two middle schools, were selected as the recruitment base for household/child participants [9]. Up to 81 children in grades K-8 were selected per community to complete the standard set of assessments, including the DSQ [9]. Letters requesting participation and study interest forms were sent home and those households returning an interest form were scheduled for home visits [9]. A random subsample of about 14% of the households consenting to participate in the HCS were selected for the enhanced protocol which included more detailed measures of dietary intake, such as the ASA24-Kids recall, in addition to the standard set of assessments. The study population for this paper comprises the sub-sample of the HCS participants with a completed DSQ who completed at least one ASA24-Kids recall.

The research protocol was approved by OMB and the Battelle Memorial Institute IRB. Parents provided written informed consent for child participation [2].

Dietary assessment measures

The 27-item DSQ was adapted from the version administered to those aged 2–69 years of age in NHANES 2009–2010 [2]. An expert committee was convened by the lead investigators to advise on dietary assessment methods. Modifications of the DSQ for the HCS were minimal. In an attempt to reduce time required for home visits, DSQ items were prioritized and variables selected that have shown moderate to strong evidence of a relationship to obesity, including sugar sweetened beverages, fruits and vegetables among others. Questions about meat intake were excluded and a question about the frequency of chip and cracker consumption was added, although the latter was used only to assess the frequency of intake of discretionary foods, and, as such, was not used for quantitative intake estimates analyzed in this paper. [2]. While fast food intake was not assessed by the DSQ, a question about frequency of intake was included. Participants or their proxy reported the frequency of consuming selected foods and food groups over the past 30 days in number of times per day, week, or month. Quantitative estimates for intakes of each of the food groups and nutrients were calculated from the DSQ using publically available scoring algorithms derived from NHANES 2009–2010 data. To calculate scores, the reported frequencies of consumption were standardized to daily frequencies, to which age and sex specific median quantities of selected food groups were applied, and calibration equations were computed, derived from regressing DSQ with 24-hour recalls [15].

The ASA24-Kids is a 24-hour recall web-based program which was developed by the NCI for use with children and is available in English and Spanish [16, 17]. It is based on the Automated Multiple Pass Method used in interviewer administered recalls for NHANES, and adapted for online administration [18,19,20]. The ASA24-Kids consists of an ‘interview’ conducted by an animated character that queries the respondent in stages about all foods and beverages consumed in the previous 24 hours. During each stage or “pass”, the respondent is probed for forgotten foods and beverages; time and eating occasion; and amounts consumed using photographs of graduated serving sizes [21]. The ASA24-Kids (2012 and 2014 versions) utilized the Food and Nutrient Database for Dietary Studies 4.1 as the basis for calculating intakes of nutrients, and food groups. “MyPyramid” serving equivalents were calculated within the ASA24-Kids program using the MyPyramid Equivalents Database 2.0 [16].

Data collection

Dietary data was collected during home visits, with the parent/guardian and child present. The DSQ and ASA24-Kids recalls were collected in English or Spanish, according to participant preference [2]. For both dietary assessments, the primary respondent was selected based on the child’s age as follows: the parent/adult proxy with child assistance for children 4–8 years; the child with parent/adult proxy assistance for children 9–11 years; the child with parent/adult proxy assistance only if needed for children 12–15 years [2].

Trained field data collectors (FDCs) administered the DSQ, giving a neutral, scripted introduction and directions to the parent or adult caregiver and child concerning who should respond and how to obtain clarification from parents when the child was the primary respondent [2]. The FDC entered respondents’ answers into a pre-programmed electronic tablet [2]. Out of range responses prompted a neutral scripted probe by the interviewer to verify or correct responses [2].

Respondents in the enhanced protocol sub-sample completed the ASA24-Kids on an HCS tablet computer supplied by FDCs during home visits [2]. Study participants were asked to complete two recalls. The first was collected on the same day as the interviewer administration of the DSQ, and the second recall occurred at a follow up home visit approximately 7–10 days later. The FDCs’ role in administering the ASA24-Kids was restricted to logging onto the program, troubleshooting, and providing minimal assistance [2]. To avoid potential reactivity, respondents were unaware that they would be completing 24-hour recalls prior to the home visit [2].

Participant sex, age, race, and ethnicity were determined by survey responses and BMI was calculated from measured height and weight recorded by trained FDCs.

Training and quality assurance

FDCs were trained to administer the DSQ and oversee completion of the ASA24-Kids [2]. To assure adherence to the protocol, field supervisors and senior research staff conducted quality assurance checks of FDCs on selected home visits [2].

Data quality was monitored monthly for missing data, irregularities in administration time, extreme values at the low and high ends of the distribution, and percentage of ‘don’t know’ and ‘refusal’ responses for the DSQ [2]. Data obtained from the ASA24-Kids was also reviewed monthly, for response rates, incompletes, documentation about reasons for missing or incomplete recalls, comparison with age and sex specific values from NHANES 2010, and review of outliers [2]. FDCs with incomplete or other data issues participated in refresher training and were monitored subsequently [2].

Statistical analyses

Box-Cox transformations were used to find the best transformation to achieve normal distributions for each food group or nutrient intake on the completed recalls (n = 656). Transformed data were reviewed, and extreme outliers were excluded from analysis for those particular nutrients or food groups. Extreme outliers were defined as those with nutrients or food groups at least two times the interquartile range lower or higher than the lower and upper bounds of the interquartile range (25th and 75th percentiles). For each intake of interest, full records were excluded if intake estimates from both recalls were missing or identified as extreme outliers, or if the DSQ intake estimate was missing (n = 7 for total added sugars; n = 4 for added sugars from SSBs; n = 9 for fruits and vegetables; n = 4 for whole grains; n = 18 for fiber; and n = 3 for dairy). All statistical analyses were conducted using SAS (version 9.4, 2013, SAS Institute Inc.).

Mean differences between the DSQ and ASA24-Kids intake estimates (mean of two recall days if available) were assessed by paired t-tests. A measurement error model using structural equation modelling, similar to that described by Freedman et al. [22], was applied to estimate relationships between estimated usual intake, as derived from the recalls, and the DSQ intake estimates. ASA24-Kids intake estimates were modelled as true usual intake plus random error. The modelling removed intra-individual variation to estimate true usual intake, with the assumption that the recalls were unbiased at the individual level and only contained intra-individual random error [23]. DSQ intakes were then modelled as a function of the estimated true usual intake plus error, both random and systematic. This model was fit by age group and sex, ethnicity, and BMI category to the DSQ and ASA24-Kids recall data and yielded estimates of measurement error characteristics of the DSQ. Models were not adjusted for covariates because we sought to assess the direct relationship between these two measurement methods and how these measures relate to a theoretical estimate of true intake using a structural equation modelling approach; the addition of covariates to the model would have distorted these relationships. The slope (β1) in the regression of reported intake from the DSQ on estimated true usual intake, using the recalls, as well as the deattenuated correlation coefficient between reported and estimated usual intakes (R), and the attenuation factor (λ = R2/ β1) were then estimated using full information maximum likelihood estimation.

If the DSQ were unbiased, the regression of reported intake on true usual intake would have a slope of one and an intercept of zero [23]. Deattenuated correlations assess the agreement between reported intake and estimated usual intake, adjusted for intra-individual variation. The attenuation factor indicates the degree to which the observed log relative risk between the observed exposure of a dietary intake derived from the DSQ and an outcome of interest is biased due to measurement error [23]. Attenuation factors typically range from 0 to 1, indicating bias towards the null (underestimation of true association), with values closer to zero indicating more severe attenuation [23]. Attenuation factors greater than one indicate bias away from the null (overestimation of true association) [23].

Results

Of the 5138 participants in the HCS, 5123 completed a DSQ, and 712 (14%) were randomly selected for the enhanced protocol. Of those, 656 participants (92.1%) completed one recall and 551 participants (77.4%) completed both recalls. The sample size for the analysis of each intake variable ranged from 638 to 653. The sample comprised approximately 50% females (Table 1). A higher proportion of participants were 4–8 years old, than 9–11 year olds, or 12–15 year olds. Over half of the participants were white, non-Hispanic, were of normal weight, and had a family income of less than $35,000 per year. Approximately 40% had parents who had attained a high school education or less. Overall there were no statistically significant differences in sociodemographic characteristics between the enhanced protocol subsample and the entire HCS sample.

Table 1 Comparison of sociodemographics of Enhanced Protocol sub-sample and Total sample, Healthy Communities Study, USA, 2013–2015
Full size table

Relative to the ASA24-Kids recalls, the mean intake estimates from the DSQ for all nutrients/foods were significantly higher for the total sample and for nearly all subgroups (Table 2). Differences in means between the two measures were not significantly different for girls aged 9–11 years on added sugars from SSBs, for girls aged 9–15 years on fruits and vegetables, and for girls aged 4–8 years for whole grains.

Table 2 Differences between children’s dietary intakes measured via DSQ and ASA24-Kids, Healthy Communities Study, USA, 2013–2015
Full size table

Deattenuated correlation coefficients of 0.4 or above were observed for the overall sample on three of the six nutrient/food intakes measured by DSQ and ASA24-Kids; the highest deattenuated correlation was observed for sugars from SSBs (0.54), followed by dairy (0.50), and fruits and vegetables (0.40) (Table 3). Values for the other three intake variables ranged from 0.34 for whole grains and fiber to 0.37 for total added sugar. Deattenuated correlations varied among the 14 sub-groups as defined by children’s age and sex (6 groups), race (3 groups) ethnicity (2 groups), or BMI (3 groups). The greatest number of deattenuated correlation coefficients at or above 0.4 among sub-groups were seen for added sugars from SSBs (11), dairy (11), and fruits and vegetables (9). Lowest deattenuated correlations (0.15 or below) were seen among children aged 9–11 years for several dietary intakes including total added sugars (both sexes), whole grains (girls), and dairy (boys). African American children tended to have lower deattenuated correlations as compared to White children or children of other races. Hispanic children had notably higher deattenuated correlations than non-Hispanic children for intakes of total added sugars and sugars from SSBs, but substantially lower deattenuated correlations for intakes of fruits and vegetables, and whole grains. Compared to normal weight children, deattenuated correlations were higher among overweight or obese children for intakes of dairy, added sugars from SSBs, and whole grains; and lower for intake of fruits and vegetables.

Table 3 Slopesa, deattenuated correlationsb, and attenuation factorsc of DSQ dietary intakes, Healthy Communities Study, USA, 2013–2015
Full size table

Attenuation was greater than 0.2 for more than 90% of the age and sex-, race-, ethnicity-, and BMI category-specific food group/nutrient intakes (Table 3). Attenuation factors for intake of sugars from SSBs were higher, while those for fruit and vegetable intake were lower, among children with higher BMI. For all intake estimates, there was a greater degree of attenuation among African American children compared to White children and children of other races. For all intake estimates other than those for total added sugars and fiber, there was a greater degree of attenuation for Hispanics than for non-Hispanics. Attenuation was most severe (λ = 0.01–0.18) for added sugars among boys and girls 9–11 years and among overweight children; whole grain intake among females 9–11 years; and dairy intake among boys 9–11 years.

Intake patterns

Although most of the intake estimates derived from the DSQ were significantly different from those from ASA24-Kids recalls, some patterns of intake among subgroups were similar using both methods. For total added sugars, fiber, and dairy foods, males consumed more than females on both measures (Table 2). While older males consumed more added sugars than younger males in both the DSQ and ASA24-Kids, this pattern was not seen among females. For added sugars from SSBs, males consumed more than females, and older children consumed more than younger ones on both measures. Intake patterns by sex and age group for fruits and vegetables, and for whole grains were inconsistent between the two methods. In addition, both instruments showed that, on average, obese children had greater intakes of total added sugars and added sugars from SSBs than overweight or normal weight children, and, that non-Hispanic children had higher consumption of these than Hispanic children. Normal weight children had greater intake of fruits and vegetables than overweight or obese children as measured by both instruments.

Discussion

This is the first study we are aware of which evaluates the performance of the DSQ among children. Unlike other dietary screeners which only yield consumption frequency, the DSQ yields quantified intake estimates based on reported frequencies of consumption. For three dietary outcomes examined in the HCS, added sugars from SSBs; fruits and vegetables; and dairy foods, deattenuated correlations were 0.4 or greater. This is an important finding for two reasons: i) many community policies and programs for obesity prevention target these foods, and ii) the level of agreement between the two dietary assessment methods suggests that the DSQ appears to be an acceptable tool for measuring these dietary outcomes for purposes of the analysis of the association between community programs and policies and dietary outcomes in studies such as the HCS and potentially for similar study populations and research questions.

The lower deattenuated correlations observed for total added sugars, compared with those for added sugars from SSBs may reflect the greater ease with which respondents can report beverage frequency than they can in reporting a wider range of foods they consume containing added sugars. Some research also suggests that foods consumed very frequently and foods consumed rarely, characteristic of many beverages, are more consistently reported than are foods that are consumed moderately frequently [24].

The variable levels of agreement seen between the two assessment methods for different age groups, in part, may be a function of the accuracy with which the selected primary respondents in the study are able to report dietary intakes. The higher correlations, suggestive of less measurement error, observed among children aged 12–15 years on several dietary outcomes may indicate that these youth were cognitively mature enough to serve as their own primary respondents. Reasonable agreement between both methods was also seen among children aged 4–8 years for several dietary outcomes, suggesting that caregiver adults were an adequate proxy reporter for this group. The higher degree of measurement error among children aged 9–11 years, especially males, as indicated by lower attenuation factors and deattenuated correlations for several dietary outcomes, suggests that they may have had difficulty in reporting their food and beverage intakes accurately, even with the assistance of a parent, findings which are consistent with those of numerous other studies [6, 25, 26].

There is substantial evidence that higher BMI is associated with higher measurement error in reporting diet, particularly underreporting of total energy intake [26,27,28,29,30]. Thus, it might be expected that overweight or obese children would underreport sugars from SSBs and other foods high in energy. However, relatively little is known about differential bias in reporting particular food groups among adults or children by BMI status, or the tendency for more biased reporting by overweight or obese respondents using different dietary assessment methods. Surprisingly, our study found that intake estimates of sugars from SSBs were less attenuated and had greater deattenuated correlation coefficients among overweight and obese children, indicating closer agreement between the two methods for overweight or obese children than for other children. By contrast, the degree of attenuation/measurement error in estimates of fruit and vegetable intake was higher in successively higher BMI categories. Further research is needed to identify the effect of BMI status, and hence misreporting bias on estimated intakes of selected food groups (beyond its effect on the accuracy of reporting energy intake).

We found that race and ethnicity were associated with the level of agreement between the two dietary assessment methods. Hispanic and African American children showed a higher degree of measurement error than children of other races and ethnicities. Previous validity studies have shown mixed results regarding performance of various dietary assessment methods among Hispanics compared with non-Hispanics; some studies found no significant differences in measures of agreement [23, 31,32,33], while others have found differences [1, 25, 34]. While efforts were made to design the DSQ to accommodate various Hispanic and Asian subgroups, it may not have included sufficient number or variety of important ethnic-specific foods. Alternatively, the scoring algorithms did not take into account the relative portion sizes of food items within a food group typical of Hispanic groups. Tucker et al. found that among Hispanic (Puerto Rican) adults, there was a tendency to consume larger portions of particular fruits and smaller portions of particular vegetables than the general U.S. population, comparing estimates from the 1982–1984 Hispanic Health and Nutrition Examination Survey with NHANES [34]. It is also possible that Hispanic respondents may have had more difficulty navigating the online ASA24-Kids recall due to language difficulty, or less familiarity with computers than adult and child respondents of other ethnicities. Although the Spanish language version of ASA24-Kids was offered, it was not often selected by respondents. Similar to previous studies comparing methods between African Americans and other races, we found that there was more measurement error in intake estimates among African Americans [35,36,37]. This may be due to similar factors to those we suggested for Hispanics (differences in foods consumed, and in portions of particular types of foods).

A limitation of our study was the uncertain validity of our reference method. For practical reasons, the ASA24-Kids was the best available method for collecting detailed dietary intake data in the HCS against which to compare data from the DSQ. The assumptions that 24-hour recalls provide unbiased estimates of usual food intake, and, further, that the ASA24-Kids, as a self-administered method, provides no additional bias over a standard interviewer-administered 24-hour recall are relatively unexplored assumptions. The use of an imperfect reference method, the ASA24-Kids recalls, may have led to inflated estimates of deattenuated correlation coefficients if the error within recall intake estimates was correlated with the error present in DSQ intake estimates.

With regards to the first assumption, it is possible that the ASA24-Kids recall method underestimated dietary intakes. We found that almost all of the intake estimates from the HCS recalls were significantly lower than those from the DSQ. In addition, the majority of mean nutrient/food intakes from the ASA24-Kids were lower than the NHANES 2009–2010 age and sex specific means, but most means were within 20% (see Additional file 2). It is widely known that 24-hour recalls underestimate energy intake [27, 38,39,40,41], but the presence and amount of reporting bias for food groups and many nutrients are currently unknown, in part due to the lack of a true gold standard. Other possible reasons for the lower estimates of dietary intakes from the ASA24-Kids relate to differences in the time periods covered by the two methods and the different types of memory needed for each method. Responses to the DSQ rely on participants’ generic memory, which provides general information about usual intake over the reference time period. By contrast, 24-hour recalls rely on specific shorter term memory to provide detailed information about particular intakes and behaviors [42].

The second assumption, that the ASA24-Kids as a self-administered method produces no additional bias over an interviewer-administered 24-hour recall, may be of concern. Unlike dietary interviewers, study children are not trained to navigate the software and identify foods that best match their intakes. One study showed that when children between the ages of 9 and 11 years reported their own dietary intakes over the past 24 hours, the ASA24-Kids was less accurate than the interviewer-administered AMPM [6]. However, other studies have shown that among computer-literate English-speaking adults, the ASA24 performed similarly to the AMPM [7, 8]. In this study, parent or adult proxies were available to assist children aged 9–11 years, but were instructed by the FDC to provide assistance only if asked by the child. More consistent and active assistance from parents about the foods eaten at home and portion sizes may be needed to improve accuracy. However, the lack of an in-person interviewer in the ASA24-Kids may also confer some advantages such as a possible reduction in social desirability bias. Smith et al. found that, among fourth-grade children, reporting accuracy decreased with social desirability when 24-hour recalls were administered in person by interviewers [43].

There are many methods used to compare performance of and agreement between dietary assessment methods. A key strength of this study was the use of measurement error modelling utilizing structural equation modeling to remove the within-person variation in food/nutrient intakes from recalls to estimate usual intake. This method made the estimates derived from the DSQ, designed to measure usual intake over a month, and those derived from up to two days of 24-hour recalls more comparable to each other. From this modelling approach, we then derived deattenuated correlations to examine the agreement between reported intake and estimated usual intake as they are a more useful and rigorous assessment than the traditional Spearman or Pearson correlation coefficients and kappa statistics, which do not adjust for intra-individual variation. The derived attenuation factors, which express the amount of potential bias due to measurement error in an exposure-outcome relationship, were also useful in assessing which dietary outcome variables would be candidates for correction of measurement error. We observed a wide range of attenuation factors, λ = 0.01–1.02. While it is true that as the attenuation factor gets further from 1 it becomes increasingly important to adjust measures of association between observed dietary intakes and outcomes of interest for measurement error, as the attenuation factor gets closer to zero, adjusted estimates become overinflated, making adjusting for error less desirable. Thus, among children 9–11 years old, intakes of total added sugars (among both sexes), whole grains (among girls), and dairy (among boys) may not be good candidates for measurement error correction. Similarly, it may not be advisable to correct for measurement error for intakes of total added sugars among overweight children, or for total added sugars, fruits and vegetables, and fiber among African American children.

Conclusions

The importance of understanding the extent of measurement error in the dietary intake estimates used in the HCS, as measured by the DSQ, is fundamental to interpreting the results of associations between community policies and programs and child dietary outcomes. This study found higher deattenuated correlations and attenuation factors, overall, for intakes estimates of sugars from SSBs, dairy, and fruits and vegetables (foods often targeted in obesity prevention community policies and programs). Based on comparison with intake estimates derived from up to two ASA24-Kids recalls, our findings suggest that the DSQ may be an acceptable tool to measure key obesity-related dietary behaviors of interest among children for the purposes of studies like the HCS. However, further study exploring the assumptions made in using the ASA24-Kids as a reference method is warranted. In addition, our results suggest that measurement error modelling and regression calibration are viable methods for future studies to correct for measurement error within the DSQ for most of the food groups/nutrients among most of the strata examined in this study, which would otherwise lead to underestimating the magnitude of associations between nutrition exposures and outcomes of interest.

Abbreviations

ASA24:

Automated Self-Administered 24-hour Recall

ASA24-Kids:

Automated Self-Administered 24-Hour Recalls for Children

DSQ:

Dietary screener questionnaire

FDC:

Field data collector

HCS:

Healthy Communities Study

NCI:

National Cancer Institute

NHANES:

National Health and Nutrition Examination Survey

SSBs:

Sugar-sweetened beverages

References

  1. Thompson FE, Subar AF. Dietary assessment methodology. In: Coulston AM, rock CL, Monsen ER, editors. Nutrition in the Prevention and Treatment of Disease, Elsevier; 2013. p. 5–46.

    Chapter  Google Scholar 

  2. Ritchie LD, Wakimoto P, Woodward-Lopez G, Thompson FE, Loria CM, Wilson DK, et al. The healthy communities study nutrition assessments: child diet and the school nutrition environment. Am J Prev Med. 2015 Oct;49(4):647–52. https://doi.org/10.1016/j.amepre.2015.06.016.

    Article  PubMed  PubMed Central  Google Scholar 

  3. National Cancer Institute (NCI). Dietary screener questionnaire in the NHANES 2009–10. http://epi.grants.cancer.gov/nhanes/dietscreen/. Updated October 15, 2015. Accessed July 12, 2016.

  4. Woodward-Lopez G, Ritchie LD, Gerstein D, Crawford PB. Obesity: dietary and developmental influences. Boca Raton: CRC Press; 2006. https://doi.org/10.1201/9781420008920

    Book  Google Scholar 

  5. Dietary Screener Questionnaire (DSQ) in the NHANES 2009–10: Dietary Factors, Food Items Asked, and Testing Status for DSQ. http://epi.grants.cancer.gov/nhanes/dietscreen/evaluation.html. Updated January 4, 2016. Accessed 12 July 2016.

  6. Diep CS, Hingle M, Chen TA, Dadabhoy HR, Beltran A, Baranowski J, et al. The automated self-administered 24-hour dietary recall for children, 2012 version, for youth aged 9 to 11 years: a validation study. J Acad Nutr Diet 2015 Oct;115(10):1591–1598. doi: https://doi.org/10.1016/j.jand.2015.02.021. Epub 2015 Apr 14.

    Article  Google Scholar 

  7. Kirkpatrick SI, Subar AF, Douglass D, Zimmerman TP, Thompson FE, Kahle LL, et al. Performance of the automated self-administered 24-hour recall relative to a measure of true intakes and to an interviewer-administered 24-h recall. Am J Clin Nutr 2014 Jul;100(1):233–240. doi: https://doi.org/10.3945/ajcn.114.083238. Epub 2014 Apr 30.

    Article  CAS  Google Scholar 

  8. Thompson FE, Dixit-Joshi S, Potischman N, Dodd KW, Kirkpatrick SI, Kushi LH, et al. Comparison of Interviewer-Administered and Automated Self-Administered 24-Hour Dietary Recalls in 3 Diverse Integrated Health Systems. Am. J. Epidemiol. doi:https://doi.org/10.1093/aje/kwu467 Epub May 10, 2015.

  9. John LV, Gregoriou M, Pate RR, Fawcett SB, Crawford PB, Strauss WJ, et al. Operational implementation of the healthy communities study: how communities shape Children’s health. Am J Prev Med. 2015 Oct;49(4):631–5. https://doi.org/10.1016/j.amepre.2015.06.019.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Fawcett SB, Collie-Akers V, Schultz J, Kelley M. Measuring community programs and policies and their intensity in the healthy communities study. Am J Prev Med. 2015 Oct;49(4):636–41. https://doi.org/10.1016/j.amepre.2015.06.027.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Sroka CJ, McIver KL, Sagatov RD, Arteaga SS, Frongillo EA. Weight status measures collected in the healthy communities study: protocols and analyses. Am J Prev Med. 2015 Oct;49(4):642–6. https://doi.org/10.1016/j.amepre.2015.07.001.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Strauss WJ, Sroka CJ, Frongillo EA, Arteaga SS, Loria CM, Leifer ES, et al. Statistical design features of the healthy communities study. Am J Prev Med. 2015 Oct;49(4):624–30. https://doi.org/10.1016/j.amepre.2015.06.021.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Pate RR, McIver K, Colabianchi N, Richard TP, Jared RP, Dianna CD, et al. Physical activity measures in the healthy communities study. Am J Prev Med. 2015 Oct;49(4):653–9. https://doi.org/10.1016/j.amepre.2015.06.020.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Arteaga S, Loria C, Crawford P, Fawcett SB, Fishbein HA, Gregoriou M, et al. The healthy communities study: its rationale, aims, and approach. Am J Prev Med. 2015 Oct;49(4):615–23. https://doi.org/10.1016/j.amepre.2015.06.029.

    Article  PubMed  Google Scholar 

  15. National Cancer Institute (NCI). Dietary screener questionnaire in the NHANES 2009–10: Data Processing & Score Procedures. http://epi.grants.cancer.gov/nhanes/dietscreen/scoring/. Updated October 15, 2015. Accessed July 12, 2016.

  16. National Cancer Institute (NCI). Instructions for the Researcher Web site for: ASA24™-2014, ASA24Tm-Kids-2014, and ASA24™-Canada-2014. https://asa24.nci.nih.gov/researcherSite/assets-r/Researcher_Instructions.pdf#nameddest=DataAnalysisSection8. Updated March 2016. Accessed July 12, 2016.

  17. Baranowski T, Islam N, Douglass D, Dadabhoy H, Beltran A, Baranowski J, et al. Food intake recording software system, version 4 (FIRSSt4): a self-completed 24-h dietary recall for children. J Hum Nutr Diet 2014 Jan;27 Suppl 1:66–71. doi: https://doi.org/10.1111/j.1365-277X.2012.01251.x. Epub 2012 May 23.

    Article  Google Scholar 

  18. United States Department of Agriculture (USDA). AMPM - USDA Automated Multiple-Pass Method. http://www.ars.usda.gov/Services/docs.htm?docid=7710#what. Updated October 2, 2014. Accessed August 19, 2016.

  19. National Cancer Institute (NCI). ASA24™-Kids 2014 and ASA24™-Kids 2012. http://epi.grants.cancer.gov/asa24/respondent/childrens.html?&url=/asa24/respondent/childrens.html. Updated March 29, 2016. Accessed July 12, 2016.

  20. National Cancer Institute (NCI). ASA24™. http://epi.grants.cancer.gov/asa24/. Updated July 6, 2016. Accessed July 12, 2016.

  21. Conway JM, Ingwersen LA, Vinyard BT, Moshfegh AJ. Effectiveness of the US Department of Agriculture 5-step multiple-pass method in assessing food intake in obese and nonobese women. Am J Clin Nutr. 2003 May;77(5):1171–8.

    Article  CAS  Google Scholar 

  22. Freedman LS, Carroll RJ, Wax Y. Estimating the relation between dietary intake obtained from a food frequency questionnaire and true average intake. Am J Epidemiol. 1991;134:310–20.

    Article  CAS  Google Scholar 

  23. Thompson FE, Midthune D, Subar AF, Kahle LL, Schatzkin A, Kipnis V. Performance of a short tool to assess dietary intakes of fruits and vegetables, percentage energy from fat and fibre. Public Health Nutr. 2004 Dec;7(8):1097–105.

    Article  Google Scholar 

  24. Thompson FE, Lamphiear DE, Metzner HL, Hawthorne VM, Oh MS. Reproducibility of reports of frequency of food use in the Tecumseh diet methodology study. Am J Epidemiol. 1987 Apr;125(4):658–71.

    Article  CAS  Google Scholar 

  25. Baranowski T, Islam N, Baranowski J, Martin S, Beltran A, Dadabhoy H, et al. Comparison of a web-based versus traditional diet recall among children. J Acad Nutr Diet 2012 Apr;112(4):527–532. doi: https://doi.org/10.1016/j.jada.2011.10.002. Epub 2012 Feb 1.

    Article  Google Scholar 

  26. Livingstone M, Robson P, Wallace J. Issues in dietary intake assessment of children and adolescents. Br J Nutr. 2004;92(Suppl 2):S213–22.

    Article  CAS  Google Scholar 

  27. Freedman LS, Commins JM, Moler JE, Arab L, Baer DJ, Kipnis V, et al. Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake. Am J Epidemiol 2014 Jul 15;180(2):172–188. doi: https://doi.org/10.1093/aje/kwu116. Epub 2014 Jun 10.

    Article  Google Scholar 

  28. Burrows TL, Martin RJ, Collins CE. A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labeled water. J Am Diet Assoc 2010 Oct;110(10):1501–10. doi: https://doi.org/10.1016/j.jada.2010.07.008. Review.

    Article  Google Scholar 

  29. Magarey A, Watson J, Golley RK, Burrows T, Sutherland R, McNaughton SA, et al. Assessing dietary intake in children and adolescents: considerations and recommendations for obesity research. Int J Pediatr Obes 2011 Feb;6(1):2–11. doi: https://doi.org/10.3109/17477161003728469. Epub 2010 Sep 28.

    Article  Google Scholar 

  30. Freisling H, van Bakel MM, Biessy C, May AM, Byrnes G, Norat T, et al. Dietary reporting errors on 24 h recalls and dietary questionnaires are associated with BMI across six European countries as evaluated with recovery biomarkers for protein and potassium intake. Br J Nutr 2012 Mar;107(6):910–920. doi: https://doi.org/10.1017/S0007114511003564. Epub 2011 Jul 27.

    Article  Google Scholar 

  31. Kristal AR, Feng Z, Coates RJ, Oberman A, George V. Associations of race/ethnicity, education, and dietary intervention with the validity and reliability of a food frequency questionnaire: the Women's Health Trial Feasibility Study in Minority Populations. Am J Epidemiol. 1997;146(10):856–69 Erratum in: Am J Epidemiol 1998 Oct 15;148(8):820.

    Article  CAS  Google Scholar 

  32. Mochari H, Gao Q, Mosca L. Validation of the MEDFICTS dietary assessment questionnaire in a diverse population. J Am Diet Assoc. 2008 May;108(5):817–22. https://doi.org/10.1016/j.jada.2008.02.021.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Mayer-Davis EJ, Vitolins MZ, Carmichael SL, Hemphill S, Tsaroucha G, Rushing J, et al. Validity and reproducibility of a food frequency interview in a multi-cultural epidemiology study. Ann Epidemiol. 1999 Jul;9(5):314–24.

    Article  CAS  Google Scholar 

  34. Tucker KL, Bianchi LA, Maras J, Bermudez OI. Adaptation of a food frequency questionnaire to assess diets of Puerto Rican and non-Hispanic adults. Am J Epidemiol. 1998 Sep 1;148(5):507–18.

    Article  CAS  Google Scholar 

  35. Champagne CM, Delany JP, Harsha DW, Bray GA. Underreporting of energy intake in biracial children is verified by doubly labeled water. J Am Diet Assoc. 1996;96:707–9.

    Article  CAS  Google Scholar 

  36. Jaceldo-Siegl K, Knutsen SF, Sabaté J, Beeson WL, Chan J, Herring RP, et al. Validation of nutrient intake using an FFQ and repeated 24 h recalls in black and white subjects of the Adventist health Study-2 (AHS-2). Public health Nutr. 2010;13(6):812–9. https://doi.org/10.1017/S1368980009992072 Erratum in: Public Health Nutr 2011 Nov;14(11):2079–80.

    Article  PubMed  Google Scholar 

  37. Jaceldo-Siegl K, Fan J, Sabaté J, Knutsen SF, Haddad E, Beeson WL, et al. Race-specific validation of food intake obtained from a comprehensive FFQ: the Adventist health Study-2. Public health Nutr. 2011;14(11):1988–97. https://doi.org/10.1017/S1368980011000735 Erratum in: Public Health Nutr. 2012 Nov;15(11):2165–6.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Subar AF, Kipnis V, Troiano RP, Midthune D, Schoeller DA, Bingham S, et al. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the observing protein and energy nutrition (OPEN) study. Am Journal of Epidemiol. 2003;158:1–13.

    Article  Google Scholar 

  39. Tran KM, Johnson RK, Soultanakis RP, Matthews DE. In person vs telephone-administered multiple-pass 24-hour recalls in women: validation with doubly labeled water. J Am Diet Assoc. 2000;100:777–80 783.

    Article  CAS  Google Scholar 

  40. Kipnis V, Subar AF, Midthune D, Freedman LS, Ballard-Barbash R, Troiano RP, et al. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;158:14–21.

    Article  Google Scholar 

  41. Park Y, Dodd KW, Kipnis V, Thompson FE, Potischman N, Schoeller DA, et al. Comparison of self-reported dietary intakes from the automated self-administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers. Am J Clin Nutr. 2018;107(1):80–93. https://doi.org/10.1093/ajcn/nqx002.

    Article  PubMed  Google Scholar 

  42. Smith AF. Cognitive psychological issues of relevance to the validity of dietary reports. Eur J Clin Nutr. 1993;47(Suppl 2):S6–18 Review.

    PubMed  Google Scholar 

  43. Smith AF, Baxter SD, Hitchcock DB, Finney CJ, Royer JA, Guinn CH. Cognitive ability, social desirability, body mass index and socioeconomic status as correlates of fourth-grade children's dietary-reporting accuracy. Eur J Clin Nutr 2016 Sep;70(9):1028–1033. doi: https://doi.org/10.1038/ejcn.2016.43. Epub 2016 May 25.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

We thank Patricia Wakimoto at UC Nutrition Policy Institute, for assisting with training and quality control/oversight of the interviewers/data collection and data processing; and Shelly Mandel at UC Nutrition Policy Institute for assisting with data processing.

Funding

The Healthy Communities Study was funded by the National Heart, Lung, and Blood Institute, in collaboration with the Eunice Kennedy Shriver National Institute of Child Health and Development, the National Institute of Diabetes and Digestive and Kidney Disorders, the National Cancer Institute, and the National Institutes of Health (NIH) Office of Behavioral and Social Sciences Research; DHHS, under Contract No. HHSN268201000041C.

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the U.S. government. No financial disclosures were reported by the authors of this paper.

Availability of data and materials

The datasets used for this study will be available in the BioLINCC repository, https://biolincc.nhlbi.nih.gov/home/ as of June 18, 2018.

Author information

Authors and Affiliations

  1. Nutrition Policy Institute, University of California, Agriculture and Natural Resources Division, 2115 Milvia Street, Third floor, Berkeley, CA, 94704, USA

    Sridharshi Chintha Hewawitharana, Lorrene Ritchie & Karen Lucy Webb

  2. Risk Factor Assessment Branch, Epidemiology and Genomics Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Rm. 4E124, 9609 Medical Center Drive, MSC 9763, Bethesda, MD, 20892, USA

    Frances Elizabeth Thompson

  3. Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, 6701 Rockledge Drive, Suite 10018, Bethesda, MD, 20892, USA

    Catherine M. Loria

  4. Battelle, 505 King Ave, Columbus, OH, 43201, USA

    Warren Strauss & Jyothi Nagaraja

Authors
  1. Sridharshi Chintha Hewawitharana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frances Elizabeth Thompson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Catherine M. Loria
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Warren Strauss
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jyothi Nagaraja
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lorrene Ritchie
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Karen Lucy Webb
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sridharshi Chintha Hewawitharana.

Ethics declarations

Ethics approval and consent to participate

The Battelle Memorial Institute’s Institutional Review Board provided oversight for the study, operating under its Federal Wide Assurance No. 4696, on file with the NIH-HHS Office of Human Research Protection. All data collection forms and protocols were approved by the U.S. Office of Management and Budget (OMB #0925–0649). All participants included in this study consented/assented to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Table S1. Definitions of dietary intake variables of interest by assessment method, Healthy Communities Study, USA, 2013–2015. (DOCX 12 kb)

Additional file 2:

Table S2. Percent difference between mean intakes for dietary intakes as estimated by the Automated Self-Administered 24 Hour Recall for Children (ASA24-Kids) and Dietary Screener Questionnaire (DSQ) in the Healthy Communities Study (HCS), (n = 656) USA, 2013–2015, and as estimated by the National Health and Nutrition Examination Survey (NHANES), (n = 2166) USA, 2009–2010, by sex and age group. (DOCX 17 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hewawitharana, S.C., Thompson, F.E., Loria, C.M. et al. Comparison of the NHANES dietary screener questionnaire to the Automated Self-Administered 24-Hour Recall for Children in the Healthy Communities Study. Nutr J 17, 111 (2018). https://doi.org/10.1186/s12937-018-0415-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12937-018-0415-1

Keywords

Nutrition Journal

ISSN: 1475-2891

Contact us