Validity of a self-administered food frequency questionnaire (FFQ) and its generalizability to the estimation of dietary folate intake in Japan

Background In an epidemiological study, it is essential to test the validity of the food frequency questionnaire (FFQ) for its ability to estimate dietary intake. The objectives of our study were to 1) validate a FFQ for estimating folate intake, and to identify the foods that contribute to inter-individual variation of folate intake in the Japanese population. Methods Validity of the FFQ was evaluated using 28-day weighed dietary records (DRs) as gold standard in the two groups independently. In the group for which the FFQ was developed, validity was evaluated by Spearman's correlation coefficients (CCs), and linear regression analysis was used to identify foods with large inter-individual variation. The cumulative mean intake of these foods was compared with total intake estimated by the DR. The external validity of the FFQ and intake from foods on the same list were evaluated in the other group to verify generalizability. Subjects were a subsample from the Japan Public Health Center-based prospective Study who volunteered to participate in the FFQ validation study. Results CCs for the internal validity of the FFQ were 0.49 for men and 0.29 and women, while CCs for external validity were 0.33 for men and 0.42 for women. CCs for cumulative folate intake from 33 foods selected by regression analysis were also applicable to an external population. Conclusion Our FFQ was valid for and generalizable to the estimation of folate intake. Foods identified as predictors of inter-individual variation in folate intake were also generalizable in Japanese populations. The FFQ with 138 foods was valid for the estimation of folate intake, while that with 33 foods might be useful for estimating inter-individual variation and ranking of individual folate intake.


Introduction
Owing to their ease of administration and low burden on the subject, the assessment of dietary intake in epidemiological studies is often assessed by means of food frequency questionnaires (FFQs) [1]. The chief limitation of FFQs, however, is that serious errors can occur if foods governing inter-individual differences in intake of certain nutrients are omitted from the list. Folate is particularly prone to such error, because it is derived from a variety of foods of both animal and plant origin, not all of which can be included in an FFQ. Foods that contribute to interindividual differences may fail to be included in food lists, potentially confounding estimation of folate intake by the FFQ.
Another implication of inter-individual variation in the intake of folate or any other nutrient is that such variation may be the underlying determinant of associations between food intake and disease. The preventive effect of certain foods on specific diseases is usually the effect of a particular nutrient contained in the food. Such associations are more likely to be detected when inter-individual differences in intake of the nutrient are larger. Conversely, even if a food contains high levels of a particular nutrient, association will be weak if consumption among individuals is similar. The identification of foods that contribute to inter-individual variation in nutrient intake among the population is therefore an important component of any investigation of nutrients responsible for associations between food intake and disease.
According to the National Nutrition Examination Survey of the United States Population [2] and the National Nutrition Survey in Japan [3], folate intake between the countries is similar among both middle-aged and older Study sites of the JPHC Study Cohort I Cohort II age groups. Intake among younger groups, however, is much lower in Japan. This trend is presumably due to differences in the foods that contribute to folate intake. Specifically, folate-fortified food and supplements are large contributors to folate intake for individuals in some Western countries, and questionnaires especially designed to assess folate intake include these items [4,5]. Because folate-fortified foods are not available in Japan and supplement consumption is low, however, folate intake is almost exclusively from natural sources. Nevertheless, little is known about those foods that are the main sources of folate intake among Japanese.
A FFQ was developed and validated for the estimation of dietary intake for the JPHC study. Spearman's correlation coefficients (CCs) between serum folate and folate intake estimated by this FFQ using a biomarker as reference was 0.26 in men [6]. In the present report, we evaluated the validity of this FFQ in the population subgroup for which the FFQ was originally developed using dietary records (DRs) as standard. We then attempted to identify foods that most contributed to folate intake, and those responsible for the differences in intake between individuals. We subsequently repeated these analyses in a second subgroup that was independent of the population for which the FFQ was originally developed, i.e., an external population, to assess its generalizability in Japan. The objectives of this study were: 1) to validate this selfadministered FFQ as a means of estimating folate intake, and identify foods that contribute to individual intake and inter-individual variation in folate intake in the population for which the FFQ was developed; and 2) to determine the validity of this FFQ, designed for a study cohort, in estimating the intake of folate and identifying foods Sequence of data collection for the JPHC FFQ Validation Study Figure 2 Sequence of data collection for the JPHC FFQ Validation Study.

Study Subjects
Two validation studies of the FFQ were conducted in a subsample of participants in the JPHC Study, a large population-based prospective study that involved medical examination of the study participants. The target population of the JPHC Study consisted of two cohorts, the first started in 1990 (Cohort I) and the second in 1993 (Cohort II). The aim of the cohort study was to investigate the association between various lifestyle factors such as diet and chronic diseases. Despite Japan's small geographical area, considerable regional variation in diet is seen, and study sites were selected to be representative of the whole country. Cohort I was drawn mainly from the northeast part of the country and Cohort II from the southwest ( Figure 1). The study design and participants in the entire cohort have been described previously [7].
The present FFQ validation study was conducted in subsamples of Cohort I and Cohort II. The Cohort I study was initiated in February 1995 to validate the FFQ for use in a 5-year follow-up survey [8], given that it was originally developed based on data from 3-day weighed DRs in a random sample from this Cohort [9], while the Cohort II study was to evaluate the generalizability of the FFQ independent of the population for which it was originally developed [10]. Respective numbers and recruitment areas were 247 volunteers from the Ninohe, Yokote, Saku and Chubu (previously named Ishikawa) public health center areas, and 392 volunteers from Mito, Kashiwazaki, Chuo-higashi, Kamigoto, Miyako and Suita. For the present report, we analyzed the data of the 215 and 350 subjects in Cohorts I and II, respectively, for whom the 28day DR and FFQ data were complete.

Data Collection
Data collection has been described in detail elsewhere [8,10]. In brief, each subject completed two FFQs and 28day DRs ( Figure 2). The first FFQ was administered to provide data to compare with the second FFQ as a means of evaluating reproducibility, and the second FFQ was administered to obtain data to compare with the DRs to evaluate its validity. Only data from the second FFQ for validity has been used in this paper.
DRs were collected over 7 consecutive days in each of the 4 seasons, except in Chubu (2 seasons). Local dietitians instructed the subjects to weigh all foods and beverages with the scales and measuring utensils provided, and to record results in a specially designed booklet. The subjects in Cohort I, however, were instructed to use standardized portion sizes for some foods that were difficult to weigh (semi-weighed DRs). The subjects described each food, method of preparation, and the names of the dishes in detail. They also reported all dietary supplements used, if any. At the end of each season, the DRs were reviewed in a standardized manner, and each food was coded by local dietitians.
The self-administered semi-quantitative FFQ consisted of 138 food items and 14 supplementary questions on dietary habits and use of supplements. The validity of the questionnaire in regard to the intake of energy, other nutrients and foods, as well as the use of dietary supplements is described elsewhere [10][11][12][13].
Dietary intakes of folate according to the DRs and the FFQ were calculated using the Standardized Tables of Food Composition, 5th ed. [14]. Mean daily intake of folate for the 28 days (14 days in Chubu) was calculated based on the DRs of each subject. Because none of the subjects used folic acid supplementation, dietary supplements were not included in the calculation.

Statistical analysis
Means and standard deviation of folate intakes from the DRs and FFQ were calculated by sex for Cohort I and Cohort II. Spearman's rank CCs were calculated for crude intake and energy-adjusted intake by the residual method, and were corrected for the attenuating effect of random intra-individual error (deattenuation) in Cohort I subjects to evaluate internal validity in the population for which the FFQ was developed. Deattenuation was done using the following formula: Deattenuated , where r is the observed correlation, λ x is the ratio of intra-to inter-subject variation, and n x is number of dietary records for each subject [15]. The same analysis was performed for Cohort II subjects to evaluate external validity The percentage contribution of each food to total folate intake was computed based on the DRs of Cohort I subjects, and the 20 foods contributing most were listed based on their percentage contribution. Percentage contributions of the same 20 foods in the DRs of Cohort II were calculated and their actual rank of percentage contribution in Cohort II was determined.
Linear regression analysis with stepwise selection was used to identify foods that contributed to inter-individual variation, with folate intake from each food item according to the FFQ of Cohort I used as the explanatory variable, and total folate intake according to the DRs as the response variable. A model (partial) R-square value for the selected food items was computed. Cumulative mean intake from each food item on the list was calculated, and CC = (1+( n x x λ / )) compared to total intake according to the DRs by Spearman's CCs to evaluate validity. Cumulative intake from the same food items in the Cohort II subjects and their CCs from the DR data were calculated to evaluate external validity.

Results
Daily folate intake as assessed by DRs and FFQ as well as the Spearman's rank CCs between the two measurements by cohort group and sex are shown in Table 1. Mean daily intake of folate based on the FFQ in Cohort I was significantly overestimated compared to the DR data. Both crude and adjusted CCs were higher in men than women in Cohort I but were similar in Cohort II ( Table 1).
The 20 foods that made the greatest contribution to total folate according to the DR in Cohort I are listed in Table  2. The list consists of various foods, mainly from plant sources such as vegetables, with spinach making the highest contribution followed by rice, green tea, cabbage, eggs and beer. These 20 foods contributed 55.2% of total intake in men and 52.9% in women. The contribution of the same 20 foods to folate intake in Cohort II subjects was 44.9% for men and 43.2% for women. Gyokuro, a type of green tea, made the second highest contribution in Cohort II, but was not among the 20 in Cohort I. Other food items with the highest contribution in Cohort II but not Cohort I were kamairi-cha (pan-fried green tea), bread, tomatoes, and pumpkins in both sexes; purple laver in men; and sweet potatoes and komatsuna (a green leafy vegetable) in women. When both kinds of green tea were excluded, however, these foods accounted for only 4.6% of total intake in men and 6.2% in women.
The foods that best predicted inter-individual variation in dietary folate and the validity (correlation coefficients) of folate intake based on those foods are shown in Table 3. A total of 33 foods are listed with the cumulative R-square value of 0.59. The food that best predicted variation of intake was green tea, which contributed 12-15% of total intake. No other food predictive of variation contributed more than 1% of total folate intake. The cumulative folate intake from the 33 foods contributed approximately 30% of total intake in both men and women. The CC of intake from the 33 foods in the internal population (Cohort I) was 0.46 in men and 0.28 in women (indicated as "internal" in Table 3), and had approximately the same level of validity as the data from the full 138-food FFQ.
When the same 33-food list was used to compute intake in the external population (Cohort II), the cumulative folate intake contribution was again 30% (indicated as "external" in Table 3), and CC was 0.30 in men and 0.35 in women, showing approximately the same level of validity as in the internal population.

Discussion
In this study, we evaluated the validity of a FFQ as a means of estimating dietary folate intake in the population for which the FFQ was originally developed. We also attempted to identify foods that differentiated the level of folate intake in individuals by stepwise regression, and tested the validity of assessing folate intake based on the intake of these foods. The results were also cross-validated in an independent population to evaluate generalizability.
Validity of the FFQ in estimating folate intake was moderate in both the internal and external populations. In previous studies, the validity of questionnaires in estimating energy-adjusted dietary folate intake varied from 0.2 to 0.6, depending on the study population [15][16][17][18][19][20][21]. In studies that reported intake from the diet and from dietary supplements separately, CC was 6-21% higher for folate intake that included supplements than for intake from diet alone. Although none of the subjects used supple- ments that contained folic acid, which had only just become available at the time of the study, the validity of our FFQ in estimating folate from the diet was relatively high, probably because the dietary folate intake of our subjects was as high as that of supplement users in some of the previously cited studies.
Although the largest proportion of folate intake was from vegetables, these did not necessarily explain the differences in intake between individuals. For example, spinach, which is very rich in folate and was one of the highest contributors to mean folate intake, could not explain inter-individual variation because it was consumed by almost every subject. In contrast, green tea contributed greatly to both individual intake and to inter-individual variation, probably because consumption was strongly dependent on individual preference. Although the analysis provided us with information about foods that predicted inter-individual variation in folate intake, some foods which had a moderate partial R-square value con-tributed less than 0.1% to total folate intake, such as luncheon meat, ham, and so on. These may have been selected by chance alone. In this kind of analysis, even unimportant contributors to the cumulative R-square value may be statistically significant, but can nevertheless be ignored [1]. In any case, it is noteworthy that individuals could be ranked by folate intake based on only 33 foods, with the same level of validity as with the long FFQ.
Any analysis of the possible effects of folate intake on disease also requires analysis of the effect of food items which contribute to total intake and inter-individual variation of folate. It is of great research interest to determine whether the association between food intake and disease is the result of folate intake. For example, we might hypothesize that folate intake may help explain the protective effect of green tea on gastric cancer described in the recent report of the JPHC study [22]. The mechanism of carcinogenesis through DNA instability and methylation abnormalities as a result of folate deficiency has been  studied in animal and in vitro studies [23][24][25], and an association between folate and gastric cancer has been reported in a number of case-control studies [26][27][28][29].
Other prospective studies, however, have failed to show a consistent association between green tea and gastric cancer [30][31][32]. We speculate that the association was strong in the JPHC population owing to the large contribution of green tea to the variation in folate intake, which was not seen in the other populations.
One of the strengths of our study is the precision of the reference data. The ratio of intra-to inter-individual variation in our data was somewhat higher (1.9-4.8) than in several studies in Western countries [33,34]. When intraindividual variation is larger, an increased number of dietary assessment days is required to obtain a valid standard. Although intra-individual variation among our subjects was high, we had more than a sufficient numbers of days (28 days) of data to represent the true intake of the individuals, because the number of DR days needed to estimate true intake within 20% of the true mean with our data was only about 17 to 19 days according to our analysis. By comparison, the greatest number of days of dietary assessment used as standard in previous studies was 14 days [35].
In addition, our analysis is unique because we used regression analysis to identify the foods most predictive for inter-individual variation in dietary folate, and then evaluated both the internal and external validity of the intake of those foods. It is important to test external validity, because there is no assurance that explanatory variables selected by regression analysis are valid in an external population [1]. To our knowledge, this is the first study to attempt to identify foods that contribute to inter-individual variation of folate intake in Japan, where folate intake is almost exclusively from natural sources. We developed a list of foods that contribute to inter-individual variation in folate intake in the population for which FFQ was originally developed, and tested the generalizability of the results to an external population. The two populations covered various geographic areas throughout Japan. Our results imply the possibility that a shorter questionnaire which specifically targets folate intake in Japanese populations can be developed.
One limitation of our study is that because the subjects needed to be highly motivated to complete the 28-day DRs, they were not a randomly selected sample. Mean folate intake based on DRs was slightly higher than in the entire cohort, probably because the validation study subjects were likely more health conscious and consumed more vegetables. If all subjects had consumed a similar amount of certain foods, inter-individual variation in the food might have been falsely low. The generalizability of questionnaire results needs to be established with care.

Conclusion
Our FFQ is valid for estimating folate intake and is generalizable to the Japanese population. Although some foods such as spinach, rice and cabbage contributed more to total intake of folate, they did not necessarily contribute to inter-individual variation. Validity of the estimation of folate intake based on intake of the 33 foods that most contributed to inter-individual variation was about the same as that based on the original 138-food FFQ. Although folate is contained in a wide variety of foods, 33 foods in the FFQ were sufficient to account for the interindividual variation of intake and ranking of individuals' intake in Japan. We concluded that the FFQ with 138 foods was valid for the estimation of folate intake, and that the FFQ with 33 foods might be useful for estimating inter-individual variation and the ranking of individual folate intake.