Clinical records are an important source of information and medical record review is a commonly used data collection method in epidemiological studies. However, data registered in clinical records are not originally collected for research purposes and may not faithfully reflect all the events that happen during a medical consultation1,2. These records are less likely to explicitly include negative self-reports or diagnoses3 and systematically tend to report incomplete information about lifestyles4.

Epidemiological studies frequently use personal interviews and self-administered questionnaires as the sole information sources on exposures and outcomes. These methods are regarded as valid tools and provide many advantages for research. Nonetheless, the quality of information obtained through self-report depends substantially on the type of illness5,6. the participants’ characteristics5,7,8, the method used to administer the questionnaire9 and the questionnaire's design10,11.

Information obtained by self-report and medical record review may not be consistent. Several studies have compared self-reported data collected by questionnaires with medical record abstraction for the assessment of history of cardiovascular diseases (CVD)3,7, pregnancy-related events and birth characteristics12–14, and show that the agreement between the two data sources depends on the data being collected15. Additionally, any data collection method may be affected by interobserver variability, which might influence medical record reviews when multiple abstractors are involved2,16. In recent years, little attention has been paid to examining interobserver variability in data abstraction from clinical records, even though these methodological aspects influence the quality of the data and are critical for obtaining reliable results17,18.

Given the extensive use of these two data collection methods, the practical implications of their use must be understood, especially when both data sources are complementarily used in the same study. In a Portuguese birth cohort, data on several variables from the baseline evaluation were missing for a large number of participants. We planned to complete these data by abstracting them from medical records at delivery. To the best of our knowledge, no previous Portuguese study has evaluated the yield of missing data recovery by medical record review or agreement between questionnaire and clinical records and between multiple raters in medical record review.

The aim of the present study was to assess the success of missing data recovery by medical record review. Additionally, we evaluated the agreement between data recovered from clinical records and data previously obtained through a structured questionnaire administered to mothers of the cohort, as well as interobserver variability in data abstraction from clinical records.

This study was based on the birth cohort Geração XXI, to which 8,127 mothers were consecutively invited after delivery. These women gave birth to 8,270 infants. Recruitment was conducted between April 2005 and September 2006 at all public maternity units covering the metropolitan area of Porto, Portugal. All the maternity units (Table 1), except Maternidade Júlio Dinis (MJD), are included in a general hospital, with a variety of medical and surgical specialties, and all correspond to level III maternity units, with differentiated perinatal support. At birth, 91.4% of the invited mothers accepted to participate.

Information on family and personal history of disease and the mothers’ anthropometric parameters before pregnancy was collected during a face-to-face interview conducted by trained interviewers using structured questionnaires within 72hours of delivery, during the hospital stay. Data on pregnancy complications, gestational age and neonatal characteristics were abstracted from clinical records by the same interviewers.

Missing data recovery by clinical record review and agreement with questionnaire

Of the 8,127 mothers, 3,771 had at least one missing value in personal history of disease, anthropometric parameters, pregnancy complications, blood glucose or oral glucose tolerance test results during pregnancy, or neonatal characteristics at birth (Fig. 1). This was the sample used in the current study.

Figure 1.

Definition of the samples for missing data recovery, assessment of agreement between data collected by questionnaire and medical record review, and evaluation of interobserver agreement in medical record review.

a A family history of diabetes mellitus or cardiovascular disease was considered present when participants reported at least one parent or sibling affected by diabetes or by stroke and/or myocardial infarction, respectively.

b A personal history of pre-pregnancy hypertension and diabetes mellitus was considered present when participants recalled having received a medical diagnosis of these conditions.

c Usual weight in the 2years preceding pregnancy and weight immediately before delivery were both obtained through recall information to the nearest 0.1 Kg. Height was measured without shoes by the interviewers to the nearest 0.1cm. When measurement was not possible, height was reported by the mother as registered in the identity card (35.4% of women with data on height).

d Gestational hypertension, eclampsia/pre-eclampsia and gestational diabetes were considered only when explicitly recorded as a diagnosis during the current pregnancy.

e Gestational age was considered as that determined by ultrasound or, when this information was not available, as length of amenorrhea, to the nearest 0.1weeks.

f Birth weight, length and head circumference of the newborn were registered to the nearest 1g and 0.1cm, respectively.

g A family history of CVD and diabetes mellitus was classified using the same criteria as those used in the baseline questionnaire. When the affected relative was not clearly identified as a parent or sibling, the family history was considered positive.

h A personal history of hypertension and diabetes was considered present only when explicitly recorded as a diagnosis and not inferred from blood pressure values, serum glucose or drug use.

i Pre-pregnancy weight was considered to the nearest 0.1 Kg as self-reported to a health professional, either at early appointments during pregnancy or at birth, while the weight at the end of pregnancy was considered as the weight registered in records at admission for delivery or, when this information was unavailable, at the last medical appointment before birth. Height was considered as registered to the nearest 0.1cm at admission for delivery or, when this information was unavailable, in the clinical records before birth.

(0.69MB).

There were no significant differences between participants with complete and missing data in age [mean (standard deviation, SD): 29.4 (5.5) vs. 29.6 (5.7) years; p=0.306], education [median (interquartile range, IQR): 10 (7-12) vs. 10 (7-12) years; p=0.122], gravidity (first pregnancy: 48.7% vs. 47.1%; p=0.156) and marital status (married/living with a partner: 93.5% vs. 94.3%; p=0.165).

Between October 2008 and June 2009, we reviewed the delivery medical records of 3,657 women to recover missing data. The remaining 114 records were not available for review during the period for which we were authorized to consult them, either because they were being used for subsequent patient care or for administrative purposes (Fig. 1).

The clinical records were reviewed by a trained abstractor (EA), who had been an interviewer at the baseline evaluation of Geração XXI, using standardized criteria (Fig. 1). From each medical record, we abstracted data on all the variables of interest, whether or not they were missing in the questionnaire. Since most women had missing values in one or very few variables, we used those who had data available from both the baseline interview and the medical record review to assess agreement between the two data sources in each variable. Therefore, the sample size was not the same for all the variables (Fig. 1).

In this study of pregnant Portuguese women, data on pathological complications of pregnancy and maternal anthropometrics were successfully recovered from medical records when missing from the baseline questionnaire, while the yield of the medical record review was low in recovering past family or personal history of disease.

The overall cost of recovering information from clinical records is an important finding of this study. This procedure required one abstractor over an extended period of time. Clinical records at these maternity units were not electronic and the fact that paper records had to be reviewed likely decreased the yield and increased the time involved. The cost-benefit of performing this process should be considered case-by-case, taking into account the study aims and the expected yield for distinct variables in each setting. However, this is a time-consuming and expensive solution for correction of methodological errors that should not have occurred in the first place, as when using records to retrieve information that was originally also obtained from medical records.

The agreement between self-reported and medical record data was highly variable. Data directly related to pregnancy and well-known risk factors were concordant. Family history of CVD was underreported in the clinical records of the five hospitals. CHVNG showed the highest estimate of agreement for family history of both diabetes and CVD. This difference can be explained by the clinical record format, which varied between maternity units, including a standardized and pre-formatted section for registration of family history of disease at the CHVNG. In general, there was greater consistency between data sources in family history of diabetes than in family history of CVD. Acknowledgement of family history of diabetes as a major risk factor for the development of gestational diabetes21 may lead physicians to register these data more systematically and, simultaneously, to increase awareness of these risk factors among pregnant women. A family history of stroke, myocardial infarction and diabetes mellitus can be accurately reported by the participants, when compared with the relatives’ self-reports or their death certificates and general practitioners’ or hospital notes22–24.

We showed that the agreement between data collected by questionnaire and those abstracted from medical records is highly dependent on the data recording procedures and practices used and the extent to which these are standardized in each hospital. Our overall results are locale-specific, despite reflecting issues that may be present to a greater or lesser extent in each setting; while the overall yield is likely to be similar in distinct settings, the limited external validity of our results needs to be acknowledged. However, we also present the results by institution and interpret them by taking into account our knowledge of the data collection procedures in each hospital, which provides important information that may be generalized with a smaller number of prior assumptions. Recent mothers are likely to provide more accurate information on personal and family history of diseases than the general population. Therefore, the results of this study cannot be generalized to other types of population.

Few studies have previously focused on the agreement between self-reported data and clinical record review for CVD and risk factors in young women before pregnancy15,25. Although the low prevalence and the lack of awareness of hypertension and diabetes diagnosed before pregnancy in young women increase the difficulty of analyzing the agreement26,27, our results are concordant with previous findings. Ramadhani et al25 have reported a substantial agreement between medical records and maternal interviews for non-gestational diabetes data (k=0.75, 95%CI: 0.64-0.86) and similar estimates were found for pre-pregnancy hypertension among pregnant Latin women (k=0.68, 95%CI: 0.46-0.90)15.

The accuracy of self-reported weight and height, in comparison with objective measurements, has been extensively reported. Overall, weight and body mass index tend to be underreported and height overreported by women28,29, and the same pattern has been observed among pregnant women30–32. In our study, weight and height were systematically higher in the clinical record than in the questionnaire. The longer time frame considered in the questionnaire could translate into underestimation of weight before pregnancy. The questionnaire data on weight at the end of pregnancy was always self-reported when the cohort was assembled, while in some hospitals weight may have been measured by nurses before delivery. Although we have no possibility of knowing if the weight registered in the clinical records was measured or self-reported, the higher value obtained by the former seems to support our assumption32. Pregnant women tend to overreport their height31. At birth, most participants were measured, which could explain the differences found.

Studies that have examined the accuracy of self-reported height and weight employed to determine body mass index categories have concluded that approximately 80% of women were correctly classified29,33, which is in accordance with our observations.

The interobserver variability was low and did not threaten data precision. Standardized training of abstractors and rigorous quality assurance were proposed as critical criteria to improve the quality and accuracy of clinical record review16–18,34,35. When research involves data collection by distinct observers, the extent to which different observers perceive and record the same information should be evaluated. In the literature, most medical record review studies fail to report on interrater agreement34. Our study showed a low interrater variability, and the discrepancies found in family history of diabetes and CVD between the two reviewers can be attributed to a combination of factors, from the different structure of the clinical records and the data registered in them to specific problems in accessing information from the charts. When records are not pre-formatted and standardized, professionals are more likely to underreport or to register data in different locations, increasing the difficulty of the data abstraction procedure, which precludes the generalization of these findings to other settings.

This study provides important information for the planning and interpretation of epidemiological studies on pregnant women, but some limitations should be discussed. Despite the large sample size, the evaluation of conditions such as hypertension and diabetes before pregnancy is limited by their very low prevalence. The absence of reference to a medical condition in the clinical record was considered as the absence of the condition itself. We do not believe that this assumption compromises our conclusions as it is unlikely that physicians did not enquire about such conditions or failed to record them when present. When clinical records did not specify which relative had developed the outcomes studied, we considered the family history of disease as positive, which could lead to a higher proportion of family history in clinical records than in the interview. However, given the unusual occurrence of this situation, we do not expect that this factor had a quantitatively significant influence on the results. In this study, there was no gold-standard and differences between data sources could reflect misclassification in the questionnaires, in the clinical records data or both. We were unable to draw conclusions on which method provides higher validity for the data assessed.

In conclusion, data directly related to pregnancy and well-known risk factors can be safely recovered from medical records when missing from self-reported data. The need to implement structured and standardized methods to abstract data from clinical records cannot be overemphasized to enhance data quality, which can subsequently improve the interpretation and generalization of the results obtained.

Elisabete Alves collaborated in the acquisition, analysis and interpretation of the data and draft the article. Nuno Lunet collaborated in the analysis and interpretation of the data reviewed and revised the article critically for important intellectual content. Sofia Correia helped to supervise the field activities and reviewed the article. Ana Azevedo designed the study, directed its implementation and reviewed and revised the article critically for important intellectual content. Henrique Barros designed and supervised the birth cohort Geração XXI, on which our study is based, and made a substancial contribution to the revisions made alter the requests of the reviews. All authors approved the final version.

The authors declare that they have no conflict of interests.