Background
One of the most usual approaches to studying the situation of cancer worldwide is to analyse the geographic distribution of mortality rates and their trends. Information on the quality of cancer mortality data is thus essential for interpreting differences in mortality statistics.
In Spain, mortality represents the only comprehensive and homogeneous source of information on cancer for the whole country. The source of mortality statistics is the medical death certificate (DC), a compulsory administrative document completed by the practitioner who certifies the death. This certificate is subsequently transcribed onto a second document, the Statistical Bulletin of Death (SBD), and both are sent to the Municipal Civil Registry. Usually, the Civil Registry forwards the SBDs to the regional offices of the National Statistic Institute (Instituto Nacional de Estadística [INE]) on a monthly basis, where all items except cause of death are digitally recorded. When the data have been duly screened to detect errors and ensure quality control, the underlying causes of death (Causa básica de defunción) are coded at the Regional Authority Mortality Registries by trained teams, applying common criteria in accordance with International Classification of Diseases (ICD) guidelines. National coding protocols have been established to guarantee homogeneity of data1, while specific methods have also been implemented to validate accuracy systematically. Digital data files are then sent to the INE head office, which releases them once they have been rendered anonymous.
Quantitative data that would enable to asses the quality of cancer death certificates in Spain are relatively scarce. Several studies have addressed this topic, but all refer to areas of the country or to specific populations, and some have been published in local epidemiological bulletins or in symposium proceedings, which are not easily accessible. In this paper, we sought to synthesize information derived from these studies and to complement such data with a short description of the temporal distribution of the proportion of deaths due to ill-defined tumours in Spain, as an indirect indicator of the quality of cancer death certification.
Methods
For review purposes, DC and SBD were deemed to be death certificates, as a very high concordance between both documents has been reported2. Studies into the quality of cancer death certification in Spain were identified through: 1) a MEDLINE and EMBASE search using broad search criteria (January 1966 to January 2006); 2) a similar search in Spanish bibliographical databases, the Spanish Medical Index (Índice Médico Español [IME]), and IBECS; and 3) references in identified papers. In two studies3,4, only abstracts of poster presentations at scientific meetings were published. In these cases, quality-indicator data presented in the poster were specifically searched for.
Studies were considered eligible if they reported quantitative estimates of the accuracy of death certificates containing any mention of cancer. In these studies, cancer death certificates were compared against a second source of information, mainly comprising clinical or anatomo-pathological reports (which were taken as the «gold standard»). Agreement between both sources was measured using detection and confirmation rates.
According to Percy et al5, the detection rate (DR) or sensitivity for a specific site is defined as the proportion of hospital diagnoses (available clinical/anatomo-pathological information) with cancer of a certain site, with a death certificate where this disease is considered to be the basic cause of death, whereas the confirmation rate (CR) or positive predictive value is the proportion of cancer deaths in which the underlying cause specified in the death certificate is confirmed by hospital diagnosis.
DRs and CRs can be computed as: a) site-specific cancer rates at three digits of the ICD, that is considering an indicator per cancer-site; b) all-site three-digit rates, an overall indicator for cancer where the rates' numerator contains all cancer cases (ICD-9 codes 140-208) in which the site specified in the death certificate and clinical information agree; and c) all-tumour rates, also an overall indicator where the rates' numerator includes all cases that just mention «cancer» in both the death certificate and gold standard, even though there might be site misclassification. These figures are logically expected to be higher than overall comparisons at a three-digit level. A graphical scheme that might help understand these concepts is shown in figure 1.
Figure 1. Cross-tabulation of cause of death according to clinical diagnoses and medical death certificates and quality-indicator formulae.
Identified papers were classified into three main categories:
I. Studies focusing on all-cause death certificates, which provide accurate detection and confirmation rates (fig. 1).
II. Studies based solely on death certificates that mention cancer. These could be subdivided into two groups:
a. Studies with an additional source of information on cg or bg (fig. 1) that allows them to estimate accurate detection and confirmation rates. In some cases, these data6 were not considered for the purpose of calculating site-specific cancer rates, thereby leading to overestimated rates.
b. Studies without information on cancer deaths not certified as such, or on non-cancer cases erroneously certified as cancer (cg or bg). This rules out the possibility of computing all-tumour or three-digit agreement detection rates, and means that site-specific detection rates and confirmation rates are likely to be overestimated (fig. 1). In some cases4,7,8, it was possible to find information on global false positives within the text of the paper. We used these data to calculate unbiased all-tumour confirmation rates.
III. Necropsy-based studies. Since necropsies are relatively scarce in this country9, the main problem with these studies is external validity, as they tend to focus on very specific populations.
Many of the detection and /or confirmation rates offered in the summary tables were directly taken from the selected studies, though, where possible, the tables were completed by computing DRs and CRs using data provided in the papers.
In a second stage, a pooled CR and DR was calculated for each specific cancer location using data from all studies that had covered that location, and then all main cancer sites were classified according to Percy's criteria, the bench-mark for these types of studies5. Finally, as a complementary approach, we calculated the percentage of deaths due to ill-defined tumour versus all-tumour deaths and the percentage of deaths of ill-defined conditions versus all-cause deaths for the period 1980-2002, using whole country mortality figures supplied by the INE.
Results
Published studies on quality of cancer death certificates in Spain
A total of 14 studies providing quantitative data on quality of cancer death certification in Spain were found. The main characteristics of these studies, their classification according to the above criteria and, where available, all-tumour (ICD-9 codes 140-208) and all-site three-digit detection and confirmation rates are shown in table 1. In addition, figure 2 depicts the geographical location of the respective study populations.
Figure 2. Geographical location of published studies furnishing data on quality of cancer death certification in Spain.
Six of these studies belonged to categories I and II(a)3,6,10-14, those furnishing the most accurate estimators, with detection rates ranging from 75.2-100% for all tumours as a whole and 64.8-100% for all-site three-digit rates. On average, confirmation rates were higher than detection rates, ranging from 91.5-99.3% for all-tumour and 68.2-80.8% for all-site three-digit rates.
Another five studies were classified as category II(b)7,8,15-18. In general, they supplied data on all-tumour and all-site three-digit confirmation rates. Yet, with two exceptions4,8, these studies failed to consider global false positives when estimating three-digit confirmation rates, an approach that implies a certain overestimation of the agreement. Where possible, therefore, we calculated a corrected confirmation rate by including global false positives in the denominator. Only one study8 had adopted this approach and, in this case, the «biased» confirmation rate (excluding global false positives) was reckoned for comparison purposes. Both «biased» denoted by «*» and corrected confirmation rates are shown in table 1. On average, biased confirmation rates were three points higher than corrected rates. In these studies, all-tumour and corrected all-site three-digit confirmation rates were comparable to those obtained for categories I and II(a).
Only two papers came within category III (necropsy-based studies)9,19. Their results were slightly worse than others, but it should be borne in mind here that in Spain most autopsies are restricted to cases with uncertain diagnosis.
Table 2 summarises available information in Spain on detection and confirmation rates for malignant neoplasms by the main specific locations. The data, sorted by the period analysed, revealed that in the interim between the early 1980's and the most recent study (1992 data), the quality of certification had improved. Highest indices were found for cancer of the stomach, colon and rectum, pancreas, lung, melanoma, female breast, brain and haematological tumours. In contrast, other sites, such as ill-defined tumours and non-melanoma skin cancer, displayed lower rates of agreement.
Furthermore, in order to have a brief overview of the quality of cancer certification for specific sites, pooled estimators were calculated to classify the accuracy of death certification for specific cancers according to Percy's criteria5, which depend on detection and confirmation rates (table 3).
Ill-defined tumours and ill-defined causes
Figure 3 shows the trend in the percentage which ill-defined tumours and ill-defined causes represent of all-cancer and all-cause deaths, respectively, over the calendar period 1980-2002, in both sexes. During the 1980's, Regional Mortality Registries became responsible for the coding process. This change led to an improvement in the quality of the information, which is reflected in the downward trend in the proportion of ill-defined causes. The initial decline in the percentage of ill-defined causes was accompanied by an increase in the proportion of ill-defined tumours, which registered a less clear pattern. Nevertheless, it should be noted that, coinciding with the introduction of ICD-10 in Spain, 1999 witnessed an increase in these indicators, with the percentage of ill defined causes and ill defined tumours rising by 31% and 10% respectively over the previous year's figures, followed by an apparent levelling-off. Interestingly, women registered higher values for these two indicators of bad certification, in all cases. When proportional mortality was computed using adjusted rates, women continued to have worse results (data not shown).
Figure 3. Percentages of deaths due to ill-defined tumours and ill-defined conditions versus all-tumour and all-cause mortality. Trends for the period 1980-2002 by sex and for both sexes.
Discussion
Though quality at a national level has not been studied, available data suggest that, overall, cancer death certificates in Spain possess an accuracy comparable to that reported for other industrialised countries5,20,21. Indirect estimations such as the proportion of ill-defined causes in Spain show similar percentages to those registered by other developed countries22.
The first Spanish study to address death certificate reliability was published in 198115. Specifically focused on cancer death certificates, this study solely covered the Barcelona metropolitan area. Several authors subsequently studied the quality of death certificates in other parts of the country. In this paper, we summarised all available information to provide a global view of the quality of Spanish cancer-mortality statistics.
In Spain, published quality estimators are basically drawn from regional studies, many of which are sponsored or undertaken by Cancer Registries. Accordingly, it should be borne in mind that, despite the existence of national coding protocols, inferring quality indicators for the country, such as our pooled estimators, might also be problematic, since decentralisation of the coding process could cause inter-regional variability, and there are huge areas of the country where death certification quality studies have not been conducted (fig. 2). Only Giménez et al3, in their study on a toxic-oil poisoned cohort, provide national data, though their results could also be misleading as they refer to a cohort of sick people, subjected to a thorough follow-up over time. The progressive increase in the number of Cancer Registries in the country might go some way towards having more representative data about quality of cancer death certification in Spain in the future. Nevertheless, results from the different studies were quite similar for most cancer sites.
Compared to other causes of death, cancer (ICD 140-208) seems to be well certified in Spain, with detection rates being as much as 9 points higher for all tumours than for all causes together, and confirmation rates over 20 points higher than for all causes11,12, which could be due to the fact that cancer is usually a well-characterised diagnosis, and in most cases has histological confirmation.
All-tumour detection rates ranged from 79.9 to 100 and the CRs exceeded 90%. Depending upon their results, some studies have classified this broad category as well-certified3,11,13, while others have viewed tumours as being slightly underreported3,12. However, the percentage of underreporting in Spain, as estimated by García-Benavides et al11 and Cáffaro et al6, seems to be around 5-6%, which is comparable to international figures20,23,24. Hence, global cancer mortality figures can be considered accurate and useful for estimating the burden of this group of diseases.
As expected, when site misclassification was taken into account, agreement estimates were lower. All-site three-digit detection rates from categories I and II(a) studies range from 64.8 to 100 and confirmation rates from 68.2 to 80.8. Based on these indicators, cancer could be deemed to be ill-certified according to Percy's criteria. It is remarkable that, in general, Ruiz-Liso et al13 obtained better results than other studies. The CR range in category II(b) studies was similar (64.4-81%). The design used in such studies excludes global false positives, thus slightly overestimating this indicator. According to three studies6-8, in which both correct and biased estimators were available, biased data were on average three points higher. In the USA5 and France20, CRs were slightly higher than in Spain (82.7 and 86%, respectively). Nevertheless, it should be noted that those studies belong to category II(b) and are also biased.
If some problematic categories are grouped (this is the case of cervical ICD 180, corpus ICD 182 and unspecified uterine cancer ICD 179; or colon ICD 153 and rectal neoplasms ICD 154), all-site three-digit agreement rates improve. This can be seen in Cirera and Navarro7, where the proportion of agreement using three-digit ICD was close on 80%, and aggregation of problematic locations raised it to 83%. A comparable increase was previously described in the USA5 (4%) and in Ontario21 (6%).
Analysis of specific anatomic locations shows that, in general, the main leading cancer sites are well certificated. Thus, lung, colon-rectum (ICD 153-154), prostate, stomach, pancreas, female breast, uterus (ICD 179-180,182) and brain cancer, as well as leukaemia, lymphomas, myeloma belong to this category. Together, they represented around 69% of all cancer deaths registered in Spain in 200225. Nonetheless, it should be noted that colon and rectal cancers are respectively ill and undercertified unless they are considered together, since mutual misclassification of the two sites has been reported7,21. A similar situation can be observed with respect to uterus. Overall, this location has good accuracy rates, but cervix and corpus uterus are under and ill certified, respectively. Mortality due to uterus cancer in Spain registered a steady decrease since 1976, contrasting with the slightly increasing trend in cervix mortality26, which has been explained mainly as a consequence of a reassignment of cases previously coded as «uterus non-specified»14. Nevertheless, if uterus is regarded as a single category, its certification has improved with time and, in more recent studies, achieves acceptable figures.
Some other frequent locations, such as larynx, bladder or ovary, which rank among the ten leading causes of cancer death in Spain, evince problems in certification. A certain degree of overreporting has been described for laryngeal cancer, due to misclassification of head and neck tumours6,8,18. Similarly, there is certain measure of overreporting for oesophageal cancer, mainly attributable to the inclusion of stomach cases6-8,18, as well as for liver, largely due to misclassification of gallbladder and ill-defined neoplasms6-8,13,18. Percy et al27 warned that inclusion in ICD-9 code 155 of liver cancer that was not specified as primary or secondary, was a possible cause of misclassification for these tumours, as it might lead to some coders registering secondary liver cancers under this category.
Furthermore, there have been reports of underestimation of urinary bladder cancer, erroneously certified as prostate neoplasm6-8,13,17,18. It should, however, be noted that, even with this underestimation, Spain had the second highest bladder mortality rate within the European Union in 200026. In addition, some Spanish authors have reported undernotification of skin cancers due to lack of information on their anatomic location6, thereby implying their inclusion as ill-defined and unspecified tumours (ICD-9 195, 199).
Accuracy of ovary-neoplasm certification seems to vary widely among studies, though it can be regarded as undercertified according to pooled estimators. While some reported cases of death due to ovarian cancer were really due to abdominal or uterine neoplasms, in some studies7,8,14, «unspecified uterus tumours» would appear to include some ovarian cancers as well as tumours of the cervix and endometrium6,8,14,17,18.
Ill-defined tumours were badly certified according to Percy's criteria. Cirera and Navarro7 reported there was clinical information that would allow for almost half of these tumours to be included in other categories. The selfsame problem has been described in other countries, such as Brazil28.
Our results show that all cancer sites classified as well certified in Spain have been reported as such in the USA by Percy et al5. However, this author also encountered good agreement rates for oesophagus, bladder, gallbladder, thyroid gland and kidney, locations that did not display a good standard in Spanish studies. In Ontario, Canada, Reynolds21 similarly observed good DRs and CRs for all those neoplasms classified as well-certified in Spain, with the exception of pancreas, and also found high rates for cervix uterus, bladder, kidney, ovary and endocrine glands. In France, Laplanche et al20 reported CRs of over 80% for breast, colon-rectum, lung and pancreas, values similar to Spanish figures. In contrast, they found CRs of under 80% for head and neck and cervix, tumours which in Spain register good CRs. Stomach cancer was also well classified in Brazil29 and Italy30. In international comparisons, however, a relevant factor to be considered are differences in international coding practice, since divergences up to 30% have been described by Percy and Muir31 among western countries using the ICD-9.
With respect to accuracy of certification, Percy mentioned the influence of several factors such as age, sex, geographic area, presence of an autopsy or place of death5. In Spanish studies, a lower quality has been reported for older ages and for women6-8,18. These sex-related differences are reflected in the percentage of ill-defined tumours and ill-defined causes, which are regularly higher in females and could in part be due, both to gynaecological neoplasms18 and to the older age of women. Insofar as place of death is concerned, a lower quality of death certification has been associated with death at home6,7, though other authors have failed to find any difference11. Finally, the quality of certification has been shown to be slightly lower in rural areas18.
As Navarro et al17 points out, clinical information is needed to validate death certificates, thus implying the exclusion of those cases where this information is not available. Death certificates lacking complementary clinical or anatomo-pathological data could be of worse quality, as they probably include more home deaths. Navarro found that death certificates excluded for this reason belonged to subjects who were, on average, seven years older than those included in her study. All this may well lead to overestimation of the quality reported in many studies.
Several strategies have been proposed to motivate and improve physicians' certification28 such as a periodic assessment of coding practices along with the education and motivation of medical students and physicians. In Spain, several Regional Health Authorities implemented specific workshops that showed their efficacy in enhancing death certificate quality indicators32. Yet, these interventions are questioned by Swift's study33, which failed to find significant changes in the state of certification after the introduction of formal education into the medical syllabus.
A further point of discussion is the effect of the introduction of the ICD-10 on the quality of mortality data. To date, we have been unable to find any validation study in Spain covering the ICD-10 coding period. In 1999, Ruiz et al34 compared ICD-9 and ICD-10 coding in a huge sample of Spanish death certificates. They reported that, whereas ill-defined condition figures increased almost a 14% with the use of this latest version, neoplasms seemed quite stable. In contrast, our data indicate that the ICD-10 effect might be greater than thought, and that it has also affected tumours coding. This could suggest a worsening in the quality of data, and careful surveillance is thus called for.
In conclusion, the quality of cancer death certification in Spain for all tumours and all main sites has improved over the last two decades and can be considered comparable to internationally published data. Thus, mortality data constitute a valid indicator to estimate the burden of cancer. However, for some locations, such as the oesophagus or bladder, death certificate information should be approached with caution. Misclassification may generate problems for studying mortality trends and planning future needs. It should be noted that, in general, most available information on the quality of death certification reflects the situation from 1970 to 1990, when the ICD-9 was in use. The relatively recent introduction of the ICD-10 may have affected quality indicators and should thus be carefully monitored. Finally, our results point to the need to improve death certification in the case of Spanish women.
Acknowledgements
The work of V. Carvajal was supported by the Carlos III Institute of Health (Grant No. 03/0007). The work of B. Suárez was funded by the Network of Epidemiology and Public Health Research Centres, International Health & Co-operation Foundation (Grant No. C03/09).
The authors wish to thank Michael Benedict for his help with the English, and Isabel Izarzugaza for supplying Euskadi's information.
Correspondencia:
Beatriz Pérez-Gómez.
Área de Epidemiología Ambiental y Cáncer.Centro Nacional de Epidemiología.Instituto de Salud Carlos III.
Sinesio Delgado, 6. 28029 Madrid. España.
Correo electrónico: bperez@isciii.es
Recibido: 25 de noviembre de 2005. Aceptado: 22 de febrero de 2006.
