Consistency of Epidemiologic Estimates

Ó Springer 2005

European Journal of Epidemiology (2005) 20: 827–832 DOI 10.1007/s10654-005-2227-9

METHODS

Consistency of epidemiologic estimates Jan J Barendregt1,2 & Alewijn Ott3 1

School of Population Health, University of Queensland, Herston, Australia; 2Department of Public Health, Erasmus MC, University Medical Center, Rotterdam, The Netherlands; 3Department of Medical Microbiology & Infectious Diseases, Erasmus MC, University Medical Center, Rotterdam, The Netherlands Accepted in revised form 16 August 2005

Abstract. Background: The epidemiology of a disease describes numbers of people becoming incident, being prevalent, recovering, surviving, and dying from the disease or from other causes. As a matter of accounting principle, the inflow, stock, and outflows must be compatible, and if we could observe completely every person involved, the epidemiologic estimates describing the disease would be consistent. Lack of consistency is an indicator for possible measurement error. Methods: We examined the consistency of estimates of incidence, prevalence, and excess mortality of dementia from the Rotterdam Study. We used the incidence and excess mortality estimates to calculate with a mathematical disease model a predicted prevalence, and compared the predicted to the observed prevalence. Results: Pre-

dicted prevalence is in most age groups lower than observed, and the difference between them is significant for some age groups. Conclusions: The observed discrepancy could be due to overestimates of prevalence or excess mortality, or an underestimate of incidence, or a combination of all three. We conclude from an analysis of possible causes that it is not possible to say which contributes most to the discrepancy. Estimating dementia incidence in an aging cohort presents a dilemma: with a short follow-up border-line incident cases are easily missed, and with longer follow-up measurement problems increase due to the associated aging of the cohort. Checking for consistency is a useful strategy to signal possible measurement error, but some sources of error may be impossible to avoid.

Key words: Bias, Dementia, Epidemiology Abbreviations: CI = Confidence interval; WHO = World health organization

Introduction The epidemiology of a disease can be described by a large number of variables. Examples are incidence, prevalence, duration, remission, survival, mortality, and case-fatality. While these variables each describe a different aspect of the epidemiology, they all describe the same disease process, and therefore they are not independent. For example, a high prevalence combined with a high case fatality will cause a high mortality. Incidence, case fatality and remission together determine the prevalence and mortality of a disease. These epidemiological variables are embedded in a causal framework, and are thus closely interrelated. The values they can take on are constrained by the values of the other variables, working through the causal relations. Consistent estimates are defined to comply to the constraints imposed by the causal framework that describes the disease process. For most diseases epidemiological data come from various sources: mortality typically from national cause-of-death statistics; many countries have cancer

registries allowing estimation of cancer incidence; survival figures come from follow-up studies, etc. With data from various sources and source populations inconsistencies are to be expected. But even within a well-defined study population consistency of estimates is not guaranteed. Some estimates are harder to get than others, resulting in more measurement error. For example, in a slowly emerging and progressive disease like diabetes, case definition depends on exceeding a threshold value (serum glucose). When diagnosing new diabetes (incidence) one will inevitably encounter borderline cases whose diagnosis is uncertain. Therefore, measuring incidence is harder than measuring prevalence, which includes a higher proportion of well-established and severe cases. Sometimes inconsistency between prevalence and incidence is striking and can easily be recognised [1, 2]. But often inconsistencies are more subtle, and can be found only by explicitly checking for them. Examining consistency can be done by relating empirical estimates to a formal model of the disease process. In this paper we illustrate this approach using a generic disease model (DisMod II) and

828 dementia data from the Rotterdam Study, and discuss possible explanations for our findings.

Data and methods The Rotterdam Study is a prospective study among residents of 55 years and older, living in the Rotterdam suburb of Ommoord. Between 1990 and 1993, 7,528 subjects were first screened for dementia [3]. These subjects were followed-up for an average of 2.34 years to establish all cause mortality [4]. In 1993 and 1994 a follow-up examination was done among 79% of the non-demented at base line: 7% had died and of the remaining 14% information was obtained from the participants’ general practitioners and medical records. Case finding was done using a three-step procedure. In a first step the Mini-Mental State Examination and the Geriatric Mental State Schedule were used. Screen-positive persons were then subjected to the Cambridge Examination for Mental Disorders of the Elderly. The third step consisted of neuro-imaging and clinical examinations. The diagnosis was established by an expert panel, using all available information, following the DSM-III-R criteria [3]. The base line measurement yielded an estimate of prevalence by sex and 5-year age group (Table 1). We calculated 95% confidence intervals assuming a binomial distribution [5]. Excess mortality was expressed as death rate ratios of demented over nondemented, by sex and 5-year age group with 95% Poisson confidence intervals (Table 2). A second follow-up examination took place in 1997–1999. Both follow-up examinations and reviewing general practitioners alerts and medical files yielded an estimate of incidence, again by sex and 5-year age group with 95% Poisson confidence intervals (Table 3) [6]. To check for the consistency of these estimates we used a generic causal disease process model (Figure 1). In this model healthy people (in this case defined as ‘non-demented’) are subject to an incidence hazard, or at risk of disease. When diseased they are subject to a case fatality hazard, or risk to die from

the disease, and a remission hazard, corresponding to recovery [7]. Both healthy and diseased people are subject to mortality risk by other causes. When this ‘other causes’ mortality hazard is the same for diseased and healthy people, i.e. when case fatality is defined to contain all excess mortality from the disease, this model is completely determined by incidence, remission, and case fatality. The model (implemented as a software package called DisMod II, running on Windows 95 and higher) is available from the website of WHO (www.who.int/evidence/ dismod). Dementia is a progressive disease, but because of misclassification remission may appear to be greater than zero. However, we assumed the remission hazard to be 0 because the Rotterdam study database was corrected whenever a misdiagnosis was discovered. Case fatality was calculated from the total mortality for the Dutch population 1990–1994, and the observed prevalence and mortality rate ratios using the following equations. Excess mortality (m) is equal to the prevalence of dementia (p) times the case fatality (C): m ¼ pc Then: m C¼ p ¼

EM p

¼

MðR
1Þ pðR
1Þ þ 1

with E: population attributable mortality risk of dementia; M: total mortality rate in the population; and R the mortality rate ratio of demented over nondemented. We used the observed incidence and calculated case fatality as inputs to the model. As the model requires single year age groups, we applied cubic spline

Table 1. Prevalence of dementia (rates per 1000), and 95% CI, The Rotterdam Study [3] .Women

Men

Age

Rate

95% CI

55–59 60–64 65–69 70–74 75–79 80–84 85–89 90+

5.8 3.7 9.5 21.1 62.0 192.9 326.9 405.7

1.5 0.7 3.8 11.8 44.1 158.7 279.5 340.6

12.9 9.1 17.8 32.9 82.7 229.5 376.1 472.4

Rate

95% CI

2.0 4.8 8.0 20.3 60.3 137.3 284.3 411.8

0.0 0.9 2.5 9.7 38.2 93.6 201.4 254.4

7.9 11.7 16.5 34.6 87.0 187.7 375.4 579.0

829 Table 2. Excess mortality of dementia, rate ratios demented/non-demented and 95% CI, Rotterdam study [4] Women

Men

Age

RR

95% CI

55–64 65–69 70–74 75–79 80–84 85–89 90+

13.3 40.0 7.5 6.2 3.6 2.2 1.4

0.3 7.0 1.4 3.1 2.3 1.5 0.9

84.0 160.6 25.6 11.5 5.6 3.1 2.0

RR

95% CI

12.0 0.0 2.8 2.9 1.9 1.6 3.2

0.3 0.0 0.3 1.1 0.9 0.8 1.3

74.0 11.0 10.9 6.5 3.4 3.1 8.7

The higher limit for women 65–69 was erronously reported to be 60.6.

Table 3. Incidence of dementia (rates per 1000), and 95% CI, Rotterdam study [6] Women

Men

Age

Rate

95% CI

55–59 60–64 65–69 70–74 75–79 80–84 85–89 90+

0.0 0.5 1.4 4.8 14.4 23.4 44.4 64.9

0.0 0.4 1.3 4.2 11.3 17.3 29.4 39.4

0.0 0.5 1.6 5.5 18.2 31.5 67.1 107.0

Rate

95% CI

0.9 0.6 2.1 5.4 16.9 21.5 45.1 24.3

0.8 0.6 1.9 4.7 13.1 16.1 29.7 17.9

0.9 0.6 2.3 6.2 21.9 28.6 68.4 32.9

Figure 1. Conceptual disease model with four states (healthy, diseased, dead from disease, and dead from other causes) and four transition hazards (incidence, remission, case fatality, and an other mortality hazard).

interpolation to convert the 5-year age group rates [8]. From the incidence and case fatality we calculated a corresponding, predicted prevalence, and compared this to the observed prevalence. We used parametric bootstrapping on the excess mortality rate ratios and incidence rates to calculate confidence intervals for the predicted prevalence. To estimate confidence intervals for the difference between observed and predicted prevalence we additionally bootstrapped from the binomial distribution of observed prevalence [9].

Results Figure 2 shows observed and predicted prevalence and 95% confidence intervals by age for women. In all age groups but one predicted prevalence is lower than the observed, and for two age groups (80–84 and 85–89) the confidence intervals even do not overlap. Table 4 (left panel) shows the difference between observed and predicted prevalence, with 95% confidence intervals. In four age groups the difference was significant (p < 0.05).

830

Figure 2. Observed and predicted prevalence of dementia by age, women, with 95% confidence intervals.

Table 4. Difference between predicted and observed prevalence (rates per 1000), with 95% CI. Females

Males 95% CI

Age 55–59 60–64 65–69 70–74 75–79 80–84 85–89 90+

95% CI

Point

Lower

Higher

Point

Lower

Higher

)1.60 )0.04 )5.62 )10.92 )23.68 )107.39 )160.52 )82.37

)8.66 )5.79 )14.13 )23.51 )46.08 )146.92 )219.10 )172.80

2.64 3.23 0.39 )1.07 )4.11 )68.33 )96.64 25.36

1.09 0.41 1.77 3.02 2.01 )16.65 )56.14 )170.75

)4.85 )6.58 )10.96 )21.79 )36.08 )83.78 )160.80 )347.19

3.09 4.44 6.94 12.25 19.82 28.90 33.43 )4.58

Figure 3. Observed and predicted prevalence of dementia by age, men, with 95% confidence intervals.

831 In Figure 3 and Table 4 (right panel) the corresponding results for men are given. For the 3 highest age groups the predicted prevalence is lower than the observed, but only for the 90+ age group the difference reached significance.

Discussion In the Rotterdam study large effort was made to estimate the epidemiology of dementia as accurate and complete as possible. Yet for most age groups, and in particular the higher age groups with large prevalences, the predicted prevalence consistent with the observed incidence and excess mortality is systematically lower than observed prevalence. For some age groups, more for women than men, this difference is statistically significant. In the following, we discuss several possible explanations, both in connection to the model and with the estimation of incidence, prevalence, and excess mortality itself. First, the model we applied to the data may not be valid. There is little cause for concern about the formal validity (analytically derived from the differential equations that describe the transitions in Figure 1), but the model assumes the markov property for survival, i.e. case fatality is not dependent on time since onset of disease, but on age. However, we tested the model using an artificial cancer data set, known to be consistent, where survival was lognormally distributed [10]. Despite this decidedly non-markovian survival the model was able to reproduce the prevalence at each year of age to within 1% of what it should have been. Since dementia survival adheres closer to the markov property than cancer survival, we expect the performance of the model in this respect to be better still. Another reason for apparent inconsistencies may be shifts in the transition hazards. Prevalence is a stock variable: currently observed prevalence is the accumulation of incident cases from the past, who still survive. If incidence in the past has been higher, current prevalence would partly consist of survivors from the previously higher incidence, and therefore would be higher than the prevalence consistent with current incidence. A similar argument can be made for increasing case fatality. In order to test whether time trends could explain the apparent inconsistency we recalculated the prevalence with trends in incidence and case fatality. We could reproduce the observed prevalence only when assuming strong trends. For example, an annual 10% decrease in incidence for the past 5 years, giving a total decline of 41% over the period, would yield a predicted prevalence comparable to the one observed. Extending the trend period further back in time has little or no effect, because the case fatality of dementia is relatively high, limiting the number of survivors and

therefore the impact on prevalence. A 41% decline in dementia incidence over 5 years seems very unlikely, however there is a possibility that (a small) part of the observed inconsistency is due to time trends in dementia epidemiology. A third reason might be migration: if demented persons migrated into Ommoord, this could have inflated the prevalence estimate. Ommoord has a number of institutional homes for the elderly (included in the study) with psycho-geriatric wards where most demented persons live. Though these homes primarily cater to the local Ommoord population, the relative high proportion of residents of these homes in the study population may be responsible for some immigration and thus part of the inconsistency. However, no psycho-geriatric nursing home was included in the study. These institutions mostly admit severe cases. The absence of a psycho-geriatric nursing home will result in some outmigration of demented persons. The net effect of this in- and outmigration is unclear, and most likely small. However, as mild to moderate dementia is more prevalent than severe dementia, requiring admission to a nursing home, migration may have resulted in a relatively high prevalence. Then there are several possible explanations for the inconsistency in the procedure of estimation of incidence, prevalence, and excess mortality. The lower than observed predicted prevalence could be explained by either an estimate of incidence that is too low, or of prevalence and/or excess mortality that is too high. The excess mortality estimate seems an unlikely candidate for measurement error: estimating all cause mortality is easy. However, as can be seen from Table 2, in the younger age groups (