Theories and Diagnoses

Last time I asked for requests, a couple readers suggested that I write about the theory-theory. I always have mixed feelings about writing about theory-theory. On the one hand, I'm a big theory-theory fan, so I like to spread the good word, but on the other hand, theory-theory is notoriously difficult to describe, so I'm also a little reticent. So it's taken me a while to get to it. The theory-theory began as a theory of concepts, loosely (or at least opaquely) described in a paper by Greg Murphy and Doug Medin titled "The Role of Theories in Conceptual Coherence"1. Bot the gist of and motivation for that paper are captured in this quote from Ahn and Luhmann2 (emphasis mine):

When cognitive psychologists first started studying concept learning, they relied on artificial stimulus materials that were not meaningful to participants in their studies. The idea was that if we want to study how people acquire novel concepts, we should use completely novel categories, thus controlling for the influence of people's already possessed concepts. In the last 20 years or so, however, many researchers have argued that this practice misses one of the most important components of concept learning processes. Concepts are not represented in isolation, but are instead linked to and defined in relation to other concepts. In order to understand the concept of "shoes", for instance, we need to understand the concept of legs and walking. Therefore, it is no surprise that people have a strong tendency to apply their existing knowledge when learning a new concept because that new concept must be embedded in a complex network of existing knowledge. This emphasis on the influence of existing knowledge on concept learning has been called a theory-based approach to concept learning. The name theory-based is derived from the idea that our existing knowledge is represented like scientific theories such that concepts are causally related to each other and there are explanations underlying what we can directly observe. (p. 277)

Reread that last sentence. The insight therein includes both the strength of theory-theory, in its recognition of the interrelatedness of concepts, and its biggest problem: the description of how they are interrelated ("represented like scientific theories") is too vague to be of much use empirically. Hence the difficulty in describing theory-theory.

This difficulty hasn't stopped theory-theory from spreading out of concept research proper into other areas of cognitive psychology, most notably "theory of mind," which is so named because of its connection to theory-theory. In theory of mind research, the theory-theory states, in essence, that our knowledge of other people's thoughts and behaviors is represented "like scientific theories such that concepts are causally related to each other and there are explanations underlying what we can directly observe." In other words, it's just the theory-theory of concepts used in a particular conceptual domain: thinking about other people's thoughts. I suspect that, when people requested that I talk about theory-theory, it's this application that they wanted to hear about. But I study concepts, so my first inclination is to talk about theory-theory at a more abstract level. And really, it's in concept research that the theory-theory has itself begun to cohere in an empirically viable way in the last few years. By way of compromise then, I thought I'd start a discussion of theory-theory with a post on an example of the use of theory-theory that bridges the more general application to concepts at large and the more specific application to theory of mind. It's a bit backasswards, for me at least, to begin with an example and then move to the theoretical discussion, but I think it might just work in this case. So, here goes.

A Brief History of the DSM

For those of you who don't know, the Dagnostic and Statistical Manual of Mental Disorders (DSM) is the nearly universally recognized manual for the diagnosis of mental disorders. If you are interested in the history of psychology, then you will probably find the history of the DSM fascinating (see here for a short history, and here for a slightly longer one), because it encapsulates many of the trends and debates in clinical psychology and psychiatry over the last half century. The most widely known debate is probably that over the inclusion of homosexuality as a mental disorder, but more important than debates over individual disorders was the long debate over the very philosophy that would shape the diagnostic manual. The first version, the DSM-1 (strangely enough), was published in 1952, and compiled mainly from data gathered from surveys of members of the American Psychological Association (APA). In the 1950s, the APA was dominated by psychoanalysts who adhered to the psychodynamic approach. So as you might imagine, the DSM-I was heavily influenced by that approach. A revised edition, the DSM-II, was published in 1968, and was also heavily influenced by the psychodynamic approach.

But during the period between the first and the second, changes were already taking place in clinical psychology and psychiatry. The medical model was slowly gaining a foothold, and psychodynamic theory was losing popularity. The DSM had also been highly criticized for its lack of scientific validity, which led to a lack of reliability in diagnoses. So, when the APA commissioned a third version of the DSM, they required that it be based on solid scientific research, including massive studies of statistical reliability, and also that it be "atheoretical." By atheoretical, they meant that it should take "a descriptive approach that attempted to be neutral with respect to theories of etiology"³, because at the time most disorders did not have widely agreed upon causes. Furthermore, the new addition was to be committed to a family resemblance approach to representing mental disorders. Here is how Ahn and Kim describe this approach⁴:

Another significant shift in the assumptions underlying the representation of mental disorders is that starting from the DSM-III, a probabilistic model has been explicitly adopted. According to this model, boundaries of categories are fuzzy and categories do not necessarily have defining features. For example, the prototypical patient with Schizophrenia has five symptoms, but a presenting patient need only have two of those five symptoms for a diagnosis of the disorder. In some sense, this is a natural consequence of taking the atheoretical symptom-based approach, because although patients with the same disorder might share a common underlying cause, the manifestations at the symptomatic level might vary depending on individual patients' genetic tendencies and/or life experiences.

In 1980, the DSM-III was published, followed by a major revision called the DSM-IIIR, published in 1987, and finally the current version, the DSM-IV, published in 1994 (which has been revised many times since it was first published). Each of these versions has adopted the same basic, atheoretical, family-resemblance approach to representing and diagnosing mental disorders. The process of compiling the DSM-V is just getting underway.

The Theory-Theory and Clinical Diagnoses
Given the philosophy of defining and diagnosing mental disorders in the DSM-IV, an important question for both clinical training and further revisions of the DSM is, do clinicians actually make atheoretical diagnoses using DSM-like prototype representations? If this is the case, then as Kim and Ahn⁵ note, clinicians should weigh each diagnostic criterion equally, without considering causal relations between them. In this case, the probability of a clinician diagnosing a patient with disorder X when that patient exhibits 4 out of 6 of the diagnostic criteria for disorder X will be the same regardless of which 4 criteria the patient exhibits.

On the other hand, if clinicians' representations of mental disorders are theory-like, consisting of not only features (diagnostic criteria), but also beliefs about the causal and other relations between those features, then those relations between features may influence their diagnosis. Kim and Ahn describe two ways in which the theory-like structure of mental disorder representations might influence diagnosis:

• The Causal Status Effect: "The causal status effect occurs when
  features causally central to an individual's theory of that category
  are treated as more important in categorization than less causally
  central features. For instance, if Symptom A causes Symptom B in
  a clinician's theory, then A is more causally central than B, and A
  is thereby predicted to have greater diagnostic importance than B." (p. 453)
• Relational Focus: In analogical reasoning, people tend to focus on relations between objects (and other relations) over the objects themselves and their attributes⁶. For example, in Rutherford's analogy between the atom and the solar system, the relation REVOLVES AROUND receives more focus than the objects involved in that relation (the sun and the planets, or the nucleus and electrons), or the attributes of those objects, such as the size of the objects. If clinicians' representations of mental disorders are theory-like, then the relations between features may be treated as more important in this way.

The causal centrality of a particular feature of a concept can be determined using the following formula (from Sloman et al.)⁷:

In the formula, C_i,t represents the causal centrality in a conceptual representation of feature i at time t, which is the sum of the co-occurrence (dependence) of feature j and feature i, represented by d_ij, multiplied by the causal centrality of feature j, represented by c_j,t, for all of the features in the representation. In other words, the causal centrality of a given feature is dependent on the co-occurrence of that feature with every other feature, along with the causal centrality of the other features.

Kim and Ahn derived two main predictions from the two potential influences of theory-like representations on diagnosis. First, the causal status effect predicts that clinicians will be more likely to diagnose patients with a particular disorder if the symptoms they exhibit are more causally central in the clinicians' representation of that disorder. Second, relational focus predicts that, while symptoms that are "causally peripheral" will have less influence on diagnoses than causally central symptoms, they will have more influence on diagnosis than symptoms that don't participate in any relations with other symptoms.

To test these predictions, Kim and Ahn conducted several experiments using the same basic methodology. For simplicity (ha!), I'll just describe their first experiment, and then summarize the results of the series of experiments. In the first of two sessions, eleven clinicians (psychologists) and 10 clinical trainees (PhD students in a clinical psychology program) were given the names of five disorders, along with a list of the DSM-IV's symptoms for those disorders, and asked to complete three tasks. First, they took the lists of symptoms and for each disorder they added any others they believed to be associated with those disorders, and subtracted those they believed were not actually associated with the disorders. They were then given the names of each disorder again, along with the lists of symptoms they had created in the first task, and asked to draw a diagram for each disorder with arrows between the symptoms, or groups of symptoms, indicating the causal relations between them. They then rated the causal strength of each connection on a 5-point scale. Here is a sample diagram (Kim & Ahn, Figure 2, p. 456):

In the third task, participants were given the names of each disorder and the list of symptoms they had provided in the first task, and for each disorder, they were asked to rate the probability of a patient having the disorder if he or she had all of the symptoms except one. They did this for each symptom of each disorder.

In the second session, participants were given ten descriptions of patients, two for each disorder. Each patient was described as exhibiting three symptoms of the disorder. The three symptoms were either all causally central, all "causally peripheral," or all not causally related to other symptoms. The centrality of each symptom was determined using the formula above applied to the data from the causal strength ratings and the co-occurrence rates from the second and third tasks in the first session. For each description, participants were asked to rate the probability that a patient with the described symptoms (and only those symptoms) had the disorder. Finally, after completing an unrelated task, participants were asked to recall as many symptoms as they could for each of the ten patients in the descriptions.

Recall the two predictions from the causal status effect and relational focus. Participants should rate the probability that a patient has a disorder as higher when the symptoms were all causally central than when they are causally peripheral or not causally related to other symptoms ("isolated symptoms"), and causally peripheral symptoms should result in higher probabilities of diagnosis than isolated symptoms. This is in fact what Kim and Ahn found (see the graph below, which is Kim and Ahn Figure 3, p 457). The pattern was the same for both clinicians and clinical trainees, though the trainees were more conservative in their diagnoses overall. Both types of participants were also more likely to recall causally central symptoms than causally peripheral or isolated symptoms in the memory task, further indicating that the causally central symptoms were more salient.

In several follow up experiments, they found analogous results, including for unfamiliar personality disorders (Experiment 4). They also rated patients described as exhibiting causally central symptoms as more typical of the particular disorder than those exhibiting peripheral or isolated symptoms (Experiment 2). In each case, the same pattern was found for clinicians (experts) and students (novices), indicating that from early in clinical training, diagnoses are affected by theoretical beliefs about the causal relations between symptoms.

The take home message of these studies is that when there are relationships between features and concepts, those relationships will influence our classification and conceptual reasoning, even when it is specified that they should not have any influence. The implications for the DSM are profound, in that the reliability of diagnostic criteria across different clinicians is partially dependent on the extent to which they agree about the causal relations between those criteria. Kim and Ahn found widespread agreement in causal centrality ratings for familiar disorders, but for unfamiliar pesonsonality disorders (e.g., schyzotypal personality disorder), there was some disagreement between clinicians about the causal centrality of particular features. This could be why personality disorders are both controversial and difficult to diagnose using DSM-IV criteria. However, recent research by Ahn et al.⁸ has shown that clinicians' confidence in their theories may influence the strength of the causal status effect. When clinicians are less confident in theor causal theories, in cases where there is no widespread agreement about the etiology of a disorder for example, the causal status effect is attenuated. This may imply that in practice, clinicians' diagnoses are more atheoretical than they were in the Kim and Ahn studies for some disorders (e.g., personality disorders), and thus reducing the potential of the theoretical beliefs on the reliability of diagnoses across clinicians.

For our purposes, these studies serve as an excellent example of what the theory-theory means for conceptual reasoning. Obviously, causal relations are not the only relations that can exist in concepts, and thus they do not exhaust the scope of the theory-theory, but as these experiments and others by Ahn and her colleagues have shown⁹, causal relations are of central importance in categorization and category use.

¹Murphy, G. L., & Medin, D. L. (1985). The role of theories in conceptual coherence. Psychological Review, 92, 289-316.
²Ahn, W., & Luhmann, C. C. (2004). Demystifying theory-based categorization. In L. Gershkoff-Stowe & D. Rakison (Eds.) Building Object Categories In Developmental Time (pp. 277-300). Mahwah, NJ: Lawrence Erlbaum Associates.
³American Psychiatric Association. (1994). Diagnostic and Statistical Manual of Mental Disorders (4th ed.). Washington, DC. Quoted in Ahn, W., & Kim, N. (2005). The Effect of Causal Theories on Mental Disorder Diagnosis. In W. Ahn, R. Goldstone, B. Love, A. Markman, & P. Wolff (Eds.) Categorization Inside and Outside the Laboratory: Essays in Honor of Douglas L. Medin (pp. 273-288). Washington D.C.: APA.
⁴ Ahn, W., & Kim, N. (2005). The Effect of Causal Theories on Mental Disorder Diagnosis. In W. Ahn, R. Goldstone, B. Love, A. Markman, & P. Wolff (Eds.) Categorization Inside and Outside the Psychology: General, 131(4), 451-476.
⁶ Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7, pp 155-170.
⁷Sloman, S. A., Love, B. C., & Ahn, W. (1998). Feature centrality and conceptual coherence. Cognitive Science, 22, 189-228.
⁸Ahn, W., Levin, S., & Marsh, J. K. (2005). Determinants of feature centrality in clinicians' concepts of mental disorders. Proceedings of the 25th Annual Conference of the Cognitive Science Society. Mahwah, New Jersey: Lawrence Earlbaum Associates.
⁹See Ahn, W., & Kim, N. S. (2000). The causal status effect in categorization: An overview. In D. L. Medin (Ed.) Psychology of Learning and Motivation, V. 40, (p. 23-65) New York: Academic Press.