Another issue is that some concepts and individuals may not naturally belong in a single unique location in the ontology, being multiply classified by experts and/or auto- mated methods. While most ontology languages permit multiple inheritance, rampant multiple classification erodes the classificatory utility of the ontology and increases the computational complexity of population (see next section). We have attempted to finesse this issue by limiting concepts to a single appearance per subdiscipline while adding “semantic crosslinks” to the ontology. These crosslinks capture the relatedness of ideas deemed mutually relevant by feedback or automated methods yet which have been manually classified by experts in distant areas of the ontology.

The most serious form of this last worry centers on the semantics of the isa relation. The isa class subsumption relation is often presumed to hold in a context-general way and capture “purely ontological” relations; if it is true that “A isa B”, then any A in any context should by its nature be a B. Because our idea category is organized primarily by contingent semantic relationships holding only in the context of certain philosophical discussions, its isa links may fail to hold domain-generally and may conflate taxonomic and non-taxonomic relations.

We take a computational ontology to be a directed acyclic graph where nodes represent concepts and the links between concepts represent the taxonomic “isa” relation (e.g. in the graph where ‘Wine → Red Wine → Brunello di Montalcino, everything that “is a” instance of Brunello di Montalcino “is a” instance of Red Wine, and everything that “is a” instance of Red Wine “is a” instance of Wine (we are less strict as to whether this subsumption relation must hold in all contexts of discourse—see Sect. 6.2). A knowledge base is an ontology that has been populated with individuals; mathematically, knowledge bases contain another kind of link denoting the “instance of” taxonomic relation, and a new kind of node denoting individuals (with the restriction that no individual can have any children). Thus we may populate our toy ontology with an individual, ‘Campogiovanni Brunello’, which is an instance of the concept Brunello di Montalcino. A computational ontology will also contain declarations for a number of non-taxonomic relations, which can either hold between individuals and constants (properties) or between two (or more) individuals (relations). Instances of these non-taxonomic relations may also be encoded in the process of ontology population. For example, a knowledge base might record the property ‘alcohol_content (Campo- giovanni Brunello, 7%)’ and the relation ‘pairs_with (Campogiovanni Brunello, Parmigiano Reggiano)’. Finally, a computational ontology may also contain a number of inference rules and axioms that can be used to reason about objects in the domain (e.g. infer the presence of certain relations on the basis of others, enforce consistency and default properties, and so forth).

Traditional methods for obtaining a description of the intellectual space of the discipline take the form of a less than a handful of philosophers testing each other's intuitions about ways to organize the content of the discipline. There are many reasons to be skeptical about this approach. First, there is the matter of individual bias; philosophers are likely to feel that the particular issues they work on are the most important. Second, the learning histories and intellectual trajectories which shape one's metaphilosophical standpoint are idiosyncratic and politically charged. Third, the expert's drive for simplicity and elegance comes at a cost: it may tempt philosophers to artificially impose a normalized structure on distant areas of philosophy. Fourth, once settled on an approach or a set of organizing principles, overconfidence and confirmation biases may set in, leading the philosopher to feel the chosen approach is appropriate in diverse settings. And finally , given the explosion in the number of significant philosophical publications mentioned in the introduction, it is unlikely that even a reasonably-sized subset of extremely well-read philosophers would be sufficiently qualified in all areas of philosophy to produce a thorough conceptualization.

Many of the automated metadata management tools available today operate primarily on term co-occurrence statistics. Term co-occurrence approaches attempt to recover semantic information about terms from the textual context in which they appear (whether it be sentence, paragraph, or entire document). As anyone who has used a search engine can attest, however, co-occurrence information alone is often not enough to intelligently infer semantic relevance. [...] Rather than searching for a fully-automated solution to our metadata needs, we seek to utilize the SEP’s most valuable resource—regular access to domain experts in philosophy. Our approach to dynamic ontology begins with a small amount of initial manual ontology construction. Once the initial structure is in place, a variety of automated methods are used to structure feedback solicitation forms and deploy that feedback in data validation, ontology population, and the semi-automatic extension of its taxonomic structure.

In numerous places in the expert-supplied taxonomies, we also noticed that cat- egory names sometimes included a “philosophy of” prefix and sometimes did not; examples include “artificial intelligence” vs. “philosophy of artificial intelligence,” “connectionism” vs. “philosophy of connectionism,” and so on.

In short, scholars and students don’t just need the reference works—they also need the means to search and navigate them effectively. To preserve the utility of encyclopedias as they grow, we must also improve our ability to represent their contents in meaningful ways accessible to novice and expert alike. The dynamic nature and increased scale of digital reference works, however, render traditional editorial methods of gathering and organizing metacontent (indices, cross-references, tables of contents) so resource-intensive and inefficient as to be practically inapplicable.

Let us call a metaphilosophical position descriptively adequate if it portrays the discipline as it is currently practiced in such a way as to support practical applications. Let us call a metaphilosophical position normatively adequate if it outlines the best way to arrive at philosophical truth, provides philosophy with a firm epistemological grounding, or otherwise describes how philosophy ought to be organized to maximize our chances of progress. While we do not defend the InPhO's normative adequacy , we believe that the collaborative, distributed, and empirical approach used to construct it makes it more likely than standard alternatives to produce a representation of the discipline which is descriptively adequate.

(1) There is no one correct way to model a domain-there are always viable alternatives. The best solution almost always depends on the application that you have in mind and the extensions that you anticipate. (2) Ontology development is necessarily an iterative process. (3) Concepts in the ontology should be close to objects (physical or logical) and relationships in your domain of interest. These are most likely to be nouns (objects) or verbs (relationships) in sentences that describe your domain.

[by Noy and McGuinness (2001)]

The most common problematic category names were those which seemed to be grab bag categories. A 'grab bag' category here is understood as one which groups together possibly heterogeneous elements which are 'leftovers' from more significant sibling decompositions [...] The frequency of such categories in the taxonomies supplied by experts suggests to us that, rather than simply demonstrating a lack of a label due to ‘laziness’, they seem to be doing some real classificatory work. Unfortunately, it isn’t clear how to replace these categories with something more informative and statistically tractable, even when they are considered on a case-by-case basis. The problem is not crippling for our methods where the grab bag category decomposes into more meaningful sub- categories; where they do not, however, we have excluded them from the current build of the ontology

Another important aspect of ontology design concerns the drawing of a line between categories and individuals. For instance, should connectionism be considered an instance of the category philosophy of articial intelligence, or should it be treated as a subcategory with instances of its own? [...] there can be no right answers without considering the intended applications. We have been guided by a rough rule of thumb that we should approximate a one-to-one correspondence between the most specific titles of SEP articles and individuals in the ontology; thus, whether connectionism will be a category or individual will depend on the amount of treatment it receives in the current version of the SEP.

Another, more focused line of objection has come from those who prefer to reserve the use of the term for a particular kind of a formal representation used in the over- arching project of the semantic web. These critics tend to focus on our idea category, indicating that the kind of semantic taxonomy expressed there is somehow not properly described as ontological. There are a number of things one might note here: it isn’t clear how the semantic divisions we focus on will enhance automated reasoning, or how the category of philosophical ideas will plug into higher-level ontologies to ensure interoperability

The purpose of such an ontology is to assist humans and automatic agents in understanding the contents of the domain (especially in terms of properties, relations, and subsumption/inheritance relationships which hold between the domain’s types) and to allow data generated in one project to be interoperable with others.

Perhaps the most noteworthy aspect of our approach is its classification of ideas according to semantic inheritance relationships holding between the contents of ideas rather than more formal inheritance relationships observed in their types (e.g. social or structural roles). [...] Moreover, we prefer a decomposition focusing on taxonomic structure that is as stable and non-controversial as possible, and thus would be familiar to most of the SEP’s authors and editors. No widely-accepted social/structural decomposition of philosophical ideas currently exists, so one would have to be engineered. For these reasons, we chose to focus instead on a decomposition classifying ideas according to their locations in the semantic space of the discipline. Thus, our category philosophical idea breaks down into idea about epistemology, idea about metaphysics, idea about ethics, idea about logic, idea about philosophy of mind, and so on. (Note that hereafter, the “idea_about...” prefix is omitted from all category names, and should be implicitly assumed to avoid confusion.)

Information about the “idea type” is nonetheless useful for inferential purposes. For example, knowing whether an idea is of the type position or of the type distinction can constrain the types of relationships philosophers can have to it. The InPhO thus repre- sents this information as non-taxonomic relations (e.g. ‘is_idea_type(connectionism, position)’ ).

The innovative nature of the work, however, brings with it a host of new difficulties not faced by traditional encyclopedias. It is increasingly impractical, for instance, to have editorial staff manually manage cross-references, tables of contents, search keywords, and other metacontent due to the asynchronous submission and revision of articles. There is also a pressure to minimize the editorial burden placed on volunteer contributors, who cannot be expected to constantly monitor the massive, ever-changing contents of the SEP and update metadata themselves.

Some readers may be skeptical of our statistical approach, supposing that we are engaged in a very dubious metaphilosophical methodology: to be arguing that because two terms frequently co-occur in the SEP , they ought to be seen as mutually relevant. We want to make clear that we do not take the InPhO to show how philosophy ought to be organized. Such an ideal conceptualization is neither recoverable by currently- available automated means nor required for our purposes. Even human experts in metaphilosophy would be hard-pressed to produce a comprehensive decomposition of the conceptual space of philosophical ideas, and no doubt any such scheme would be the subject of controversy. However, we have real and pressing information management needs, and, for the purpose of meeting those needs, having an imperfect formal representation of the structure of the field is better than having none at all.

How much semantic information about terms is recoverable from co-occurrence information alone, and how much, instead, can only be recovered by agents possessing human perceptual apparatus, physical or causal interaction with the subject domain, memory , emotions, human cognitive dispositions, neural architecture, and so on? Despite great advances in the last few years on both co-occurrence models and other models of semantic structure, and on language learning and semantic memory , no general answer to this question is currently known. [...] Thus far, we have discussed two ways in which the InPhO is a dynamic ontology: the dynamic population of instances and the dynamic extension of the ontology' s category structure when sections become too large. An alluring future direction which could make the InPhO even more dynamic involves the use of more ambitious automated reasoning techniques to infer large sections of the ontology' s taxonomic structure using semi-automated means.

Encyclopedias have always occupied a precarious position in academia. On the one hand they taxonomize human knowledge and provide valuable entry points for scholars and students into the intellectual worlds of academic disciplines, covering their subject matters in more breadth and detail than could any single person or even any reasonably-sized university department. On the other hand, they carry a risk of congealing knowledge into a cold and quickly-obsolete imitation of living scholarship, stultifying the thought of beginners who might be better off wrestling with multiple, recent perspectives than the predigested orthodoxy of a designated expert. The tension in being encyclopedic is especially acute today, given the recent explosion in the number of universities, scholars, and academic publications. While the explosion has made faith- ful and succinct summarization even more elusive, encyclopedias have perhaps never been more relevant. If we wish to prevent disciplines from disintegrating into collections of highly-technical cottage industries in which specialists speak only amongst themselves, the development and maintenance of reference works offering accessible, up-to-date summaries is imperative.

In the design, implementation, and long-term deployment of computational ontologies, knowledge modelers face several enduring challenges. For one, computational ontologies have often been designed without sufficient logical rigor, which may come with pragmatic costs in terms of the expressive power, clarity, and interoperability of the scheme (Guarino 1995; Smith 2006; Arp, this volume). The economics of ontology design is also a problem, generally requiring significant time from scholars specially-trained in both the target domain and the principles and methods of computational ontology design (hereafter, “double experts”). Obsolescence looms large, as change in the problem domain or our understanding of it can render all that design effort useless, in the best case requiring more time from double experts to manually evolve the ontology (Flouris et al. 2006; Ceusters and Smith 2006) and in the worst cases taking a project back to the drawing board.

The first line of criticism comes from those who believe the word ‘ontology’ has been stretched too far from its original metaphysical roots, feeling that it is more misleading than useful when referring to a formal data representation rather than an actual hierarchy of being. This group of critics—which has included at least one lexically-conscious information scientist, as well as philosophers—tends to object to the use the term has acquired in the computer and information sciences in general.

Such a system would facilitate interaction with the encyclopedia for expert and novice users alike. Experts could see the “big picture” of an idea or philosopher in a glance, capturing a large amount of information in an efficient manner, and novices could surf through the encyclopedia’s content by following paths hewn by an expert- level understanding of the domain. Students engaged in research could instantly see which other topics are importantly related to their current search item, and the seman- tic labels of those links would tell them not only which other items are related to their current search but also how. [...] What can be visualized, and what visual metaphors guide our comprehension of philosophy? These are questions whose answers affect the development of philosophy as a discipline in the digital age.

A key challenge facing philosophers in the digital age is to discover how best to use computers to support our understanding of the discipline. As should be clear by now , we do not expect our text processing tools to write a compelling philosophy paper any time soon. Rather, we recognize what has long been true: that humans and computers work better together than either do in isolation. With careful collaboration, reliable represen- tations of the discipline can be created facilitating a wide range of future tasks in digital philosophy (several discussed below). Keeping future tasks such as the design of visu- ally-effective conceptual and thematic navigation tools in mind, we have tried not to 'OVERFIT' our methods and ontology to the present needs of the editorial staff of the SEP.

After all, it isn't like philosophers to allow standards or accepted truths to stifle such progress; in fact, the history of philosophy shows that stating something precisely and making it widely available is instead an invitation to more intelligent critique. Given the contentious nature of philosophy and its practitioners, therefore, it seems more likely to these authors that codifying philosophy's self-conceptualizations will enhance metaphilosophical critique rather than nullify it.

An advantage of our expert-supervised approach to ontology population is that it provides ample quality-control while still allowing for the possibility of novel discovery. Our co-occurrence statistics have turned up a number of connections that might easily have been overlooked by area experts. To take one example, the methods ranked anaphor as one of the highest hypernym candidates for propositional attitudes. Though two of the authors of this article are philosophers of mind, we initially thought this connection was due to error. A quick SEP search revealed, however, that the ranking could be explained by the fact that anaphoric sentences pose a challenge to the Fregean theory of propositional attitudes. This interesting connection would likely not be discovered by novices; and though experts might not think of the connection off the tops of their heads, they can easily uncover its validity with a cursory inspection of the relevant SEP articles.

Our approach to representing the discipline derives from what computer and information scientists call “formal ontology.” However, we will avoid using this term because some researchers sensitive to Husserl’s distinction between “formal ontology” and “material ontology” prefer to reserve the former for a stricter kind of ontology than is satisfied by some aspects of our approach (Poli 1995; and see 6.2 below). Instead, we prefer to refer to our representation as a “computational ontology” and “dynamic ontology.”

We concede this last point; the classifications under the idea category will likely not hold domain-generally. For these divisions, we propose that the isa relation be under- stood to express conditional subsumption: A isa B, but only in the context of C, ..., Z, B’s ancestors. Let us call this conditional subsumption relationship ‘isa’.29 Rather than supposing that this prevents the InPhO from being a proper ontology, we think the move from isa to isa helps capture precisely the sort of information required for our metadata needs. We believe that this sort of conditional, hierarchically-structured knowledge is important in modeling the ability of philosophical experts to say which topics are most relevant to the examination of particular ideas in particular scholarly contexts. [...] While these observations clarify the issue, it may remain a point of dispute whether a representation based on the isa* relation is properly called an ontology. Our general response to the terminological worries of this section is to acknowledge the need to regiment language for purposes of clarity and precision, but note that all we mean by “ontology” and “dynamic ontology” is precisely what we say in Sect. 2 above. [...] Though we would be reluctant to give up the word “ontology” entirely, readers suffi- ciently bothered by this issue may call the idea section of the InPhO what they like—we place no special importance on its terminological status.

One broad response to these challenges [of knowledge modeling], emphasized by the “formal ontologists,” is to attempt to produce a “once and for all” description of the underlying reality of the subject domains, and to link the types of those subject domains into a standardized upper-level ontology describing the most basic, enduring features of reality. While this approach can hope to minimize the amount of change needed in future iterations, when change is called for it is usually performed manually. Another approach might be characterized by the phrase “dynamic ontology.” On this approach, more effort is placed on automating as much of the design and evolution process as possible rather than on attempting to produce a final description in the initial stages of a project.

Viewed against these challenges, our semi-automated approach may be thought to possess significant empirical advantages over this traditional method. Granted, our reliance on expert conceptualizations for the InPhO’s basic framework renders us vulnerable to some of these worries—but we do not ask experts to represent any- thing outside of their own area of specialization. Furthermore, our feedback system is deployed with redundancy in mind. We propose the same hypotheses to multiple experts in the feedback process, and the nonmonotonicity of our logic programming methods allows us to flexibly respond to expert disagreement. OUR hope is that this will afford the InPhO a degree of intersubjectivity. [...] We record not just what philosophers think they are writing about, but also study statistical properties of their actual output .

There seems to be no way around the conclusion that large-scale paradigm shifts in philosophy will require fresh re-conceptualizations of the Þeld from the top down. Rather than being fatalistic, however, we can note the advantages of the 'dynamic ontology' approach to these issues, for which ontology evolution can be seen as a natural extension of the initial process of ontology creation and population. First, re-conceptualizations which result from large-scale paradigm shifts may not have to be coded entirely from scratch; much of the old ontology may be reincorporated or used as inspiration where relevant, and metrics about the unsuitability of the obsolete ontology may guide the creation of the new one. Second, smaller changes to the ontol- ogy are tractable by modest semi-automatic means. For example, we will be able to automatically detect when some categories become too large, and then use automated methods to divide the contents of the category into two or three more closely-related clusters;