Modelling spreading activation in Word Grammar

1. Brief description of the research (below)
2. Non-technical description (pages 2-4)
3. Full technical description (pages 5-12)

1. Brief description 

The proposed project is the second in a series. The first project (funded by ESRC) produced software (WGNet) for handling the network which, according to Word Grammar, constitutes a grammar; that project also applied WGNet in a small-scale analysis of English inflectional morphology. The proposed project will extend WGNet so that it can also model spreading activation in a larger network of morphological information. It will work in much the same way as Levelt's WEAVER++ model of speech production, allowing the user to modify a single general formula which will predict the activation of each node at each (notional) moment of time, on the basis of the activation of its neighbouring nodes. It will also display these activation levels on screen. The immediate aim is a model of inflectional morphology which can be used either in perception or in production; but the ultimate aim (for a later project) is to apply this model in parsing entire sentences.

2. Non-technical description 

The aim of this project is to build a computer program which will take a grammar as its database, display it on the screen as a network, and show 'activation' spreading from node to node in response to a query; the program will be called WGNet++, because it is built on an existing program called WGNet (web site at www.phon.ucl.ac.uk/home/dick/wgnet/wgnet.htm). 

Language as a network
Language is often thought of as a collection of rules - a grammar - and a list of words - a dictionary - but it can also be seen as a network of interconnected words, word classes (what used to be called 'parts of speech'), meanings and forms. This is actually a much more productive view because it explains some well-known facts about how we use language; for example, we sometimes aim at one word but say (or write) a closely related word - we aim for tram but say bus (similar meaning), pram (similar form) or train (similar meaning and form). This view is fairly uncontroversial when applied to vocabulary, but even the rules of grammar can be expressed as patterns in a network; for example, the rule for forming plural nouns is a cluster of relations between a general word class (Plural noun) and two forms, the word's stem and the suffix {s}. The same is true for the rules of syntax; for example, the link between a verb and its subject is a link in the network which 'carries' further links such as the word order. 
	The idea that language is a network of interconnected (linguistic) concepts is the basis for a number of theories of language structure including Word Grammar, which is the theory which will be used (and tested) in this project.  The network view is controversial among linguists, so this project will provide an important opportunity to evaluate it. 

Spreading activation
Psychologists commonly think of the human mind as a vast network of interconnected concepts. One attraction of this view is that it corresponds to the networks of neurons that are found in the brain, though nobody knows how close this correspondence is. Another attraction is that it allows a simple theory of how we think - i.e. how we exploit the network of knowledge that we call long-term memory. When we think about a concept, we 'activate' the node for it. In neural terms, activation involves electrical and chemical changes, but psychologists think of it simply as some degree of activation. We also know that when neurons are sufficiently active, they 'fire' and pass their activity to other neurons to which they are connected. The corresponding idea in psychology is that activation spreads, willy nilly, from one concept to its neighbours. This spreading activation is not under our conscious control, and may produce unwanted effects such as the speech errors mentioned above. 
	Spreading activation is highly relevant to our use of language in speaking/writing (production) and listening/reading (perception) because it helps to explain how we retrieve language items - how we decide which word we should use next in production, and which word we have just heard in perception. Basically, the next word to use is the one which is currently the most active, and what we have just heard is the currently most active word in our vocabulary. Arguably the same is true for syntax: the preceding words which are syntactically related to the current one are those which are most active in our minds.  We shall not be able to test this hypothesis in the present project, but we hope to do so in later work.
	In this project, activation will be shown numerically by a number which is attached to each node, and which the operator can adjust node by node. It will be possible to check the activation level of any node on screen, and to use some system such as colours to give an overview of the entire system at any given moment. As activation spreads from node to node, this will be displayed 'in slow motion', at a speed controlled by the operator.  The spreading of activation will be controlled by a single formula which applies in just the same way to every node. The formula will be built into WGNet++, but it will be designed as an experimental tool so that the operator can vary the parameters freely. For example, the operator will be able to decide what the 'firing threshold' for nodes should be, and what effect firing has on the fired node - does its activation return to zero, or to some higher level? 

Focussed activation
Although activation spreads in a completely blind and mechanical way, it obviously is not randomly distributed: we focus attention, and activation, on the concepts that currently matter to us. In language, this means that we focus on the words that we are trying to understand (perception) or to plan (production).  These words are temporary concepts which we can represent in WGNet++ by new network nodes. We know something about them - about their spelling or pronunciation (perception) or about their meaning (production), but no more. Our target is to work out the missing information - their meaning (perception) and their form (production) - so they constitute a problem and receive correspondingly heavy activation. 

Problem solving by classification and inheritance
	WGNet++ will contain a general-purpose problem-solving mechanism for dealing with this kind of situation: in order to 'enrich' a temporary concept, classify it - i.e. relate it to some permanent concept. This model concept is chosen by a Best Fit Principle which selects the permanent concept whose characteristics best fit the known characteristics of the temporary concept. Here too activation is relevant, because the best fit will almost certainly be with the currently most active concept, because the activation of the temporary concept will have spread via its known features to whichever permanent concept shares those features. For example, you read yelow, and decide that this temporary concept is an example of the permanent word this temporary concept yellow (in spite of the spelling mistake). The Best Fit Principle can produce two or more 'winners' that are compatible with each other - for example, yelower would be classified as an example of both yelow and 'Comparative adjective'.
	Once a temporary concept has been classified, it can be enriched by means of default inheritance. This is the logic of ordinary thinking, whereby one concept inherits the characteristics of another if the first is an example of the second - for example, if we know that a sparrow is a bird, and we know that birds lay eggs, we can infer that sparrows too lay eggs. That is, the concept 'sparrow' inherits the egg-laying from 'bird'. This logic allows exceptions, so higher concepts are only inherited by default; thus we know that birds fly, but we also know that some kinds of bird (e.g. penguins and ostriches) do not fly. In the case of the word yelow, we can classify it as an example of the permanent word yellow without ignoring the spelling mistake. 

The goals of WGNet++
WGNet++ will apply the processes of best-fit classification, spreading activation and default inheritance to enrich temporary concepts, and by hypothesis humans do the same. The goal of this project will be to apply the system to English inflectional morphology, which is a relatively unambitious target, but in a future project we hope to show that the same process can be applied to syntax (where the parser will link each word to the most active words that have already been parsed).
	To be a little more specific about the present project, we shall build the following software elements:
WGNet++, an expansion of WGNet which will allow spreading activation and default inheritance;
a database of information about how English verbs and nouns relate to 'forms' - the particular roots and affixes out of which they are built - including general rules for inflectional morphology. As explained above, all this information can be expressed in the same format, as nodes and links in a network.  To be realistic, the database will have to contain several thousand words, so we shall have to use a combination of manual and automatic methods for building the database. 
Our aim is to be able to use WGNet++ to enrich temporary words as described above,  working in either direction - either from word to form or from form to word. In the first case, the temporary concept will be the word's form, and in the second it will be the word's inflectional and lexical identity. 
	The main unknown element in this work will be the formula for spreading activation. In the technical  description of the project we identify nine independent parameters which might turn out to be relevant, and we can also vary the initial activation levels of all the nodes in the database. We shall have to determine suitable settings empirically, by trial and error, but of course we shall start from the simplest assumptions and keep the formula as simple as possible. 
	As we expained earlier, networks are probably found throughout cognition, and not just in language. This means that the formula which we arrive at for inflectional morphology may turn out to control all spreading activation throughout the entire network, including not only other parts of language (most obviously, syntax), but also other parts of cognition. This may eventually turn out to be wrong, but if so the discovery will be illuminating and important.

Originality 
Network models are common in psychology, but rare (and controversial) in linguistics. However even in psychology, most network models are neural networks whose aim is to learn new patterns and whose outcome is a network in which the information about each concept is 'distributed' across the entire network, rather than located in a single node (as in the Word Grammar model). The most successful and relevant non-distributed network models are in psycholinguistics, where Levelt and his colleagues in Nijmegen have built Weaver++ as a model of single-word production. However Weaver++ divides the production process into a number of separate steps, each controlled by a different activation formula. The most original features of the present project are:
1. the idea that all language use, whether in production or in perception, and whether in word-choice or in syntax, is controlled by a single formula and procedure, 
2. the idea of combining spreading activation with default inheritance. 

3. Full technical description 

Background
The proposed project is part of a larger attempt to discover whether it is possible to model parsing (by a human processor) as a combination of spreading activation and default inheritance in a network model of language. The basic outlines of this model already exist on paper, but remain to be implemented (and tested) on a computer. At least in theory, each word activates a part of the language network and inherits properties from the relevant super-categories; and part of this activation survives while later words are processed, so when later words look for grammatical links they can select the most active ones as potential candidates. Furthermore this parsing takes place in the context of a larger network of knowledge in which some active nodes represent the current (cognitive) context. Since these contextual nodes are intimately linked (via semantics) to the linguistic nodes, the contextual activation should spread to the linguistic structure and help to resolve some ambiguities. 
	I submitted a proposal for this entire project to the ESRC in 2000 under the title "Parsing by spreading activation in Word Grammar". Although the ESRC rejected this proposal, the report encouraged resubmission in the form of a series of smaller projects. The Board Assessor commented:
This is a project that has considerable promise in that it may well provide a more realistic model of human syntactic processing than the mainstream approach that has been provided in theoretical linguistics. However the project as it stands is very ambitious, very expensive particularly given the number of potential problems that developing it may well imply. It is thought that a more piecemeal approach, which inevitably will mean a more modest proposal, may be a better way forward for this very innovative and interesting approach to modeling.
In response to this advice, I have broken the original plan down into three separate stages:
Stage 1. Produce basic software for displaying and editing networks and for default inheritance.
Stage 2. Add the facility for modelling spreading activation.
Stage 3. Apply this software to the task of parsing.
	Stage 1 was funded by ESRC and is now complete. The ESRC received the final report at the end of November 2002. The present proposal is for Stage 2 and if this is successful there may be a further proposal to ESRC to complete the programme with Stage 3. I shall refer to each of these stages as a 'project' and to the entire set of projects as the 'programme' of research.

Theoretical background: Networks in Linguistics and in Psychology
The research programme is based on the theory of Word Grammar (WG). WG belongs to the family of linguistic theories called Cognitive Linguistics which has become increasingly popular during the past two decades, alongside Langacker's Cognitive Grammar, Lamb's Stratificational Grammar and Fillmore and Kay's Construction Grammar. One of the assumptions shared by all these theories is that language is a network, and more precisely a portion of the even larger network of general knowledge, rather than a collection of modules - a dictionary, a grammar, a phonology and so on. This non-modular approach has been applied in WG to several different areas of language, including syntax and semantics as well as morphology, and the theoretical foundations are relatively well developed and documented. Consequently, the focus of this research programme is on implementation rather than on theory development. However the implementation will inevitably raise new theoretical questions which have to be addressed, and especially so in Stage 2, where the focus will be on dynamic activation levels in the network rather than on the overall architecture of the system. The only published work on activation in a WG network is in the area of sociolinguistics, where some tentative suggestions have been made for modelling quantitative variation (Hudson 1997b; Hudson 1997a). The project will therefore have to develop a theory of activation as well as building a computer implementation. We outline the foundations for this theory below.
	Meanwhile, of course, cognitive networks have received a great deal of attention in psychology and psycholinguistics. Computer simulations have been produced for a large number of psycholinguistic processes in perception, production and learning, so this project will have plenty of earlier work to build on, including algorithms for modelling spreading activation. A particularly helpful source of ideas will be Levelt and Roelof's WEAVER++ simulation of speech production (Levelt et al 1999), which assumes an architecture of linguistic knowledge very similar to that of WG and includes a well-developed equation for spreading activation; it is also particularly compatible with our model because the network is symbolic rather than a distributed connectionist one. However we shall also learn from other such networks developed specifically for speech perception such as the logogen, cohort and TRACE models (Marslen-Wilson 1984; McClelland and Elman 1986; Morton 1964). 
	Most psycholinguistic models of processing are more concerned with the mechanisms for using language than with the architecture of the language faculty itself, whereas linguistic theories have the reverse strengths. However desirable this division of labour may be, it has the undesirable consequence that the two disciplines have produced models of language which are hard to relate to one another. The present programme can therefore be seen as an innovative attempt to build a bridge between one particular theoretical linguistic model of language structure and some psycholinguistic models of language processing. 
	The programme also applies the logic of default inheritance (and more specifically, multiple default inheritance), which has rarely, if at all, been combined with spreading activation. (A search on the Web of Science yields zero intersection of these terms.) Default inheritance has been used extensively in artificial intelligence and (more recently) in theoretical linguistics (Flickinger 1987; Fraser and Corbett 1997; Goldberg 1995; Langacker 2000; Lascarides and Copestake 1999; Malouf 1998; Sag 1997), and provides an easy explanation for the idea that cognitive categorisation is based on prototypes (Rosch 1976). The disadvantage of default inheritance for computer models is that it is non-monotonic so it raises serious tractability problems, but these should vanish when spreading activation can guide the search for possible exceptions. This is explained below.
	
Theoretical background: the architecture of language
According to WG, all of linguistic knowledge is a network, and one of the main arguments for the network structure is precisely the need to model the effects of spreading activation in speech errors, priming experiments and so on (Hudson 1984; Hudson 2000). This very general idea is fleshed out by specific claims about the nature of the network - claims which can be independently justified and which have specific consequences for a theory of processing. 
	Some of these claims are similar to those which underlie the WEAVER++ program:

•	As noted earlier, networks are local ('symbolic') rather than distributed - each node corresponds to some identifiable linguistic element such as a word, a morpheme or a general class of words or morphemes.
•	Every fact of language is represented by links in this network, so there is no lexicon/grammar distinction - no qualitative contrast between general rules and specific lexical items. 
•	The theory distinguishes words (aka lemmas - i.e. the units of syntax) from forms (including morphemes), so meanings are at two removes from observable sounds or letters: meanings relate to words, words relate to forms and forms relate to observables.
•	The network links (relations) are themselves independent elements which are classified.
•	The network of language is closely integrated into the total network of knowledge, so contextual information can easily interfere with processing. 
	
	However WG does embody some claims which distinguish it from WEAVER++:
•	The relation of classification ('isa') is fundamentally different from other relations because it carries the basic logic of the system, multiple default inheritance. As noted above, default logic is problematic in computer systems because it is non-monotonic, so the database has to be searched for potential overriders for every inferred fact. However, when default inheritance is combined with spreading activation this problem vanishes because any overriding fact must, by definition, be highly active and the system therefore need only consider the small number of highly active nodes. 
•	
•	Every concept has a distinct node, so a new 'token node' is created for each token concept - for each token word, form or meaning as it is encountered by the processor. The aim of processing is not simply to activate the relevant section of the network, but to use this activation to help enrich some token node. This is done by first attaching the token to some type (or set of types), then applying default inheritance - for example, if a token isa the type DOG, it must inherit all the links of this type including its sense, 'Dog'. This idea is explained more fully below.
•	The token nodes should solve the 'binding problem' noted by Levelt et al - how to ensure that all the activated 'bits' of speech are attached to each other in the right order. The WEAVER++ solution is to include procedures which check for consistency, but such procedures are unnecessary in WG.
•	The classification of links follows just the same logic as the classification of nodes: they are organised into 'isa' hierarchies of relations which permit default inheritance. This is quite easy to see in syntax, where everyone accepts, for example, that the direct object relation isa complement, which isa dependent; but it seems to play an important part throughout cognition. (This theoretical feature seems to be unique to WG.)
•	

	This is the theoretical background to the proposed programme. The ultimate goal is a model in which a single very general algorithm handles the whole of language processing by allowing activation to spread through a network which includes morpho-lexical, syntactic and semantic information, and even contextual knowledge. The more modest goal of the proposed programme is to explore the feasibility of this architecture via small-scale models.

Processing in WG: the morpholexicon
In WG, processing is the application of an existing network to a new word either in comprehension or in production. In comprehension the new word is defined by its  'signal' (sounds or letters produced by someone else), and the target is the relevant word's meaning and syntax. In production, the word is defined by its meaning and the target is the word's signal. These relations are shown schematically in diagram (a), where 'signal X' is the input to an act of perception and 'meaning X' is the input for production. The unknown elements are shown as queries (e.g. 'word = ?').

	However elementary it may seem, this model already incorporates theoretical assumptions which are contentious. One of the most basic issues in any theory about the architecture of language is the question of how many links stand between a meaning and the corresponding signal. The answer ranges from one - i.e. the meanings and sounds are directly related - to several. In WG, as in WEAVER++, two intervening 'levels' are recognised:

•	a word (or 'lemma') which carries the meaning and also syntactic properties such as a word class and valency information;
•	
•	a form, consisting of one or more morphemes, which carries the signal and some morphological information (such as declension class).
This more complex model is shown in figure (b) above.
	Another controversial feature of the WG model is the strict distinction between types and tokens. Each word token is represented as a separate node, rather than simply as the effect of activating an existing 'type' node. For example, suppose the current input to a perception model is signal X; the first thing the model does is to create a new token node 'form X', as shown in figure (c) below, for which signal X can be the signal. Then (because every form belongs to some word) it creates 'word X', as the word token of which form X is the form, and (for similar reasons) 'meaning X', as the meaning of word X. The aim of the processing is then (in the terminology of Relevance Theory) to 'enrich' these concepts, which it does by finding the stored types which best match the tokens. Once these best matches are found, the system inserts an 'isa' link from the token to the type, shown in the diagram as a line with a small triangle whose base rests on the super-category. Figure (d) shows how the three letters <d>, <o>, <g> acting as the signal activate the meaning 'Canine'.

	This approach has three theoretical advantages over the simpler models in which signal X would simply activate the relevant parts of the network:

•	It allows the logic of default inheritance to apply, because each token 'isa' its selected type. This guarantees a cleaner treatment of exceptions than in a simple spreading-activation model in which the default and exceptional features may be equally active. In particular:
•	It allows the system to accommodate minor deviations in the input such as spelling mistakes or even (eventually) extensions such as metaphor. This is possible because the token's features are stored separately from those of the stored types; for example, in some circumstances word X may be recognised as a token of the word FOUR even if it is spelt <for>.
•	It allows the database to include the complex defaults and exceptions which occupy linguists at all levels. (Indeed it can be argued that default inheritance is the appropriate treatment for all the phenomena for which linguists invoke special apparatus such as transformations or competition for 'optimality' - Hudson 2003.)
•	It allows activation to be clearly focussed on a specific outcome: the properties of the token nodes. These are the items of special interest to the user, so we may assume that these nodes receive maximum activation. This avoids the problem of activation dissipated on irrelevant targets. If activation spreads out from 'meaning X' and 'word X' as well as from 'signal X', then:
•	any nodes which receive activation from both directions will be favoured, in contrast with those whose activation is merely from one direction;
•	in comprehension, top-down activation from the context can influence the choice of higher nodes.
•	Although token nodes in general are temporary, a token which is novel (i.e. has no type) may be of such interest that it becomes permanent - i.e. it may be learned. In short, the distinction between token and type nodes may be the basis for learning new words, as in all 'usage-based' theories (Ellis 2002; Langacker 2000; Tomasello 2000). 
•	
•	Thanks to default inheritance, the token nodes 'bind' all the relevant properties together in an orderly way, so even if two similar nodes are both highly active, their properties are kept separate because they belong to distinct nodes. (except in slips of the tongue). 
The formula for spreading activation
One of the aims of the project is to develop a formula for calculating the activation of each node in a network through an imaginary sequence of times. We are aware of the impressively simple formula developed for WEAVER++, and we expect to take this as a starting point. However one crucial factor in that formula (called 'spreading rate') is different for different stages in the planning of an utterance, whereas we would hope to find a single formula which paid attention only to the strictly local characteristics of each node. 
	The factors which we believe may be relevant to the activation of two nodes m and n, related by relation r, are as follows. In the following we distinguish only two times, called t1 and t2, but it may prove necessary to identify further times between these:

•	The activity levels of m and n at t1; the overall aim is to calculate their respective activity levels at t2.
•	The respective types of m, n and r, which we can call M, N and R; it may be, for example, that activity transmits differently along 'isa' links compared with other relations, and it is even possible that different kinds of node behave differently - e.g. that nouns transmit differently from verbs - though we assume this is not the case.
•	The firing threshold of m (if any - WEAVER++ does not assume thresholds).
•	The 'fan' size of m - i.e. how many other nodes receive its activation when it fires. (WEAVER++ seems to subsume this variable under the 'spreading rate'.)
•	The transmission efficiency and speed of r when transmitting in the direction from m to n - i.e. the rate fixed for their respective types R, M and N.
•	A combination function (for all nodes or just for those of type N), which determines how incoming activation is combined with its existing activation.
•	
•	A 'back-channel function' which decides whether activation is allowed to pass back to a  node from which the current node's activation originated; contrary to the strict 'feed-forward' movement of activation in WEAVER++, we expect to allow activation to pass freely in any direction.
•	A decay function for activation over time. 

The stage-2 project
The project for which funding is requested in this application is stage 2 of the total programme described above. The main outcome of stage 1 was WGNet, a computer program for displaying and editing networks (as defined by WG), together with a small database for inflectional morphology in English. The aim of stage 2 will be to develop this work in the direction of the programme's ultimate goal, parsing by spreading activation. 
	The project will have six objectives:

•	To become familiar with the existing code and gradually adapt it to make WGNet more transparent in its treatment of relations and inheritance. (Programmer: 3 months) 
•	To extend WGNet so as to allow a researcher to define a processing query such as 'what is the meaning associated with the letters <d, o, g>?' This will create the required token nodes with minimal links to the existing database. (Programmer: 1 month)
•	To add spreading activation to WGNet by allowing the user to set the initial activation level for each node (or to set this on the basis of a database of frequencies), and also by providing a formula to control the spread of activation to other nodes. (Programmer: 2 months)
•	To add a visual display of the progress of activation during a query session; the activation will be shown on screen at a convenient speed.  (Programmer: 3 months)
•	To experiment with equations for spreading activation in order to reduce the range of possibilities, and if possible to find one which replicates at least some of the experimental findings quoted by Levelt (and similar findings in the comprehension literature). (Programmer, Linguist and PI: notionally 3 months; but experimentation will in fact start as soon as the basic facilities are available.)
•	To populate the database with sufficient morpho-lexical and syntactic information to allow thorough testing of the morphology and some pilot testing of the syntactic parsing machinery. At least some of this information will be input by hand, though it will also be possible to import some simple lexical information from a large list of words with frequencies which will be supplied by the Survey of English Usage. (Linguist: 9 months)
The PI will take the main responsibility for writing up the results, so this can start as soon as results start to appear.

The project will produce an advanced version of WGNet (to be called WGNet++) and a large grammatical database which will be made available to other researchers and presented in conferences and articles. If the outcome is as expected, this material will probably form the basis of a further application to ESRC for the funding of stage 3.

Annex
References

Ellis, Nick C.: 2002, 'Frequency effects in language processing: a review with implications for theories of implicit and explicit language acquisition.', Studies in Second Language Acquisition 24, 
Flickinger, D: 1987, Lexical rules in the hierarchical lexicon. Stanford PhD dissertation. unpublished PhD dissertation, Stanford University.
Fraser, Norman and G. G. Corbett: 1997, 'Defaults in Arapesh', Lingua 103, 25-57.
Goldberg, Adele: 1995, Constructions. A Construction Grammar Approach to Argument Structure., University of Chicago Press, Chicago.
Hudson, Richard: 1984, Word Grammar., Blackwell., Oxford.
Hudson, Richard: 1997, 'Inherent variability and linguistic theory', Cognitive Linguistics 8, 73-108.
Hudson, Richard: 1997, 'The rise of auxiliary DO: Verb-non-raising or category-strengthening?', Transactions of the Philological Society 95, 41-72.
Hudson, Richard: 2000, 'Language as a cognitive network', in Hanne Gram Simonsen and Rolf Theil Endresen (eds.), A Cognitive Approach to the Verb. Morphological and Constructional Perspectives, Mouton de Gruyter,Berlin, pp. 49-72.
Hudson, Richard: 2003, 'Mismatches in Default Inheritance', in Elaine Francis and Laura Michaelis (eds.), Linguistic Mismatch: Scope and Theory, CSLI,Stanford, pp. 
Langacker, Ronald: 2000, 'A dynamic usage-based model.', in Michael Barlow and Suzanne Kemmer (eds.), Usage-Based Models of Language, CSLI,Stanford, pp. 1-63.
Lascarides, A. and A. Copestake: 1999, 'Default representation in constraint-based frameworks', Computational Linguistics 25, 55-105.
Levelt, W. J. M., A Roelofs, and A. S. Meyer: 1999, 'A theory of lexical access in speech production', Behavioral And Brain Sciences 22, 1-45.
Malouf, Robert: 1998, Mixed Categories in the Hierarchical Lexicon. unpublished PhD dissertation, Stanford University.
Marslen-Wilson, William: 1984, 'Function and structure in spoken word recognition.', in H Bouma and D Bouwhuis (eds.), Attention and Performance. Vol X: Control of Language Processes., Lawrence Erlbaum,Hillsdale, NJ, pp. 
McClelland, J and Jeffrey Elman: 1986, 'The TRACE model of speech perception.', Cognitive Psychology 18, 1-86.
Morton, John: 1964, 'A preliminary functional model for language behaviour.', International Audiology 3, 216-225.
Rosch, Eleanor: 1976, 'Classification of real-world objects: origins and representations in cognition. (Reprinted in P. Johnson-Laird and P. C. Wason (eds.) (1977) Thinking: Readings in Cognitive Science. Cambridge: Cambridge University Press. 212-222.)', in S Ehrlich and E Tulving (eds.), La Mémoire Sémantique., Bulletin de Psychologie,Paris, pp. 
Sag, Ivan: 1997, 'English relative clause constructions',  Journal of Linguistics 33, 431-483.
Tomasello, Michael: 2000, 'First steps toward a usage-based theory of language acquisition', Cognitive Linguistics 11, 61-82.