MindOntology:Self-Modifying Evolving Probabilistic Hypergraph
From OpenCog
Contents |
Self-modifying, Evolving Probabilistic Hypergraphs
Patternist philosophy is extremely general, which is both a strength and a weakness. In order to more effectively apply it to the AGI problem, I have created an intermediate formalism called Self-Modifying, Evolving Probabilistic Hypergraphs (SMEPH). SMEPH is a more specific formalism for describing intelligent systems, which is consistent with patternist philosophy but provides more guidance regarding the analysis and construction of particular intelligent systems.
The basic ideas underlying SMEPH are threefold, as the acronym would suggest:
- To use a specific mathematical structure called a generalized hypergraph to model intelligent systems.
- To study the way hypergraphs change over time (i.e. the way they evolve — the word evolution is used here in a general sense, rather than specifically in the sense of evolution by natural selection, although that is an aspect of SMEPH as well when one delves into the details.)
- To use probability theory to study the relationships between the parts of the hypergraph.
A hypergraph is an abstract mathematical structure (Bollobas, 1998), which consists of objects called Vertices and objects called Edges, which connect the Vertices. In computer science, a graph traditionally means a bunch of dots connected with lines (i.e. vertices connected by edges, or nodes connected by links). A hypergraph, on the other hand, can have Edges that connect more than two Vertices; and SMEPH's hypergraphs extend ordinary hypergraphs to contain additional features such as Edges that point to Edges instead of Vertices; or Vertices that, when you zoom in on them, contain embedded hypergraphs. Properly, SMEPH's hypergraphs should always be referred to as generalized hypergraphs, but this is cumbersome, so we will persist in calling them hypergraphs instead. In a hypergraph of this sort, edges and vertices are not as distinct as they are within an ordinary mathematical graph (for instance, they can both have edges connecting them), and so it is useful to have a generic term encompassing both Edges and Vertices; for this purpose, in SMEPH and OCP, we use the term Atom.
A weighted, labeled hypergraph is a hypergraph whose Edges and Vertices come along with labels, and with one or more numbers that are generically called weights. The label associated with an Edge or Vertex may sometimes be interpreted as telling you what type of entity it is. On the other hand, an example of a weight that may be attached to an Edge or Vertex is a number representing a probability, or a number representing how important the Vertex or Edge is to the system.
Hypergraphs may come along with various sorts of dynamics. Minimally, one may think about:
- Dynamics that modify the properties of Vertices or Edges in a hypergraph (such as the weights attached to them.)
- Dynamics that add new Vertices or Edges to a hypergraph, or remove existing ones.
The SMEPH approach to designing and implementing intelligent systems is centered on a particular collection of Vertex and Edge types. The key Vertex types are
- ConceptVertex, representing a set — for instance, an idea or a set of percepts
- SchemaVertex, representing a procedure for doing something (perhaps something in the physical world, or perhaps an abstract mental action).
The key Edge types are
- ExtensionalInheritanceEdge (ExtInhEdge for short: an edge which, linking one Vertex or Edge to another, indicates that the former is a special case of the latter)
- ExtensionalSimilarityEdge (ExtSim: which indicates that one Vertex or Edge is similar to another)
- ExecutionEdge (a ternary edge, which joins {S,B,C} when S is a SchemaVertex and the result from applying S to B is C).
So, in a SMEPH system, one is often looking at hypergraphs whose Vertices represent ideas or procedures, and whose Edges represent relationships of specialization, similarity or transformation among ideas and/or procedures.
ExtInh and ExtSim Edges come with probabilistic weights indicating the extent of the relationship they denote (e.g. the ExtSimEdge joining the cat ConceptVertex to the dog ConceptVertex gets a higher probability weight than the one joining the cat ConceptVertex to the washing-machine ConceptVertex). The mathematics of transformations involving these probabilistic weights becomes quite involved — particularly when one introduces SchemaVertices corresponding to abstract mathematical operations, a step that enables SMEPH hypergraphs to have the complete mathematical power of standard logical formalisms like predicate calculus, but with the added advantage of a natural representation of uncertainty in terms of probabilities, as well as a natural representation of networks and webs of complex knowledge.
Derived Hypergraphs
SMEPH hypergraphs may be used to model and describe intelligent systems (such as human mind/brains, for example). One can (in principle) draw a SMEPH hypergraph corresponding to any individual intelligent system, with Vertices and Edges for the concepts and processes in that system's mind. This leads to what is called the derived hypergraph of that system.
SMEPH Vertices
A ConceptVertex in the derived hypergraph of a system corresponds to a structural pattern that persists over time in that system; whereas a SchemaVertex corresponds to a multi-time-point dynamical pattern that recurs in that system's dynamics. Drawing the derived hypergraph of an intelligent system is one way of depicting the mind of that system — this follows from the definition of a mind as the set of patterns in an intelligent system, and the fact (which follows from mathematical pattern theory) that the patterns in the system can be read off from the derived hypergraph.
To phrase it a little differently, we may say that a ConceptVertex, in SMEPH, refers to the habitual pattern of activity observed in a system when some condition is met (this condition corresponding to the presence of a certain pattern). The condition may refer to something in the world external to the system, or to something internal. For instance, the condition may be observing a cat. In this case, the corresponding Concept vertex in the mind of Ben Goertzel is the pattern of activity observed in Ben Goertzel's brain when his eyes are open and he's looking in the direction of a cat. The notion of pattern of activity can be made rigorous using mathematical pattern theory, as is described in The Hidden Pattern (Publications#HiddenPattern)
Note that logical predicates, on the SMEPH level, appear as particular kinds of Concepts, where the condition involves a predicate and an argument. For instance, suppose one wants to know what happens inside Ben's mind when he eats cheese. Then there is a Concept corresponding to the condition of cheese-eating activity. But there may also be a Concept corresponding to eating activity in general. If the Concept denoting the activity of eating X is generally easily computable from the Concepts for X and eating individually, then the eating Concept is effectively acting as a predicate.
A SMEPH SchemaVertex, on the other hand, is like a Concept that's defined in a time-dependent way. One type of Schema refers to a habitual dynamical pattern of activity occurring before and/or during the time that some condition is met. For instance, the condition might be saying the word Hello. In that case the corresponding SchemaVertex in the mind of Ben Goertzel is the pattern of activity that generally occurs before he says Hello.
Another type of Schema refers to a habitual dynamical pattern of activity occurring after some condition X is met. For instance, in the case of the Schema for adding two numbers, the precondition X consists of the two numbers and the concept of addition. The Schema is then what happens when the mind thinks of adding and thinks of two numbers.
Finally, there are Schema that refer to habitual dynamical activity patterns occurring after some condition X is met and before some condition Y is met. In this case the Schema is viewed as transforming X into Y. For instance, if X is the condition of meeting someone who is not a friend, and Y is the condition of being friends with that person, then the habitually intervening activities constitute the Schema for making friends.
SMEPH Edges
SMEPH edge types fall into two categories: functional and logical. Functional edges connect Schema vertices to their input and outputs; logical edges refer mainly to conditional probabilities, and in general are to be interpreted according to the semantics of Probabilistic Logic Networks.
Let us begin with logical edges. The simplest case is the Subset edge, which denotes a straightforward, extensional conditional probability. For instance, it may happen that whenever the Concept for cat is present in a system, the Concept for animal is as well. Then we would say
Subset cat animal
(Here we assume a notation where "R A B" denotes an Edge of type R between Vertices A and B.)
On the other hand, it may be that 50% of the time that cat is present in the system, cute is present as well: then we would say
Subset cat cute <.5>
where the <.5> denotes the probability, which is a component of the Truth Value associated with the edge. There is a collection of roughly a dozen different logical edge types in SMEPH, which are derived from the Probabilistic Logic Networks framework. We will discuss some of these types in more depth in a later section, in the context of OCP's closely related usage of PLN.
Next, the most basic functional edge is the Execution edge, which is ternary and denotes a relation between a Schema, its input and its output, e.g.
Execution father_of Ben_Goertzel Ted_Goertzel
for a schema father_of that outputs the father of its argument.
The ExecutionOutput (ExOut) edge denotes the output of a Schema in an implicit way, e.g.
ExOut say_hello
refers to a particular act of saying hello, whereas
ExOut add_numbers {3, 4)
refers to the Concept corresponding to 7. Note that this latter example involves a set of three entities: sets are also part of the basic SMEPH knowledge representation. A set may be thought of as a hypergraph edge that points to all its members.
In this manner we may define a set of edges and vertices modeling the habitual activity patterns of a system when in different situations. This is called the derived hypergraph of the system. Note that this hypergraph can in principle be constructed no matter what happens inside the system: whether it's a human brain, a formal neural network, Cyc, OCP, a quantum computer, etc. Of course, constructing the hypergraph in practice is quite a different story: for instance, we currently have no accurate way of measuring the habitual activity patterns inside the human brain. fMRI and PET technologies give only a crude view, though they are continually improving.
Pattern theory enters more deeply here when one thoroughly fleshes out the Inheritance concept. Philosophers of logic have extensively debated the relationship between extensional inheritance (inheritance between sets based on their members) and intensional inheritance (inheritance between entity-types based on their properties). A variety of formal mechanisms have been proposed to capture this conceptual distinction; see (Wang, 2006, 1995) for a review along with a novel approach utilizing uncertain term logic. Pattern theory provides a novel approach to defining intension: one may associate with each ConceptVertex in a system's derived hypergraph the set of patterns associated with the structural pattern underlying that ConceptVertex. Then, one can define the strength of the IntensionalInheritanceEdge between two ConceptVertices A and B as the percentage of A's pattern-set that is also contained in B's pattern-set. According to this approach, for instance, one could have
IntInhEdge whale fish <0.6> ExtInhEdge whale fish <0.0>
since the fish and whale sets have common properties but no common members.
Patternist philosophy comes in here and makes some definite hypotheses about the structure of derived hypergraphs. It suggests that derived hypergraphs should have a dual network structure, and that in highly intelligent systems they should have subgraphs that constitute models of the whole hypergraph (these are self systems). SMEPH does not add anything to the patternist view on a philosophical level, but it gives a concrete instantiation to the general ideas of patternism.
Probabilistic and Evolutionary Dynamics
The logical edges in a SMEPH hypergraph are weighted with probabilities, as in the simple example given above. The functional edges may be probabilistically weighted as well, since some Schema may give certain results only some of the time. These probabilities are critical in terms of SMEPH's model of system dynamics; they underly one of SMEPH's three key principles of the dynamics of intelligence,
Principle of Implicit Probabilistic Inference: In an intelligent system, the temporal evolution of the probabilities on the edges in the system's derived hypergraph should approximately obey the rules of probability theory.
The basic idea is that, even if a system's underlying dynamics has no explicit connection to probability theory, nevertheless it must behave roughly as if it does, if it is going to be intelligent. The roughly part is important here — it's well known that humans are not terribly accurate in explicitly carrying out formal probabilistic inferences. And yet, in practical contexts where they have experience, humans can make quite accurate judgments — which is all that's required by the above principle, since it's the contexts where experience has occurred that will make up a system's derived hypergraph.
The next key dynamical principle of SMEPH is evolutionary, and states
Principle of Implicit Evolution: In an intelligent system, new Schema and Concepts will continually be created, and the Schema and Concepts that are more useful for achieving system goals (as demonstrated via probabilistic implication of goal achievement) will tend to survive longer.
Note that this principle can be fulfilled in many different ways. The important thing is that system goals are allowed to serve as a selective force.
Another SMEPH dynamical principle pertains to a shorter time-scale than evolution, and states
Principle of Attention Allocation: In an intelligent system, Schema and Concepts that are more useful for attaining short-term goals will tend to consume more of the system's energy. (The balance of attention oriented toward goals pertaining to different time scales will vary from system to system.)
Next, there is the
Principle of Autopoesis: In an intelligent system, if one removes some part of the system and then allows the system's natural dynamics to keep going, a decent approximation to that removed part will often be spontaneously reconstituted.
And the
Cognitive Equation Principle: In an intelligent system, many abstract patterns that are present in the system at a certain time as patterns among other Schema and Concepts, will at a near-future time be present in the system as patterns among elementary system components.
The Cognitive Equation Principle, named after the Cognitive Equation from the 1994 book Chaotic Logic, basically means that Concepts and Schema emergent in the system are recognized by the system and then embodied as elementary items in the system — so that patterns among them in their emergent form become, with the passage of time, patterns among them in their directly-system-embodied form. This is a natural consequence of the way intelligent systems continually recognize patterns in themselves.
Note that derived hypergraphs may be constructed corresponding to any complex system which demonstrates a variety of internal dynamical patterns depending on its situation. However, if a system is not intelligent, the evolution of its derived hypergraph can't be expected to follow the above principles.
These principles follow from the psynet model of mind, but they are more precise than the psynet model can be, because they assume a particular formalism for representing the contents of a mind (SMEPH hypergraphs). Of course, no particular mind will be completely described by this sort of hypergraph model; the idea is that this level of approximate description is good enough for many purposes.
The relationship between the human brain/mind and OCP may be explored in a SMEPH context, by considering that both OCP and the human mind can be modeled as SMEPH hypergraphs that obey the principles of implicit probabilistic inference and evolution. Below we will use this approach to help organize our discussion of various concrete results from the cognitive sciences and their relevance for OCP and AGI in general.
From SMEPH to OCP
While SMEPH is a general approach to modeling any intelligent system, it is also possible to create intelligent systems bearing a special relationship to SMEPH. OpenCog Prime falls into this category, as described on the page FromSMEPHToOCP, as did its predecessor systems Webmind and the NCE. This special relationship makes it particularly easy to analyze these AI systems in SMEPH terms, but it also gives rise to potential confusions.
