VisualizationToolIdeas
From OpenCog
This page has rough ideas and plans for visualization tools for the AtomSpace of OpenCog.
Contents |
Conceptualization
I envisage the view of the atomspace looking somewhat like the maps of the Opte Project - at least from a zoomed out and distant viewpoint.
Requirements
One option is for the tool to have a 3d view implemented in OpenGL. This view displays nodes, and the links between them using a intuitive colouring scheme to visualise AtomTypes, their TruthValues, and AttentionValues. Not all this data need be displayed at once (see #Visual themes below).
Another, more adventurous option, would be to use a 3D virtual world like OpenSim or Multiverse or Second life to display the data. The basic 3D view notion would be the same as in the OpenGL idea, but, the nodes and links would exist in a 3D virtual world, and the user's avatar could walk around and inspect different parts of the data. Data viz in virtual worlds is an up and coming area. Potentially, one could implement this using a modification of the proxy currently used to interface between the Novamente Pet Brain and virtual world servers like Multiverse.
The tool should be a separate process to OpenCog, accessing a OpenCog instance through a UDP connection. OpenCog should fire updates to the visualisation tool, and the tool should communicate to OpenCog what sort of updates it is interested in.
The tool should also provide some ability to control the execution of OpenCog, providing a debugging tool of sorts. This is for dynamics debugging rather than code debugging.
Vis update
Updates should be able to occur at a number of resolutions. The simplest is after every cycle, but a finer resolution where one can visualise the state after each MindAgent is run would be useful. This obviously wouldn't work for multithread/multiprocess OpenCog instances, but for single instances while debugging OpenCog dynamics, this would be extremely useful. Whatever the update step is, the tool should display either the cycle number and/or the last MindAgent run.
Updated atoms should maintain their positions in the 3d representation of the AtomSpace. New atoms, should be dynamically added to the view as they appear.
Atoms that change their TruthValue, AttentionValue, or other characteristics should visually indicate this, perhaps by flashing/glowing briefly (exactly what causes a flash/glow should be configurable).
What atoms to display
The AtomSpace has the potential to become extremely complex and huge. For this reason, the filtering of what atoms to display is important.
- One way is to only show atoms in the AttentionalFocus (or only those above a specifiable threshold of importance).
- A search box that can filter atoms on a number of characteristics, or by matching a subgraph. Perhaps through the use of a HyperGraph query language.
Selection of atoms
By clicking the user should be able to select atoms. Using shift should allow a rectangular region to be selectable, ctrl-click should add atoms to the existing selected atoms. The suer should then be able to group these, such that they are represented by a singular atom on the screen (as well as being able to ungroup these atoms too!).
Editing atoms
When atoms are selected, their characteristics should be editable.
Visual themes
Nodes and links can have different themes. These differ in the amount of data they can display visually.
For example, the simplest representation of a node is a filled circle. The colour inside the circle can be mapped to any characteristic of a node. It could be the type (each type should have a different colour associated with it, and this should be changeable in the preferences), it could be one of ShortTermImportance, LongTermImportance, TruthValue, TruthValueConfidence, etc.
More complex representations of nodes would be two half circles joined together, each side representing a different chracteristic. A 3 sectioned circle, 4 sectioned etc. Another way to represent a value would be to make it represent the size of the node.
Each characteristic should be able to map to a colour gradient. E.g. you could have a filled circle with the colour representing TruthValue. A gradient could be associated with TruthValues like Yellow->Green - atoms that are more green have higher TruthValues.
The basic idea is that the visualisation is highly configurable, since depending on what you are investigating, you'll be interested in certain characteristics more than others.
Python shell
An integrated python shell should allow the user to make AtomSpace manipulations directly. This might be better as a separate tool, or at least acting on a OpenCog server through a separate interface.
Other Ideas
The z clipping plane could indicate the AttentionalFocusThreshold.
Implementation
There are a number of visualisation toolkits available. VTK is often used in visualizing scientific data, and Paraview is a generic viewer based on it.
It may however be worth designing the viewer from scratch (although making use of relevent OpenGL based libraries e.g. Ogre) since the interaction with the CogServer is quite specific and is visualizing highly dynamic data - rather than relatively static or slowly changing datasets which are more commonly used with VTK (as far as I can tell, it might not have this limitation!).
See the log visualisation done in Ruby here - this could possibly be used as a prototype. There is also the visualisation language Processing] which is gaining popularity.
--Joel Pitt 06:27, 24 April 2008 (CDT)
Applications
Ideas for tools that may be used to inspect and/or manipulate a system's state or dynamics, or other purposes:
- A teaching tool for the concepts of system dynamics of self-organizing complex systems, combining guided exercises with free-form self-directed exploratory learning. dhart 19:35, 1 August 2008 (CDT)
