Visual languages

I am trying a new format, where each point has both good and bad aspects.

Comments on "Complete Visualizations of Concurrent Programs and their Executions," by Kenneth M. Kahn and Vijay A. Saraswat, Xerox Palo Alto Research Center Technical Report Series, SSL-90-38, August 1990.

  1. For languages that do not rely on side-effects, a visual representation can represent both the program and the current state of execution. This is an appealing extension of textual substitution principles.
  2. I like the notion of "topological completeness." I never thought about it before, but this an important characteristic of most successful visual languages (that I know of). The authors appear, however, to use it as an excuse for producing incomprehensible visual programs. Surely elements that are syntactically non-essential, like shape and color, are nonetheless critical for recognition?
  3. Successful visual languages, formal and otehrwise, seem to share a common feature: they are not general-purpose, but can be very effective if applied within a limited domain. Dataflow and finite-state automata diagrams, which the authors mention in section 5, are two examples. Pictorial Janus, and the authors' approach in general, seems to be at the other end of the spectrum.

Comments on Building Visual Language Parsers, by Richard Helm, Kim Marruitt, Martin Odersky, in Proceedings of CHI '91:. Human Factors in Computing Systems, pp. 105-112, April, 1991.

  1. The topological syntactic relationship is more general than the containment relationship that I understood until now as being one of the three fundamental ones. Having said that, in their example grammars, the authors use only contains as a topological relationship, and touches as an instance of the network relationship.
  2. The authors state that VL parsing takes hand-written gestures as input, but they don't say how they recognize what those gestures represent. In the presence of touching elements or of crossing lines (StateCharts, for example), how do they get from gestures to terminal symbols such as circle and arrow?
  3. I am never quite sure what to think about visual language parsers: for box-and-arrow languages at least, building the syntax tree while the program is being constructed is simpler and more robust than extracting it later.

John Reekie, March 5th, 1998.