abu@software-lab.de

16. Jun. 2002

A Unifying Language for Database And User Interface Development

(c) Software Lab. Alexander Burger

A language framework is presented which closes the semantic gap between database, application logic, and user interface. We introduce the concepts of Prefix Classes and Relation Maintenance Daemon Objects to suggest a unified development style reducing software development time and cost.

Introduction

Ever since computers were used in commercial applications, there was a growing demand to reduce software development time and cost.

Business models are quite different from each other. Off-the-shelf software usually does not fit the individual needs for enterprise workflow data processing, real world modeling, and multimedia applications. For that reason, many companies are forced to develop their own solutions. Software is a significant cost factor.

Due to the competitive nature of business, the redundancy inherent in such individual developments can hardly be avoided. Concerning the development costs, however, there seems to be a lot of room for improvements.

Typically, these projects implement some kind of database and a set of application programs. The task of these programs is to manipulate the data in the database, implement the general application logic, and provide for some kind of user interface.

We observe a large semantic gap between the database structure, and the application logic with its user interface. The current standard database language is SQL, which operates on the table level, while general-purpose programming languages like Cobol, C/C++ or Java are used to implement the application logic and user interface.

In numerous attempts to remedy the situation, object-oriented paradigms are applied to both ends, by extending the relational database to object-oriented databases, and by building user interface frameworks in object-oriented languages.

There's no doubt that object-oriented databases provide for more intuitivity and productivity, and modern graphical user interfaces (GUI) cannot not be imagined without those frameworks.

But the semantic gap does not appear to diminish significantly. Databases and user interfaces are separate worlds: Existing class libraries are concerned about visual effects and event handling, but not about application logic and database maintenance. It is the programmer's responsibility to write glue code that displays data in corresponding GUI fields, detects modifications by the user, validates them, writes changes back to the database, and does other housekeeping.

This paper introduces a language and programming environment that closes the semantic gap, by unifying database and user interface into a single application server framework.

Traditional DB and GUI Development

The mainstream database format today is still the relational model, which packs the application data into two-dimensional tables, and relies on the application program logic to correctly access these data and to maintain their integrity via proper SQL statements.

A single data record on the application level (like, for example, a customer or an article) - which is presented to the user within a single GUI window - is usually spread out over several tables in the database.

The application program has to select these data, explicitly supplying knowledge about the relations between the tables, the data types and sizes, then have them copied to variables in the application scope, and move them to the GUI window. (Note: There are tendencies to "hide" this code into the methods of an object-relational database. This serves well for better structuring the application program design, but does not decrease the actual amount of programming work. The correct select-statements still have to be written, and a change in the relations or table structures may require individual modifications.)

The GUI components have access to these variables in the application scope; they are notified after a successful select to display these data.

Now comes the most tedious part. The user interface cannot simply wait until a user has done all his changes to the data record and hits some "submit"-button. It is necessary to provide immediate feedback on field entry, field exit, and often on every key stroke or mouse click. Depending on the context and the internal state of the application, individual GUI components or program features have to be enabled or disabled.

When the focus leaves a GUI component, its data have to be validated (possibly displaying error messages to the user), certain side-effects carried out (which might influence other GUI components), and modifications written to the database. A simple change in a single text field can cause an update to several database tables.

All these things involve SQL statements which again must contain extensive knowledge about the database structure. And they cannot easily be abstracted into reusable DB- or GUI-classes, because the requirements tend to differ for each view on the data record and each combination of GUI components. Thus, they have to be written individually for each GUI window.

A Unified Approach

We use the PicoLisp interpreter to build a vertical, unified solution to these problems. It allows to describe a direct mapping between application structures and database objects, so that the underlying machine can handle most of the above issues automatically.

The solution is "vertical", because it extends from the virtual machine level up into the application and GUI levels. And it "unifies", in a consistent way, the Lisp interpreter with

by using the same syntax and philosophy all along the way. We will describe its details in the following sections, and give some practical examples.

It turned out that certain requirements for the virtual machine are not met by existing languages like C++ and Java. These requirements include dynamic I/O of persistent objects, and a garbage collector handling these objects accordingly.

PicoLisp was chosen as the base language, because its interpreter is simple and completely written in C. This makes it easy to incorporate the necessary extensions to the virtual machine. Besides this, two other features (of Lisp in general)

are considered essential. They are both needed for the intended descriptive syntax of the resulting application development system.

Object Architecture

PicoLisp employs a very simple object architecture. It uses symbols for the implementation of both classes and objects. There are many ways to implement symbols in Lisp (e.g. John Allen: "Anatomy of Lisp", McGraw-Hill, 1978); in PicoLisp each symbol has a value cell, a property list, and a name.

For a symbol representing an object, the value cell holds a list of the object's classes, the property list holds the object's attributes (instance variables), and the name is usually empty (anonymous symbol).

For a symbol representing a class, the value cell holds an association list with the class' methods, concatenated with a list of the class' superclasses, the property list may hold some class attribues (class variables), and the symbol's name is the name of the class.

When a message (also a symbol) is sent to an object, that object's list of classes - and recursively those classes' superclasses - is searched from left to right (in a depth-first manner) for that message, and the corresponding method body is executed. (In effect, this is a multiple-inheritance late-binding strategy.)

In that way, a class +MyCls can be defined to inherit from three classes +Cls1, +Cls2 and +Cls3 (by convention, class names start with a `+'):


(class +MyCls +Cls1 +Cls2 +Cls3)

Then, an object can be created with


(new '(+MyCls))

or another object with equivalent behavior:


(new '(+Cls1 +Cls2 +Cls3))

In both cases the resulting objects will inherit method definitions from +Cls1, +Cls2 and +Cls3. Because of the depth-first and left-to-right search order, however, methods in the class hierarchy of +Cls1 will override methods with the same name anywhere in the class hierarchies of +Cls2 and +Cls3.

This is a "horizontal" inheritance, as opposed to - and in addition to - the normal "vertical" inheritance. +Cls1 and +Cls2 can surgically alter the behavior of +Cls3, in a very fine-grained manner. Thus - as +Cls3 will typically define the general behavior - +Cls1 and +Cls2 are called "Prefix Classes" of +Cls3.

This object architecture is used throughout the whole system, including the DB and GUI. And prefix classes are an essential part of it: The expressive power of Lisp's equivalence of code and data is augmented in combination with prefix classes.

Database

The PicoLisp database is built upon that object architecture.

On the lowest level, a database is a collection of persistent objects. PicoLisp supports persistent symbols, called external symbols, as a first-class data type. These symbols are known to - and handled in special ways by - the interpreter.

External symbols are stored in a file, in linked blocks of fixed size, where each symbols's starting block address is computable from the symbols's name.

A read or write access to an external symbol's value or properties causes that symbol to be automatically fetched from the database file. At the end of any transaction, modified symbols are written back (commit) or reverted (rollback). The garbage collector knows about the state of external symbols, and purges currently unused symbols from memory (Note: These symbols are only temporarily removed from memory, not from the database. The latter is done separately by a DB-level garbage collector.).

On the higher levels, these external symbols are organized into class hierarchies, reflecting the application's organizational structures.

As opposed to simple two-dimensional tables, they form arbitrary data structures like lists, stacks, trees and graphs.

The connections (relations) between objects in the database are not established by index lookup, but by explicit inclusion. That is, an object referring to another object can explicitly hold (i.e. contain a pointer to) that object, and can get access to it very rapidly. There is no explicit select operation, everything is simply available when it is needed.

Relation Daemons

Generally, instances of persistent database objects are called "Entities". In our system, there is an additional separate class hierarchy of "Relations": Instances of these classes we call "relation maintenance daemon objects".

Relation daemons are a kind of metadata, controlling the interactions between entities, and maintaining database integrity. They are the concrete realization of an abstract Entity/Relationship.

Like other classes, relation classes can be extended and refined, and in combination with proper prefix classes a fine-grained description of the application's structure can be produced.

Besides some primitive relation classes, like +Number, +String or +Date, there are

A defining function rel is provided, which is used in the context of an entity class to assign relation daemon objects to that class:


(rel attr (+Cls ..) Arg ..)

rel is called with an attribute name attr, a list of relation classes (+Cls ..) and - depending on these classes - other optional arguments. Basically, this function simply does a


(new '(+Cls ..) Arg ..)

and assigns the resulting daemon object to the attr-property of the current entity class.

Relation Prefix Usage

For example, a simple entity "Person" can be defined, having just a "name" attribute:


(class +Person +Entity) # class '+Person'
(rel name (+String))    # relation 'name'

If the name relation needs a unique index, it is written as:


(rel name (+Key +String))

And - for an extended example of prefix classes - if name should be a mandatory list of names, each with an index using the soundex algorithm:


(rel name (+Need +List +Sn +Idx +String))

This demonstrates the power of combined prefix classes, which allow to define complex object behavior "on the fly", without the need to leave the current programming focus. This is even more important in user interface programming (see section GUI Integration).

Entity Linkage

A uni-directional link to another entity might be, for example, the person's address:


(rel adr (+Link) (+Address))

This assumes that some entity class +Address is defined:


(class +Address +Entity)

In reality, the relation will probably be bi-directional, with several persons living at some address:


(class +Person +Entity)
(rel adr (+Joint) prs (+Address))

(class +Address +Entity)
(rel prs (+List +Joint) adr (+Person))

This says: The adr-attribute of +Person (a +Joint) is an address (connected to the prs-attribute of +Address), while the prs-attribute of +Address (a +List +Joint) is a list of persons (connected to the adr-attribute of each person).

So, when a person's adr is assigned to some address, the +Joint relation daemon will take care of updating the list of persons in that address, and vice versa.

Query Language

For extensive searches in the database, as they are needed for reports or user-specified queries, a Prolog engine was incorporated into PicoLisp. Prolog is similar to SQL, due to its declarative nature, but much more powerful because of its rule-deriving and backtracking capabilities.

Prolog is easy to implement in Lisp (see J. A. Campbell: Implementations of Prolog, Ellis Horwood Limited, 1984). We extended the basic inference machine to iterate also through facts in the database, and the basic search/backtracking algorithm to a self-optimizing parallel search through multiple index trees.

The details of the query language are beyond the scope of this paper. It is mentioned here because our production system would be incomplete without it.

GUI Integration

The connection between database objects and GUI components is established with only two prefix classes: +E/R and +Obj.

Normally, GUI components are created at runtime with the gui function, e.g.


(gui '(+TextField) "Text" 8)
(gui '(+NumField) "Number" 8)

for a text field and a numeric field, each 8 characters wide.

+E/R (Entity/Relation)

The +E/R prefix connects the GUI field with a given relation of an entity:


(gui '(+E/R +NumField) '(n . Obj) "Number" 8)

The specification (n . Obj) is passed as an additional argument. It indicates that this numeric field is "connected" to the n-property of the database object Obj.

Nothing else has to be done by the programmer. The field will automatically display the value of n from the database, and modifications entered by the user into this field will be written automatically to the database value of n, in the object Obj.

+Obj (Object)

The +Obj prefix extends the GUI field types, from primitives like numbers or strings, to database objects.

That is, a GUI field can "contain" a database object, just like a text field contains a string, and a numeric field contains a number. An object can be "set" into (assigned to) the field, and retrieving the value of the field will result in that object.

The field cannot, of course, directly display the complete database object. But it can show a typical attribute of that object, e.g. the object's name in a text field, or the object's ID number in a numeric field:


(gui '(+Obj +NumField) '(id +Cls) "ID" 8)

The +Obj prefix extends the capabilities of the basic field type (here a +NumField), in such a way that

As a consequence, when +E/R and +Obj are combined


(gui '(+E/R +Obj +NumField) '(x . Obj) '(id +Cls) "Number" 8)

they effectively establish the GUI for an entity linkage (+Link, +Joint etc.), as described in section Entity Linkage.

An Example

Putting it all together, the total effect of the described concepts can best be explained with the help of a detailed and complete example.

Assuming a simple family database, we represent a network of family relationships. For each person, we have attribues for name, sex, date of birth, and we want to have links to father, mother, husband/wife and children.

For a better understanding, we first present the traditional, relational representation (Note that we have to introduce a unique ID number for each record. Also, in such a tabular representation, it is more convenient to store the ID of father and mother (instead of the children).):

ID nm sex dat mate pa ma







1 John M 22jan1954 2
2 Mary F 01feb1958 1
3 Thomas M 26jan1988 1 2
4 Claudia F 15jul1989 1 2
5 Michael M 03jan1992 1 2

When viewing the these family members as a graph, we get the following object structure:

Example object structure

In the figure, we omitted the date of birth attribute.

The bi-directional relations, connecting the parents to the children and to each other, lend themselves to the +Joint entity linkage, resulting in the following definition for a person:


(class +Person +Entity)
(rel nm   (+Key +String))          # Name
(rel pa   (+Joint) kids (+Man))    # Father
(rel ma   (+Joint) kids (+Woman))  # Mother
(rel mate (+Joint) mate (+Person)) # Partner
(rel dat  (+Date))                 # born

From this base class, we can derive two classes +Man and +Woman:


(class +Man +Person)
(rel kids (+List +Joint) pa (+Person))

(class +Woman +Person)
(rel kids (+List +Joint) ma (+Person))

To produce a corresponding GUI

Example GUI display

allowing to view and edit the family members in the database, the following code is sufficient:


(row
   (gui '(+E/R +TextField) '(nm : home obj) "Name" 20)
   (gui '(+E/R +DateField) '(dat : home obj) "born" 10)
   (gui '(+ClassField)
      '(: home obj) "Sex"
      '(("Male" +Man) ("Female" +Woman)) ) )
(----)
(row
   (gui '(+E/R +Obj +TextField)
      '(pa : home obj) '(nm +Man)
      "Father" 20 )
   (gui '(+E/R +Obj +TextField)
      '(ma : home obj) '(nm +Woman)
      "Mother" 20 ) )
(gui '(+E/R +Obj +TextField)
   '(mate : home obj) '(nm +Person)
   "Partner" 20 )
(---- T)
(gui '(+E/R +Chart)
   '(kids : home obj)
   4 '("Children" "born" "Father" "Mother")
   (quote
      (gui '(+Obj +TextField) '(nm +Person) "" 15)
      (gui '(+Skip +Lock +DateField) "" 10)
      (gui '(+ObjView +TextField) '(: nm) "" 15)
      (gui '(+ObjView +TextField) '(: nm) "" 15) ) )

The above block of code will produce exactly the layout and functionality of the example GUI display. Without explaining all details here, suffice it to say that the row function arranges the components horizontally (while otherwise the default is vertically), (----) groups components into separate panels, and a +Chart creates an array of its argument components.

The point is that this is the complete program, not just some important details. It specifies the whole database and GUI application.

Discussion

The previous sections and the example show that application programming does not need to involve any concerns about database access (select, insert, update etc.) and database integrity maintenance.

The advantages are derived from the use of prefix classes and relation daemons. They allow to specify the complete program behavior and appearance in a single place of definition, and in a very concise form. Typically, the names of prefix classes are simply chained together, and intermix freely with Lisp's formal indifference of code and data.

This removes the need of maintaining separate resource files, class and data declarations, and program code.

Conclusion

Achieving low program development costs is an old claim. It has been stated for manyfold methodologies and paradigms.

The system described in this paper has proven its practical value in commercial applications during several years. Among them are sales, accounting, report, and logistics applications. Research projects include graphics/animation and speech synthesis systems.

This shows that it is possible to employ the concepts of prefix classes and relation maintenance daemons successfully to commercial application development.

We observed a significant decrease in program development time. What used to be the major part of a project's development effort showed to reduce to about 5 percent.

Download

The PicoLisp system can be downloaded from the PicoLisp Download Page.


Top (Software Lab.)