Supporting interaction design with the UML

Panos Markopoulos (P.Markopoulos@tue.nl), IPO-Center for User System Interaction, Eindhoven University of Technology, The Netherlands

Abstract

This  paper discusses the use of the UML to support interaction design.  We argue that any extension to the UML that aims to support interaction design in general, should make none or minimal assumptions concerning the interaction and the software design processes, and the context or scale of the target applications.  We summarise our view of earlier research on using the UML for interaction design, we discuss what appear to be the core representations that may be recruited for interaction design.  Finally, we focus on task models and discuss how activity diagrams can support their specification. 

1. Introduction

This paper discusses the use of the UML to support interaction design.  The intended users of such an extension are interaction designers, also called sometimes, interaction architects or user-experience architects.  In some cases, particularly in the context of small projects, or small development teams, this role is assumed by software engineers; in larger development teamsspecialists fulfill this role.  

Our intention to support interaction design is emphasized in order to delimit the scope of the paper.  We are not (here) concerned with the specification of software architecture and software components and we do not assume familiarity of interaction designers with software development.  Rather, we wish to investigate which subset and extensions of the UML can support the articulation of design ideas and their communication to software developers.

But what, if not software, do interaction designers need to model? (the term 'model' is used here in the sense adopted within the UML).  It is hard, if not impossible, to give an unambivalent answer.   In many cases (probably most cases) interaction designers may choose not to model at all, but to build prototypes of an interactive system or simply to program it.  Scenarios pose the lowest threshold for representing design ideas and may be the only representation used.  The set of models discussed are commonly recommended representations for interaction design.  (See for example the representations used in the Usability Engineering lifecycle of [17], or the set of 'descriptions' involved in interaction design as discussed at the CHI'97 workshop [1] on object modelling and the design of interactive systems).  The representations discussed here are not suggested to be sufficient to cover all  modelling needs of interaction designers.  Each application domain sets its own requirements to  interaction designers, e.g., to specify animations, sound effects, web page layout, etc.  Our starting position is that a good proportion of interaction design problems may be served by scenarios, as perhaps the primary design representation, and by the following six set of models (1+6 design representations):
  1. Model of the user population,
  2. model of the semantics of the problem domain,
  3. model of the user tasks,
  4. model of the display content and structure (layout),
  5. model of the navigation between and within displays,
  6. model of the 'look and feel' of individual interactive objects.
The notation that is appropriate for each of these models varies according to context and purpose.  Below we discuss evolving ideas concerning the specification of each of these models, and we  focus on modelling user tasks.
A major strength of the UML is that it makes minimal assumptions concerning the software design process.  This facilitates the assimilation of the UML within in-house development methods, allows the independent evolution of notations and of the design process, and minimises the learning curve for an organisation wishing to adopt the UML.  In extending the UML to cover interaction design we wish to maintain this trait.  As far as this is possible, we try not to tie a proposal for extending the UML to a particular design method. However, the modelling needs of interaction designers and the design process are not orthogonal.
Traditionally, interaction design methods have been tied to modelling activities and corresponding notations.  Consider, for example, interactive system design approaches such as ADEPT [8,10] and Usage Centred Design [5].  ADEPT emphasized the correspondence between task decomposition and sequencing to the organisation of interactive objects on the display in terms of containment, layout and navigation relationships.  Consequently the design approach and the tool support emphasized the hierarchical decomposition of tasks to sub-tasks and temporal ordering relations between task related activities.  Similarly, Usage Centered Design is founded upon the concept of Essential Use Cases, a reductionist version of Jacobson's use cases, that models user-goal related interaction with the system under design.
In the following section we review current proposals for  extending the UML to support interaction design, with the following two points in mind: The remaining of the paper is structured as follows:

2. Related Work

This research aims to support a model based approach to the design of interactive systems.  Model based design (see for example[24, 26]) supports  interaction design through the construction, manipulation and transformation of a set of models, which capture salient aspects of users, their task related activity and task domain, and the interface to the system.  This field of research has produced notations, design principles and, mostly, experimental tools for the generation of user interfaces from higher level models (usually task models).  There have been a few early results concerning the automatic generation of user interfaces for limited application domains but, in large, model based design of user interfaces been slow in gaining industrial acceptance [24].  We can identify two main reasons for this:
  • Design representations are usually 'exotic' notations particular to the tool support,
  • tool support is bound to one target technology.  It becomes outdated by the time it is built.
Model based design still holds some promise, and recently, there have been larger scale efforts, involving industrial partners to develop user interface generators from higher level models (task or domain models).
The use of models in the design, 'by hand', is by now a recommended component for interaction design.  See, for example, [17] and [3].  The rise of the UML clearly provides a new momentum to the development of model based design, as it can reach a wider audience and can be embedded within recommended practices for software development.  The UML provides the opportunity of at least overcoming the first of the two limitations of model based design approaches listed above.
The CHI'97 workshop on object models in user interface design [1], addressed the question raised in the introduction, of what models are needed to support interaction design.  The workshop focused on domain and task models, but acknowledged a list of other models, ('descriptions' in their own terms), which can be recruited during interaction design.  The overall agreement was supportive of model based design, and encourages the present research but, in our view, is too optimistic concerning the uptake and utilisation of models in current practice.
The CHI'98 workshop continued the discussion, bringing the UML on the agenda for the first time.  It focused on the relation between use cases, scenarios and task models.  The problematic of using use cases and scenarios is very much in line with our own views presented in [12, 13] and those of Constantine and Lockwood [6].  The workshop proposed a plausible meta-model to relate tasks, scenarios and use cases, which could guide a UML-based interaction design method.
Wisdom  [20, 21] is such a design method, which targets small software development teams.  This target domain allows several simplifying assumptions.  The most important is that the interaction designer is a software developer, which means that  (s)he can manipulate equally well dialogue models, domain models and software architecture models.  Wisdom, is in its essence, a task based approach, although in Wisdom task models are synonymous to dialogue models, i.e. they specify the dynamic aspects of interaction with the system.  Tasks are represented in a UML based adaptation of Paterno's ConcurTaskTree notation [22] (which we see more of below).
Constantine's and Lockwood's [5], usage centred design, is a task based design method, which makes no attempt to be compatible with the UML.  It revolves around three models, a task model, a role model and a content/navigation model (see discussion below), which compliment nicely the modelling capabilities of the UML.  Its own version of use cases, is not consistent with that of the UML, but the different conception of use cases enables the focus on user tasks which is essential  for interaction design.
Recent extensions to the UML for web applications are also very relevant to the topic of this paper.  Particularly interesting is a UML based notation for specifying navigation within hyper-documents [2].  The possibility for extending this approach to modelling navigation for interactive systems in general is very attractive.  Similar to Wisdom, the notation is most useful within the context of their own design method: it encodes a data centric view of the design of interactive systems, which is very appropriate for hyper-documents, and which is complementary to the dialogue centred view embodied within Wisdom, or other task based design methods, e.g., see [22].

3.  1+6 interaction design representations in the UML

In this section we discuss the problematic of using the UML to represent scenarios (design representation 1) and the other 6 design representations listed in the introduction.

Scenarios

In the field of 'Human Computer Interaction', the term scenario is usually associated with a narrative, contextualised description of an interaction episode (although no single definition, representation or use is universally accepted).  The purpose of the scenario can be to envision, to communicate, to analyse or even to evaluate interaction designs (for a range of definition and uses of scenarios see [4]).  The term scenario, is thus understood differently than in the context of the UML, where it represents a sequence of interactions instantiating a use case.
The relation between scenarios and use cases was dealt with inconclusively in the CHI'98 workshop [1].  A thorough treatment of the topic can be found in [6].
[12] and [13] discussed the relation of scenarios and use cases, highlighting the differences in content and purpose, which are sometimes obscured by the variety of representation forms (structured text, narrative, activity diagrams, etc.).
  • Use cases describe the behaviour of the system under design not of the user using it.  In the field of interaction design a scenario captures primarily user rather than system behaviour.  Even if system behaviour only is described, an interaction scenario describes functionality from the viewpoint of a single user – who may be one of the many actors related to a use case.
  • the UML's use cases are performed by actors who, apart from human users, can also be systems.
  • Use cases specify system functionality.  At a semantic level they can serve as a specification of interactions, but they normally abstract away from user interface specifics.
There is little we can stipulate in a modelling language for the content of scenarios, apart from the possibility of traceabilty relations between the scenarios of interest to the users (and therefore the interaction designers), and the use cases they relate to.  Fulfilling this traceability requirement touches on design process issues and is outside the scope of this paper.

User Population Model

A methodical approach to the design of an interactive system should first try to identify and classify the intended and potential users of the system.  This will guide the user requirements elicitation process, the task analysis, but also activities such as the generation and elaboration of scenarios (see for example [9, 18].  In its simplest form, such a model partitions the user population, in terms of capabilities, habits, experience with technology, knowledge, etc.; in other words everything that impacts the way they will use the system. the UML's actors provide a rudimentary representational framework for representing user roles, but these 'actors' are classifiers that apply to non humans as well.  Constantine and Lockwood [5, 6] discuss the construction of a 'role model'.  The techniques they suggest to identify different roles, also discriminate users by competence and knowledge.  Representationally, no special extension to the UML is necessary, although for clarity some stereotype for modelling the partitioning into user groups would be straightforward to define and useful..

Domain Model

The domain model, as understood within model based design approaches, see for example [23, 24, 26] can be represented simply as a class diagram, and is the same as the object model produced for software development, is a class diagram that coincides with the analysis model of the Unified Process.  For some approaches, the domain model information is captured as a set of objects associated with the task model [10, 16, 22].  In either case, this is simply an object model which captures the ontology of the problem domain and structural relationships between the relevant entities.   the UML provides sufficient modelling capacities for this class of models.

Display Content and Layout

This model identifies the objects constituting the user interface, their grouping and specifies their layout, e.g. by indicating approximate placement or by indicating topological relations between groups.  Interactive objects are named after content and purpose, e.g., 'input a name', 'select a broadcast channel', etc.   Constantine and Lockwood [5] call this type of model a content model, and this is perhaps a very good indication of how it is used.
Such models can simply be sketched out on paper, with a rough indication of placement of objects on the screen, or they are more formally defined within tool supported model based design environments, e.g., as a tree structure in the Adept approach [8] or using a formal grammar as in Puerta's Mecano system [23].  There is a significant overlap with the concept of an interactor, see for example [22], although the content model does not model the software architecture, i.e. how the interactive objects are 'glued' together.
the UML's object diagrams can serve well to specify these groupings as aggregations.  The main requirement though in such a formalisation would be that it should be traceable to the navigation model discussed below, or, more likely, that they should be combined in a single model.  In this case the combined models can be thought of as an abstract interface model in the sense that the term is more widely used by the Model Based Design community: it is a model of the interface content and behaviour, abstracting away from low level  details, such as 'look and feel' of individual interactive objects, or detailed behaviour such as history, undo, etc.

Task Modelling

Human Computer Interaction research has spawned a number of design approaches, which share a commitment to base design on task analysis.  The theoretical foundations of task analysis techniques vary, but the representations used converge in the following ways:
  • The decomposition of user tasks to sub-tasks is represented as a tree structure.
  • Task trees are decorated with temporal ordering of task activity.
  • Elementary task activity is associated with task-domain representations in terms of objects.
The relation of use cases and task models is characterised by the following:
  • Use cases confound functionality used by multiple actors of the system, while task models describe separately the goal oriented activity of individual users.
  • Use cases describe the system under design unlike a task model which represents the activity of its users and their goals.
  • Use cases are performed by actors who, apart from human users, can also be systems.
We can restrict the style of writing use cases to address these points.  For example, we can write use cases only for human users relating those to their goals.  This is the approach adopted by Constantine and Lockwood with their essential use cases.
Seeking to represent tasks with use cases, in a manner more consistent with the UML, we have discussed in [13] the specification of tasks using activity diagrams and use cases.  As a carrier example, the same exemplar task was specified using a special purpose notation for task modelling, called ConcurTaskTrees [22] and a combination of use cases and activity diagrams.  The conclusions were that:
  • Use cases and activity diagrams are quite intuitive representations of temporal ordering of tasks.
  • Different use cases should specify the task of each actor using the same system functionality.
  • Both use case and activity diagrams do not show clearly task decomposition structure.  The use of an external to the UML notation like ConcurTaskTrees is a useful but not always necessary addition to the UML.
The need to represent in succinct and summative form (such as a tree structure) the decomposition relations between user tasks, was based on what is most commonly associated with task specification in the literature on user system interaction design.  As mentioned in the introduction, this is related to using task models to design the containment of corresponding interaction objects.  Parenthetically, we note that interactive objects do not have to map to tasks or subtasks.  Tasks and subtasks may simply determine the temporal ordering in which the user interacts with on screen representations of domain objects [11].

Navigation Models

In some applications, e.g., hypertext systems, a navigation model is the most relevant abstraction for the interaction designer.  In general terms it models the successive display of different groupings of interactive objects.  In the previous section where related work was discussed, it was mentioned that grouping specified by association between abstractions of interactive objects.  The navigation model of Constantine, is an annotation of a content model with screen transitions, which amounts to a state transition diagram (which is one of the most common techniques recruited by the most software-minded interaction designers, when having to design navigation).  Similarly, the Wisdom  navigation model, consists in adding navigation links between view objects.  In a similar ilk, the purpose specific notation for describing hyperdocuments, by Baumeister et al. [2] also models navigation as associations between 'nodes'.
In [13] we discussed how the UML statecharts can be used to model navigation.  In the context of the UML metamodel, this raises the question to which model element would we attach the navigation statechart.  (Note that we could consider it simply as an activity graph; however, navigation does not represent the transition of control when activities are completed, but is a user-event driven process).  A state diagram modelling navigation can be associated with a display component, e.g., the abstract drawing device for a graphical user interface, or the browser window in web applications, etc.  A state of this diagram indicates that a group of interactive components are displayed.
In conclusion, it is not clear whether navigation should be seen as a grouping or a behavioural abstraction; in the latter case we need to explicate the associations between state diagrams and interactive components.

Modelling the 'look and feel'

The 'look and feel' of an interactive system concerns the detailed reactive behaviour of interactive objects which constitute the interface.  Using the UML these objects can be described in a range of forms.  Use cases can provide a description that abstracts away from the software structure, while statecharts and interaction diagrams can provide a more detailed account of this interaction.  Special purpose notations like UAN [7] provide more expressive power to interaction designers, and are less verbose than use cases.  Compared to interaction object centred approaches, they seem to be dual capturing similar information from a user centred viewpoint. [14].

4. Towards a UML profile for modelling tasks

The approach we recommend here is summarised by the following:
  • Use cases are described in the standard fashion of the UML.
  • Each use case is associated with an activity diagram, showing on a separate 'swimlane' the activities of each its users.
  • All user activities in the use case model are traced to activities in a separate task model, associated with each class of system users.
  • The  task model for each user class is modelled by an activity diagram, which combines the activities associated with all use cases.
  • To show decomposition, we represent explicitly the expansion relation between an activity and an activity diagram.  This relates an activity in one activity diagram, with the activity diagram that expands it further.  In this manner, we can show several levels of decomposition of the task in one diagram and, also important for interaction design, name the intermediate level (sub-) tasks.
Expansion is modelled as a dependency relation between an activity diagram and and a sequential node containing the expanding activity diagram.  Normally, the UML does not explicitly show this relation, relying on name binding to represent the relation between the activity node in one diagram and its expansion in a separate activity diagram. To illustrate this relation, we use as an example the (imaginary) task of handling warrantee claims at manufacturer's service desk.  This task model is an abstraction and simplification of a more realistic task model discussed in [13].  Figure 1 shows the specification of this task using the ConcurTaskTrees formalism.

Figure 1.  An example task specification in the ConcurTaskTrees notation [22].  The diagram was produced with the aid of the CTTE editor provided by CNUCE.

The task consists in three subtasks, performed in sequence.  The tree describes a task where a service department of a manufacturing company handles warrantee claims, repairs faulty products and judges claims for warrantee cover.  These activities are performed in sequence with information flowing between them (the []>> symbol).   The resolution of the complaint can be broken down to activities that are independent of each other (the ||| symbol), one of which is optional (shown in square brackets).
This example, does not show the full features of this notation which, it must be noted, verges on the most complex and high level notations among those proposed for task modelling, aiming to bridge the gap between the needs of the analyst and the developer.  The analyst relies on the tree structure to communicate with the user, while making economical use of temporal ordering operators and the attributes associated with each task.  More detail and precision is required by developers, who need the full expressive power of the notation, plus simulation tools to validate their specification.
The following activity diagram specifies the temporal sequencing of the task related activities.  This diagram could be combined with a more extensive use case based task specification, as was shown in [13]

.

Figure 2.  An activity diagram modelling the task 'Handle Complaint'.

As mentioned in the previous section, this activity diagram does not capture the decomposition of the task to subtasks.  The intermediate nodes of the ConcurTaskTree (Record Claim, Resolve Claim, and Arrange Payment) do not map to any activity node in the diagram of figure2.
The required extension to the UML will allow mixing the hierarchical decomposition information with activity diagrams, as in figure 3.

    Figure 3.  A task model, showing the expansion of activities at middle levels of the tree to activity diagrams at the leaves of the task tree.
The activity diagram, captures essentially the same information as the concurTaskTrees diagram; note however that the complete specification of a task  in ConcurTaskTrees captures more information about task attributes and task objects.  In general, it is possible to map any binary relation between subtasks as expressed in concurTaskTrees, with activity diagrams.  In most cases, the activity diagrams tend to clutter although they are not correspondingly complex.

4.  Discussion

Representing the expansion relation as a dependency relation between activity diagrams, allows us to model the decomposition of tasks.  The resulting diagrams, tend to be very large and complex, something which should be avoided in any diagrammatic specification.  On the other hand, appropriate tool support can help manage and present this complexity in similar ways as in any tree structure; in any case this is not more cumbersome than how activity nodes are related to activity diagrams by reference in the UML.
It is debatable whether adding these hierarchical relations between activity nodes and activity diagrams is a necessary extension to the UML.  The literature on task modelling notations would certainly support this view.  However, whether it is necessary in practice to observe and manipulate these relations, as opposed to inferring them or exploring them as in current activity diagrams, is not at all certain.
Earlier works on the specification of tasks, relied on a tree structure to show task decomposition relation, which could be decorated with process algebraic operators to specify temporal sequencing.  This allows the fine details of the specifications to be gradually elaborated during the design process, possibly reflecting different concerns of the people reading and using task models.  The use of process algebraic operators promotes an extensional rather than intensional specification of temporal relations between subtasks, which leads to more abstract and parsimonious representations.
We find the activity diagrams shown above a rather clumsy rendition of the information captured in the ConcurTaskTrees notation.  The advantage of the proposed solution, is that it stays within the framework of the UML meta-model; adding a process algebraic component to specify dynamic behaviours of systems, would be an interesting variant to the UML but a much more drastic departure from the current notation.
In the example of the previous section, we have indicated how we think decomposition relations can be captured.  Several other aspects of task models, e.g., frequency of tasks, relations to task objects, allocation to humans or machine, must be captured within a task model.  We are currently working on an extension for the UML that will capture all such salient task related information.

5. References

  1. Artim, J., et al, Incorporating Work, Process and Task Analysis Into Commercial and Industrial Object-Oriented System Development, SIGCHI Bulletin, 30(4), 1998.
  2. Baumeister, H., Koch, N. and Mandel, L., Towards a UML Extension for Hypermedia Design, the UML 1999, pp.614-629, 1999
  3. Bomsdorf, B., and Szwillus, G., From Task to Dialogue: Task Based User Interface Design, SIGCHI Bulletin, 30(4), 1998.
  4. J. Carrol (Editor) Scenario Based Design, Wiley, 1995.
  5. Constantine, L.L. and Lockwood, L.A.D, Software for Use: A Practical Guide to the Models and Methods of Usage Centered design, Addison-Wesley, 1999.
  6. Constantine, L.L. and Lockwood, L.A.D,  Structure and Style in Use Cases for User Interface Design, in van Harmelen, M., (Editor), Object-Oriented User Interface Design, (in press).
  7. Hix, D. and Hartson, R.H Developing user interfaces, Wiley, 1993.
  8. Johnson, P, Wilson, S., Markopoulos, P., and Pycock, J. (1993) ADEPT - Advanced design environment for Prototyping with Task Models, INTERCHI'93, Addison-Wesley, pp.55-56, 1993.
  9. Kaindl, H., Combining goals and functional requirements in a scenario-based design process.  In Johnson, H., Nigay, L. and Roast, C. (Eds.) Proceedings BCS-HCI’98, Springer, 1998, pp. 101-120.
  10. Markopoulos P, Wilson S, Pycock J & Johnson P, Adept-A task based design environment, in Shriver BD (Editor) HICSS-25, Vol. II,  IEEE Computer Society Press (California), 1992, pp. 587-596.
  11. Markopoulos, P.,  Johnson, P., and Rowson, Formal Aspects of Task Based Design, in Harrison, M., and Torres, J-C, (Editors), DSV-IS'97, Springer, 1997.
  12. Markopoulos, P.,  Bekker, M., and Marijnissen, P, Interaction Design and the USDP: Should the UML do it all?  DSV-IS 2000, Limerick Ireland, June 2000. http://www.tue.ipo.nl/pmarkopo
  13. Markopoulos, P. and Marijnissen, P, the UML as a representation for Interaction Design, paper accepted for publication in OZCHI 2000,  http://www.tue.ipo.nl/pmarkopo
  14. Markopoulos, P., Papatzanis, G., Johnson, P., & Rowson, J., Validating semi-formal specifications of interactors as design representations, in Markopoulos, P., & Johnson, P., (Editors), DSV-IS ’98, Springer-Verlag, 1998, pp.102-116. 
  15. Markopoulos, P., Shrubsole, P., and de Vet, J., Re-finement of the PAC-model for the component-based de-sign and specification of  television based user inter-faces, Duke, D.J. and Puerta, A. (Eds.)  DSV-IS ’99, Springer, 1999, pp. 117-132.
  16. Markopoulos P, Wilson S & Johnson P, Representation and Use of Task Knowledge in a User Interface Design Environment,  IEE Proceedings~E, Computers and Digital Techniques, Vol. 141, No. 2, 1994, pp. 79-84.
  17. Mayhew, D.J., The Usability Engineering Lifecycle, A Handbook for User Interface Design, Morgan Kaufmann, 1999.
  18. Nielsen, J., Scenarios in Discount Usability Engineering.  In  [4], 1995, 59-84.
  19. Nunes, N., et al, Workshop on Interactive System Design and Object Models, in Moreia, A., (Editors), ECOOP’99 Workshop Reader, Springer-Verlag LNCS, 1999.
  20. Nunes, N., and Gunha, J-F, A Bridge too Far:  The Wisdom Approach, in [ 19 ]
  21. Nunes, N., and Gunha, J-F, Wisdom: a UML based Architecture for Interactive Systems, in DSV-IS 2000, June, 2000.
  22. Paterno, F., Model Based Design and Evalluation of Interactive Applications, Springer-Verlag, London, 1999.
  23. Puerta, A., A model based interface development environement, IEEE Software 14(4) July/August 1997, pp.41-47, 1997.
  24. Szekely, P., Retrospective and Challenges for Model Based Interface Development, in Vanderdonckt, J., (Editor), CADUI'96, Presses Universitaires de Namur, Namur, 1996. 
  25. van Harmelen, M., et al, Object Models in User Interface Design: A CHI 97 Workshop, SIGCHI Bulletin, 29(4), ACM; 1997.
  26. Wilson S, Markopoulos P, Pycock J & Johnson P, Models in User Interface Design, in Gornostaev J (Editors) EWHCI’92, St. Petersburg, Russia, 4-8 August, 1992, International Centre for Scientific and Technological Information, Moscow, 1992, pp. 210-217.