It's the journey and the destination:
Shape and the emergent property of genre in evaluating digital documents.
Andrew Dillon and Misha Vaughan
This item is not the definitive copy. Please use the following citation when referencing this material:
Dillon and Vaughan (1997) It's the journey and the destination: Shape and the emergent property of genre in digital documents.
New Review of Multimedia and Hypermedia, 3, 91-106.
To anyone versed in the literature on hypermedia, it is clear that the
last 10 years' worth of research on usability since Conklin's (1987) seminal
article has largely been ignored by web designers. Surfing web sites even
casually will likely expose a user to screens of badly formatted text,
superfluous graphics, mixed fonts, unreadable color combinations, and
dangling or dead links. While the issue of knowledge transfer between
research disciplines and design practice is fraught with problems and
is a fascinating topic in and of itself (see e.g., Klein and Eason, 1993),
this is not the focus of the present paper. Instead we wish to extend
work that started with the birth of hypertext systems and continues to
demand attention in these days of free-for-all web design: the evaluation
of user behaviour in electronic space.
Specifically, this paper will extend the analysis
of 'user navigation' to the evaluation of user behaviour in web environments.
In so doing, the present authors will attempt to unify work in the area of
structural representation of content with models of navigation based on physical
movement. To do this, we will partly jettison the metaphor of navigation as
it has generally been employed in hypermedia literature in favor of the concept
of shape, which Dillon and Schaap (1996) invoked to explain expert user
performance in information structuring tasks. As we will attempt to
demonstrate, shape offers a more cohesive construct for understanding user
behavior in digital domains, and reconciles the logical division in the hypermedia
literature between navigation through physical and semantic domains. Throughout,
we will provide examples of our work which tap this construct in user testing.
2. The state of our knowledge on human use of hypermedia
Despite the rush to digitize the world of information,
few examples of well-designed hypermedia exist. While such a statement may
appear unreasonable to most people, we need to clarify what we mean by well-designed.
In HCI terms, a well-designed system is one that offers utility and usability
to its intended users at a reasonable price. By utility is meant the support
of meaningful tasks which the user wishes to or needs to perform. By usable
is meant that intended users can exploit the utility of the technology in
an efficient, effective and satisfying manner (Shackel, 1991).What constitutes
reasonable with respect to cost will not be discussed further here.
To claim there is little evidence for the existence
of well-designed hypermedia is like trying to prove a null hypothesis, no
matter what you demonstrate, there is always room for another view. However,
it is remarkable that the published literature on usability evaluations of
hypermedia have produced but a handful of results that are encouraging. Perhaps
the most pointed critique in recent years has been Landauer's (1995) review
of the evidence for hypermedia. He concluded that:
"There have been nine scientifically satisfactory studies...In almost all
cases, users were quicker and more successful using paper and print than the
electronic form" (p. 260).
While the conclusion is counter-intuitive to most
people, we should perhaps be more surprised by the apparent paucity of studies
on this technology that Landauer found scientifically acceptable. Reports
on hypermedia design and use fill journals and conference proceedings, frequently
suggesting powerful improvements in learning and efficiency through the use
of this technology, yet Landauer can find only nine that are scientifically
satisfactory. Chen and Rada (1996) identified 23 studies of hypermedia
use for their review of effect sizes, and reached slightly less disappointing
conclusions, but their criteria appear to have been much less stringent than
Landauer's, and they included unpublished dissertations and studies that have
been critically received elsewhere.
Dillon and Gabbard (in press) report a detailed
review of the findings on learning outcomes from hypermedia use in educational
studies, an area where the utility of hypermedia is taken for granted. They
concluded that the new technology most often failed to yield any significant
difference in learner performance. Like Landauer (1995) they noted that many
of the experimental studies in the literature are poorly designed, fail to
adequately manipulate variables, utilize controls, or report statistical analyses.
These authors summarized the literature as indicating that hypermedia had
largely failed to demonstrate any practical advantage to learners in most
critical thinking tasks - a conclusion completely at odds with the positive
endorsement of this technology at the practical level.
If controlled experimentation with constrained hypermedia
is demonstrating the difficulties involved in designing usable hypermedia,
then the emergence of the web only exacerbates matters. While an evolutionist
might argue that time will select the best forms of digital documentation,
the scientist cannot sit idly by. If we are interested in improving human
performance with computing technologies, then we must learn to design more
useful and usable digital documents.
McKnight et al (1991) argued that the major problem
with hypertext was navigation, and despite the occasional dismissals of this
as an issue that truly affects users (e.g., Landow, 1992), this topic is continually
raised in the writings on hypermedia use, and regarded as a serious form of
cognitive overhead for users of extended digital documents. Furthermore, the
literature reviews cited above all note that user difficulties in locating
and organizing information are major sources of the performance deficit in
hypermedia. Thus, in the present paper we will examine what is known about
navigation, and then suggest an alternative view of information organization
that may help designers and evaluators improve matters.
2. The navigation issue - an update
Conceptually, navigation in hypermedia environments
has been likened to movement through physical space. However, Dillon, McKnight,
and Richardson (1993) argue that in terms of navigating physical spaces, theory
and research suggest that individuals form three types of mental representations
over time (landmarks, routes, and surveys) which have never been adequately
measured in electronic environments. In the physical world, a person begins
to represent their surroundings by developing landmark knowledge of a geographical
location (Siegel & White, 1975; Siegel, Kirasic, & Kail, 1978).
Landmark knowledge is primarily a function of visual stimuli and a person
who has just entered a new location will attend to the local topography (Mandler,
1984) and identify unique spatial configurations (Siegel & White, 1975).
Dillon et al describe landmarks as "any features of the environment which
are relatively stable and conspicuous" (p. 173). For example, a woman
arrives in a new city and takes a cab from the airport to her hotel.
She sees a restaurant on the way to the hotel and decides to stop there for
dinner. To get there, she will recall that the restaurant was across
from building X but past the park. After traversing to the restaurant
a few times, she will begin to develop route knowledge.
Route knowledge is developed as a function of physically
navigating a path to a destination (Siegel, Kirasic, & Kail, 1978; Thorndyke
& Hayes-Roth, 1982). Dillon et al (1993) define route knowledge
as "the ability to navigate from point A to point B, using whatever landmark
knowledge we have acquired to make decisions about when to turn left or right"
(p. 173). Our woman, when leaving the restaurant to return to the hotel, would
begin to get a sense that the hotel lies around the corner and up that road
a ways. After visiting this particular restaurant a few more times,
she would begin to develop a sense of survey knowledge regarding the terrain
between the hotel and the restaurant.
Survey knowledge has been described as a gestalt
sense of a physical environment that allows an individual to visualize the
environment in terms of an external map (Siegel & White, 1975; Thorndyke
& Hayes-Roth, 1982). Dillon et al (1993) argue that this ability
"allows us to give directions or plan journeys along routes we have not directly
traveled as well as describe relative locations of landmarks within an environment"
(p. 174). Our traveler at this point, would begin to navigate the distance
to the restaurant in terms of "west on main to the intersection at 5th."
The literature on navigation frequently assumes
an invariant developmental sequence of landmark to route to survey which
might not be correct. Conceivably, a person may develop survey
knowledge independent of actually navigating a physical space through the
use of external representations such as maps. Furthermore, several studies
suggest that landmark, route, and survey knowledge are each best suited to
different types of tasks (Thorndyke, 1980; Thorndyke & Hayes-Roth, 1982),
and that individual differences such as spatial ability (Pellegrino, Alderton,
& Shute, 1984; Stanney & Salvendy, 1995), sex differences (Geary,
1995), or learning techniques (Thorndyke & Stasz, 1980) may influence
the use of different mental representations. The physical theory also
falls short of explaining how other human factors such as value judgments
about a location or affective responses to a location, such as symbolic landmarks
(Presson & Montello, 1988), may influence the formation of mental representations.
Dillon et al (1993) argue that one way to tie all
these representations together is to consider each of them a form of schematic
representation. Schema theory suggests that a person's knowledge is
generally organized and structured, and that a person develops this mental
structure through repeated interactions with the environment (Mandler, 1984).
These mental structures are abstract "models" that guide future interactions
with the real world. In turn, these models are then updated based on
new encounters. This general approach to knowledge representation would seem
to fit well with the development of knowledge for physical spaces. Landmark,
route, and survey knowledge each guides a person's interactions with 3 dimensional
space and develops through repeated interactions with the environment.
There is evidence to support this interpretation. Knowledge for places
(e.g. a room and its contents) (Mandler, 1984), and memory for maps (Okabayashi
& Glynn, 1984) have both been interpreted as schematic in form, and there
is evidence that spatial knowledge may be organized into hierarchically ordered
chunks based on perceived boundaries (Johnson & Hasher, 1987).
If schematic representations are involved, then
it is likely that navigation through semantic space is a more complicated
than simply acquiring a sense of physical place. Dillon et al (1993) point
out that any physical form of information is at best a three-dimensional representation
of n-dimensional concepts. However, this point is rarely addressed in the
literature where the analogy between navigating physical space and electronic
space is common (though see an interesting attempt in Kaplan and Moulthrop,
1994). As testament to the perceived importance of navigation and hypertext,
navigational strategies have become a means of assessing the usability of
hypermedia products (e.g. Leventhal, Teasley, Instone, Rohlman, and Farhat,
1993). In addition, several studies have sought to improve the navigational
performance of users through better system and interface design (e.g. Gray
& Shasha, 1989; Kerr, 1990; Kim & Hirtle, 1995; Parunak, 1989), as
well as to characterize users' navigational strategies (Canter, Rivers, &
Storrs, 1985). For more understandable reasons, navigation in physical spaces
remains the dominant model for work in virtual world wayfinding also (see
e.g., Darkin and Silbert, 1996)
However, the conceptualization of hypermedia navigation
as emerging through landmark, route, and survey representations has rarely
been the subject of research in the human-computer interaction (HCI) community.
A fine example of the empirical tradition of HCI research on hypermedia is
the study Simpson and McKnight (1990) who found evidence for a link between
the accuracy of a user's mental map and her efficiency in navigating a hypertext
space. In this study they used a hierarchically organized hypertext,
and two forms of a table of contents to aid navigation, a hierarchical list
of contents and an alphabetical index. Subjects using the hierarchical list
provided more accurate graphical maps of the hypertext and proved to be more
efficient than subjects using the alphabetical index. The hierarchical
list, which reflected the hierarchically organized hypertext, apparently provided
users with a more accurate representation of the hypertext space. In terms
of hypermedia spaces, this suggests that the tighter the coupling between
an access device and the underlying information structure, the more effectively
a person will be able to use it. While the authors utilize the navigation
construct to describe the process of interaction, they are not wed to it as
a means of explaining knowledge development or user behavior at the interface
and their findings are explicable without needing to mention navigation as
it is understood in the physical world.
There have been some specific studies of the development
of navigation knowledge in electronic space and the relationship between this
and user performance, but the results are mixed. Kerr (1990) examined cues
in a manner which could be regarded as assessing the nature of landmark characteristics
in an information space. Subjects used a database with one of four different
sets of navigational cues (text, color, graphics, and a combination of all
three). No significant differences were found on speed, accuracy, or
efficiency measures for the set of tasks, but subjects reported being significantly
more aware of and using more often textual and graphic cues, to a lesser extent
combination cues (textual, color, and graphic cues), and least of all color
cues. Kerr also asked subjects to give their impression of the database,
verbally and/or graphically, to capture the subjects' conceptualizations of
the database. The descriptions, both verbal and graphic, were coded
for complexity and degree of graphic detail. The faster searchers tended
to give more detailed and graphic impressions, while the slower searchers
tended to give simpler, more verbal impressions. This suggests that
the faster searchers had a more accurate and rich mental representation of
the database and could reproduce this representation. However, since these
differences were observed independent of environmental cues manipulated, it
suggests that the form of knowledge manifest by users was driven more by individual
differences in cognition than any characteristic of the electronic space.
Leventhal et al (1993) examined the navigational
strategies of users working with a hypertext system over three trials.
The system was designed to facilitate hierarchical navigation, linear navigation,
text searching, and hyperlinked navigation. Their data demonstrate that
users made significantly more use of hierarchical navigation aids in the earlier
stages of using the system and with more complicated task types, suggesting,
according to these authors, the emergence of distinct forms of navigational
knowledge in line with the evolutionary model of physical navigation.
Certainly number of cards visited dropped over time, suggesting greater efficiency
of use but use of the hierarchical navigation aids was not significantly related
Heffron, Dillon, & Mostafa (1996) explicitly
tackled that notion of what constitutes a landmark in hypertext. They
asked subjects to complete an in-depth searching task on a web site and afterwards
to recall features of the web site they remembered seeing. Interestingly,
half of the subjects were unable to recall even one feature, and while the
authors note that this may have been a function of the manner in which the
recall task sought to elicit responses, these data are a surprise to anyone
who thinks that visual features are automatically encoded for recall.
Features of the web site that were recalled included specific locations, i.e.
two home pages, as well as color, high-lighted terms, and keywords.
Evidence so far has suggested that users do
make use of some landmark-type information, although what constitutes landmarks
in information space remains unclear, and common sense ideas of memory for
details may not accurately describe what is happening. Missing from these
data is evidence which suggests that users create distinct stages of landmark,
route, and survey representations of an information space over time, or a
clear indication that more sophisticated navigational knowledge (however measured)
leads to better performance. Thus, while navigation may be a useful process
measure to take in evaluation in order to better understand what users are
doing, the theoretical framework that this metaphor carries with it is largely
unsupported in the literature on hypermedia. Furthermore, it seems mostly
related to efficiency measures of performance at best i.e., how fast a user
can find information, therefore tapping knowledge of physical structures.
In terms of building a map of the semantic space, the navigational
model sheds no light.
3. Shape - a reasonable alternative?
While the physical model of navigation has many
attractions as a lens for viewing the process of hypermedia use, there are
some serious shortcomings to its theoretical value. In the first instance,
the user of a hypermedia system is largely static, with a view that changes
in front of him or her. If we are to invoke navigational literature based
on studies in physical environments. then it would appear logically more appropriate
to do so for findings based on studies of drivers or pilots (see e.g., Wickens,
1992) where cues for movement are exploited differently and relative position
cannot be related easily to kinesthetic memory for direction. To date,
few hypermedia researchers have made this link, and given our general reservations
about the utility of the physical framework, the present authors will not
extend this analysis here though it appears ripe for further review.
In the second instance, and perhaps more importantly,
the physical navigation framework fails to tackle the issue of semantic space
which Dillon et al, (1993) suggested to be the more important of the senses
of space employed in hypermedia. Certainly from the user's perspective, semantic
space has immediate resonance. Arguments must be followed, information has
sequence and structure, and while the mapping between physical and semantic
space might be tight in restricted domains such as menu hierarchies, this
coupling can be very loose in extended, detailed documents incorporating multiple
However, all metaphors extend only so far. In HCI
is it apparent that the metaphors of desktops and trashcans have limitations
for encouraging use of computers, and the information superhighway metaphor
for the internet is already a cliche. In our view, the metaphor of navigation
in hypermedia use is perhaps better than most, but it also serves to
limit our analyses of interaction more than inform them in general research
studies. As we evaluate usability, a concern with the process of movement
through the information space is important, but by thinking always of navigation
there is a tendency to concentrate on physical behaviors that are reflections
or indices, at least in part, of semantic processes, without directly examining
the semantic component of information use (comprehension, finding meaning
and relevance) that for most hypermedia designers is central.
Dillon (1994) proposed the TIMS framework for conceptualising
hypermedia for evaluation purposes. This framework posits four general components
of user activity in information usage: task formation and monitoring, information
modeling, manipulation of resources, and standard reading activities of visual
processing. Each of these components is manifest in the verbal protocols of
digital document users (see Dillon, 1996 for extended analysis). A central
issue in TIMS is the information model users employ, which it is argued can
be based on experience (the invocation of schematic structures from long-term
memory) or on the perception of structure in the current environment. The
latter example will occur where there exist no previous structural equivalents
for the user in their knowledge base, but even then, experience is likely
to drive the search for structural cues on the part of the user. Regardless
of primary source, the user's information model mediates task activities and
psychophysiological interaction with the digital document by serving to direct
attention and assimilate data on the organization of the information resource
with which the user is interacting.
In purely physical navigation terms, the cognition
underlying the information modeling act could be seen as supporting the formation
of a mental map of surroundings, influenced by knowledge of previous environments
one has encountered, but in so doing, the physical world is represented only
in terms of the connectivity and clustering of landmarks into routes and complete
maps. In the conceptual space of hypermedia such a map is not enough, equating
as it does to knowledge of where information is located, not what it means.
Physical navigation has as its raison d'etre the arrival of a user at some destination. Once there,
the model or map has served its purpose. In hypermedia, such a model would
only be useful in part, if at all, since the object of the user's interaction
is to gain some information for use, not merely to arrive at a screen containing
it. This might not matter if use could be factored out, but it is not sensible
to do so since processing the information even partially serves to impact
the user's model.
A case can still be made for maintaining the physical
model (and indeed we shall do so in the conclusion of this paper, albeit on
more limited grounds) but doing so as our primary evaluative strategy requires
a blurring of distinctions between physical and semantic forms that is more
likely to confuse than enlighten (if the previous 10 years of writing on this
topic is any guide). One alternative we would like to propose is the re-focusing
of efforts on this topic away from the physical issues associated with navigation
towards the potentially more fruitful concept of shape.
The concept of shape in information space has several
antecedents that are worth considering here to establish what we mean by this
term. Bazerman (1987) spoke of shaping information as part of the writing
process scientists engage in, and while this term is close to the meaning
intended here (and has practical significance for design that we will allude
to later), a closer approximation can be found in the literature on human
cognition in discourse processing. Particularly, the work of van Dijk and
Kintsch (1983) and Mandler (1984) which focused on schematic superstructures
is closer to the meaning intended here.
Shape is a property of information that is conveyed
both by physical form and information content. Separating these elements completely
is perhaps impossible but one can talk of the distinction between the layout
and sequencing of information as viewed by the consumer (user or reader) and
the cognitive representation of meaning that employs (at least in theoretical
terms) knowledge structures such as schemata, mental models and scripts. The
representation of meaning is crucial to any analysis of hypermedia design
and use as it gives the task real human value, yet it is precisely this component
that is frequently by-passed in evaluations rooted directly in physical navigation
By invoking shape as a construct to consider in
design and evaluation of hypermedia we anticipate tapping more directly the
variables that determine the success or failure of a hypermedia - namely the
facility with which a user can exploit the information resource to satisfice
their task requirements. For users to gain from their interaction, they must
do more than locate target information, at least in most real-world tasks.
They must assimilate information into their own knowledge structures, an activity
that extends processing beyond layout (the classic physical navigation concern)
Meaning emerged as an important variable in our
research into experts' abilities to exploit cues in isolated paragraphs of
text to determine location in an unseen article (Dillon 1990). The capability
to perform this task with approximately 80% accuracy without reading for meaning
demonstrated that members of a discourse community were capable of determining
organization on the basis of perceptual cues. When novices where shown to
be unable to perform this task as well (Dillon and Schaap, 1996) it suggested
a phenomenon that was more than the classic form of navigation considered
heretofore in the hypermedia literature. Instead, it appears that physical
properties of information structure are tightly linked to conceptual properties
according to rules that are more complex than those of pure physical organization,
and that experts seem to be aware of these and to respond to many of them
A further set of investigations by the present authors
(Dillon and Vaughan, 1997) sought to tap into this process by examining the
verbal protocols of expert users of this document form. Manipulating the presence
or absence of obvious perceptual cues in the paragraphs employed in Dillon
and Schaap (1996), it emerged that experts were using more complex rationales
for their classification than simple perceptual cues. Information content,
wording, relationships to a process were all seen as indicators of location.
While in part this may result from the necessity to generate a rationale rather
than merely classify as before, the protocols frequently alluded to the manner
in which authors were referring to points, or the inferences that authors
seemed to be drawing. In other words, the semantic content of the paragraphs
was important in gaining a sense of location, not just the perceptual cues
inherent in the display. Experts employed both. As such, the
scientific article appeared to have a shape similar to that shown in Fig 1.
Here, the article seems to build relationships between elements into a structural
form that starts broad and narrows before broadening again over time.
All well and good one might say, but surely this
only means people are comprehending as well as navigating? One may proceed
with this line of argument, but to us it misses the point. Comprehension is
not something 'other than' navigation, some form of task that is independent
of the process of moving through the information space. Rather it is an intrinsic
component of information use. Furthermore, in an unpublished study, the first
author attempted to mimic expert performance with novice users who were provided
with a set of rules for allocating paragraphs (i.e., visual cues rather than
semantic processing was involved), but novices were unable to do so at levels
anyway close to expert performance. So, while physical navigation might be
neatly divorced from the purpose of the journey, interactions with digital
documents are not so easily divided. The purpose of moving though the information
space is frequently the same purpose as the journey, to reach an end point
of comprehension - and in this case the journey is the destination.
For sure there are scenarios where the user is navigating
in a form that is more typical of the physical model but these occasions are
not the norm, and are more likely to reflect situations where the user is
trying to locate a specific detail, or place in the document that is not marked
or linked, occasions that mark a breakdown in the ideal form of interaction
between information user and medium. To limit our conceptualizations (and
by extension our evaluations) of interaction to issues of this kind is unlikely
to serve us well. Furthermore, it is rare that a user would develop a model
of the information space that attains full map or survey knowledge status
without a commensurate gain in knowledge of content, but we fail to appreciate
that knowledge gain if we concentrate solely on physical form, and it is precisely
that approach which leads to speculation about novices acquiring an expert's
knowledge by following the experts links in a semantic network (see Jonassen,
So how does the concept of shape help? Clearly,
it embraces much of what we know about navigation in information environments
by acknowledging the value of physical attributes in aiding or hindering access
to material. Thus shape has a very physical component to it as a construct
for information design and it subsumes what is known from research on navigation.
However, in terms of content, shape invokes concerns with discourse organization,
such as form, flow, genre, and argument. It assumes users must manipulate
and move through information space, but instead of by-passing concerns with
the nature of the material being read or viewed, shape issues focus attention
on the regularities of discourse that are followed by both creator and user
in an ideal sense.
In so doing, shape draws on both social and cognitive
theories of reading, writing, communication and design. It seeks to focus
on the traditions, expectations and stylisms of discourse communities as well
as to focus on the principles and findings of the HCI community to create
usable hypermedia. While users can still be thought of as navigators in some
sense, it is not as models of physical space, but as travelers in semantic
environments that interest us here - and shape would appear a stronger hypothetical
Applying shape to hypermedia evaluations
It is important to re-iterate that the metaphor
of navigation is not being rejected wholesale. It is still our view that capturing
process measures of interaction is vital to improving hypermedia design, and
data relating to user movement through an information space is always necessary
to capture. Thus, in one very important way, nothing has changed by shifting
the emphasis to shape. In the usability lab or the field, we still capture
data on selections, paths, route to goal etc.
What shape does bring to the evaluation however
is a new focus on the form of content - the semantic relationship between
information elements. Rather than merely examining the navigation aids present
in the interface, e.g., ensuring that there are easily marked exists on every
screen (Nielsen,1993), this construct also draws attention to the logic of
order that a discourse community might share. This could be the superstructure
of the scientific article, the plot of a narrative, or the clustering of a
newspaper. In this way, an understanding of genre becomes central to the evaluation
and design process, and user expectancies are not violated, no matter how
easy it might be for users to move around.
Of necessity, the concept of genre invokes ideas
of discourse communities, traditional forms and emerging digital document
types. Users are assumed to base their decisions on movement through information
space in part on these attributes of genre. Where the genre is violated, ignored,
or may not yet exist (a likely occurrence in hypermedia forms of document)
then the user is left to rely on the more traditional interface features of
back-forward buttons and hot links to gain perspective. In such a situation,
we would capture navigational data in a manner typical of the physical model
of navigation. Where the users have experience with the genre, evaluation
would seek to identify the presence or absence of design cues that enhance
the user's perception of order. Raw navigational data would only tell part
of the story here and verbal protocols, clustering exercises, drawings (e.g.,
see Figure 2) and assembly tasks are all techniques that have been usefully
Clearly, this approach does not establish a new
method or require that evaluations are performed differently. Rather the construct
of shape draws attention explicitly to the mix of perceptual, cognitive and
social variables at the interface between people and information. Process
measures of navigation such as time spent on screens, buttons selected, menu
options taken, back-forward movements etc. are all still taken but they are
complemented with other measures as suggested here. Furthermore, analysis
of navigation is placed in the context of semantic as well as physical relationships
by considering the shape of information. Hence, our suggestion is to keep
navigation but in a less central analytic role than before, and more a supplement
to or even an index of shape. In figure 2 for example, we would employ the
user's diagram to solicit comments on organisation, semantic relationships,
and sense of order perceived by the user. In short, we gain all we would from
a typical navigation analysis and extend it to assess semantics.
The present paper has explored the most common process
metaphor in hypermedia evaluation and shown the inconsistencies and limitations
inherent in its use. As an alternative, the concept of shape is offered. In
practical terms this does not mean that we reject navigation measures in our
evaluations but that we resist the temptation to rely on them as crucial indices
of the outcome of interaction. At best, navigation measures yield insight
into the efficiency with which users can move through the information space,
but the relationship between this and the value of the interaction to the
user in terms of information retrieved or comprehended is at best oblique.
Navigation measures can yield insight into interface usability in efficiency
terms only but understanding the information space meaningfully requires more
than an ability to navigate it.
Thus we advocate navigation's retention as a measure
but only one in a mix of measures needed to gain the full impression of a
hypermedia's worth. By subsuming it under the umbrella term 'shape' we seek
to establish an appropriate context for performance measures to be taken and
for re-design options to be considered. Thus, we can consider research at
the interface level to establish the form of features that aid users locating
themselves and moving through the information space. We are strongly of the
view that within this framework, longitudinal studies of users' interactions
with digital documents or sites are needed to determine how well the physical
metaphor holds up to learning the organization of digital spaces. However,
as far as understanding the true utility of hypermedia systems, the concept
of shape demands we focus on the evolution of semantic knowledge of content
which, while it may be influenced by and partly rest on navigational knowledge,
is much more than just a sense of organization and requires integration of
meaning and form in a manner that is guided by socio-cultural forces.
Shape remains a somewhat elusive construct and we
recognise our own limitations in attempting to extract it for analysis, however
we believe evidence for its existence is strong and it's pragmatic utility
is clear. Future research efforts must concentrate on developing a reliable
and valid means both of extracting users' conceptualizations of shape and
providing designers with the tools for applying shape characteristics to their
The authors would like to thank comments from three
anonymous reviewers that led to significant improvements in this paper.
Canter, D., Rivers, R. & Storrs, G. (1985). Characterizing user navigation
through complex data structures. Behaviour and Information Technology, 4, 93-102.
Chen, C. and Rada, R. (1996). Interacting with Hypertext: A Meta-Analysis
of Experimental Studies. Human-Computer Interaction.
11, 2, 125-156
Conklin, J. (1987, September). Hypertext: An introduction and survey. Computer, p.17-41.
Darkin, R. and Sibert, J. (1996) Wayfinding strategies and behaviors in large
virtual worlds. Proc. CHI'96. ACM SIGCHI Conference on Human Computer Interaction. New York: ACM Press, 142-149.
van Dijk, T and Kintsch, W. (1983) Strategies of Discourse Comprehension, New York: Academic Press.
Dillon, A. (1991). Readers' models of text structures: The case of academic
articles. International Journal of Man Machine Studies,
Dillon ,A. (1994) Designing Usable Electronic Text: An Ergonomic
Analysis of Human Information Usage. London: Taylor and Francis.
Dillon, A. & Schaap, D. (1996). Expertise and the perception of shape
in information. Journal of the American Society for Information Science,
Dillon, A. and Gabbard, R. (in press) Hypermedia as an educational
technology: a review of the empirical literature on learner comprehension,
control and style. [contact email@example.com for updates]
Dillon, A. & Vaughan, M. W. (1997). Mental representations of complex
information spaces. Poster Sessions: Abridged Proceedings of the HCI International
'97 7th International Conference on Human-Computer Interaction jointly with
13th Symposium on Human Interface (Japan)
(pp. 96). West Lafayette, Indiana: School of Industrial Engineering, Purdue
Dillon, A., McKnight, C., & Richardson, J. (1993). Space - the final
chapter or why physical representations are not semantic intentions.
In C. McKnight, A. Dillon, & J. Richardson (Eds.), Hypertext:
A psychological perspective (pp. 169-191). New York: Ellis Horwood.
Edwards, D. W. & Hardman, L. (1989). 'Lost in hyperspace': Cognitive
mapping and navigation in a hypertext environment. In R. McAleese (Ed.),
Hypertext: Theory into practice (pp. 90-105).
Geary, C. (1995). Sexual selection and sex differences in spatial cognition.
Learning and Individual Differences, 7, 289-301.
Gray, S. H. & Shasha, D. (1989). To link or not to link? Empirical guidance
for the design of nonlinear text systems. Behavior Research Methods, Instruments,
& Computers, 21, 326-333.
Heffron, J. K., Dillon, A., & Mostafa, J. (1996). Landmarks in the World
Wide Web: A preliminary study. In S. Hardin (ed.), Proceedings of the 59th
Annual Meeting of the American Society for Information Science (pp. 143-145). Medford, New Jersey: Information Today,
Jonassen, D. and Wang, S. (1993). Acquiring Structural Knowledge from Semantically
Structured Hypertext. Journal of Computer-Based Instruction.
Johnson, M. K. & Hasher, L. (1987) Human learning and memory. Annual
Review of Psychology, 38, 631-668.
Kaplan, N. and Moulthrop, S. (1994) Where no mind has gone before: ontological
design for virtual spaces. ECHT'94 Proceedings: ACM European Conference
on Hypermedia Technology, New York: ACM,
Kerr, S. T. (1990). Wayfinding in an electronic database: The relative
importance of navigational cues vs. mental models. Information Processing
& Management, 26, 511-523.
Kim, H. & Hirtle, S. C. (1995). Spatial metaphors and disorientation
in hypertext browsing. Behaviour and Information Technology,
Klein, L. and Eason, K. (1993) Putting social science to work.
Cambridge: Cambridge University Press.
Leventhal, L. Teasley, B. Instone, K, Rohlman, D. and Farhat, J. (1993) Sleuthing in HyperHolmes: an evaluation
of using hypertext vs. a book to answer questions. Behaviour and
Information Technology, 12(3) 149-164
Mandler, J. M. (1984). Stories, scripts, and scenes: Aspects of schema
theory. Hillsdale, NJ: Lawrence Erlbaum.
Nielsen, J. (1993) Usability Engineering.
Cambridge MA: Academic Press.
Okabayashi, H. & Glynn, S. N. (1984). Spatial cognition: Systematic
distortions in cognitive maps. The Journal of General Psychology,
Landow, G. (1992). Hypertext: The Convergence of Contemporary Critical
Theory and Technology. Baltimore: Johns
Hopkins University Press
Landauer, T. (1995) The Trouble with Computers
Cambridge MA: MIT Press
McKnight, C., Dillon, A. and Richardson, J. (1991) Hypertext in Context, Cambridge: Cambridge University Press.
Parunak, H. V. D. (1989). Hypermedia topologies and user navigation. Hypertext
'89 Proceedings (pp. 43-50). New York: The Association
for Computing Machinery.
Pellegrino, J. W., Alderton, D. L., & Shute, V. J. (1984). Understanding
spatial ability. Educational Psychologist,
Presson, C. C. & Montello, D. R. (1988). Points of reference in spatial
cognition: Stalking the elusive landmark. British Journal of Psychology, 6, 369-393.
Shackel, B. (1991) Usability - Context, Framework, Definition, Design
and Evaluation. In: Shackel, B. and S. Richardson (eds.) Human Factors
for Informatics Usability. Cambridge:
Cambridge University Press. 21-38
Siegel, A. W., & White, S. H. (1975). The development of spatial representations
of large-scale environments. In H. W. Reese (Ed.), Advances in childhood
development and behavior (Vol. 10, pp. 9-55). New York: Academic
Siegel, A. W., Kirasic, K. C., & Kail, R. V. (1978). Stalking the elusive
cognitive map: the development of children's representations of geographic
space. In J. F. Wohlwill & I. Altman (Eds.), Human behavior and
the environment: Children and the environment
(Vol. 3, pp. 223-258). New York: Plenum.
Simpson, A. & McKnight, C. (1990). Navigation in hypertext:
Structural cues and mental maps. In R. McAleese & C. Green, Hypertext:
State of the Art (pp. 73-83). Oxford: Intellect.
Stanney, K., & Salvendy, G. (1995). Information visualization:
Assisting low spatial individual with information access tasks through the
use of visual mediators. Ergonomics, 38, 1184-1198.
Thorndyke, P. W. (1980). Performance models for spatial and locational
cognition (Report No. R-2676). Santa
Monica, CA: Rand.
Thorndyke, P. W. & Hayes-Roth, B. (1982). Differences in spatial
knowledge acquired from maps and navigation. Cognitive Psychology,
Thorndyke, P. W. & Stasz, C. (1980). Individual differences in procedures
for knowledge acquisition from maps. Cognitive Psychology,
Wickens, C. D. (1992). Engineering psychology and human performance (2nd ed.) New York: Harper Collins.