Artifacts as Theories: Convergence Through User-Centered Design
This item is not the definitive copy. Please use the following citation when referencing this material: Dillon, A. (1995) Artifacts as Theories: Convergence through User-Centered Design. Proceeding of the 58th Annual ASIS Conference, Medford NJ: ASIS, 208-210.
The present paper proposes the artifact as theory perspective which draws together models of scientific practice and design behaviour and in so doing, offers the view of any information technology system as a conjecture on the part of the design team of human and organizational requirements to be met. By adopting this perspective, information system design can be seen as an ill-structured problem best tackled by user-centered theories and methods. The present paper will outline this perspective, emphasizing the need for convergence of views at the outset of design, and demonstrate the advantages it offers to both the theory and practice of technology design and the field of information science.
Efforts at increasing the usability of information technology have led to the emergence of the user-centered design philosophy. As a result, it is now common practice to engage in user testing of new products, ideally throughout the development cycle, in order to ensure the resulting technology is acceptable.
However, rapid prototyping and iterative testing in themselves offer no guarantees of design improvement. Prototypes may rest on weakly articulated user needs, and the resulting design may still be far from appropriate, in usability and functionality terms, for the real users' requirements. Even the presence of user testing at the earliest stages of development cannot be relied on to overcome these weaknesses. Furthermore, prototyping and testing early and repeatedly in the development cycle are frequently resisted on the basis of cost, compounding the problem of meeting user requirements.
The ill-structured nature of design, particularly at its earliest stages, can render the search for methods and techniques to support a more user-centered perspective problematic. The adoption of the user-centered philosophy is certainly desirable but the catch-cry of "know the user" is insufficient if it is equated solely with the production of prototypes or the quick running of a user-trial towards the end of the design process. The present paper proposes a view of information technology design that draws strong parallels between design and science, and argues for the adoption of a more scientific model of design in our development of information technologies.
The artifact as theory perspective draws together models of scientific practice and design behaviour and in so doing, offers the view of a prototype or established system as a conjecture on the part of the design team of the human and organizational requirements to be met. By extension, usability testing is proposed as a form of attempted refutation. By adopting this perspective, usable information system design can be seen as an ill-structured problem best tackled with user-centered theories and methods. The present paper will outline this perspective and demonstrate the advantages it offers to both the practice of information technology design and the theoretical development of information science.
ARTIFACT AS THEORY-INSTANTIATION
It has become traditional to view the process of design (not just software, but all product design) as more art than science, and from this view, to posit a dichotomy between these modes of thinking (1). Hence, artists are seen as creative, solution oriented, divergent and non-rational thinkers. Their preferred style of work is supposed to reflect the non-orderly manner of these thought processes and a regard for aesthetics over mathematics. Scientists however, are seen as more methods-oriented, quantitatively skilled and rigidly objective (2).
While the first point of attack on this stereotype must be the empirical data we have on cognitive style, designerly thinking and scientific practice - much of which raises serious doubts about many aspects of this view (3, 4)- within the confines of the present paper it is more important that attention is focused on the process by which artifacts are created, in particular the class of artifacts known as information technology.
The classic software development process can be generically represented as a five stage model:
Of course, there are numerous deviations and extensions of these stages (the important and costly stage of maintenance is ignored here) depending on the product and writer involved in describing that process (Preece 1993, (5) for example, describes the development process in terms of 17 possible stages), but these generic stages appear in virtually all software engineering models of the development lifecycle. This linear stage model is in reality more a representation of the management view of the process or, at best, a high level abstraction of the approximate behaviors involved. Few software designers see themselves as being involved in all these stages.
The software design team is expected to establish, or be provided with, the requirements for the product. At this point, how the requirements are met, i.e., what the designer actually does, is something of a black box. Cognitive studies of creative thought will posit stages of incubation, divergence, or insight (6) but our abilities to describe and subsequently support this activity - the one that largely determines the form of the artifact that is produced - is extremely limited.
To date, the response from the Human-Computer Interaction (HCI) community has been to provide guidelines, principles, or experimental findings from user studies, to the design community in the hope that such material will inspire or at least help them to envisage what might be required to satisfy requirements. Where this has failed, and to a large extent the collection of context-free findings from usability laboratories and the production of lavish handbooks of interface design have failed to be such a provider of design information, the user-centered design community has tended to examine the design process and argued for greater emphasis on design-test iterations via rapid prototyping (7).
In reality, iteration is a common feature of all design models and there is good sense in attempting to incorporate user-testing into the earliest stages of design. Certainly a problem faced by many HCI advocates is the refusal of those managing the design process to include such tests until such a time as the design is set in concrete and major modification is too costly to consider (8).
However, the concern for early testing alone, and its conception as the panacea for ensuring usability is short sighted, leading as it does to a sole reliance on empirical techniques at the expense of theoretical development. Landauer (9) has argued that evaluation and re-design are sufficient for the moment and suggested that useful theory in this domain is, for the most part, impossible to find. This may be true but only if we assume that the search for useful theory is limited to what we can derive from existing conceptualizations of the human and the organization (as Landauer seems to suggest). But there is no reason to presume that we must borrow exclusively from the existing constituents of cognitive and social science. Certainly these offer useful guidance, but neither psychology nor sociology consider the development of knowledge to impact design as central to their goals as disciplines. Yet it is precisely such a discipline we require.
The process of scientific practice offers a useful perspective for those concerned with the design of more usable artifacts. Popper (10) argued that all observations are theory impregnated and that scientists make progressive attempts at understanding a problem by formulating theories that are subjected to attempted refutations. From this perspective, theory and empiricism are intertwined inseparably, and while we never demonstrate the "truth" of an answer or theory, we can improve our understanding of the world by modifying theories that are shown to be false.
For Popper, the strength of a discipline can be gauged in terms of the willingness of practitioners within it to formulate testable hypotheses about the domain of inquiry. His well-known falsifiability criterion has been interpreted as one means by which we may demarcate science and non-science. Though critics of Popper attack this perspective on the grounds of its idealizing views of scientific practice (11) the borrowing of this perspective for the design practice is less concerned with its accurate depiction of routine scientific than with its potential insight as a prescriptive guide for progress.
Popper argues that theory-generation and testing are intrinsically human activities and that all problem-solving involves these activities to a greater or lesser extent. Thus what separates science from craft is less the putatively distinctive modes of thought manifest by the practitioners and more the formalism of the theories and empirical methods involved. Thus, both approaches he claims, "may be described as fundamentally utilizing the method of trial and error" (12 p.87).
The development of information technology certainly makes use of the method of trial and error but this is an unsatisfactory state of affairs. The emphasis on ever faster prototyping and rules for minimizing the effort of empirical evaluations (13) may help HCI (as a discipline) to impact the design process earlier (as we all desire) but will prove insufficient in developing the theoretical aspects of the field which are required for long-term development, ultimately rendering it more a methodology than a science.
In the sense that design is problem solving, the artifacts that are created represent conjectures on the part of the design teams involved. That is, they are (on one level) the embodiments of theories about the users and the tasks they will be performing with the tool (artifact) being developed (14). By extension, the usability trial can be viewed as an attempted refutation of the theory, carried out in order to improve it and render it more robust. It is this construction of the process that offers HCI and Information Science a way forward.
USING THE CONCEPTUALIZATION
Obviously, the notion of artifact as theory or conjecture about users is not the typical conceptualization we have of information technologies but it can be useful for analyzing how we might develop more usable artifacts. In the first instance, it demands than we attend seriously to the formulation of the artifact at the outset of design. In other words, we do not seek to just test for usability earlier, or hope to derive a set of design guidelines from the HCI literature for the design team to follow. Important as such initiatives are, HCI advocates need to be involved even more and even earlier, shaping the conceptualization of the tool that will in turn shape and be shaped by its context of use.
What can we contribute at the earliest possible stages? The answer to that tells us more about what we know as a research discipline than any handbook or collection of journal articles, and by extension it sets out a program for research into what remains unknown. At the macro-level it forces us to make explicit our models of the user of technology and to ask how are we to conceptualize the user appropriately? Typically the literature on users from HCI borrows extensively from cognitive psychology and the user is seen as a limited capacity information processor with short-term and long-term memories and certain response biases (15). Similarly, the view of the user as a social being existing within and influencing an organizational context is borrowed from sociology (16). While both perspectives are valid, an adequate user model for HCI purposes is unlikely to be derived simply by borrowing theories, constructs and methods from other disciplines in which they were developed to answer largely non-HCI type questions.
At the micro-level, i.e., at the level of any one design project, we need to establish clear views of particular users, their tasks and the environments in which they will be utilizing the technology. In such cases, our macro-level models need to be fleshed out with specific details relevant to that context. We can often support this with appropriate stakeholder, user and task analyses. All this can and should be done before a single line of code is generated so that not only does a design team have an agreed target to meet (and experience suggests various participants in the design process do not all share the same view, hence the requirement for greater convergence) but that target is more than just a loosely compiled set of assumptions about the users' levels of intelligence, attitude and computer experience or about the social context's form such as a workplace, a library or a museum.
In effect, this stage of design necessarily precedes all else and should result in an agreed view of the required tool to be developed. Designing the tool then follows by formulating a set of functions that will meet those requirements. At this point, the HCI community can provide evidence from existing studies or run experiments to provide new data on the most appropriate means of presenting these functions. The tool or artifact then is seen as the embodiment of the theory of what is required in that context. The theoretical constructs required are those that will inform the accurate description of this context and enable impact to be predicted. It should be said, this predictive power need not be strong or equivalent to the expectations of theory in the hard sciences. We are after all dealing with humans, where even partial predictive power would be useful. If we do not even attempt to address this issue, there is little hope of altering the technological determinism of most contemporary information systems design.
IMPLICATIONS FOR INFORMATION SCIENCE
In as much as information science is concerned with the human aspects of information storage, retrieval and use, then the issues tackled by the HCI community are central to the field. In HCI there is a real need for a model of the human performing tasks which are information intensive. Rather than merely drawing solely from other disciplines, information science could take a central role in theoretical developments by offering a perspective of the human user of information technology that is located at an appropriate level of abstraction for design purposes.
The artifact as theory approach indicates that we need to find a means of adequately describing humans as existing in contexts of use (i.e., performing tasks with tools in certain environments) that draws on psychological, sociological and educational theories but is expressed in a form that enables useful conceptualization of human socio-cognitive activity at the earliest stages of design. From this, the information technologies that result can be seen as the conjectures we make. Our usability evaluations thus highlight the flaws and weaknesses of our theory of the user and serve as feedback beyond the particular tool under development, towards our underlying assumptions (theories) of the world in which we operate.
This perspective provides a unifying perspective on the range of evaluations that are performed and requires those involved in this line of inquiry to make explicit their models of the human performing these information activities. This renders cross-project comparisons possible and aids our attempts at developing an agreed model of the information user (17)
Finally, this perspective offers at least partial redress to the much vaunted perspective of information science which has it that the field is atheoretical.
(1) Lawson, B. How Designers Think. (London: Architectural Press, 1979)
(2) Pylyshn, Z. Some comments on the theory-practice gap. In J. Carroll (ed.) Designing Interaction. (Cambridge: Cambridge University Press, 1991) 39-49.
(3) Darke, J. The primary generator and the design process. Design Studies 3 (1979) 157-162.
(4) Dillon, A. and Sweeney, M. The application of cognitive psychology to CAD. In D. Jones and R. Winder (eds.) People and Computers IV. (Cambridge: Cambridge University Press, 1988) 477-488.
(5) Preece, J. A Guide to Usability. (London: Addison Wesley, 1993).
(6) Gilhooly, K. Thinking: Directed, Undirected and Creative. (London: Academic Press, 1982).
(7) Preece, p.40.
(8) Shackel, B. and Richardson, S. (eds.) Human Factors for Informatics Usability. (Cambridge: Cambridge University Press. 1991)
(9) Landauer, T. Let's get real: a position paper on the role of cognitive psychology in the design of humanly useful and usable systems. In J. Carroll (ed.) Designing Interaction. (Cambridge: Cambridge University Press, 1991) 60-73.
(10) Popper, K. The Logic of Scientific Discovery , Third Edition (London: Hutchinson, 1972).
(11) Chalmers, A. What is this thing called science? (Milton Kenes: Open University, 1976).
(12) Popper, K. The Poverty of Historicism, (Reading: ARK Edition, 1986)
(13) Nielsen, J. Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies, 41, (1994) 385-397.
(14) Carroll, J. and Campbell, R. Artifacts as psychological theories: the case of HCI. Behaviour and Information Technology, 8 (1989) 247-256.
(15) Card, S.K., Moran, T.P. and Newell, A., The Psychology of Human-Computer Interaction. (Hillsdale NJ: Lawrence Erlbaum Associates, 1983).
(16) Eason, K. Information Technology and Organisational Change. (London: Taylor and Francis, 1988).
(17) Dillon, A. Designing Usable Electronic Text: Ergonomic aspects of human information usage (Bristol, PA: Taylor and Francis, 1994)