Authors: Olusola Samuel-Ojo, Doris Shimabukuro, Samir Chatterjee, Musangi Muthui, Tom Babineau, Pimpaka Prasertsilp, Shaimaa Ewais, and Mark YoungDESRIST 2010 | School of Information Systems and Technology | Claremont Graduate University
Abstract
The knowledge of design problem and solution is obtained in the building and application of an artifact, which is the end-goals of the design science research. Our objective in this paper is to conduct meta-analysis of the research being published by DESRIST to date to better understand the paradigm of design science research, and to sense the direction that research undertakings are headed. We present a meta-analysis model and detailed analysis of applications, IT artifact types, multi-disciplinary teams, and impact results.
Our findings show that the majority of the papers are negatively skewed, systematically clustering more IT artifacts for IS development problem domain. The most cited papers clustered around those that featured better design theories. We concur that researchers need to equally focus on areas with less research including organization and market domains, as well as causal theories of design through a rigorous formative or summative evaluation of artifacts.
Introduction
In the last few years design science research (DSR) has garnered attention within the IS community. After the landmark article published by Hevner, et al. appeared in MISQ in 2004 [1], this type of research paradigm became acceptable as an alternative to traditional behavioral research published by leading journals. In 2006 DESRIST was created as a stand-alone platform to publish and showcase design science research conducted within the community.
Since the dawn of the digital revolution, IT has changed the way we live, work, play and are entertained. Designers of IT-based digital technology products play a critical role in ensuring that their designed artifacts are not just beautiful, but provide value to their users. The experiences we have when we browse the web, or visit Amazon.com sell or buy on eBay, or play amusing games on our mobile phones have a tremendous impact on how we live our lives [2]. Designing interactions in the digital space presents interesting challenges for IS professionals especially as user interactions with the digital world continue to increase.
Meta-analysis of DSR within the IS Community: Trends, Patterns, and Outcomes
The fundamental principle of DSR is that knowledge and understanding of a design problem and its solution are acquired in the building and application of an artifact. Design is often a complex process, and designing valuable artifacts is further challenged by the need for creative advances in domain areas where existing theories are often insufficient [2]. In the IS discipline, we are concerned with designing artifacts that have a direct impact on organizations and society in general. This is supported by Lee [3] distinguishes IS research from the other fields:
Research in the information systems field examines more than just the technological system, or just the social system, or even the two side by side; in addition, it investigates the phenomenon that emerges when the two interact.
The term artifact is used to describe something that is artificial, or constructed by humans, as opposed to something that occurs naturally [4]. Such artifacts must improve upon existing solutions to a problem or perhaps provide a first solution to an important problem. IT artifacts, which are the end-goal of any design science research project in our community, are broadly classified into [2]:
- Constructs (vocabulary and symbols),
- Models (abstractions and representations),
- Methods (algorithms and practices),
- Instantiations (implemented and prototype systems), and
- Better design theories.
In both Herbert Simon’s seminal work, The Sciences of the Artificial [4] and Nigel Cross, Developing a Discipline of Design Science Research [8], we clearly see the importance they place on doing (i.e., construction). Simon believed that design is concerned with how things ought to be in order to attain goals [4]. Simon saw the design process as generally concerned with finding a satisfactory design, rather than an optimum design. He believed that “both the shape of the design and the shape and organization of the design process are essential components of a theory of design” [4]. Cross, on the other hand, places less importance on theory, emphasizing instead the knowledge that is acquired through the building process [8].
We must not forget that design knowledge resides in products them- selves; in the forms and materials and finishes which embody design at- tributes. Much everyday design work entails the use of precedents or previous exemplars – not because of laziness by the designer but be- cause the exemplars actually contain knowledge of what the product should be.
The goal of this paper is to evaluate the trends, outcomes, and impact of design science research by conducting a meta-analysis of the research being published within the DESRIST community from 2006 to 2009 to better understand the paradigm of DSR. In order to achieve this goal, we generate a set of sub-questions to examine the problem domain being studied, interdisciplinary research, types of IT artifacts, evaluation methods, and impact.
Discussion
Accelerating DSR knowledge production outcome requires that research efforts embrace the cycle of theory building. This cycle includes descriptive and normative stages which must be rigorously evaluated. Evaluation allows the worth or merit of the artifact to be ascertained, and rules out alternate explanations. Evaluation helps balance performance vs. cost, usability vs. security, and timeliness vs. fashionability, which is critical to the design, procurement, and implementation of the artifact as well as accumulation of design science knowledge. The assessment techniques for performance evaluation should include analytical modeling, simulation, field measurement, and quasi-experiment in both formative and summative forms.
Building a robust IS foundation is a key to nurturing the identity. However, the IS community needs to equally focus on organizations and markets problems. While DSR has no widely recognized theories, it provides the research artifacts that help support other theories in related fields. Our research attempts to capture an understanding of the trends and evolution of DSR into a theory-based research field.
We also acknowledge limitations of this study. We obtained citation information based on a basic count from ISI Web of Science and Google Scholars. We did not investigate what the use of the citation was. Scrutinizing what was cited should be a focus of future research, as the citation of scholarly articles is often considered one of the main outcome measures. In addition, the newness of the paper will matter in terms of citation impact in that older papers have a better chance of wider exposure than newer ones and, thus, might be cited more. This newness factor was held as constant in this study.
Conclusion
We conducted a meta-analysis of the research being published by DESRIST to date using a model, and produced a classification schema for the problem domain, IT artifact and their evaluation types, interdisciplinary collaboration, and impact results.
From the meta-analysis model, design science researchers should consider dimensions including research problem environment, design research artifacts, artifact evaluation, design process through interdisciplinary collaboration, and impact outcomes in formulating their IT artifact, which is in congruence with Hevner and Chatterjee’s design science research cycles [2]. The model might also be used as search logic for IT artifacts and as a utility evaluation framework for IT artifact for intended use and intended user.
In terms of the IT artifact concentration, half of the papers that addressed IS development problem domain featured IT artifact count that is more than the average with a wider variability. The entire IS development problem domain based papers had a median paper count that is above others, thus systematically clustering more IT artifacts. This behavior is salience. However, it is desirable that the trend of research undertakings is near evenly distributed across IS problem domains.
Of a particular interest is the summary variability of the most cited papers, which negatively skewed for papers that featured better design theory. To achieve the goal of science, it is critical that we regularly supply and test theories of design for typology, prediction, explanation, a sense of understanding and control of design events, which can be achieved by focused, interdisciplinary collaboration, and autonomous research. These theories might take any of the three conceptions: set of theoretical statements of generalization or laws, set of interrelated axioms, and propositions featuring existence, relationship or causal processes.
Researchers need to focus more on organization and market problem domains while aiming at crossing from descriptive to causal theories of design through a rigorous formative or summative evaluation of artifacts. The products and processes of such undertakings will strengthen the production of design knowledge. They will inform design and refine kernel theories, which in turn will benefit practice.
Article References
1. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design Science in Information Systems Re- search. MISQ 28, 75–105 (2004)
2. Hevner, A.R., Chatterjee, S.: Design Research in Information Systems: Theory and Prac- tice. Springer Publishing, Heidelberg (2010)
3. Lee, A.S.: Editorial. MISQ 25, iii–vii (2001)
4. Simon, H.A.: Sciences of the Artificial, 3rd edn. MIT Press, Cambridge (1996)
5. Walls, J.G., Widmeyer, G.R., El Sawy, O.A.: Assessing Information System Design Theory in Perspective: How Useful was our 1992 Initial Rendition? JITTA 6, 43–58 (2004)
6. Rossi, M., Sein, M.: Design Research Workshop: A Proactive Research Approach. Presentation delivered at IRIS 26, August 9 – 12 (2003),20Rossi%20-%20design%20research%20-%20IRIS.pdf (last accessed January 18, 2010)
7. Purao, S.: Design Research in the Technology of Information Systems: Truth or Dare. GSU Department of CIS Working Paper. Atlanta. Georgia, USA (2002)
8. Cross, N.: Design/Science/Research: Developing a Discipline. In: 5th Asian Design Con- ference: International Symposium of Design Science, Seoul, Korea, Su Jeon Dang Printing Company (2001)
9. Gregor, S., Jones, D.: The Anatomy of a Design Theory. JAIS 8, 312–335 (2007)
10. Sidorova, A., Evangelopoulos, N., Valacich, J., Ramakrishnan, T.: Uncovering the Intellectual Core of the Information Systems Discipline. MISQ 32, 467–482 (2008)
11. March, S.T., Smith, G.F.: Design and Natural Science Research on Information Technology. DSS 15, 251–266 (1995)
12. Benbasat, I., Zmud, R.W.: The Identity Crisis Within the IS Discipline: Defining and Communicating the Community’s Core Properties. MISQ 27, 183–194 (2003)
13. Carlsson, S.A.: Towards an Information Systems Design Research Framework: A Critical Realist Perspective. In: Proceedings of the First International Conference on Design Science in Information Systems and Technology, DESRIST 2006 (2006)
14. Carlile,P.R.,Christensen,C.M.:TheCyclesofTheoryBuildinginManagementResearch(2005)
