Fazlollahi, Parikh, Verma/ 1
Adaptive Decision Support Systems
Bijan Fazlollahi, Mihir A. Parikh, and Sameer Verma Department of Decision Sciences College of Business Administration Georgia State University Atlanta GA 30303 Tel: (404) 651-4000 Fax: (404) 651-3498 E-mail:
[email protected]
Abstract
Effectiveness of Decision Support Systems (DSS’s) is enhanced through dynamic adaptation of support to the needs of the decision maker, to the problem, and to the decision context. We define this enhanced DSS’s as Adaptive Decision Support Systems (ADSS’s) and propose its architecture. In an ADSS, the decision maker controls the decision process. However, the system monitors the process to match support to the needs. The proposed architecture evolves from the traditional DSS models and includes an additional intelligent “Adaptation” component. The “Adaptation” component works with the traditional data, model, and interface components to provide adaptive support. The architecture also integrates enhancements proposed in the past research. In this paper, we have illustrated the proposed architecture with two examples, a prototype system, and results from a preliminary empirical investigation.
Keywords: Decision Support Systems, Active decision support, Intelligent decision support, Adaptive support, DSS architecture, Cognitive support.
Fazlollahi, Parikh, Verma/ 2
Biographical Information
Bijan Fazlollahi is an Associate Professor of Decision Sciences at Georgia State University, Atlanta, GA. His research is in the area of Decision Support Systems. He has published over 30 articles in various journals and proceedings including Journal of Information Systems Research, Interfaces, and Information and Management. He is on the Editorial Board of the Journal of Database Management. He is a former Fulbright Scholar.
Mihir A. Parikh is a doctoral candidate in Decision Sciences at Georgia State University. He has received a Bachelor of Mechanical Engineering from Gujarat University, India and a Master of Business Administration from Georgia State University. His current research interests are in the areas of decision support and end-user systems, and applications of emerging information technologies (e.g., neural networks, fuzzy logic, genetic algorithms, multimedia) in decisionmaking and business training.
Sameer Verma is a doctoral student in Decision Sciences at Georgia State University, Atlanta GA. He has a Master of Science in Decision Sciences from Georgia State University, and a Bachelor of Engineering from Osmania University, Hyderabad, India. His areas of interest and work include intelligent decision support systems, education support systems, business training systems and strategic management through decision support and guidance. His focus is on the implementation of these systems through cognitive style research, using hypermedia / multimedia technologies.
Fazlollahi, et. al./ 3
Adaptive Decision Support Systems
Abstract Effectiveness of Decision Support Systems (DSS’s) is enhanced through dynamic adaptation of support to the needs of the decision maker, to the problem, and to the decision context. We define this enhanced DSS’s as Adaptive Decision Support Systems (ADSS’s) and propose its architecture. In an ADSS, the decision maker controls the decision process. However, the system monitors the process to match support to the needs. The proposed architecture evolves from the traditional DSS models and includes an additional intelligent “Adaptation” component. The “Adaptation” component works with the traditional data, model, and interface components to provide adaptive support. The architecture also integrates enhancements proposed in the past research. In this paper, we have illustrated the proposed architecture with two examples, a prototype, and results from a preliminary empirical investigation.
1. Introduction
DSS’s have benefited from advances in software and hardware technology. The data, model and interface components of DSS’s are now much more sophisticated and powerful than they were two decades ago. The databases are larger, more current and easier to query and search, the models are more complex reflecting reality, and the interfaces are much more user-friendly. However, the evolution has been in the direction of building a
Fazlollahi, et. al./ 4
DSS to provide more effective support for the low level cognitive tasks such as, data storage & retrieval, data drilling, manipulation, and consistency checking [19]. Little has been done in developing DSS’s that provide support for the high level cognitive tasks such as, framing of problems, alternative generation [17], making tradeoffs involved in preferences, and handling incomplete information, misinformation, and uncertainty. These high level cognitive tasks involve human mental activities of reasoning, learning, and idea generation requiring human judgmental inputs.
A primary objective of DSS’s is to help the decision-maker make effective decisions by identifying what should be done and ensure that the chosen criterion is relevant [7]. Provision of support for the high level cognitive tasks (i.e. the high level cognitive support) can strengthen the capabilities for achieving the objective. This type of support extends the limits of “bounded” rationality by promoting improved understanding, better insights, and more extensive analysis [7,16, 27, 28]. The high level cognitive support is analogous to referring the decision-making tasks to human staff assistants and staff advisors. Normally, a staff assistant makes efforts to understand the changing requirements of the task, the needs of the decision-maker, and the best way to support the particular decision-maker. For this, the staff assistant constantly monitors the current status of the task, provides interim reports, and is sensitive to the needs and the peculiarities of the decision-maker and the context in which the decision is made. The high level cognitive support adds to the functionality of DSS’s, especially for situations with complex problems and expert decision-makers. As an example of the added functionality;
Fazlollahi, et. al./ 5
some of the tasks may be delegated to the intelligent agents [26]. The intelligent agents accomplish the tasks on their own initiatives while interacting with the decision-maker and the decision environment. However, the agents operate within the user control philosophy of DSS’s [14] where the decision-maker exercises human judgement and provides judgmental inputs.
The purpose of this paper is to propose an enhanced DSS, adaptive decision support system (ADSS), which provides the high level cognitive support adapted to the needs of the user, the decision task characteristics, and the decision context. The paper reviews past research and discusses ADSS’s as a prescription to the unaddressed requirements of complex decision-making situations. It proposes an architecture that identifies and incorporates key components for designing and developing an ADSS. It illustrates the architecture through building and using a prototype ADSS including results from a preliminary empirical investigation. Finally, it provides a summary of observations and recommendations for future directions of research in ADSS’s.
2. Background
DSS’s have evolved to provide more effective support for decision-making. The factors influencing DSS evolution include (1) the discovery of structure in some judgmental tasks and then assigning the task to the computer, and (2) improvements in technology allowing the computer to do more tasks. Keen and Stable as far back as in 1978 foresaw that
Fazlollahi, et. al./ 6
decision support may be achieved by exploitation of many technologies [7]. Modern database technology, graphical user interface, hypermedia, multimedia, expert systems, neural networks, fuzzy logic, genetic algorithms, distributed systems, client-server, objectoriented approach are examples of recent technologies that can carry out prescriptions which were not feasible in 1978. In recent years, some of the emerging technologies have been used in providing the high level cognitive support. Research in the area of high level cognitive support is labeled active decision support, inductive learning, decisional guidance, and adaptive interface.
Manheim [10] suggests active DSS’s which have few features which can provide the high level cognitive support. These features include: <
maintaining an explicit representation of the decision-maker's conceptual problemsolving model and using it to guide support activities;
<
providing tools for supporting the "natural heuristics", such as "do the easy things right away" as well as tools for rational model-type such as linear programing and break-even analysis model; and
<
providing tools to enhance the user's ability to balance strategic (global and long term) and opportunistic (local and short term) thinking.
The active DSS’s are capable of active participation in the decision-making processes. The systems operate almost independent of explicit directions from the users and provide support which the users may find helpful [8, 20]. Raghavan [21] identifies support features
Fazlollahi, et. al./ 7
of the active DSS’s as monitoring the user activities, making inferences, and conducting appropriate activities such as alerting, engaging in an insightful conversation, or automatically carrying out certain tasks. The active DSS’s aim at improving the decisionmaking effectiveness through stimulating creative ideas, criticizing choices, and guiding decision structuring.
The active DSS’s complement users’ problem solving abilities in the application domain [20]. The DSS’s use alternative models of the problem solving processes, ask the users to make choices at the intermediate stages allowing the users to determine the problem solving paths, and maintain updated models of the user problem-solving processes. They support the users in a number of forms such as suggesting alternative actions and indicating issues that the users may have overlooked. Rao [20] concludes that the active DSS’s should be designed as knowledge-based systems.
Piramuthu, et. al. [18] describes an adaptive DSS for real-time scheduling of a flexible manufacturing system. The DSS dynamically identifies a pattern in the scheduling environment and matches an appropriate scheduling heuristic rule to the task. The system architecture includes a “learning and refining” module capable of simulation and inductive learning for acquisition and refinement of heuristics. The module interacts with the knowledge-base to provide adaptive scheduling.
Holsapple, et. al. [4] describe an adaptive DSS which utilizes unsupervised inductive
Fazlollahi, et. al./ 8
learning, a learning through observation and discovery, to acquire problem processing knowledge for machine learning. The DSS refines the problem processing knowledge to match the existing conditions.
Holsapple, et. al. [4] summarize the relationship among DSS paradigms based on two problem processor related factors of active/reactive and adaptive/non-adaptive. The traditional DSS’s are labeled as non-adaptive and reactive. They suggest that the focus of research should be on adaptive and active DSS’s. These are the systems where problem processors acquire and eliminate knowledge through unsupervised learning and are largely self-driven. The research is concerned more with learning abilities that improve the problem processing behavior of a DSS [4].
Silver [25] proposes “decisional guidance” as an enhancement to the DSS’s. The decisional guidance enlightens or sways its users as they structure and execute their decision making processes and thus provide meta-support for judgmental activities. The guidance is implemented in the form of help facilities [12] or embedded intelligence that inform and advise users. The objective of the decisional guidance is more effective use of DSS’s leading to more effective decision making.
Several researchers propose adaptive interface, user-controlled or self-adaptive, to allow for the differences in the users and to enhance DSS’s quality and effectiveness [2, 24]. Adaptability of interface ensures that the system provides flexibility to satisfy the different
Fazlollahi, et. al./ 9
users' cognitive styles, the users’ experience level, and different decision approaches. Thus, adaptable interface allows a DSS to provide ease of learning and user control.
Dutta [3] proposes additional intelligent components to a planning DSS so that the system can adapt to the changing task requirements. He identifies a need for support in monitoring, replanning, and managing interdependencies among different temporally separated actions in the iterative process of planning. The DSS monitors the environment, handles uncertain and incomplete information, and interprets and integrates conflicting output from different models and view points.
Although the enhancements proposed in the past research provide the high level cognitive support through increased DSS functionalities, the research and development in the area of providing high level cognitive support is fragmented and technologically oriented. The methodology for providing the support is yet in the infant stage [19]. In particular, there are no frameworks to guide the identification of the necessary enhancements and addition of functionalities to the DSS’s that would provide the high level cognitive support. In the following section, we propose an adaptive decision support system (ADSS) that incorporates different ideas regarding extensions and enhancements to the traditional DSS’s for providing the high level cognitive support.
3. Adaptive decision support systems
Fazlollahi, et. al./ 10
We define ADSS’s as: “DSS’s that support human decision making judgements by adapting support to the high level cognitive needs of the users, task characteristics, and decision contexts.”
ADSS’s are enhanced DSS’s with an objective to improve the effectiveness of the decision-maker in performing tasks requiring high degree of human judgement such as framing problems, generating alternatives, making tradeoffs, and handling equivocality and uncertainty. ADSS’s due to their emphasis on the high level cognitive support will also improve user learning and understanding of the decision-making process and the domain knowledge.
ADSS’s, unlike traditional DSS’s which are adaptive systems only through evolution [29], are adaptive through adjustments to the skill level and changing needs of the decision-maker during the decision making process. The decision maker learns through interaction with the ADSS’s [7]. The learning leads to changes in problem-solving expertise and support needs. ADSS’s provide support that fits the user's current needs. Also, the progress through the intelligent, design and choice phases in a dynamic decision environment leads to changing problem-solving task. ADSS’s adapt to the changing problem solving model and provide support for the appropriate tasks. Furthermore, ADSS’s adapt to the decision contexts such as organizational structure. For example, a decision in a matrix structure would require more coordination with other decision makers
Fazlollahi, et. al./ 11
than in a hierarchical structure. In such situations, the support must also provide mechanism for coordination of decisions. Matching support to the decision-maker, the decision problem, and the decision context substantially helps the decision-maker to make effective decisions [7].
Adaptation is achieved by matching support needs with the system support. The support needs of the user are determined by monitoring the user performance and support history. The support needs of the task and the contexts are identified through monitoring the decision process and selecting the appropriate models. ADSS’s monitor the decisionmaking process, diagnose problems/ opportunities, and design and implement interventions. Such abilities rest on having knowledge of the specific user, the problem domain, an expert model of the decision process, and strategies for intervention. As the support needs change, the systems dynamically change their support to match the current needs. Dynamic adaptation enables ADSS’s to better address learning, interaction, support, and evolution-the key words in the DSS definition [7].
ADSS’s use intelligent technologies to determine the support needs and may provide an active rather than a passive participation in the decision making process [8]. The active participation includes performing tasks such as finding patterns in data, selecting appropriate models, or acting as critiquing agents [13, 26]. It further means that the user/DSS interaction (a two-way communication) is established with the decision-maker controlling the process, similar to in the case of a decision-maker with a human staff
Fazlollahi, et. al./ 12
assistant.
4. ADSS Architecture
The architecture of decision support systems was first proposed by Sprague and Carlson [30] as a macro architectural model with three components data, model, and interface. Later, Turban [32] revised this model and added expert systems/knowledge-base component to the model. Other researchers [3, 8,9, 10, 11, 23, 24, 25, 31] have proposed enhanced architectures to encompass particular functionalities not specifically identified in the original macro model.
**********
Insert Figure 1 about here ***********
Figure 1 shows the proposed architecture for ADSS’s. The architecture is an evolution of the Sprague & Carlson model [30]. In addition to the three; data, model, and interface components, of the traditional DSS’s, ADSS’s have an “Adaptation” component. The adaptation component is integrated with the other three components to generate and provide adaptive support.
ADSS’s have three subsystems: user diagnosis, problem solving, and guidance/instruction. Each subsystem incorporates data, model and adaptation component. The user diagnosis subsystem includes information regarding what the user knows and what support the
Fazlollahi, et. al./ 13
system has already communicated to the user. The problem solving subsystem includes the model derived from a theory or stated by the user for appropriately solving the problem. ADSS’s do not require the general model of human problem solving processes to guide their automatic intervention in the decision-making processes. Instead the more attainable descriptive models of specific tasks are used to guide some of the activities of ADSS’s. The guidance/instruction subsystem includes knowledge about how to intervene in the decision making processes. The ADSS architecture addresses the functionalities of ADSS’s which are (1) to monitor the decision-makers, the decision-making tasks and the decision contexts, (2) to make inferences on the basis of descriptive models, and (3) to intervene at the discretion of the decision-maker to provide decision support.
Two examples are selected from management science and personal finance domain to explain the architecture. Example A refers to an ADSS used for selecting appropriate forecasting model for a given historical data. This decision-making situation is structured with well-defined statistical models and quantitative methods to identify which forecasting model is more suitable. Example B refers to an ADSS used for determining appropriate allocation of assets for an individual investor. The process of asset allocation is dependent on subjective variables such as the degrees of risk preference, time horizons, and financial conditions of the investor. This decision-making situation is unstructured as there is a lack of quantitative models which incorporate the individual investor’s characteristics and concerns in performing asset allocation.
Fazlollahi, et. al./ 14
The components of each ADSS are described in details and are illustrated with examples A and B in the following sub-sections:
Data: This component is similar to the data component of the Sprague & Carlson model [20]. It stores raw data about the problems, the concepts and procedures, and the user history. It has three sub-components: Problem, concept/procedure, and user history.
Problem: This sub-component stores raw data about the problem or decision at hand. The details of the problem can be obtained from this database and presented to the user of the system. This sub-component can be an external database dynamically linked to the other components of the system. Example A: Problem: Forecasting sales for year 1997 Data (historical) : Year Sales 1985 $5,234,667.00 1986 $8,342,235.00 .................................. ................................... 1995 $41,564,982.00 1996 $59,002,538.00 Example B: Problem:
Investment asset allocation for a household with four members- husband, wife, two young children. Data: Household Income : $60,000 per year Major expenses: Rent+utilities $1,000 Auto loans - $600 Other $2,000 Major financial goals:Retirement - $400,000 in 30 years Education for the children - $50,000 in 14 years Buying a house - $25,000 in 2 years
Fazlollahi, et. al./ 15
Concept/Procedure: This sub-component stores information on the concepts and procedures related to the domain knowledge area. This sub-component can also be dynamically linked with an external concept/procedure library. Example A: Domain knowledge: Time-series forecasting Model: Linear forecasting Concepts: what a linear forecasting model is; when it should be used; the underlying assumptions of the model; the advantages and disadvantages of using this model. Procedures: how to find the coefficients of the independent variables; how to import data into the model. Example B: Domain knowledge: Personal finance and investments Concepts: What stocks, bonds, or money market funds are; what an emergency fund is; what a portfolio is; risk and return of a portfolio. Procedures: How to determine portfolio risk; how to determine portfolio return.
User History: Before the user uses the system, the system performs a diagnostic test and determines the knowledge level of the user in the concepts and procedures involved in the decision-making process. These diagnostic data are stored in this component. When the user uses the system, the system continuously monitors actions of the user. This sub-component stores the sequential historical actions and interactions such as performance of the user, type of help the user requested, and time it took the user to solve the problem. Example A: User: Jeff Jones, Performance history: Outcome Time Problem 1: Right 5 min. Problem 2: Right 7 min. Problem 3: Wrong 3 min. Problem 4: Right 2 min.
Fazlollahi, et. al./ 16
Support history: Concept: Procedure:
Linear model, Exponential Smoothing model, Doing a square of differences, Finding a coefficient for independent variable.
Example B: Performance history (alternate portfolio developed by the user): Iteration 1: Stocks-30%, Bonds-20%, Cash-50% Iteration 2: Stocks-35%, Bonds-35%, Cash-30% Iteration 3: Stocks-50%, Bonds-40%, Cash-10% Iteration 4: Stocks-50%, Bonds-30%, Cash-20% Support history: Iteration 1: The user did not ask for support Iteration 2: The user asked for support on: Concepts: Returns on stocks, risk involved in the Cash Investments Procedure: How to determine portfolio return Iteration 3: The user asked for support on: Concepts: Portfolio evaluation, risk involved in stocks Procedure: How to determine portfolio risk Iteration 4: The user did not ask for support
Models: This component stores models and knowledge about problem solving, guidance/instruction, and user diagnosis in three sub-components: Problem solving model, guidance/instruction model, and user diagnosis model.
Problem Solving Model: This sub-component stores descriptive problem-solving models for different problems. When a problem is presented, this sub-component has the knowledge and the models for identifying and solving the problem. This sub-component has two parts: Associated concepts and associated procedures. Associated concepts include the models about the identifying concepts involved in solving the problem. Associated procedures have the models about identifying the
Fazlollahi, et. al./ 17
procedures involved in solving the problem.
Example A: Associated concept:
Associated procedure:
Example B: Associated concept:
Associated procedure:
If the historical data shows a linear trend with no seasonality and low fluctuations, then simple linear regression model should be used. To use the linear model: (1) find the coefficient, (2) find the intercept, (3) select a future time period to forecast, (4) use the developed model to find the new level of the dependent variable for the future time period.
If the risk preference is high, time horizon is long, and financial condition is stable, allocate a larger part of the portfolio to stocks. To determine suitability of the portfolio in terms of risk, identify risk preference of the user and match that with the risk of the portfolio.
Guidance/Instruction Model: This sub-component has models about presentation of the concepts and the procedures. It has models to determine when, how, and at what level a concept or a procedure should be presented. Example A: Problem:
User:
Example B: Problem: User:
In case of the linear regression model, present the concepts and procedures related only to the linear model and not other forecasting models. If the user knowledge is strong in linear model concept, give only brief conceptual information. If the user knowledge is weak in linear model concept, give detailed conceptual information about the model along with examples/nonexamples of the model.
In case of the portfolio return, present only the concept and procedures involved in determining portfolio return. If the user knowledge is strong in the concept of determining portfolio return but weak in calculating portfolio return, then give brief conceptual information and detailed step-by-step procedure
Fazlollahi, et. al./ 18
for calculating portfolio return.
User Diagnosis Model: This sub-component stores information about how to interpret the user history. This sub-component has models and rules needed to interpret the conceptual and the procedural knowledge levels of the user and determine the level of expertise and support needs. Example A: Concepts:
Procedure:
Example B: Concepts:
Procedure:
If the user does not know what a trend is, the difference between high and low fluctuation, and what seasonality is, then the user knowledge in linear model is weak; If at least two (of the above three) concepts are clear, then the user knowledge in linear model is average; If all the concepts are clear, then the user knowledge in linear model is strong. If historically the user was never able to perform this task, then the user knowledge is weak in finding coefficient of independent variables ; If the user was able to perform the task successfully half of the time, then the user knowledge is average in the procedure; If the user was able to perform the task successfully most of the time, then the user knowledge is strong in the procedure.
If the user portfolio does not match with optimal portfolio given user’s risk preference, time horizon, and financial condition, he is conceptually weak in understanding one or more of these three dimensions. The degree of weakness can be determined by the size of the difference in the portfolios. If the user has never been able to perform the procedure of determining portfolio return, then the user is weak in the procedure. If the user has determined the portfolio return successfully about half of the time, then the user is average in the procedure.
Adaptation: This component integrates the sub-components of the data and model components to infer about the adaptive support. The component has three subcomponents: Expert problem solving evaluation, user performance evaluation, and
Fazlollahi, et. al./ 19
guidance.
Expert Problem Solving Evaluation: This sub-component represents expert's evaluation of the problem and problem solving knowledge. It matches the problem sub-component from the data component and the problem solving model subcomponent from the model component and determines the concepts and procedures related to the problem. It creates a dynamic task profile of the associated concepts and procedures for solving the given problem. For example, Let's say problem T11 is selected from the problem sub-component of the data component. The expert problem solving evaluation sub-component identifies the key features of the problem T11, (such as linear trend, low fluctuations, no seasonality for the example A and low risk preference, short time horizon for the example B). Then, it uses the models from the problem solving model sub-component of the model component and determines the associated concepts and the associated procedures as shown below: Problem: Associated concepts: Associated procedures:
T11. C1 and C8 P2, P6, P8 and P14.
User Performance Evaluation: This sub-component evaluates the performance level of the user and develops a dynamic user performance profile for both concepts and procedures. As the user uses the system, the history of interaction is recorded in the user history sub-component of the data component. The user performance evaluation sub-component, uses the user history and interprets the knowledge of the user based on the user diagnosis knowledge. For example, Let's say the user is Jeff Jones and has used the system for quite sometime and the system has accumulated a history of interaction during this time.
Fazlollahi, et. al./ 20
The user performance evaluation sub-component matches the interactions recorded in the user history sub-component of the data components with the models form the user diagnosis model of the model component. From the comparison it determines a user profile indicating Jeff’s degree of knowledge of concepts and procedures as shown below: User: Concepts:
Procedure:
Jeff Jones. Weak-C1, C3 and C7; Average- C2, C4 and C5; Strong- C6 and C8. Weak-P1, P2, P9 and P14; Average-P3, P5, P6, P7, P8, and P10; Strong-P4, P11, P12, and P13.
Guidance: This sub-component compares the task profile from the expert problem solving evaluation sub-component with the user profile from the user performance evaluation sub-component. It compares knowledge required for an associated concept with user's knowledge in the concept and determines the concept differences ()C). It also compares the proficiency required to preform an associated procedure with the user's proficiency in performing the procedure and determines the procedure differences ()P). Based on )C and )P, this subcomponent determines which concepts and procedures should be presented and at what level. For example, Jeff Jones is given the problem T11. This problem has C1 and C8 as associated concepts and P2, P6, P8 and P14 as associated procedures. Jeff is weak in concept C1 and procedures P2 and P14, average in procedures P6 and P8, and strong in concept C8. So, the guidance sub-component determines to provide Jeff the detailed information along with examples and non-examples of C1, P2 and P14, the detailed information about P6 and P8, and only the brief information about C8. Problem: T11. Associated concepts: Associated procedures:
C1 and C8 P2, P6, P8 and P14.
Fazlollahi, et. al./ 21
User: Jeff Jones Concepts:
Procedure:
Prescription: Concepts:
Procedures:
Weak-C1, C3 and C7; Average- C2, C4 and C5; Strong- C6 and C8. Weak-P1, P2, P9 and P14; Average-P3, P5, P6, P7, P8, and P10; Strong-P4, P11, P12, and P13. C1 -
Detailed information along with examples and non-examples C8 - Brief information P2 - Detailed information along with examples and non-examples P6 - Detailed information P8 - Detailed information P14 - Detailed information along with examples and non-examples.
User Interface (dialogue): The user interface component is the link between the user and the system. This component is the one that is seen and used directly by the user, so the user may think that this is the system. It is a self-adaptive interface that automatically adjusts to the users' preferences and tasks, and provides the functionality and form required to match the interface to a specific user performing a specific task. Self-adaptive interface promotes ease-of-use and consistency of features in the interface that are important factors in establishing usefulness and success of the system.
5. ADSS Prototype
Problem Description: We used the exploratory system development process methodology [21] to investigate the proposed architecture with a prototype system. We selected forecasting, specifically data analysis and model selection, as the area of domain
Fazlollahi, et. al./ 22
knowledge. In this prototype system, the user is provided with the sales data plotted against time and asked to examine the plot and select the most appropriate forecasting model to predict future sales. The system presents four cases/problems (labeled A, B, C, and D) of sales data, each of the problems requires different forecasting model. The user examines the cases one at a time in a sequential and irreversible order, and selects an appropriate forecasting model for each case. In solving the problems, the user can access information about the data and the models pertinent to forecasting. The information may help in analyzing the data plot and selecting an appropriate model. In this prototype, the user's history of interactions in solving each case is stored to or retrieved from a database.
Prototype Description: We developed the system, by mapping the conceptual components of the architecture to different files, programs and other features in KnowledgePro software package. KnowledgePro is an environment that supports rapid prototyping in rule-based programming for expert systems. The software allows reading and writing to a variety of file and graphic formats. An add-on package called KPWIN ++ generates C++ code for the KnowledgePro-based programs and compiles them into runtime executable files.
Physical representation: As described in the architecture, the system consists of the data, model, adaptation and user interface components. Each component is divided into subcomponents.
Fazlollahi, et. al./ 23
Data: This component consists of the problem, the concepts/procedures, and the user history sub-components. It has data in the form of independent data files and random access memory (temporal data). Problem The problem data are loaded by runtime programs from an independent file on a disk. The data are presented to the user in a graphical format (bitmap) as a time series plot which the user has to analyze. Concept/Procedure The concepts and procedures are assembled in text and graphics formats, in accordance with the problem type and the problem solving stage requirements. They are stored in files on a disk. User History This sub-component deals with temporal data. However, to maintain a cumulative user profile, the data from the random access memory is dumped to a trace (ASCII text/database) file, after every significant event. This file contains data regarding navigation, time stamping, results, performance, etc. In every new session, the trace file from the previous sessions of the user is accessed to adjust for the previously learned concepts and procedures.
Model: This component consists of rule-based programs (executables), which store the various models used by the system. The model component encapsulates three sub-components: the problem solving model, the guidance/instruction model, and
Fazlollahi, et. al./ 24
the user diagnosis. Problem Solving Model This sub-component contains the problem solving models, represented through associated concepts and associated procedures. We have modeled this knowledge by programming in Knowledge Pro's rule-based expert system shell. Guidance/Instruction Model This sub-component is represented by the models that determine the format of the presentation of the concepts and procedures that the user may require. The inference is based on the performance of the user. User Diagnosis Model This sub-component has rules that diagnose and interpret the user history for determining the strengths and weaknesses of the user in the domain knowledge.
Adaptation: This component has three sub-components. All sub-components are exclusively rule based. The expert problem/solving evaluation sub-component associates the problem file name with the problem solving knowledge rule block. After comparing the problem and the expert's opinion, the sub-component determines the expert's representation of the required concepts (CE) and the procedures (PE). The user performance evaluation sub-component examines user history from the trace file and the user diagnosis knowledge. Using the two, this sub-component determines the concepts (CU) reviewed and procedure (PU) performed by the user. All these values are stored as temporal data in the RAM.
Fazlollahi, et. al./ 25
Finally, the guidance sub-component compares the inferences from the expert problem solving evaluation sub-component and the user performance evaluation sub-component, and generates the deviations for concepts ()C) and procedure ()P). The guidance sub-component determines which and in what format the concepts and procedures need to be presented. The concepts and procedures are obtained from the data component's concept/procedure sub-component, and the presentation format is obtained from the model component’s guidance/instruction sub-component. The system bases its inferences of formats and concepts on the user profile and present user performance ()C and )P). In the prototype system, the outcome for each of the four cases can be either right or wrong. Therefore, as more information is gathered, the decision tree develops more branches. As an example, trees for cases A and B are shown in figures 2 and 3 respectively.
**********
Insert Figure 2 about here ***********
**********
Insert Figure 3 about here ***********
For case A (Figure 2), the system uses the user history, which indicates that the user could have been right or wrong about case A type of problems in previous sessions. The outcome here refers to concepts and the format of presentation. Therefore, while the user is in case A, he/she can get one of the two types of
Fazlollahi, et. al./ 26
outcomes. As the system proceeds to case B (Figure 3), the user's performance profile has changed. It now consists of not only the performance in previous sessions, but also of the performance on case A. This information is used in conjunction with the user's history. In case B, the outcomes increase to four. This is due to the number of combinations. It must be noted, that the user's performance is a cumulative variable. We have used this approach to decrease the search time for outcomes, and to make the process of concept retrieval and presentation more efficient. Based on the inferences made by the guidance/instruction subcomponent, the concepts are presented to the user through the user interface. Now for case C, we will have updated user history and the performance in case B.
User Interface: The user interface is designed in Microsoft Windows 3.1. By using windows and buttons, the system provides the ease of navigation. The hypertext is constrained to prevent the user from getting lost in hyperspace. For example, while using the system, only the relevant hot regions are activated. These controls are coded using KnowledgePro's event based topic controls.
Sample Session and Explanation: The user begins by registration. Registration allows the system to retrieve the user's history from the database. Then the user is presented with an information screen containing all the relevant and required information for using the system. After reading the instructions, the user continues by clicking on the Continue button. The system is designed to keep control over decision making process with the
Fazlollahi, et. al./ 27
user.
**********
Insert Figure 4 about here ***********
**********
Insert Figure 5 about here ***********
The next screen consists of two windows (shown in figures 4 and 5), which show the data plot (Figure 5) for case A, and a menu pad (Figure 4) for access to guidance/instruction, and for model selection. The user can click on any of the specific information buttons to get extended guidance/instruction on those models, in the context of the present problem. For example, by clicking on the "Information" button on "Least Squares Regression", the information provided consists of textual information on least squares regression, and a suggestion about the data. The screen also displays the current plot . When the user selects the least squares model, the system provides feedback, indicating that the choice was wrong. The system automatically proceeds to case B. This time the user selects information option for Least Squares Regression. The screen looks different this time. This is due to the fact that the user chose the wrong model for case A. Therefore the guidance is more extended. A comparison of the two information screens is shown in figures for case A and case B.
Fazlollahi, et. al./ 28
**********
Insert Figure 6 about here ***********
**********
Insert Figure 7 about here ***********
Comparing the figures 6 and 7, we see that in addition to the information in Case A (Figure 6), the figure for case B (Figure 7) provides a comparison of example and nonexample. The user selects the least squares model this time. The feedback from the system indicates the choice to be correct. The system proceeds to case C (Figure 8). In case C, by clicking on the information option for "Least Squares", it displays a third format. In this format the information includes three plots illustrating the key factors in forecasting. The reasoning for this format is based on the performance of the user on previous sessions (user history) and the performance on cases A and B (i.e. user history, and present user performance).
**********
Insert Figure 8 about here ***********
**********
Insert Figure 9 about here ***********
The user selects exponential smoothing model. The system indicates that this model is wrong. At this point, the user has two wrong and one correct choice. The system continues to case D (Figure 9). Finally, case D reveals information on moving average
Fazlollahi, et. al./ 29
model, which explains the modeling technique, shows a data plot, provides graphical comparisons, and indicates strongly that the user should not use moving average("So you should not use moving average model..."). Comparing the figures for case C and case D, we see that the graphical information and text are different. The suggestion also is different, with a more emphatic statement in case D. Hence, the system adapts to the user’s history from previous sessions, and makes fine adjustments as the user goes from one problem to the other.
6. Results from a Preliminary Empirical Investigation:
We used the prototype discussed in the previous section to conduct a preliminary empirical investigation in a laboratory environment. The study and the results are reported in detail in other publications1 and conclusions are briefly discussed in the following paragraphs. One hundred and thirty five subjects participated in the study. The preliminary results from the experiment show that the meta-support in DSS’s increases decisionmaking performance, learning and satisfaction of the users. The study examined decisional, instructional and cognitive aspects related to the support provided by the system.
On the decisional aspects, the decision quality improved, however, the decision time increased. The reason for the increase in the decision time is that the user spends more time using the DSS facility to explore more alternatives and increase understanding. The users with dynamic guidance, where the message content is tailored to the user’s needs,
Fazlollahi, et. al./ 30
performed better than the users with predefined guidance, where the users got “canned” or pre-determined messages. The users with suggestive guidance, where the system provides recommendation, did not perform better than the users with informative guidance, where the system provides detailed information without any recommendations. [Refer to Pub. 11 for details].
On the instructional aspects, the users of the DSS with guidance learned significantly more than the users of the DSS without guidance. In addition, the users of the DSS with guidance were more satisfied with the overall process of decision-making than the users of the DSS without guidance. Also, the users with dynamic guidance learned more and were more satisfied with the process than the users with predefined guidance. Furthermore, the users with informative guidance learned more than the users with suggestive guidance. However, both groups were equally satisfied with the process [Refer to Pub. 21 for details].
On the cognitive aspects, the influence of guidance on the user depended on the user’s cognitive styles. We used Jung’s Psychological Types with Myers-Briggs Type Indicators to determine cognitive styles of the users. The users with Sensing-Dominant and ThinkingAuxiliary (ST) type performed better than the users with Intuition-Dominant and Thinking-Auxiliary(IT) type. Furthermore, the ST users with dynamic guidance performed better than the ST users with predefined guidance. However, the ST users with suggestive guidance did not perform better than the ST users with informative guidance [Refer to
Fazlollahi, et. al./ 31
Pub. 31 for details].
7. Summary and Conclusion
Traditionally, DSS’s have provided support for low level cognitive tasks. To increase decision support effectiveness of DSS’s, there is a need to support high level cognitive tasks which require human mental activities of reasoning and learning. In this paper, we propose ADSS’s which provide support for high level cognitive tasks by dynamically adapting system support to the knowledge level of the user, the decision task characteristics, and the context in which the decision is made. ADSS’s are a result of integration of research in the field of decision support systems, cognitive science, and artificial intelligence.
ADSS’s monitor the problem solving processes and the human decision-maker to determine the support needs for making the judgmental inputs. The systems determine the gaps in the conceptual and procedural knowledge of the user for performing the given decision task. Based on the gaps, the systems determine the support needs and customize the support to match the needs.
ADSS’s have, in addition to the data, model and user interface components of the traditional DSS’s, an "adaptation" component. The adaptation component uses artificial intelligence techniques to identify user needs and problem solving model, and matches
Fazlollahi, et. al./ 32
them with appropriate level of support.
We developed a prototype ADSS with decisional guidance capabilities to support the decision-maker in selecting an appropriate forecasting model for a given set of data. Selection of the appropriate forecasting model required user judgement as to which forecasting model was more appropriate. The prototype system provided guidance for making the judgement, the guidance was matched to the particular user’s needs and the decision task on hand. A preliminary empirical investigation was conducted using the prototype. The investigation was designed on the basis of a research practice where the researchers invent and test ways for improving decision-making effectiveness. The results of the investigation show that the guidance for judgmental inputs improves decision quality, user learning, and user satisfaction.
This study provides an architecture to integrate the currently disparate and fragmented research efforts. By applying this architecture in the development of real world DSS’s, decision-support effectiveness of the DSS’s could be increased.
ADSS’s can be applied to other areas of decision making. We anticipate their usefulness will be optimal in the areas, (1) where the task environment is unstructured requiring more judgmental inputs from the decision-maker and (2) where the impact of the decision is high, such as strategic management and crisis management.
Fazlollahi, et. al./ 33
In strategic management and planning, top management has to develop comprehensive strategies to cope with the instability, uncertainty, and complexity of the environment. This requires sophisticated and comprehensive understanding of the internal and the external factors to develop strategic plans for long term direction of the establishment. While a traditional DSS does not adequately support tasks like problem formulation and problem structuring, an ADSS can provide support for high level cognitive tasks such as setting goals and objectives, evaluation of alternative strategies, and stakeholder analysis [15].
In crisis management situation, a tendency is to consider a limited number of alternatives and quickly reach a decision. The limited analysis reduces the decision quality by rejecting a correct course of action, accepting a wrong solution to the problem, solving the wrong problem, and solving the right problem correctly but too late [22]. ADSS’s can support the decision-making process by supporting generation and evaluation of more alternatives, identifying objectives, and evaluating the consequences.
Although we performed a preliminary investigation of the architecture using a prototype with decisional guidance capabilities, there is a need for further research. We suggest ADSS research in the following areas: 1.
Investigate ADSS’s for unstructured and complex decision-making situations.
2.
Develop and test alternative strategies for adaptive support.
3.
Study the impact of adaptive support on the expert and novice decision-makers.
Fazlollahi, et. al./ 34
4.
Investigate the use of emerging technologies (e.g., neural networks, fuzzy logic, and genetic algorithms) in developing ADSS’s. New developments in on-line analytical processing and data-mining could also be used in building ADSS’s.
5.
Investigate other related issues such as information overload, biasing behavior, and restriction on flexibility and creativity in the context of ADSS’s.
Fazlollahi, et. al./ 35
USER User Interface (Dialogue) ADAPTATION Expert Problem Solving Evaluation
Guidance Module
User History
Problem
DATA
Problem Solving Model
User Performance Evaluation
Guidance/ Instruction Model
MODELS
Figure 1: An Architecture of Adaptive Decision Support Systems
User Diagnosis Model
Fazlollahi, et. al./ 36
Outcome for Case A Profile of
Right
User Wrong
Figure 2: Decision tree for Case A
Fazlollahi, et. al./ 37
Outcome for Case B Result of A
R
Right and Right
R W
Wrong and Right R R
Profile of
W
Right and Wrong
W
User
W
Wrong and Wrong R = Right W = Wrong
Figure 3: Decision tree for Case B
Fazlollahi, et. al./ 38
Figure 4: Menu pad
Fazlollahi, et. al./ 39
Figure 5: Data plot
Fazlollahi, et. al./ 40
Figure 6: Guidance screen for Case A
Fazlollahi, et. al./ 41
Figure 7: Guidance screen for Case B
Fazlollahi, et. al./ 42
Figure 8: Guidance screen for Case C
Fazlollahi, et. al./ 43
Figure 9: Guidance screen for Case D
Fazlollahi, et. al./ 44
REFERENCES
[1]
D. Diaper (Editor), Task Analysis for Human-Computer Interaction, Ellis Horwood Limited, Hartnolls, U.K., (1989).
[2]
B.L. Dos Santos and C.W. Holsapple, A Framework for Designing Adaptive DSS Interface, Decision Support Systems, 5(1), 1-11, (1989).
[3]
S. Dutta, Decision Support for Planning, Decision Support Systems, 12, 337-353, (1994).
[4]
C. W. Holsapple, R. Pakath, V. Jacob, and J. S. Zaveri, Learning by Problem Processors, Decision Support Systems, 10, 85-108, (1993).
[5]
P.G.W. Keen, Decision Support Systems: The Next Decade, Decision Support Systems, 3, 253-265, (1987).
[6]
P.G.W. Keen, Adaptive Design for DSS, DataBase, 12(1-2), 15-25, (1980).
[7]
P.G.W. Keen and M.S. Scott Morton, Decision Support Systems: An Organizational Perspective, Addison-Wesley, Reading, MA, (1978).
Fazlollahi, et. al./ 45
[8]
M.L. Manheim, An Architecture for Active DSS, Proceedings of 21st Hawaii International Conference on System Sciences, IEEE Computer Society, 356-365, (1988).
[9]
M.L. Manheim, Issues in Design of a Symbolic DSS, Proceedings of 22nd Hawaii International Conference on System Sciences, IEEE Computer Society, 14-23, (1989).
[10]
M.L. Manheim and D. Isenberg, A Theoretical Model of Human Problem-Solving and Its Use for Designing Decision-Support Systems, Proceedings of 20th Hawaii International Conference on System Sciences, IEEE Computer Society, 614-627, (1987).
[11]
M.L. Manheim, M.L. Srivastava, S. Vlahos, N. Hsu and J. Jones, A Symbolic DSS for Production Planning and Scheduling: Issues and Approaches, 23nd Hawaii International Conference on System Sciences, 383-390, (1990).
[12]
M.D. Merrill, R.D. Tennyson and L.O. Posey, Teaching Concepts :An Instructional Design Guide, 2nd ed., Educational Technology Publications, (1992).
[13]
F. Milli, Dynamic View of Decision Domains for the Design of Active DSS, 22nd Hawaii International Conference on System Sciences, IEEE Computer Society,
Fazlollahi, et. al./ 46
24-32, (1989).
[14]
H. Mintzberg, D. Raisinghani and A. Theoret, The Structure of 'Unstructured' Decision Processes, Administrative Science Quarterly, 21(2), 246-275, (June 1976).
[15]
J. Moormann, M. Lochte-Holtgreven, An approach for an integrated DSS for strategic planning, Decision Support Systems, 10(4), 401-411, (1993).
[16 ]
A. Newell and H.A. Simon, Human Problem-Solving, Prentice-Hall, Englewood Cliffs, New Jersey, (1972).
[17]
G.I. Nierenberg, The Idea Generator (A Software Product), Experience in Software Inc., Berkeley, CA, (1987).
[18]
S. Piramuthu, N. Raman, M. J. Shaw, and S. C. Park, Integration of Simulation Modeling and Inductive Learning in an Adaptive Decision Support System, Decision Support Systems, 9, 127-142, (1993).
[19]
F.J. Radermacher, Decision Support Systems: Scope and Potential, Decision Support Systems, 12(4/5), 257-265, (1994).
Fazlollahi, et. al./ 47
[20]
H. Raghav Rao, Ramalingam Sridhar and Sudeep Narain, An Active Intelligent Decision Support System - Architecture and Simulation, Decision Support Systems, 12(1), 79-91, (August 1994).
[21]
S.A. Raghavan, JANUS - A Paradigm for Active Decision Support, Decision Support Systems, 7, 379-395, (1991).
[22]
H. Raiffa, Decision Analysis. Reading, MA: Addison Wesley, (1968).
[23]
W.E. Remus and J.E. Kottemann, Toward Intelligent Decision Support Systems: An Artificially Intelligent Statistician, MIS Quarterly, 403-418, (1986).
[24]
C.S. Sankar, F. Nelson Ford and Michael Bauer, A DSS User Interface Model to Provide Consistency and Adaptability, Decision Support Systems, 13, 93-104, (1995).
[25]
M.S. Silver, Decisional Guidance for Computer-Based Decision Support, MIS Quarterly, 15, 47-71, (March 1991).
[26]
B. G. Silverman Unifying Expert Systems and Decision Sciences, Operations Research, 2(3), 393-413, (1994).
Fazlollahi, et. al./ 48
[27]
H.A. Simon, Sciences of the Artificial, MIT Press, (1981).
[28]
H.A. Simon, The New Science of Management Decision, Harper and Row, New York, (1960).
[29]
R.H. Sprague Jr., A Framework for Development of Decision Support Systems, MIS Quarterly, 4(4), 1-26, (December 1980).
[30]
R.H. Sprague, E.D. Carlson, Building Effective Decision Support Systems, Prentice-Hall International, Englewood Cliffs, N.J., (1982).
[31]
R. Sridhar, H.R. Rao and S. Narain, An Architectural Framework for an Intelligent Decision Support System, International Journal of Mini and Microcomputers, 12(2), (1990).
[32]
E. Turban, Decision Support and Expert Systems, 2nd ed., Macmillan Publishers, (1990).
Fazlollahi, et. al./ 49
Footnote
1
Refer to the following three publications for details:
Pub. 1: Fazlollahi, Parikh and Verma (1995). Evaluation of Decisional Guidance in Decision Support Systems: An Empirical Study. Proceedings of the Third International Conference of the Decision Sciences Institute, June 1995, Pueblo, Mexico; pp. 78-80. Pub. 2: Fazlollahi, Parikh and Verma (1995). Evaluation of Alternate Instructional Strategies in Intelligent Coaching Systems: An Empirical Study. Proceedings of the 26th Annual Meeting of the Decision Science Institute, November 1995, Boston, MA; pp. 499-501. Pub. 3: Fazlollahi, Parikh and Verma (1995). Influence of Decision Making Cognitive Style on the Design Features of Intelligent Guidance/Help for DSS: An Empirical Study. Proceedings of the 1995 Information Resources Management Association International Conference, May 1995, Atlanta, GA; pp. 25-30.
