| Futures studies |
|---|
 |
| Concepts |
| Techniques |
| Technology assessment and forecasting |
| Related topics |
Real-time Delphi (RTD) is an advanced form of the Delphi method. The advanced method "is a consultative process that uses computer technology" [1] to increase efficiency of the Delphi process.
Gordon and Pease [2] define the advanced approach as an innovative way to conduct Delphi studies that do not involve sequential "rounds" and consequently lead to a higher degree of efficiency with regard to the time frame needed to perform such studies. Friedewald, von Oertzen, and Cuhls [3] underline that aspect by writing, in "a Real-Time-Delphi, the participants do not only judge twice but can change their opinion as often as they like when they see the aggregated results of the other participants". So, here it becomes clear that the Real-Time Delphi approach requires real-time calculation and provision of group responses. Friedewald et al. [3] further state that the Real-Time Delphi method has beneath its explorative and predictive elements also normative and communicative elements. These latter are investigated by Bolognini, [4] who explores the potential of computer-based Delphi as a communication technique for electronic democracy.
Comparative studies of von der Gracht and colleagues have revealed that Real-time Delphi studies are comparable to the outcome of conventional round-based Delphi surveys. [5]
The basic idea of a real-time, therefore computer-based (usually web-based), Delphi approach originates in a paper published by Turoff back in 1972 about an online Delphi conference conducted in the United States. [6] The conference was characterized by remote locations of participants, an online tool to access and give judgments, anonymity of the participants, continuous operations and analysis of results (i.e. participants were able to see given answers of the other participants in real-time), as well as asynchronous participation (i.e. participants could independently login and logout how often and when they desired). The stated aspects are some of the key characteristics of Real-Time Delphi studies, which shows that the original idea of conducting such studies can be traced back to the respective year. Today, nevertheless, technological innovations and advanced computer aided design possibilities (e.g. high-speed internet connections, high definition graphic, and advanced processor performance) facilitate more sophisticated studies in this context. [7] The general idea to develop a faster advanced form of Delphi studies by using ideas and basic concepts of Turoff, was initiated by the U.S. Defense Advanced Research Projects Agency (DARPA), which awarded a grant in 2004 to develop an approach to improve "speed and efficiency of collecting judgments in tactical situations". [2] A small software company named Articulate Software in San Francisco was awarded an innovation research grant to develop what DARPA was asking for. [2] Adam Pease, principal consultant and CEO of Articulate Software, published the findings and methodology together with Theodore Gordon in 2006. [2]
Based on the findings in this seminal paper, Heiko von der Gracht developed a second Real-time Delphi platform until 2008. In subsequent years, von der Gracht and colleagues used the platform extensively for prospective research studies. In 2011, von der Gracht and Gordon presented a joint paper at WFS Annual Summit 2011, which summarizes the lessons learned of 40 RTD studies across the two platforms. [8]
In 2017, Angenheyster and colleagues published a study, which compares various Real-time Delphi platforms. Analysis criteria include among others visual appearance, data output, user friendliness, administration, and types of questions. [9] A cross-disciplinary study by Beiderbeck et al. focused on new directions and advancements of the Delphi method, including real-time formats. The authors provide a methodological toolbox for designing Delphi surveys including among others sentiment analyses of the field of psychology. [10]
The question arises how a Real-Time Delphi study differs from a Conventional Delphi study. The basic framework is to think of a Delphi study which is conducted in form of an online questionnaire. However, a Conventional round-based Delphi study conducted via the internet is called "Internet Delphi". The basic difference to Internet Delphi is that the process of a Real-Time Delphi is not characterised by single iterated rounds. In fact, real-time calculation and provision of responses are the key characteristics of Real-Time Delphis. Various other labels for Real-Time Delphi can be found in literature and many authors are not completely aware of the differences: [11] "Electronic Delphi", "Computer Delphi", "Computer-aided Delphi", and "Technology Delphi". However, it is important to truly understand the design and process a researcher has chosen to find out whether real-time calculations and provisions have been applied or not.
The typical Real-Time Delphi process can be described in the way that participants get access to an online questionnaire portal for a certain time frame, within which they are allowed to log in and log out as often as they want. Whenever they login, they will see all their quantitative and qualitative answers of previous sessions and they can change all answers as desired within the given period of time. Besides their own answers they will see the ongoing – hence, real-time – responses of other participants, and with regard to metric assessments the group as a whole will be visualised in terms of median, average, and interquartile range (IQR). It has to be pointed out that the numerical visualisations as well as the qualitative inputs change in the course of other participants changing their responses. [2] Consequently, a participant can find out to what extent his own responses from an earlier point of time are still within the group opinion (i.e. IQR). The core innovation, then, of Real-Time Delphi studies is the real-time calculation and provision of results. [12]
The core methodological innovation of Real-Time Delphi studies are the absence of iterated rounds and the real-time calculation and provision of group responses. Whereas Conventional Delphi studies are characterised by repeating sequential rounds, the Real-Time Delphi approach is characterised by a continuous round-less procedure leading to a reduced time frame needed to conduct such studies. [13] Consequently, conducting large-scale studies of huge complexity in a relatively short period of time becomes possible. [13] Another core methodological innovation is the fact that experts may not only judge once or twice, depending on the number of rounds, as it has been usual in a Conventional Delphi study. During a Real-Time Delphi, experts can independently reassess their responses as often as they want. [3]
Hartman and Baldwin [1] discuss further advantages of the Real-Time Delphi approach: First, the number of experts participating in the real-time study can be increased due to a higher degree of automation during and improved possibilities for analysis after the study. Additionally, the Internet provides the possibility to invite a worldwide expert panel to participate in the study. Second, the degree of interaction among the experts can be increased due to the fact that they can immediately react on others' comments. Additionally, the time frame between giving own answers and getting insights into others' responses is very short, which encourages stronger cognitive examination with the respective issue in question. Hartman and Baldwin [1] argue that with the help of this procedure the validity of results is maximised.
In order to conduct a Real-Time Delphi study, computer software – usually web-based – is needed for facilitating real-time calculations and visualisation of results. It is generally proposed in the existing literature that the experts participating in the study see not only their own answers but also the median and interquartile range of all given responses immediately after answering a quantitative question. [2] Besides the quantitative assessment a qualitative judgment of participants can be shown which serves as a justification for their numerical assessment of the question. Additionally, it can be shown to the expert how many respondents have already given their answers. To examine the qualitative arguments of others participants can click on a button, and a "reasons window" opens, which shows the statements of other participants to underline their point of view. So, the respective legitimisations given by others may cause a respondent to recapture his own point of view. [2] In the next step the expert can change his own answer, add new arguments to underline his point of view, or leave his answer unchanged. In addition, the respondent will be shown an attention indicator, a so-called "flag", [13] if his answer is within or outside the interquartile range or significantly different from the median. This application helps to see and understand immediately the own assessment and to think about reasons for the deviation from group opinions or else a high degree of consent. The respondent's attention will be called by highlighting questions with a high degree of deviation with a different colour and by asking him to give further reasons for his deviation from the group opinion.
After operating a question in the described procedure, the participant can continue to the next question or press a "save"-button in the program, which leads to an immediate update of the median, interquartile range, and given arguments, and then leave the program. A second advantage of the round-less approach is the fact that, in order to take part in the study, participants can login and logout with their personalised account as often as they want during the time frame provided. Their already given answers will be saved and recalled when they login the next time. So, by design of the study, there are no explicit single rounds to answer the questions. [2] Updating and playing back the information to the other participants follow immediately in succession to the process of answering. Here, it becomes clear that the process of answering can be synchronous or asynchronous and a worldwide expert panel can be reached, which is one of the major advantages of web-based tools. Turoff and Hiltz [11] argue that the issue of asynchronous interaction is probably one of the least understood characteristics of Real-Time Delphis. Zipfinger [14] points out two advantages of asynchronous participation of the experts: First, they can login to the portal whenever they want; therefore, one could argue that the degree of convenience of taking part is increased for the participant due to a 24-hour availability of the portal. Second, panellists can contribute to whatever aspects in the questionnaire they want, especially when having gone through each question at least once. [11] Here, a substantial aspect of Real-Time Delphis becomes obvious: Turoff and Hiltz [11] explain that a Real-Time Delphi study offers a design of structured communication which allows every individual to choose the sequence and speed to contribute to the problem solution process. So, in comparison to face-to-face discussions, the Real-Time Delphi approach gives room for individuality and different cognitive abilities of the participants. [11]
A further advantage is the fact that the administrator of the study can set an arbitrary time frame in which participants have to log in and take part in the questionnaire. So, whenever the researcher or administrator of the study is satisfied with the existing answers (i.e. in terms of quantity and quality), he can declare the study to be ended and close the online tool (i.e. "freeze" the responses). [2] [13]
The key features of Delphi studies, such as anonymity, controlled feedback and group response, are also met in the context of Real-Time Delphi studies. However, the issue of iteration is, by design of the technique, not valid for Real-Time Delphi studies anymore. Instead of answering each question a first time and getting a second sheet with the group responses in the second round, the Real-Time Delphi already shows the second screen (i.e. group responses) immediately after answering each question. Having answered each question at least once, the participant can usually control which question to reassess from a "consensus portal", which serves as a kind of control panel to access single questions again. So, on the one hand, the procedure differs from a Conventional Delphi and, on the other hand, the iteration into single rounds is missing.
Having asked the question how the accomplishment of a Real-Time Delphi study differs from conducting a usual Delphi study, Gordon and Pease [2] point out that a Real-Time Delphi study can be implemented via a site on the Internet or in any other network (e.g. intra-company network, local area network) and is, therefore, not conducted in paper-and-pencil form any more.
As with all Delphi studies, the process of defining and selecting experts is still extremely important. [2] The Conventional Delphi study is then divided into several steps of response round, analysis through the facilitator, playing back the information, next response round, and so on. However, the Real-Time Delphi study is, after granting access to the online tool, rather a self-running process.
The basic strengths of a Real-Time Delphi study are its efficiency and applicability to all Delphi topics (i.e. common problem sets, decision making issues, cross impact studies, etc.). [2] Figure 2 illustrates that the process of a Real-Time Delphi differs. Important is to point out that the number of interventions of the facilitator needed during the response phases (i.e. after opening the online tool) are usually less. Having developed the online tool in advance, the intermediate analysis done by the facilitator of the study is rather uncomplicated in comparison to the Conventional Delphi. The overall shortened time period needed to conduct a real-time study underlines that the approach can be regarded as generally more efficient. [2]
Gordon and Pease [2] point out that a Real-Time Delphi study is applicable for a wide range of possible circumstances under which the consultation of experts is necessary. On the one hand, the authors give the example of a "small group operating synchronously in a conference room with laptop computers connected wirelessly to the web site where the software resides, with anticipated completion of the exercise in say 20 min.". On the other hand, it can be thought of a larger panel of experts operating asynchronously from remote locations within a longer period of time.
The greatest weakness of the real-time approach is that it is missing a wholly integrated, scientifically founded concept. The real-time Delphi idea is still a very new concept, which requires further research and application to become a tool for full-scale operations. [2] Especially the editing of the alpha (i.e. the first) inputs of respondents, the real-time presentation of group results, and the tracking of progress over time should be integrated in a kind of administrator package to make the accomplishment of a Real-Time Delphi less difficult. [2]
Numerous examples for real-time Delphi applications can be found. Among them, The Millennium Project conducted by Glenn, Gordon, & Florescu in 2009 provides a context for global thinking and improved understanding of global issues, opportunities, challenges, and strategies. More information on the project can be found on www.millennium-project.org
Another stream of projects based on real-time Delphi studies was conducted by the former Institute for Futures Studies and Knowledge Management of EBS University of Business and Law in Germany, among them the T&L2030 Series of five different studies together with PricewaterhouseCoopers: Transportation & Logistics 2030 Series. [15] Further illustrative examples in academic journals include cases from logistics, [16] social business, [17] professional services, [18] retail, [19] and automotive. [20]
An interview is a structured conversation where one participant asks questions, and the other provides answers. In common parlance, the word "interview" refers to a one-on-one conversation between an interviewer and an interviewee. The interviewer asks questions to which the interviewee responds, usually providing information. That information may be used or provided to other audiences immediately or later. This feature is common to many types of interviews – a job interview or interview with a witness to an event may have no other audience present at the time, but the answers will be later provided to others in the employment or investigative process. An interview may also transfer information in both directions.
The Delphi method or Delphi technique is a structured communication technique or method, originally developed as a systematic, interactive forecasting method that relies on a panel of experts. Delphi has been widely used for business forecasting and has certain advantages over another structured forecasting approach, prediction markets.
Scenario planning, scenario thinking, scenario analysis, scenario prediction and the scenario method all describe a strategic planning method that some organizations use to make flexible long-term plans. It is in large part an adaptation and generalization of classic methods used by military intelligence.
An electronic meeting system (EMS) is a type of computer software that facilitates creative problem solving and decision-making of groups within or across organizations. The term was coined by Alan R. Dennis et al. in 1988. The term is synonymous with group support systems (GSS) and essentially synonymous with group decision support systems (GDSS). Electronic meeting systems form a class of applications for computer supported cooperative work.
All telecommunications service providers perform forecasting calculations to assist them in planning their networks. Accurate forecasting helps operators to make key investment decisions relating to product development and introduction, advertising, pricing etc., well in advance of product launch, which helps to ensure that the company will make a profit on a new venture and that capital is invested wisely.
The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations, published in 2004, is a book written by James Surowiecki about the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group. The book presents numerous case studies and anecdotes to illustrate its argument, and touches on several fields, primarily economics and psychology.
Strategic foresight is a planning-oriented discipline related to futures studies. In a business context, a more action-oriented approach has become well known as corporate foresight.
Futures techniques used in the multi-disciplinary field of futurology by futurists in Americas and Australasia, and futurology by futurologists in EU, include a diverse range of forecasting methods, including anticipatory thinking, backcasting, simulation, and visioning. Some of the anticipatory methods include, the delphi method, causal layered analysis, environmental scanning, morphological analysis, and scenario planning.
Technology forecasting attempts to predict the future characteristics of useful technological machines, procedures or techniques. Researchers create technology forecasts based on past experience and current technological developments. Like other forecasts, technology forecasting can be helpful for both public and private organizations to make smart decisions. By analyzing future opportunities and threats, the forecaster can improve decisions in order to achieve maximum benefits. Today, most countries are experiencing huge social and economic changes, which heavily rely on technology development. By analyzing these changes, government and economic institutions could make plans for future developments. However, not all of historical data can be used for technology forecasting, forecasters also need to adopt advanced technology and quantitative modeling from experts’ researches and conclusions.
Cognitive pretesting, or cognitive interviewing, is a field research method where data is collected on how the subject answers interview questions. It is the evaluation of a test or questionnaire before it's administered. It allows survey researchers to collect feedback regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey to the full sample of people.
Synchronous conferencing or synchronous computer-mediated communication (SCMC) is any form of computer-mediated communication that occurs in real-time; that is, there is no significant delay between sending and receiving messages. SCMC includes real-time forms of text, audio, and video communication. SCMC has been highly studied in the context of e-learning.
The wisdom of the crowd is the collective opinion of a diverse and independent group of individuals rather than that of a single expert. This process, while not new to the Information Age, has been pushed into the mainstream spotlight by social information sites such as Quora, Reddit, Stack Exchange, Wikipedia, Yahoo! Answers, and other web resources which rely on collective human knowledge. An explanation for this phenomenon is that there is idiosyncratic noise associated with each individual judgment, and taking the average over a large number of responses will go some way toward canceling the effect of this noise.
Group decision-making is a situation faced when individuals collectively make a choice from the alternatives before them. The decision is then no longer attributable to any single individual who is a member of the group. This is because all the individuals and social group processes such as social influence contribute to the outcome. The decisions made by groups are often different from those made by individuals. In workplace settings, collaborative decision-making is one of the most successful models to generate buy-in from other stakeholders, build consensus, and encourage creativity. According to the idea of synergy, decisions made collectively also tend to be more effective than decisions made by a single individual. In this vein, certain collaborative arrangements have the potential to generate better net performance outcomes than individuals acting on their own. Under normal everyday conditions, collaborative or group decision-making would often be preferred and would generate more benefits than individual decision-making when there is the time for proper deliberation, discussion, and dialogue. This can be achieved through the use of committee, teams, groups, partnerships, or other collaborative social processes.
Harold Adrian Linstone was a German-American mathematician, consultant, futurist and University Professor Emeritus of Systems Science at Portland State University and a specialist in applied mathematics.
Robust decision-making (RDM) is an iterative decision analytics framework that aims to help identify potential robust strategies, characterize the vulnerabilities of such strategies, and evaluate the tradeoffs among them. RDM focuses on informing decisions under conditions of what is called "deep uncertainty", that is, conditions where the parties to a decision do not know or do not agree on the system models relating actions to consequences or the prior probability distributions for the key input parameters to those models.
Adaptive learning, also known as adaptive teaching, is an educational method which uses computer algorithms as well as artificial intelligence to orchestrate the interaction with the learner and deliver customized resources and learning activities to address the unique needs of each learner. In professional learning contexts, individuals may "test out" of some training to ensure they engage with novel instruction. Computers adapt the presentation of educational material according to students' learning needs, as indicated by their responses to questions, tasks and experiences. The technology encompasses aspects derived from various fields of study including computer science, AI, psychometrics, education, psychology, and brain science.
Corporate foresight has been conceptualised by strategic foresight practitioners and academics working and/or studying corporations as a set of practices, a set of capabilities and an ability of a firm. It enables firms to detect discontinuous change early, interpret its consequences for the firm, and inform future courses of action to ensure the long-term survival and success of the company.
An online interview is an online research method conducted using computer-mediated communication (CMC), such as instant messaging, email, or video. Online interviews require different ethical considerations, sampling and rapport than practices found in traditional face-to-face (F2F) interviews. Online interviews are separated into synchronous online interviews, for example via online chat which happen in 'real time' online and asynchronous online interviews, for example via email conducted in non-real time. Some authors discuss online interviews in relation to online focus groups whereas others look at online interviews as separate research methods. This article will only discuss online interviews.
Future-oriented technology analysis (FTA) is a collective term from futures studies for analyzing future technology and its consequences. It includes technology intelligence, technology forecasting, technology roadmapping, technology assessment, and technology foresight. Technology Futures Analysis or Technology Future Analysis (TFA) is a synonym.
Heiko Andreas von der Gracht is a German academic, futurist, author, and an expert in the fields of Foresight and Technology management.