Using a web-based survey tool to undertake a Delphi study: Application for nurse education research
Introduction
The Delphi technique is characterized as a series of surveys interspersed with controlled feedback. It has been widely used in a variety of nursing contexts to obtain previously unknown opinions of a group of informed individuals. Typical examples include identifying research priorities (Cowman et al., 2012, Fletcher-Johnston et al., 2011, Ramelet and Gill, 2012), developing quality indicators (Boulkedid et al., 2011, Uphoff et al., 2012, Wilson et al., 2012) or for gaining consensus on education assessments (Crookes and Brown, 2010, Sowter et al., 2011). The main premise is that the collective opinion of the group or panel is more valid than individual opinions. Hasson et al. (2000), identified that no universal guidelines exist for conducting a Delphi technique and historically postal surveys were commonly used to collect data (Keeney et al., 2010). Logistically, this process was both time consuming and costly.
The advent of the Internet is now increasingly being used to conduct surveys in populations who regularly use the World-Wide-Web (Allen and Roberts, 2010, Boulkedid et al., 2011, Kaplowitz et al., 2004), particularly those with a higher level of education and access to online facilities (Barrios et al., 2011). A web-based Delphi technique (eDelphi) is not only an environmentally friendly and cost effective way to carry out research, it enables participants to easily respond from diverse geographical locations, is time efficient, allows direct import into data analysis software, enables a quick turnaround time between rounds, and improves data quality (Barrios et al., 2011, Chang et al., 2010, Joinson and Reips, 2007, Kaplowitz et al., 2004, Okoli and Pawlowski, 2004). Commercially available web-based survey software offers a number of advantages over developing systems internally. They are usually quick to set up, relative low in cost and provide a high level of data security. Currently, there are over 300 web-survey commercial software products available, with SurveyMonkey (n.d.) being a market leader (Allen and Roberts, 2010, Fan and Yan, 2010). Therefore, for researchers interested in undertaking a national survey to elicit the opinions of an expert panel of Australian critical care nurses, a web-based Delphi technique seemed an attractive option.
The aim of this eDelphi study was to survey a national panel of experts to elicit their opinion on critical care course graduate practice standards. SurveyMonkey (SM) was selected to administer the eDelphi process as it was user-friendly, had been used with different web browsers, computer configurations and Internet services, supported SPSS for data importation and employed high level data protection measures that were consistent with industry standards (Allen and Roberts, 2010, Fan and Yan, 2010, Funke et al., 2011, SurveyMonkey, 2011). This paper reports on the experience and value of using web-survey software to conduct an eDelphi study.
Section snippets
Design
Using SM, the eDelphi technique consisted of the familiar iterative process of administering three rounds of surveys to a national panel of critical care nurses. The approach used has been entitled a ‘reactive eDelphi’ as in the first survey round panel members were asked to rate the importance of predetermined statements rather than provide responses to open questions (Keeney et al., 2010, McKenna, 1994).
Survey Development
Generating first round statements from earlier research using Delphi technique has
Ethical Considerations
Ethics approval was obtained from the University Human Research Ethics Committee (HREC) (SON&M 23-2011). Consent was inferred by panel members' agreement to participate in the study and submission of the completed surveys. Research conducted over the web poses additional threats to privacy and confidentiality in that the data exist as virtual records and although owned solely by the researcher, are potentially accessible by others (Ahern, 2005, Duffy, 2002). This potential threat is less of a
Discussion
Using SM to undertake this eDelphi technique provided a number of advantages over traditional survey approaches; these included the high quality of the data collected, ease and speed of administering the surveys, communicating with the panel and the generation of panel feedback. These advantages enabled data collection to be undertaken in 12 weeks.
The panel response rate and attrition rate over three rounds have the potential to impact on the rigor of a Delphi study (Keeney et al., 2010). Online
Conclusion
This eDelphi study aimed to obtain the opinions of a national panel of experts on critical care course graduate practice standards and was achieved using an online platform. The three rounds of surveys, including panel feedback between rounds, were facilitated by the ability to easily communicate with panel members located across Australia. Preparing feedback to provide to the panel collectively and individually was achieved using a combination of SM and Word software features. This enabled the
Acknowledgments
Thank you to the study participants who supported this study by completing the three survey rounds.
References (44)
- et al.
24/7 in-house intensivist coverage and fellowship education
Chest
(2012) The Delphi technique: a comparison of results obtained using two expert panels
International Journal of Nursing Studies
(1993)- et al.
Factors affecting response rates of the web survey: a systematic review
Computers in Human Behavior
(2010) Using the Delphi method to engage stakeholders: a comparison of two studies
Evaluation and Program Planning
(2010)- et al.
A review of critical care nursing staffing, education and practice standards
Australian Critical Care
(2012) - et al.
Personalized salutation, power of sender and response rates to Web-based surveys
Computers in Human Behavior
(2007) - et al.
Construction of a parent satisfaction instrument: perceptions of pediatric intensive care nurses and physicians
Journal of Critical Care
(2009) - et al.
The Delphi method as a research tool: an example, design considerations and applications
Information Management
(2004) - et al.
A Delphi study on National PICU nursing research priorities in Australia and New Zealand
Australian Critical Care
(2012) - et al.
The development of evidence based guidelines for clinical practice portfolios
Nurse Education Today
(2011)
Using the internet to conduct research
Nurse Researcher
The Ethics of Outsourcing Online Survey Research
Response rates and data quality in web and mail surveys administered to PhD holders
Social Science Computer Review
Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review
PLoS One
Using thematic analysis in psychology
Qualitative Research in Psychology
Methodological issues in online data collection
Journal of Advanced Nursing
A Delphi study to validate an Advanced Practice Nursing tool
Journal of Advanced Nursing
An international eDelphi study identifying the research and education priorities in wound management and tissue repair
Journal of Clinical Nursing
The Development of a Pre-registration Nursing Competencies Assessment Tool for Use Across Australian Universities
Do data characteristics change according to the number of scale points used?
International Journal of Market Research
Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes
Methodological issues in Web-based research
Journal of Nursing Scholarship
Cited by (86)
Self-management in heart failure using mHealth: A content validation
2023, International Journal of Medical InformaticsCommunity composting: A multidisciplinary evaluation of an inclusive, participative, and eco-friendly approach to biowaste management
2022, Cleaner Environmental SystemsCitation Excerpt :Two iterations were considered sufficient by many qualitative forecasting studies (Lloyd, 2011). During the consultation activities of the participatory process, an open structured interview format was chosen to collect information (Gill et al., 2013) from stakeholders, through a set of questions and relevant issues starting from the evidence from literature review. The questions, focused on drivers and obstacles influencing CC adoption and management, required narrative responses.
Design and content validation of an instrument measuring user perception of the persuasive design principles in a breastfeeding mHealth app: A modified Delphi study
2022, International Journal of Medical InformaticsA multi-stage decision making model for determining a suitable innovation structure using an open innovation approach
2022, Journal of Business Research