Elsevier

Nurse Education Today

Volume 33, Issue 11, November 2013, Pages 1322-1328
Nurse Education Today

Using a web-based survey tool to undertake a Delphi study: Application for nurse education research

https://doi.org/10.1016/j.nedt.2013.02.016Get rights and content

Summary

Background

The Internet is increasingly being used as a data collection medium to access research participants. This paper reports on the experience and value of using web-survey software to conduct an eDelphi study to develop Australian critical care course graduate practice standards.

Methods

The eDelphi technique used involved the iterative process of administering three rounds of surveys to a national expert panel. The survey was developed online using SurveyMonkey. Panel members responded to statements using one rating scale for round one and two scales for rounds two and three. Text boxes for panel comments were provided.

Collecting data and providing feedback

For each round, the SurveyMonkey's email tool was used to distribute an individualized email invitation containing the survey web link. The distribution of panel responses, individual responses and a summary of comments were emailed to panel members. Stacked bar charts representing the distribution of responses were generated using the SurveyMonkey software. Panel response rates remained greater than 85% over all rounds.

Discussion

An online survey provided numerous advantages over traditional survey approaches including high quality data collection, ease and speed of survey administration, direct communication with the panel and rapid collation of feedback allowing data collection to be undertaken in 12 weeks. Only minor challenges were experienced using the technology. Ethical issues, specific to using the Internet to conduct research and external hosting of web-based software, lacked formal guidance.

Conclusions

High response rates and an increased level of data quality were achieved in this study using web-survey software and the process was efficient and user-friendly. However, when considering online survey software, it is important to match the research design with the computer capabilities of participants and recognize that ethical review guidelines and processes have not yet kept pace with online research practices.

Introduction

The Delphi technique is characterized as a series of surveys interspersed with controlled feedback. It has been widely used in a variety of nursing contexts to obtain previously unknown opinions of a group of informed individuals. Typical examples include identifying research priorities (Cowman et al., 2012, Fletcher-Johnston et al., 2011, Ramelet and Gill, 2012), developing quality indicators (Boulkedid et al., 2011, Uphoff et al., 2012, Wilson et al., 2012) or for gaining consensus on education assessments (Crookes and Brown, 2010, Sowter et al., 2011). The main premise is that the collective opinion of the group or panel is more valid than individual opinions. Hasson et al. (2000), identified that no universal guidelines exist for conducting a Delphi technique and historically postal surveys were commonly used to collect data (Keeney et al., 2010). Logistically, this process was both time consuming and costly.

The advent of the Internet is now increasingly being used to conduct surveys in populations who regularly use the World-Wide-Web (Allen and Roberts, 2010, Boulkedid et al., 2011, Kaplowitz et al., 2004), particularly those with a higher level of education and access to online facilities (Barrios et al., 2011). A web-based Delphi technique (eDelphi) is not only an environmentally friendly and cost effective way to carry out research, it enables participants to easily respond from diverse geographical locations, is time efficient, allows direct import into data analysis software, enables a quick turnaround time between rounds, and improves data quality (Barrios et al., 2011, Chang et al., 2010, Joinson and Reips, 2007, Kaplowitz et al., 2004, Okoli and Pawlowski, 2004). Commercially available web-based survey software offers a number of advantages over developing systems internally. They are usually quick to set up, relative low in cost and provide a high level of data security. Currently, there are over 300 web-survey commercial software products available, with SurveyMonkey (n.d.) being a market leader (Allen and Roberts, 2010, Fan and Yan, 2010). Therefore, for researchers interested in undertaking a national survey to elicit the opinions of an expert panel of Australian critical care nurses, a web-based Delphi technique seemed an attractive option.

The aim of this eDelphi study was to survey a national panel of experts to elicit their opinion on critical care course graduate practice standards. SurveyMonkey (SM) was selected to administer the eDelphi process as it was user-friendly, had been used with different web browsers, computer configurations and Internet services, supported SPSS for data importation and employed high level data protection measures that were consistent with industry standards (Allen and Roberts, 2010, Fan and Yan, 2010, Funke et al., 2011, SurveyMonkey, 2011). This paper reports on the experience and value of using web-survey software to conduct an eDelphi study.

Section snippets

Design

Using SM, the eDelphi technique consisted of the familiar iterative process of administering three rounds of surveys to a national panel of critical care nurses. The approach used has been entitled a ‘reactive eDelphi’ as in the first survey round panel members were asked to rate the importance of predetermined statements rather than provide responses to open questions (Keeney et al., 2010, McKenna, 1994).

Survey Development

Generating first round statements from earlier research using Delphi technique has

Ethical Considerations

Ethics approval was obtained from the University Human Research Ethics Committee (HREC) (SON&M 23-2011). Consent was inferred by panel members' agreement to participate in the study and submission of the completed surveys. Research conducted over the web poses additional threats to privacy and confidentiality in that the data exist as virtual records and although owned solely by the researcher, are potentially accessible by others (Ahern, 2005, Duffy, 2002). This potential threat is less of a

Discussion

Using SM to undertake this eDelphi technique provided a number of advantages over traditional survey approaches; these included the high quality of the data collected, ease and speed of administering the surveys, communicating with the panel and the generation of panel feedback. These advantages enabled data collection to be undertaken in 12 weeks.

The panel response rate and attrition rate over three rounds have the potential to impact on the rigor of a Delphi study (Keeney et al., 2010). Online

Conclusion

This eDelphi study aimed to obtain the opinions of a national panel of experts on critical care course graduate practice standards and was achieved using an online platform. The three rounds of surveys, including panel feedback between rounds, were facilitated by the ability to easily communicate with panel members located across Australia. Preparing feedback to provide to the panel collectively and individually was achieved using a combination of SM and Word software features. This enabled the

Acknowledgments

Thank you to the study participants who supported this study by completing the three survey rounds.

References (44)

  • N.R. Ahern

    Using the internet to conduct research

    Nurse Researcher

    (2005)
  • P.J. Allen et al.

    The Ethics of Outsourcing Online Survey Research

    (2010)
  • M. Barrios et al.

    Response rates and data quality in web and mail surveys administered to PhD holders

    Social Science Computer Review

    (2011)
  • R. Boulkedid et al.

    Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review

    PLoS One

    (2011)
  • V. Braun et al.

    Using thematic analysis in psychology

    Qualitative Research in Psychology

    (2006)
  • M.A. Cantrell et al.

    Methodological issues in online data collection

    Journal of Advanced Nursing

    (2007)
  • A. Chang et al.

    A Delphi study to validate an Advanced Practice Nursing tool

    Journal of Advanced Nursing

    (2010)
  • S. Cowman et al.

    An international eDelphi study identifying the research and education priorities in wound management and tissue repair

    Journal of Clinical Nursing

    (2012)
  • P. Crookes et al.

    The Development of a Pre-registration Nursing Competencies Assessment Tool for Use Across Australian Universities

    (2010)
  • J. Dawes

    Do data characteristics change according to the number of scale points used?

    International Journal of Market Research

    (2008)
  • A. Delbecq et al.

    Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes

    (1986)
  • M.E. Duffy

    Methodological issues in Web-based research

    Journal of Nursing Scholarship

    (2002)
  • Cited by (86)

    • Self-management in heart failure using mHealth: A content validation

      2023, International Journal of Medical Informatics
    • Community composting: A multidisciplinary evaluation of an inclusive, participative, and eco-friendly approach to biowaste management

      2022, Cleaner Environmental Systems
      Citation Excerpt :

      Two iterations were considered sufficient by many qualitative forecasting studies (Lloyd, 2011). During the consultation activities of the participatory process, an open structured interview format was chosen to collect information (Gill et al., 2013) from stakeholders, through a set of questions and relevant issues starting from the evidence from literature review. The questions, focused on drivers and obstacles influencing CC adoption and management, required narrative responses.

    View all citing articles on Scopus
    1

    Tel.: + 61 8 9266 2070.

    2

    Tel.: + 61 8 83022754.

    3

    Tel.: + 31 107037174.

    View full text