Querying Rural Content Experts Using an Online Questionnaire


content validity, rural, content experts, instrument development, lack of anonymity


Purpose:  Obtaining feedback from rural content experts is critical in developing valid and reliable instruments to advance the science of rural health. However, traditional methods, i.e., focus groups are impractical due to location and distance. Using an online questionnaire combined with telephone and email contacts to obtain content experts’ feedback is discussed. Item statement analysis and efficiency and effectiveness of the process are presented. 

Methods:  The process included the development of an online questionnaire, asking experts to rate 51 item statement for their relevancy, sufficiency of description, and clarity and readability.  To increase the response rate, a series of four contacts (one telephone and three email) were planned and implemented.  An item content validity index (I-CVI) was calculated for all items.

Results:  Distribution of the online questionnaire to rural content experts separated by geographic distance was efficient and effective in gathering feedback on item statements for content validity.  Content experts completed the questionnaire in less than one hour suggesting the overall efficiency of the process; an 84% response rate supports process effectiveness.  Following I-CVI evaluation, item statements were reduced from 51 to 24.  The analysis resulted in retaining, with or without revision, 47% of the item statements. 

Conclusions:  The online questionnaire and four-contact strategy were effective in gathering input from a representative sample of rural content experts separated by great distances; thereby, strengthening the content validity of the item statements.  The process demonstrates new opportunities for using online technologies to reach rural content experts.


Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share (for non-commerical purposes) the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).