Online Primary Research: From Exploring to Creating

 

Image Credit: https://sites.google.com/site/wrdsbresearchandinquiryplan/home/rationale-2

 

Conducting Primary Research Online: A Primer

1. Start by writing your research question(s) for the overall research topic.
2. Which research questions can best be answered with primary research?
3. Categorize your primary research as interview, survey, or observation.
4. Determine if the primary research is possible in person, online, or both.

 

Online Survey	Online Interview	Online Observation (including audience analysis and reception studies)
SurveyMonkey.com	E-mail Correspondence	Google Reviews
Google Forms	Twitter Threads	TrustPilotYelp!/Better Business Bureau/Review Forums
Facebook/Twitter Polls/Instagram	Facebook Interviews	Online Text/Content Analysis Tools.  For example: Voyant-Tools.org  OmniSci Twitter Map
Texting/Apps/Forums- (Reddit Survey Forum)	Video/FaceTime/Skype/ Zoom/GoToMeeting

Background for Online Observations: Reception Studies

In Chapter 8 of RAAWr, you read about the importance of the “user experience” of a product or document.  You also read about usability testing, protocol analysis, focus groups, interviews, and surveys as methods to learn how users feel about a product or document.  All of these are valuable methods that could be used as part of Project 3.

However, you have another primary research method available for Project 3 called a “reception study.”  A reception study is a type of online observation that involves the collection and analysis of discourse (often text, but also images, emoji, gif) that real users have written or produced about their experience or opinion of something.  Harris (2005) and Ceccarelli (2005) have each argued for the importance of reception studies in understanding how actual users, not ideal or imagined users, experience the world. Additionally, Iser’s (2006) Reception Theory calls for text, context, and the reader. Put another way, reception theory should be coupled with critical reception (what are experts saying?), allowing for a richer understanding of the topic.

In academic research, reception studies are used to learn how “real” audiences have written about literature, film, media, science, technology, and products and services. The melding of critical and audience reactions broadens perspective in research. In Project 3, you may design and complete a reception study to better understand what real users or audiences have posted online about your research problem, which is likely a product, service, or situation. Remember to consider the tone and credibility of real-world user opinion, especially placed in context with secondary (or critical) sources.

The steps below are intended to guide you through the collection and analysis of online texts as part of a reception study using Voyant Tools.

A Brief Guide to Using Voyant Tools to Analyze Text

1. Begin by brainstorming where you could look for online reviews, comments, posts, or feedback about your research problem for Project 3.
   1. Consider Google Reviews, Yelp, Bing Reviews, Comments Sections, Online Forums, Twitter Hashtags, Facebook Reviews, etc.
2. Take a brief look at each of the locations you have brainstormed above.
   1. Note the number of reviews or posts.  Are there enough to draw conclusions?
   2. Note what type of reviews, ratings, or feedback is posted.  Do they include text?
3. Select from the options above or select a combination from the above options.
   1. Which source(s) of audience reception is/are most representative?
   2. Which source(s) has/have enough responses to analyze and interpret?
   3. How old or new are the posts on the sources, or how long do they span?
4. Copy ALL of the selected user comments/reviews and paste them into a separate file.
5. “Clean” the text: remove usernames and label information such as “stars” or “date.”
6. Copy and paste the entirety of your “cleaned” text into Voyant-Tools.org and “Reveal.”
7. Use the “context,” “correlations,” and “document terms” options to suggest relationships between users’ comments based on words and phrases.
8. Finally, analyze how the use of words and phrases in user comments informs your understanding of the research problem in Project 3.  How does user reception help you understand the causes, extent, and impact populations of your research problem?

Example of Online Reception Study Using Voyant Tools

Figure 1. Screenshot of Google Reviews of USF Bull Runner.

Figure 2. Screenshot of the Voyant Tools landing page (Voyant-Tools.org)

Figure 3. Screenshot of the Voyant Tools visualization interface (Voyant-Tools.org).

Note: As Figure 3 illustrates, you can select or deselect terms to analyze and compare (top right).  You can also compare contexts and correlations between words and phrases that appear frequently (bottom right). 

Please see Figure 4 on the next page for how a Fall 2019 student used Voyant Tools to present a reception study in their Project 3 Research Summary (this student granted permission for their anonymous work to be presented as a teaching example).

Figure 4. Student example of online reception study within Project 3.  This research was presented under the Impacted Populations heading of the Project 3 memo.

Questions?

Please feel free to reach out to us if you have any questions about conducting online primary research for Projects 3 and 4 using Voyant Tools.

Dr. R.J. Lambert Dr. Gina Anderson-Lopez

rjl2@usf.edu andersonlope@usf.edu