The below abstract was sent to the SACNAS National Conference.
Under the field of human computer interaction (HCI), the subfield of captology guides how technology can influence behavior [1]. In this research, we extend research from Project Acceso 2015 and perform an analysis of previous online conversations and improve upon the design of an existing online discussion board. More specifically, this research focuses on how simple algorithms can be used to influence the quality and flow of online conversations resulting in greater topic-focus and more readable discussion posts.
In total, 1,629 conversations were mined for readability and keyword density. Readability was accessed using Readability Metrics, an open-source application programming interface for managing texts and their readability scores [2]. Key-word density was calculated as a ratio of total keywords found over total words posted minus all stop-words. Our analysis found that while readability increased from originations to responses, there were decreases in response readability and keyword density. Additionally, there was the tendency for users to move away from topics as discussions aged.
A social network analysis (SNA) was also performed using NodeXL, which is an open source extension for MS Excel that provides a range of basic network analytics and visualization features [3]. Our analysis discovered that users who fell within an acceptable level of quality tended to be more central to the network, while users with lower quality scores tended to be situated on the outskirts of the network. This suggests that as the quality of user posts increases, the number of responses that user receives also increases, thus increasing the density of the social network.
To remediate these issues and persuade users to deliver more on-topic, readable discussion posts, SPEQ-DB provides users with a more responsive way to judge the quality of their posts. More specifically, users can view a quality report, which dynamically analyzes and rates the content of their posts and provides users with a subset of trending keywords. Additionally, an overall group quality index is calculated and provides users with a comparison of their individual post as it is measured against the group’s average. Finally, discussion pins allow users to keep track of posts they are most interested in or find most helpful. Below is the screenshot, or you can try it out here: socialxyz.com/SPEQ. A sample poster can be found here: socialxyz.com/pubs/QualityPoster-draftv3.1.pdf.
To test SPEQ-DB, a design science research methodology is proposed. In design science, researchers are concerned with the way things ought to be in order to attain goals and they construct artifacts as a way of achieving these goals [4]. Building atop [5], our research asks the following questions:
R1: To what extent will SPEQ-DB enhance the quality of both origination and response posts in online conversations?
R2: To what extent will SPEQ-DB increase levels of network density within the online community?
REFERENCES
[1] Fogg, B., & Nass, C. (1997). “Silicon sycophants: The effects of computers that flatter,” Int’l Journal of Human Computer Studies, 46(5).
[2] Ipeirotis, P. (2012). “Readability Metrics API,” Mashape. Accessed June 19, 2016 from https://market.mashape.com/ipeirotis/readability-metrics.
[3] Hansen, D., Shneiderman, B. and Smith, M. (2011). Analyzing Social Media Networks with NodeXL: Insights from a Connected World, Burlington: Morgan Kaufmann.
[4] Simon, H. (1996). The Sciences of the Artificial Third Edition, Cambridge, MA : MIT Press.
[5] Thoms, B., Eryilmaz, E. (2014). “How Media Choice Affects Learner Interactions in Distance Learning Classes,” Computers & Education, v75, pp.112-126