tl;dr – 5-min Video Presentation
Open-ended Courses in Distance Learning
As the latest iteration of a storied effort to enhance access to high-quality educational resources through distance learning, in recent years, universities are offering their vast catalog of STEM and non-STEM courses online and at scale. In this paper, we ask: how can we effectively teach open-ended university-level courses – mostly from the humanities, the arts, and the social sciences – online or at scale?
We define open-ended courses (OE courses) as those which involve a significant amount of content that:
- incorporates introspection and criticism,
- raises questions with more than a small number of “correct” responses, and
- is amenable to subjective interpretation.
In our university, a significant number of departments offer mostly OE courses. And about one-third of the Massive Open Online Courses (MOOCs) today are from these disciplines.
Motivation
The following question motivates our work:
Should instructors teach OE courses such as Jazz Improvisation and non-open-ended (NOE) courses like Intro to Programming in the same way?
Surprisingly, despite the different characteristics between OE and NOE courses, prior work (Peng) indicated that instructors teach OE courses at scale in a manner similar to NOE courses on mainstream MOOC platforms (e.g., Coursera, Udacity, EdX), through video lectures, quizzes, assignments, and peer assessment. In addition, prior studies (Shea et al. and Evans & Myrick) found that instructors in traditionally open-ended subjects had lower satisfaction and more trouble offering online courses.
So, despite the prevalence of OE courses, we do not know much about how to teach them at scale and online effectively. These findings motivated us to better understand the challenges and concerns of university faculty teaching and students taking open-ended university courses offered at scale. Prior work that aimed to scale OE courses focused primarily on lowering the cost of assessing open-ended assignments (e.g., PeerStudio by Kulkarni et al., Juxtapeer by Cambre et al., Automatic essay grading by Balfour). While this is a critical concern to address, there is a need to better understand how to effectively teach OE courses at scale or online.
Mixed-Methods Study Design
To better understand the effective teaching practices for open-ended, university-level courses at scale or online, we first contrasted the teaching and student learning experiences of open-ended and non-open-ended university courses:
-
RQ1: Based on how instructors teach open-ended courses differently from non-open-ended courses, what can we infer about the critical pedagogical elements unique to teaching open-ended courses effectively in higher education, from the instructors' perspective?
-
RQ2: From the students' perspective, what pedagogical elements do students prefer to have in open-ended courses, i.e., what are students’ preferences for how instructors should teach open-ended courses?
We approached these two RQs with two surveys administered in our institution, a large, public research-oriented (R1) university: (1) an instructor survey (121 faculty) to compare how instructors operationalized OE and NOE courses in our university, and (2) a student survey (55 students) about students’ positive and negative experiences with OE courses. Primarily grounded in statistical analysis, we summarized five key pedagogical elements for the effective teaching of OE courses, incorporating both the instructors’ and students’ perspectives. We hope this result can help instructors and institutions better allocate resources when offering OE courses at scale.
Besides identifying the key pedagogical elements, we took one step further to understand what challenges and opportunities instructors may encounter when incorporating these elements into online or large-scale OE courses in a university setting. Thus, we ask our third research question:
- RQ3: What are the challenges and opportunities in teaching university-level, open-ended courses effectively at scale or online in a university setting?
To answer RQ3, we conducted in-depth interviews with 11 university instructors from 9 departments about their perspective on and experience with large or online OE courses, ranging from a 30-student studio class to a 300-student MBA class. We extracted themes of challenges and opportunities in the interviews based on a grounded-theory-like approach. Then we organized them into categories using the five pedagogical elements identified in RQ1 and RQ2. By comparing instructors’ needs against the state-of-the-art online learning-at-scale environment, we suggested six future research directions that may guide the development of infrastructure that supports large or online OE courses.
Results Highlights
Overall, we found that supporting students’ self-expression is critical to OE courses. Figure 1 summarizes the six future directions needed to address the challenges and opportunities to incorporate the five key pedagogical elements when teaching OE courses at scale or online in a university setting. We now share some of the most interesting findings.
Highlight I: In-depth Conversations
Unlike the Q&A-type interactions in NOE courses, in-depth conversations between instructors and students, and among peers played a significantly more essential role in OE courses than in NOE courses based on our survey results. Our interviews identified a significant challenge of having sufficient interaction in a large class – the scalability of conversations with limited instructors’ time and in a non-discussion-friendly or non-studio-friendly environment.
“It’s all about conversation. . . . So, then the core question about scalability is about conversation.” (P8)
Besides the challenge of scaling in-depth conversations, instructors also had concerns about the lack of real engagement when students converse on an online forum:
“So it’s like here, marking it off the list. Oh, you want me to do this? Okay, we’ll do this.” (P7)
Exiting in-person and online discussion channels in educational settings do not support conversations-at-scale well. Future research should find ways to better incentivize in-depth conversations and increase instructors’ involvement in conversations without overwhelming the instructors in a large scale open-ended course.
Highlight II: Studio-friendly Environment
The studio is an indispensable component of many OE courses, especially in Art & Design. Our survey results showed that 70% of the studio sections involved in-the-moment one-on-one interaction with instructors and over-the-shoulder peer interaction.
“… it’s those little things that you miss. The ability to look at the person next to you and say I like what you’re doing, how do you do that?” (P9)
Instructors suggested that both types of interactions were missing in their asynchronous online studios. Furthermore, current MOOC platforms support neither of them. Developing solutions for easy context-sharing is a critical next step to enable in-the-moment or over-the-shoulder interaction. In addition, future work can create asynchronous approaches to compensate for the lack of these interactions.
Highlight III: A Welcoming, Unbiased Environment
Based on our interviews and surveys, learning tasks and assignments in OE courses incorporated more personal opinions, experience, and emotion than NOE courses due to their value-laden nature. As a result, instructors of OE courses highly valued the establishment of trust in the class, and students in OE courses preferred an open, unbiased environment for comfortable self-expression. Instructors shared that diminished trust both in an online setting and in a large-scale setting may hinder students’ willingness to share their work and opinions.
“The small class is intensely personal. They are hanging work that came from their soul, twice a week. And that’s really threatening … my students and I trust each other … Seldom will that happen in the big class.” (P6)
Current MOOC platforms have few tools or measures to address issues of trust and inclusion in OE courses. Much more research is needed to investigate how to moderate a learning environment and set up norms to encourage students to self-express in front of a broad audience that potentially has drastically different backgrounds. We suggest developing proactive measures that guide healthy debates and critiques and create more opportunities to establish trust and a sense of community.
Other Findings
In the interest of time, we briefly summarize the remaining major challenges, opportunities, and the corresponding future directions for offering OE courses at scale or online in a university setting. For more details, please refer to the paper.
- Personalized Instruction and Benefits of Diversity: The majority of instructors of OE courses rated having knowledge of students’ backgrounds and personalizing instruction as necessary instead of nice-to-have. On the one hand, a large class brings challenges to offering personalized instruction, interaction, and feedback to students due to limited attention and loss of personal connection. On the other hand, we may realize the value of diversity within a large student body in OE courses via learner-sourcing.
- Flexible Assignments and Assessments: In a large class, it is difficult to scale an open-ended assignment due to an increased grading workload. In an online course, incorporating artifact creation that requires specific physical equipment or spaces is challenging.
- Extensive Open-ended Feedback: When teaching a large class, instructors faced the challenge of providing individual feedback to students as an organic part of the development process of a deliverable, since instructors were hesitant to rely on peer feedback as the only feedback source.
Implications
While we recognize the benefits of enhancing the accessibility to higher education, we urge universities to carefully consider the challenges and concerns raised by instructors and students in our study and guard against over-scaling OE courses. We hope our work can guide instructors’ teaching practices, support student learning, and inform institutions’ resource allocation and infrastructure development processes to achieve positive learning outcomes in large-scale, online OE courses in a university setting.
Resources
@article{li2021s,
title={" It's all about conversation" Challenges and Concerns of Faculty and Students in the Arts, Humanities, and the Social Sciences about Education at Scale},
author={Li, Tiffany Wenting and Karahalios, Karrie and Sundaram, Hari},
journal={Proceedings of the ACM on Human-Computer Interaction},
volume={4},
number={CSCW3},
pages={1--37},
year={2021},
publisher={ACM New York, NY, USA}
}