Beth Benker has been at UWE for almost ten years, primarily studying the sociology of food. During this time teaching on the CHSS Social Science Foundation Year alongside teaching on undergraduate programmes, Beth has become increasingly interested in pedagogies that avoid disadvantaging SpLD students.
Sociology lecturer Beth Benker uses Copilot to encourage critical reading, thinking and reasoning capacities in CHSS Social Science Foundation Year students while avoiding disadvantaging students with SpLD (Specific Learning Disabilities/Difficulties/Differences).
The problem
The assignment in this case study is the result of a great deal of personal research in facilitating criticality and reasoning capacities without requiring significantly more time, energy, and emotional energy from students with SpLD. Often criticality and critical reasoning is taught alongside extensive close reading which requires significant attention to detail, capacity to read unfamiliar documents discussing abstract ideas, reliance on working memory and long stretches of concentration – things that many students will struggle with. There is a good amount of pedagogical/psychological evidence suggesting that familiarity and practice often enable greater engagement with all of these skills, but it can be overwhelming to begin. Therefore, this assignment enables students to engage critically with a Copilot-drafted assignment and improve it using peer-reviewed sources and critical reasoning. It also enables students to understand the significant flaws present in many LLMs (Large Language Models) as a whole and discourages inappropriate use (read: plagiarism) for future assignments.
The solution
Copilot and Microsoft Word are the digital learning tools used in this assignment. These were chosen because the assignment was designed to be as accessible as possible. Students have access to both Copilot and Word with their student registration, so the vast majority of students with their own computers will be able to access these pieces of software easily. Students were given several essay titles to use as prompts and they were asked to use Copilot to generate an essay answer. They were then invited to prompt Copilot to make certain changes, such as adding references or changing the length. From here, they then copied their final output into Microsoft Word and this was then annotated using the comments function with various improvements, including logical errors, inadequate, inappropriate and/or absent sources, and errors in structure and presentation. The assignment is informed by an inquiry-based learning pedagogy which is grounded in the constructivist philosophy, and values the importance of student’s own investigations. This pedagogy also encourages skills required in both undergraduate and postgraduate study, and feeds into the graduate skills that are so important for our students’ futures, such as IT fluency, self-motivation and critical reasoning capacities.
Any challenges
The biggest challenge in this assignment was and continues to be user resistance. This came as a surprise to me during the first year of the assignment, given my own understanding of students as generally very comfortable with technology. Students were apprehensive about using LLMs given their mixed media coverage, and students were also apprehensive about using Microsoft Word to make comments. With regards to the former, the discomfort is largely due to the unfamiliarity of Copilot, but also LLMs in general. Additionally, there are a small number of students who are concerned about the more ethical considerations of LLMs, including data privacy concerns and the disastrous environmental impact. To address both of these, the teaching sessions around the assignment were adapted to include time devoted to both showing students what the process looks like and for students to have a go themselves so that the assignment was not the first time they needed to use either learning tool.
Feedback from staff and students
In verbal feedback, students explicitly mentioned that it was an enjoyable assignment and that they liked that they could still ‘pick things apart’ without needing to write a full essay. The development and introduction of this assignment has been met with college-wide support from a wide variety colleagues both for the pedagogical orientation and the appropriate engagement with Copilot (and other LLMs) that it promotes.
Next steps
This assignment will be used for a piece of empirical research with my colleague Dan Gray on the use of AI in higher education contexts. There are accessibility benefits to these technologies which are under-utilised in HE contexts.
Top 5 tips
- Resist showing students too much: allowing them to explore both LLMs and the literature in their own time enables them to engage much more easily later on.
- Encourage students to talk to each other: as students are getting used to both pieces of software their learning community can reinforce their learning better than the teaching staff in the room.
- Ensure that prompt questions are not too advanced: questions that are too hard will lose students immediately.
- Do your research on UWE’s AI guidance: given how new LLMs are, guidance can change to reflect new developments quit quickly.
- Do your research on LLMs just before the assignment, too: they are often changing, and so their capacities are also changing – for example, ChatGPT had a major update in 2023, so now it can use new information in its answers. This could catch you out in the teaching room!