AI in ED: Equity, XAI and learner agency

Alexa Steinbrück / Better Images of AI / Explainable AI / CC-BY 4.0

The AI in education theme continues to gather momentum, resulting in a non stop stream of journal; articles, reports, newsletters. blogs and videos. However, while not diminishing, there seem to be some subtle change in directions in the messages.

Firstly, despite many schools wary of Generative AI, there is a growing realisation that students are going to use it anyway and that the various apps claiming to check student work for AI simply don't work.

At the same time, there is an increasing focus on AI and pedagogy (perhaps linked to the increasing sophistication of Frontier Models from Gen AI but also the realisation that gimmicks like talking to an AI pretending to be someone famous from the past are just lame!). This increased focus on pedagogy is also leading to pressure to involve students. in the application of Gen AI for teaching and learning. And at recent students two ethical questions have emerged. The first is unequal access to AI applications and tools. Inside Higher Ed reports that recent research from the Public Policy Institute of California on disparate access to digital devices and the internet for K-12 students in the nation’s largest state public-school system. Put simply, they say. students who are already at an educational and digital disadvantage because of family income and first-generation constraints are becoming even more so every day as their peers embrace AI at high rates as a productivity tool—and they do not.

And while some tools will remain free, it appears that the most powerful and modern tools will increasingly come at a cost. The U.K. Jisc recently reported that access to a full suite of the most popular generative AI tools and education plug-ins currently available could cost about £1,000 (about $1,275) per year. For many students already accumulating student debt and managing the rising cost of living, paying more than $100 per month for competitive AI tools is simply not viable.

A second issue is motivation and agency for students in using AI tools. It may be that the rush to gamification, inspired by Apps like DuoLingo, is running thin. Perhaps a more subtle and sustained approach is needed to motivate learners. That may increase a focus on learner agency which in turn is being seen as linked to Explainable AI (or XAI for short). Research around Learning Analytics has pointed to the importance of students understanding the use purpose of LA but also being able to understand why the Analytics turns out as it does. And research into Personal Learning Environments has long shown the importance of learner agency in developing meta-cognition in learning. With the development of many applications for personalized learning programmes, it becomes important that learners are able to understand the reasons for their own individual learning pathways and if necessary challenge them.

While earlier debates about AI in Ed ethics, largely focused on technologies, the new debates are more focused on practices in teaching and learning using AI.

Homework Apocolypse?

Catherine Breslin & Tania Duarte / Better Images of AI / AI silicon clouds collage / CC-BY 4.0

November marks two years since the release of Open AI's GPT large language model chatbot. Since then AI, or more specifically Generative AI has dominated the discourse over the future of education. And of course it has spawned hundreds of project resulting in an increasing torrent of research results. Yet on one critical issue - does the use of AI improve learning - there appears little consensus. This is probably because we have no good ways of measuring learning. Instead we use performance in tests and exams as a proxy for learning. And its probably true to say that the debates over AI are turning the heat on the use of such a proxy, just as it is on the essay as the dominant form of assessment in schools and universities.

Last week in his newsletter, One Useful thing, Ethan Mollick talked about the use of AI, cheating and learning in an article entitled 'What comes after the Homework Apocalypse'. It is probably fair to say Ethan is a big fan of AI in education.

To be clear, AI is not the root cause of cheating. Cheating happens because schoolwork is hard and high stakes. And schoolwork is hard and high stakes because learning is not always fun and forms of extrinsic motivation, like grades, are often required to get people to learn. People are exquisitely good at figuring out ways to avoid things they don’t like to do, and, as a major new analysis shows, most people don’t like mental effort. So, they delegate some of that effort to the AI. In general, I am in favor of delegating tasks to AI (the subject of my new class on MasterClass), but education is different - the effort is the point.

He postulated that fall in grades achieved by students in the USA between 2008 and 2017 had been caused by the increasing use of the Internet for homework. Students were simply copying homework answers. And in an experiment in ma high school in Turkey with students using GPT4 grades for homework went up but final exam grades fell. But giving students GPT with a basic tutor prompt for ChatGPT, instead of having them use ChatGPT on their own, boosted homework scores without lowering final exam grades. 

Ethan says this shows "we need to center teachers in the process of using AI, rather than just leaving AI to students (or to those who dream of replacing teachers entirely). We know that almost three-quarters of teachers are already using AI for work, but we have just started to learn the most effective ways for teachers to use AI."

He remains convinced to the value of Generative AI in education. The question now, he says "is not whether AI will change education, but how we will shape that change to create a more effective, equitable, and engaging learning environment for all."

AI Competency Framework for teachers

At last week's Digital Learning Week 2024, UNESCO formally launched two AI Competence Frameworks, one for teachers and the other for students. These frameworks aim to guide countries in supporting students and teachers to understand the potential as well as risks of AI in order to engage with it in a safe, ethical and responsible manner in education and beyond.

Above is a copy of Tim Evans popular poster  summarizing the AI Competency Framework for Teachers. He says "I've taken the extensive, lengthy report and attempted to gather my take on the 10 key points, and areas of focus." Tim has also made a copy of the poster available on Canva.

AI: What do teachers want?

Yutong Liu & Kingston School of Art / Better Images of AI / Talking to AI / CC-BY 4.0

A quick post in follow up to my article yesterday on the proposals by the UK Department for Education to commission tech companies to develop an AI app for teachers to save them time. The Algorithm - a newsletter from MIT Technology Review picked up on this today, saying "this year, more and more educational technology companies are pitching schools on a different use of AI. Rather than scrambling to tamp down the use of it in the classroom, these companies are coaching teachers how to use AI tools to cut down on time they spend on tasks like grading, providing feedback to students, or planning lessons. They’re positioning AI as a teacher’s ultimate time saver."

The article goes on to ask how willing teachers are to turn over some of their responsibilities to an AI model? The answer, they say, really depends on the task, according to Leon Furze, an educator and PhD candidate at Deakin University who studies the impact of generative AI on writing instruction and education.

“We know from plenty of research that teacher workload actually comes from data collection and analysis, reporting, and communications,” he says. “Those are all areas where AI can help.”

Then there are a host of not-so-menial tasks that teachers are more skeptical AI can excel at. They often come down to two core teaching responsibilities: lesson planning and grading. A host of companies offer large language models that they say can generate lesson plans that conform to different curriculum standards. Some teachers, including in some California districts, have also used AI models to grade and provide feedback for essays. For these applications of AI, Furze says, many of the teachers he works with are less confident in its reliability. 

Companies promising time savings for planning and grading “is a huge red flag, because those are core parts of the profession,” he says. “Lesson planning is—or should be—thoughtful, creative, even fun.” Automated feedback for creative skills like writing is controversial too. “Students want feedback from humans, and assessment is a way for teachers to get to know students. Some feedback can be automated, but not all.” 

TeacherMatic

The AI pioneers project which is researching an developing approaches to the use of AI in vocational and adult education in Europe is presently working on a Toolkit including analysis of a considerable number of AI tools for education. Indeed a problem is that so many new tools and applications are being released it is hard for organisations to know what they should be trying out.

In the UK, JISC has been piloting and evaluating a number of different applications and tools in vocational colleges. Their latest report is about TeacherMatic which appears to be adapted in many UK Further Education Colleges. TeacherMatic is a generative AI-powered platform tailored for educators. It provides an extensive toolkit featuring more than 50 innovative tools designed to simplify the creation of educational content. These tools help in generating various teaching aids, such as lesson plans, quizzes, schemes of work and multiple-choice questions, without users needing to have expertise in prompt engineering. Instead, educators can issue straightforward instructions to produce or adapt existing resources, including presentations, Word documents, and PDFs. The main goal of TeacherMatic, the developers say, is to enhance teaching efficiency and lighten educators’ workloads. To allow teachers to dedicate more time to student interaction and less to repetitive tasks.

For the pilot, each participating institution received 50 licenses for 12 months, enabling around 400 participants to actively engage with and evaluate the TeacherMatic platform.

The summary of the evaluation of the pilot is as follows.

The pilot indicates that TeacherMatic can save users time and create good quality resources. Participants commended the platform for its ease of use, efficient content generation, and benefits to workload. Feedback also highlighted areas for improvement and new feature suggestions which the TeacherMatic team were very quick to take on board and where possible implement.

Participants found TeacherMatic to be user-friendly, particularly praising its easy-to-use interface and simple content generation process. The platform was noted for its instructional icons, videos, and features such as Bloom’s taxonomy, which assists in creating educational content efficiently. However, suggestions for enhancements include the ability to integrate multiple generators into a single generator. It also remains essential for users to evaluate the generated content, ensuring it is suitable and accessible to the intended audience.

TeacherMatic was well-received across institutions, for its capabilities, especially beneficial for new teaching staff and those adapting to changing course specifications. Feedback showed that TeacherMatic is particularly valuable for those previously unfamiliar with generative AI. Pricing was generally seen as reasonable, aligning with most participants’ expectations.

TeacherMatic has been well-received, with a majority of participants recognising its benefits and expressing a willingness to continue using and recommending the tool.