Who promotes education technology as a precondition for education transformation?

UNESCO has published the 2023 edition of the The Global Education Monitoring Report (GEM) entitled "technology in education: a tool on whose terms?" The report is an independent annual publication. The funded by a group of governments,multilateral agencies and private foundations and facilitated and supported by UNESCO. It comes in at a weighty 418 pages with much of interest (I have yo admit I have only read the 32 page summary.

Unusually for a report of this type, it received considerable media attention, at least in the UK. But this was focused on the section of the use of smartphones in school which concluded that "Mere proximity to a mobile device was found to distract students and to have a negative impact on learning in 14 countries, yet less than one in four have banned smartphone use in schools."

The report says that "good, impartial evidence on the impact of education technology is in short supply" adding that here is little robust evidence on digital technology’s added value in education. Technology evolves faster than it is possible to evaluate it: Education technology products change every 36 months, on average. Most evidence comes from the richest countries. In the United Kingdom, 7% of education technology companies had conducted randomized controlled trials, and 12% had used third-party certification. A survey of teachers and administrators in 17 US states showed that only 11% requested peer-reviewed evidence prior to adoption. A lot of the evidence comes from those trying to sell it. Pearson funded its own studies, contesting independent analysis that showed its products had no impact.

The 2018 PISA found that 65% of 15-year-old students in OECD countries were in schools whose
principals agreed that teachers had the technical and pedagogical skills to integrate digital devices in instruction and 54% in schools where an effective online learning support platform was available; these shares are believed to have increased during the COVID-19 pandemic. Despite this the report found that teachers often feel unprepared and lack confidence teaching with technology. Only half of countries have standards for developing teacher ICT skills. While 5% of ransomware attacks target education, few teacher training programmes cover cybersecurity.

The report found that online content has grown without enough regulation of quality control or diversity. Online content is produced by dominant groups, affecting access to it. Nearly 90% of content in higher education repositories with open education resource collections was created in Europe and Northern America; 92% of content in the OER Commons global library is in English. Massive open online courses (MOOCs) mainly benefit educated learners and those from richer countries.

Higher education is adopting digital technology the fastest and being transformed by it the most. There were over 220 million students attending MOOCs in 2021. But digital platforms challenge universities’ role and pose regulatory and ethical challenges, for instance related to exclusive subscription deals and to student and personnel data.

While such technology has tremendous potential, many tools have not been designed for application to education. Not enough attention has been given to how they are applied in education and even less to how they should be applied in different education contexts.

To understand the discourse around education technology, it is necessary to look behind the language being used to promote it, and the interests it serves. Who frames the problems technology should address? What are the consequences of such framing for education? Who promotes education technology as a precondition for education transformation? How credible are such claims? What criteria and standards need to be set to evaluate digital technology's current and potential future contribution to education so as to separate hype from substance? Can evaluation go beyond short-term assessments of impact on learning and capture potential far-reaching consequences of the generalized use of digital technology in education?

Exaggerated claims about technology go hand in hand with exaggerated estimates of its global market size. In 2022, business intelligence providers’ estimates ranged from USD 123 billion to USD 300 billion. These accounts are almost always projected forward, predicting optimistic expansion, yet they fail to give historic trends and verify whether past projections proved true. Such reporting routinely characterizes education technology as essential and technology companies as enablers and disruptors.
If optimistic projections are not fulfilled, responsibility is implicitly placed on governments as a way of maintaining indirect pressure on them to increase procurement. Education is criticized as being slow to change, stuck in the past and a laggard when it comes to innovation. Such coverage plays on users’ fascination with novelty but also their fear of being left behind.

AI and Assessment

This podcast on AI and assessment was recorded for the Eramus+ eAssessment project. It forms part of an online course including four main sections:

  • Assessment Strategies
  • Analysing evidence
  • Feedback and Planning
  • Community of practice.

The course, hosted on the project Moodle site, is free of charge. Participants are welcome to follow the full course or can dip in and out of the sixteen different units.

The podcast discusses the impact of Generative AI and advanced chatbots like ChatGPT and Bing on education and how they are changing the way we assess learning.

Below is a transcript of the podcast.

Scene 1

In late November 2022, OpenAI dropped ChatGPT, a chatbot that can generate well-structured blocks of text several thousand words long on almost any topic it is asked about. Within days, the chatbot was widely denounced as a free essay-writing, test-taking tool that made it laughably easy to cheat on assignments. The response from schools and universities was swift and decisive. Los Angeles Unified, the second-largest school district in the US, immediately blocked access to OpenAI’s website from its schools’ network. Others soon joined. By January, school districts across the English-speaking world had started banning the software, from Washington, New York, Alabama, and Virginia in the United States to Queensland and New South Wales in Australia. Several leading universities in the UK, including Imperial College London and the University of Cambridge, issued statements that warned students against using ChatGPT to cheat.

Scene 2

This initial panic from the education sector was understandable. ChatGPT, available to the public via a web app, can answer questions and generate unique essays that are difficult to detect as machine-written. It looked as if ChatGPT would undermine the way we test what students have learned, a cornerstone of education.

However, there is a growing recognition that advanced chatbots could be used as powerful classroom aids that make lessons more interactive, teach students media literacy, generate personalized lesson plans, save teachers time on admin, and more. Educational-tech companies including Duolingo and Quizlet, which makes digital flashcards and practice assessments used by half of all high school students in the US, have already integrated OpenAI’s chatbot into their apps. And OpenAI has worked with educators to put together a fact sheet about ChatGPT’s potential impact in schools. The company says it also consulted educators when it developed a free tool to spot text written by a chatbot (though its accuracy is limited). Turnitin – a company leading in the provision of anti plagiarism software – already controversial – designed a new plug in supposed to alert to the possibility that assessments were using AI. Its accuracy is also questioned and Turnitin had to backtrack on not allowing users to opt out of new AI plugin.

Scene 3

“We need to be asking what we need to do to prepare young people—learners—for a future world that’s not that far in the future,” says Richard Culatta, CEO of the International Society for Technology in Education (ISTE), a nonprofit that advocates for the use of technology in teaching.

Take cheating. In Crompton’s view, if ChatGPT makes it easy to cheat on an assignment, teachers should throw out the assignment rather than ban the chatbot.

Scene 4

In response to the limitations of traditional assessment methods, there is a growing movement towards authentic and personalized assessment methods in courses. These methods include using real-life examples and contextually specific situations that are meaningful to individual students. Instructors may ask students to include their personal experience or perspectives in their writing. Students can also be asked to conduct analysis that draws on specific class discussions. Alternatives to essay-based assessment also need to be further explored. These methods can include using (impromptu) video presentations for assessments or using other digital forms such as animations. Through self-assessment or reflective writing, students could discuss their writing or thinking process. Additionally, peer evaluations or interactive assessment activities could be integrated into grading by engaging students in group discussions or other activities such as research and analysis in which students are expected to co-construct knowledge and apply certain skills. Instructors may consider placing an emphasis on assessing the process of learning rather than the outcome.

Scene 5

Most of the commentary and research into the impact of Generative AI so far comes from school and higher education. What bout vocational education and training. Well, firstly I would argue that VET has been better at formative assessment than in general education. And in many countries VET is based on students being able to demonstrate outcomes, usually through practical tests – this is much closer to authentic assessment. Of course, in many systems VET students are also required. To produce evidence of knowledge as well as practice and this may need some changes.  But for vocational education and training, the challenge may not be so much how we are assessing but what we are assessing. Leaving aside the discussion about whether and AI is going to threaten jobs and affect employment – and to be honest no one is really sure to what extent this will happen and how many new jobs may be generated. But it is pretty clear that    AI is going to impact on the content and tasks of many jobs…….. Vocational Education and Training has a key purpose in preparing students for the world of work. If that work world include people using AI in their occupation then that will form a key part of the curriculum. And authentic assessment means assessinghow they are working with AI in the work tasks of the future.

Scene 6

Lets sum up. We need to change how we assess learning. Did ChatGPT kill assessments? They were probably already dead, and they’ve been in zombie mode for a long time.