Taccle AI

Improving skills and competences of VET teachers and trainers in the age of Artificial Intelligence

technologies

News 

Do we need specialised AI tools for education and instructional design?

Graham Attwell 172 Views , , , , , , , , , ,
Photo by Amélie Mourichon on Unsplash

In last weeks edition of her newsletter, Philippa Hardman reported on an interesting research project she has undertaken to explore the effectiveness of Large Language Models (LLMs) like ChatGPT, Claude, and Gemini in instructional design. It seems instructional designers are increasingly using LLMs to complete learning design tasks like writing objectives, selecting instructional strategies and creating lesson plans.

The question Hardman set out to explore was: “how well do these generic, all-purpose LLMs handle the nuanced and complex tasks of instructional design? They may be fast, but are AI tools like Claude, ChatGPT, and Gemini actually any good at learning design?” To find this out she set two research question. The first was sound the Theoretical Knowledge of Instructional Design by LLMs and the second to assess their practical application.She then analysed each model’s responses to assess theoretical accuracy, practical feasibility, and alignment between theory and practice.

In her newsletter Hardman gives a detailed account of the outcomes of testing the different models from each of the three LLM providers, But the The headline is that across all generic LLMs, AI is limited in both its theoretical understanding and its practical application of instructional design. The reasons she says is that they lack industry specific knowledge and nuance, they uncritically use outdated concepts and they display a superficial application of theory.

Hardman concludes that “While general-purpose AI models like Claude, ChatGPT, and Gemini offer a degree of assistance for instructional design, their limitations underscore the risks of relying on generic tools in a specialised field like instructional design.”

She goes on to point out that in industries like coding and medicine, similar risks have led to the emergence of fine-tuned AI copilots, such Cursor for coders and Hippocratic AI for medics and sees a need for “similar specialised AI tools tailored to the nuances of instructional design principles, practices and processes.”

News 

What are Learning Tools?

Graham Attwell 237 Views , , , , , , , ,
Yutong Liu & Kingston School of Art / Better Images of AI / Talking to AI 2.0 / CC-BY 4.0

There's an interesting post from Philippa Hardman in her newsletter today. Entitled Are ChatGPT, Claude & NotebookLM *Really* Disrupting Education?  her research asks how much and how well do popular AI tools really support human learning and, in the process, disrupt education?
She created a simple evaluation rubric to explore five key research questions: 

1. Inclusion of Information

2. Exclusion of Information

3. [De]Emphasis of Information

4. Structure & Flow

5. Tone & Style

Philippa Hardman used her own research articles as the input material, which she fed into what she says are considered to be the three big AI tools for learning: 

  1. ChatGPT 4o
  2. Claude 3.5
  3. NotebookLM

She prompted each tool in turn to read the article carefully and summarise it, ensuring that it covered all key concepts, ideas etc ensuring that I get a thorough understanding of the article and research.

She provides a detailed table of the results of each of the three applications, and additionally of the NotebookLM podcast application, assessing the strengths and weaknesses of each. she says that "while generative AI tools undoubtedly enhance access to information, they also actively “intervene” in the information-sharing process, actively shaping the type and depth of information that we receive, as well as (thanks to changed in format and tone) its meaning. "

She goes on to say:

While popular AI tools are helpful for summarising and simplifying information, when we start to dig into the detail of AI’s outputs we’re reminded that these tools are not objective; they actively “intervene” and shape the information that we consume in ways which could be argued to have a problematic impact on “learning”.

Another thing is also clear: tools like ChatGPT4o, Claude & Notebook are not yet comprehensive “learning tools” or “education apps”. To truly support human learning and deliver effective education, AI tools need to do more than provide access to information—they need to support learners intentionally through carefully selected and sequenced pedagogical stages.  

Her closing thoughts are about Redefining the “Learning” Process . She says:

It’s clear that AI tools like ChatGPT, Claude, and NotebookLM are incredibly valuable for making complex ideas more accessible; they excel in summarisation and simplification, which opens up access to knowledge and helps learners take the first step in their learning journey. However, these tools are not learning tools in the full sense of the term—at least not yet.

By labelling tools like ChatGPT 4o, Claude 3.5 & NotebookLM as “learning tools” we perpetuate the common misconception that “learning” is a process of disseminating and absorbing information. In reality, the process of learning is a deeply complex cognitive, social, emotional and psychological one, which exists over time and space and which must be designed and delivered with intention.

News 

TeacherMatic

Graham Attwell 302 Views , , , , , , ,

The AI pioneers project which is researching an developing approaches to the use of AI in vocational and adult education in Europe is presently working on a Toolkit including analysis of a considerable number of AI tools for education. Indeed a problem is that so many new tools and applications are being released it is hard for organisations to know what they should be trying out.

In the UK, JISC has been piloting and evaluating a number of different applications and tools in vocational colleges. Their latest report is about TeacherMatic which appears to be adapted in many UK Further Education Colleges. TeacherMatic is a generative AI-powered platform tailored for educators. It provides an extensive toolkit featuring more than 50 innovative tools designed to simplify the creation of educational content. These tools help in generating various teaching aids, such as lesson plans, quizzes, schemes of work and multiple-choice questions, without users needing to have expertise in prompt engineering. Instead, educators can issue straightforward instructions to produce or adapt existing resources, including presentations, Word documents, and PDFs. The main goal of TeacherMatic, the developers say, is to enhance teaching efficiency and lighten educators’ workloads. To allow teachers to dedicate more time to student interaction and less to repetitive tasks.

For the pilot, each participating institution received 50 licenses for 12 months, enabling around 400 participants to actively engage with and evaluate the TeacherMatic platform.

The summary of the evaluation of the pilot is as follows.

The pilot indicates that TeacherMatic can save users time and create good quality resources. Participants commended the platform for its ease of use, efficient content generation, and benefits to workload. Feedback also highlighted areas for improvement and new feature suggestions which the TeacherMatic team were very quick to take on board and where possible implement.

Participants found TeacherMatic to be user-friendly, particularly praising its easy-to-use interface and simple content generation process. The platform was noted for its instructional icons, videos, and features such as Bloom’s taxonomy, which assists in creating educational content efficiently. However, suggestions for enhancements include the ability to integrate multiple generators into a single generator. It also remains essential for users to evaluate the generated content, ensuring it is suitable and accessible to the intended audience.

TeacherMatic was well-received across institutions, for its capabilities, especially beneficial for new teaching staff and those adapting to changing course specifications. Feedback showed that TeacherMatic is particularly valuable for those previously unfamiliar with generative AI. Pricing was generally seen as reasonable, aligning with most participants’ expectations.

TeacherMatic has been well-received, with a majority of participants recognising its benefits and expressing a willingness to continue using and recommending the tool.