Can Pedagogical Innovations Be Evaluated? André Tricot, Professor of Cognitive Psychology.

The speaker, affiliated with Paul Valéry University and based in Paris, directs the National Center for Educational Studies (Cnesco). This center is responsible for organizing consensus conferences and international comparison studies within the field of education. A significant aspect of Cnesco's work is ensuring that all resulting productions, including findings on pedagogical innovations, are freely accessible to the public.

Educational myths are defined as assertions lacking empirical substantiation, often lacking a theoretical basis as well. These statements are frequently repeated, such as the concept of multiple intelligences. A notable example is the 'Digital Natives' idea, which generated over ten thousand citations despite having no supporting data or theory initially. It took several years for counter-evidence to emerge and challenge this widespread belief.

It is marvelous that one has the right to say anything, and it is up to others to do the work to show that it is nonsense.

The process involves confronting a general pedagogical idea with the current state of scientific knowledge and available data. This critical approach was documented in a book published in 2014 concerning digital learning, acknowledging that previous foundational work in this specific area originated with Dutch colleagues a year prior.

Evaluating pedagogical innovation fundamentally relies on experimental methodology. This typically involves dividing students randomly into two groups. Both groups are assessed before and after instruction using identical content, such as studying second-degree equations or Thales' theorem, ensuring the core material remains consistent across both groups.

The variable introduced is the teaching method or support material, where one is considered innovative and the other represents the usual approach. It is crucial to ensure instruction occurs over the same duration, as time allocation is a significant bias factor. After measuring the resulting learning gain through various calculation methods, results must be replicated and compiled into meta-analyses if promising.

The researcher's responsibility within this domain centers on evaluating broad, general ideas or established dogmas within pedagogy. While individual teacher actions in the classroom are generally not the focus, concern arises when institutions or figures of authority mandate a specific method as the sole correct approach, such as promoting the investigation approach in science education.

What seems dangerous to me is when an institution, when people who have power, tell you this is the right way to teach.

A commonly held general idea suggests that manipulating materials helps students learn better, implying that students must be active to acquire knowledge. This notion is surprisingly old, having been debated for over twenty-five centuries. Historical texts, such as Ferdinand Buisson's dictionary entry on grammar teaching, reveal that this debate is not new, contrasting countries that emphasize practice over explicit rule instruction.

The literature surrounding 'learning by action' suffers from immense confusion regarding terminology. Sometimes 'action' refers to physical activity, like learning a gesture or manipulating an object. Conversely, it can refer to cognitive activity, involving reflection, hypothesis generation, and questioning. This semantic overlap complicates the literature, making coherent conclusions difficult to draw when lumping these distinct concepts together.

Furthermore, the literature frequently blurs the line between action as a means of learning and action as the learning objective itself. For example, a philosophy teacher might have students practice writing essays to work through philosophical concepts—where writing is the means. Alternatively, the goal might simply be mastering the skill of essay composition. Research results often diverge based on whether the study focuses on the means or the end goal.

This literature that goes in all directions and says anything arrives at the conclusion that to make something learned, one must and it is enough to do something.

A critical distinction exists between cognitive engagement—the depth of mental processing—and the performance of a task. Many conflate high levels of cognitive engagement with simply being busy performing a task. Researchers have categorized engagement into levels, where higher levels involve greater student input, such as questioning and adding information to presented material.

While Level 4 engagement typically results in better learning outcomes than lower levels, it demands significantly more preparation time from the instructor and greater cognitive cost for the students. Therefore, Level 4 and Level 3 engagement should be reserved for the most crucial learning objectives. Conversely, simpler tasks, like listening to a lecture or reading a text, can be approached with Level 1 or 2 engagement, which is less taxing.

A notable meta-analysis from 2003 examined the efficacy of problem-solving for mathematics instruction, differentiating between learning to perform tasks (procedural skills) and learning to understand concepts. The results strongly supported problem-solving's effectiveness for teaching students how to execute mathematical procedures.

However, when assessing problem-solving's impact on conceptual understanding in mathematics, the evidence becomes substantially less convincing, with effect sizes approaching zero. This finding is often overlooked in contemporary recommendations provided by educational ministries, suggesting a disconnect between empirical evidence and current policy implementation regarding mathematics instruction.

Reviewing the empirical and experimental literature suggests several consensual points regarding active learning approaches. The great advantage of modern research over historical philosophical speculation is the availability of concrete data upon which to base discussions. Despite this, pedagogy often struggles to acknowledge its own history and existing body of evidence before promoting new ideas.

Most pedagogical innovations encountered in the 20th century appear to be recycled concepts, with ideas often traceable to the 18th or 19th centuries, or even ancient thinkers like Socrates. The most recent concepts, such as cooperative learning, find their roots in the late 19th century. Frequently, these ideas are promoted by individuals lacking expertise in pedagogy, a situation not always observed in fields like medicine or biology.

A study involving 134 teachers compared those trained (80 hours) in the investigation approach against untrained peers over three years, tracking 3,000 students. The only significant effect observed was that trained teachers spent an additional ten minutes per quarter on science instruction. Crucially, this training resulted in improved factual knowledge but showed no improvement in student motivation, scientific reasoning, or mastery of the investigation method itself.