testing

Improving academic performance with a simple psychological intervention

Stereotype threat is a factor not only for some ethnic groups, but for women in certain areas (e.g., math, engineering), and also for older adults. Interventions that help reduce stereotype threat are also potentially helpful for those who suffer from test anxiety, or math anxiety.

Dave Nussbaum talks in the Scientific American about research he’s been involved in, showing how a small intervention aimed at reducing stereotype threat had significant long-lasting benefits for Latino American middle school students.

... In both schools, the intervention improved core course grades (Science, Social Studies, English, and Math) among Latino American students by the end of the school year by an average of roughly 0.3 points (on a 4.33 scale), reducing the achievement gap by 20-30% (the intervention had no effect on the White students’ grades). The intervention also sharply reduced the downward performance trajectory. Moreover, students in one school were followed for two years after the intervention had been completed and its effect on their grades persisted, even as some students made the difficult transition from middle school to high school. …

 Read the article

tags memworks: 

tags problems: 

tags study: 

Desirable difficulty for effective learning

When we are presented with new information, we try and connect it to information we already hold. This is automatic. Sometimes the information fits in easily; other times the fit is more difficult — perhaps because some of our old information is wrong, or perhaps because we lack some of the knowledge we need to fit them together.

References: 

D’Mello, S., Lehman B., Pekrun R., & Graesser A. (Submitted). Confusion can be beneficial for learning. Learning and Instruction.

tags memworks: 

tags strategies: 

tags study: 

The changing nature of literacy. Part 4: Models & Literacies

This post is the fourth and last part in a four-part series on how education delivery is changing, and the set of literacies required in today’s world. Part 1 looked at textbooks; Part 2 at direct instruction/lecturing; Part 3 at computer learning.. This post looks at learning models and types of literacy.

 

Literacy. What does it mean?

tags strategies: 

tags study: 

The most effective learning balances same and different context

I recently reported on a finding that memories are stronger when the pattern of brain activity is more closely matched on each repetition, a finding that might appear to challenge the long-standing belief that it’s better to learn in different contexts. Because these two theories are very important for effective learning and remembering, I want to talk more about this question of encoding variability, and how both theories can be true.

tags memworks: 

tags strategies: 

tags study: 

Successful remembering requires effective self-monitoring

We forget someone’s name, and our response might be: “Oh I’ve always been terrible at remembering names!” Or: “I’m getting old; I really can’t remember things anymore.” Or: nothing — we shrug it off without thought. What our response might be depends on our age and our personality, but that response has nothing to do with the reason we forgot.

We forget things for a number of short-term reasons: we’re tired; we’re distracted by other thoughts; we’re feeling emotional. But underneath all that, at all ages and in all situations, there is one fundamental reason why we fail to remember something: we didn’t encode it well enough at the time we learned/experienced it. And, yes, that is a strategy failure, and possibly also a reflection of those same factors (tired, distracted, emotional), but again, at bottom there is one fundamental reason: we didn’t realize what we needed to do to ensure we would remember it. This is a failure of self-monitoring, and self-monitoring is a crucial, and under-appreciated, strategy.

I’ve written about self-monitoring as a study skill, but self-monitoring is a far broader strategy than that. It applies to children and to seniors; it applies to remembering names and intentions and facts and experiences and skills. And it has a lot to do with cognitive fluency.

Cognitive fluency is as simple a concept as it sounds: it’s about how easy it is to think about something. We use this ease as a measure of familiarity — if it’s easy, we assume we’ve met it before. The easier it is, the more familiar we assume it is. Things that are familiar are (rule of thumb) assumed to be safe, seen as more attractive, make us feel more confident.

And are assumed to be known — that is, we don’t need to put any effort into encoding this information, because clearly we already know it.

Familiarity is a heuristic (rule of thumb) for several attributes. Fluency is a heuristic for familiarity.

Heuristics are vital — without these, we literally couldn’t function. The world is far too complex a place for us to deal with it without a whole heap of these rules of thumb. But the problem with them is that they are not rules, they are rules of thumb — guidelines, indicators. Meaning that a lot of the time, they’re wrong.

That’s why it’s not enough to unthinkingly rely on fluency as a guide to whether or not you need to make a deliberate effort to encode/learn something.

The secret to getting around the weaknesses of fluency is effective testing.

Notice I said effective.

If you intend to buy some bread on the way home from work, does the fact that you reminded yourself when you got to work constitute an effective test? Not in itself. If you are introduced to someone and you remember their name long enough to use it when you say goodbye, does this constitute an effective test? Again, not in itself. If you’re learning the periodic table and at the end of your study session are able to reel off all the elements in the right order, can you say you have learned this, and move on to something else? Not yet.

Effective testing has three elements: time, context, and feedback.

The feedback component should be self-evident, but apparently is not. It’s no good being tested or testing yourself, if your answer is wrong and you don’t know it! Of course, it’s not always possible to get feedback — and we don’t need feedback if we really are right. But how do we know if we’re right? Again, we use fluency to tell us. If the answer comes easily, we assume it’s correct. Most of the time it will be — but not always. So if you do have some means of checking your answer, you should take it.

[A brief aside to teachers and parents of school-aged students: Here in New Zealand we have a national qualifying exam (actually a series of exams) for our older secondary school students. The NCEA is quite innovative in many ways (you can read about it here if you’re curious), and since its introduction a few years ago there has been a great deal of controversy about it. As a parent of students who have gone through and are going through this process, I have had many criticisms about it myself. However, there are a number of good things about it, and one of these (which has nothing to do with the nature of the exams) is a process which I believe is extremely rare in the world (for a national exam): every exam paper is returned to the student. This is quite a logistical nightmare of course, when you consider each subject has several different papers (as an example, my younger son, sitting Level 2 this year, did 18 papers) and every paper has a different marker. But I believe the feedback really is worth it. Every test, whatever its ostensible purpose, should also be a learning experience. And to be a good learning experience, the student needs feedback.]

But time and context are the important, and under-appreciated, elements. A major reason why people fail to realize they haven’t properly encoded/learned something, is that they retrieve it easily soon after encoding, as in my examples above. But at this point, the information is still floating around in an accessible state. It hasn’t been consolidated; it hasn’t been properly filed in long-term memory. Retrieval this soon after encoding tells you (almost) nothing (obviously, if you did fail to retrieve it at this point, that would tell you something!).

So effective testing requires a certain amount of time to pass. And as I discussed when I talked about retrieval practice, it really requires quite a lot of time to pass before you can draw a line under it and say, ok, this is now done.

The third element is the least obvious. Context.

Why do we recognize the librarian when we see her at the library, but don’t recognize her at the supermarket? She’s out of context. Why does remembering we need to buy bread on the way home no good if we remember it when we arrive at work? Because successful intention remembering is all about remembering at the right time and in the right place.

Effective encoding means that we will be able to remember when we need the information. In some cases (like intention memory), that means tying the information to a particular context — so effective testing involves trying to retrieve the information in response to the right contextual cue.

In most cases, it means testing across a variety of contexts, to ensure you have multiple access points to the information.

Successful remembering requires effective monitoring at the time of encoding (when you encounter the information). Effective monitoring requires you not to be fooled by easy fluency, but to test yourself effectively, across time and context. These principles apply to all memory situations and across all ages.

 

Additional resources:

If you want to know more about cognitive fluency and its effect on the mind (rather than memory specifically), there's nice article in the Boston Globe. As an addendum (I'd read the more general and in-depth article in the Globe first), Miller-McCune have a brief article on one particular aspect of cognitive fluency -- the effect of names.

Miller-McCune have have a good article on the value of testing and the motivating benefits of failure.

tags memworks: 

tags strategies: 

tags study: 

Regulating your study time and effort

  • Knowing how well or how poorly you know something is critical to effectively allocating your study time and effort.
  • The more difficult the material being learned, the worse we tend to be at estimating how well we know it.
  • Various learning strategies improve our awareness of how well we know something.
  • Learner attributes are also important, particularly our attitudes to learning and beliefs about our abilities.

In general, the weight of the research evidence suggests that college students tend to have a poor sense of how prepared they are for testing, and having been tested, they have a poor sense of how well they did! (This, of course, is even more true of younger students).

Does it matter?

Well, yes, it does. Being able to accurately estimate how well you've learnt something (monitoring) allows you to better allocate your time and energy (self-regulation). You don't want to spend more time than you need on particular topics; you also don't want to short-change topics that need more work.

We tend to be better at regulating our time and effort when the material to be learned is simple.

Obviously, also, some people are much better than others at knowing how well they know something. What distinguishes those people who have a good metacognitive sense and those who don't?

Well, partly, it's about the strategies used in learning. Taking notes, for example, tends to make you more aware of what you know and what you don't know.But not only note-taking; any strategy that causes you to process the material more thoroughly should have this result.

Studies have found that your monitoring accuracy can be improved:

  • when you monitor your learning after a short delay, rather than immediately after studying the material [1]
  • when items are actively generated and not simply passively read [2]
  • by having practice tests of the material [3]
  • by summarizing the material [4]
  • by generating keywords -- but only if, again, you delay a little while before generating them [5]

In general, it seems that students tend to be better at predicting their ability to recall information than their understanding -- as evidenced by their ability to apply the information and make inferences about it. It is of course easier to test your memory than your understanding, and it may well be that students tend not to clearly distinguish between these two aspects of learning. However, certain strategies, such as taking notes (although it depends on the nature of the notes!), do lend themselves to helping develop understanding more than memory.

One final thing is worth noting. It's not only about strategies. Monitoring accuracy is also affected by learner attributes — which doesn't mean you can excuse yourself on the grounds you're "not smart enough"! Studies have found that IQ rarely is a significant factor once background knowledge and other factors (such as socioeconomic status) are accounted for [6]. What looks like being of importance is the student's chronic dispositional status toward learning -- that is, their general attitude to it. For example, those who believe intelligence is malleable and can be increased are more likely to work on increasing their skills, compared to those who believe intelligence is fixed, who tend to focus more on demonstrating good performance, often by choosing only those sort of tasks at which they can do well [7].

References: 

  • Peverly, S.T., Brobst, K.E., Grahan, M. & Shaw, R. 2003. College adults are not good at self-regulation: A study on the relationship of self-regulation, note taking, and test taking. Journal of Educational Psychology, 95 (2), 335-346.
  • Thiede, K.W., Anderson, M.C.M. & Therriault, D. 2003. Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66-73.
  1. Dunlosky, J. & Nelson, T.O. 1992. Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory & Cognition, 20, 374-380.
  2. Mazzoni, G. & Nelson, T.O. 1993. Metacognitive monitoring after different kinds of monitoring. Journal of Experimental Psychology: Learning, Memory and Cognition, 21, 1263-1274.
  3. King, J.F., Zechmeister, E.B. & Shaughnessy, J.J. 1980. Judgments of knowing: The influence of retrieval practice. American Journal of Psychology, 93, 329-343.
    Lovelace, E.A. 1984. Metamemory: Monitoring future recall ability during study. Journal of Experimental Psychology: Learning, Memory and Cognition, 10, 756-766.
    Shaughnessy, J.J. & Zechmeister, E.B. 1992. Memory monitoring accuracy as influenced by the distribution of retrieval practice. Bulletin of the Psychonomic Society, 30, 125-128.
    Ghatala, E.S., Levin, J.R., Foorman, B.R. & Pressley, M. 1989. Improving children's regulation of their reading PREP time. Contemporary Educational Psychology, 14, 49-66.
    Pressley, M., Snyder, B.L., Levin, J.R., Murray, H.G. & Ghatala, E.S. 1987. Perceived readiness for examination performance (PREP) produced by initial reading of text and text containing adjunct questions. Reading Research Quarterly, 22, 219-236.
  4. Thiede, K.W. & Anderson, M.C.M. 2003. Summarizing can improve metacomprehension accuracy. Contemporary Educational Psychology, 28,
  5. Thiede, K.W., Anderson, M.C.M. & Therriault, D. 2003. Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66-73.
  6. Bjorklund, D.F. & Schneider, W. 1996. The interaction of knowledge, aptitude, and strategies in children's memory performance. In H. Reese (ed.), Advances in child development (vol. 26, pp. 59-89). New York: Academic press.
    Ceci, S.J. 1996. On intelligence: A bioecological treatise on intellectual development. Cambridge, MA: Harvard University press.
  7. Dweck, C. 1999. Self-theories: Their role in motivation, personality, and development. Philadelphia: Psychology Press.

tags study: 

Asking better questions

Questions — especially why questions — help us make connections to existing anchor points — facts we know well. But some questions are better than others.

To decide whether a question is effective, ask:

  • does it make the information more meaningful?
  • does it make the information more comprehensible?
  • does it increase the number of meaningful connections?

Consider our facts about blood:

  • arteries are thick and elastic and carry blood that is rich in oxygen from the heart.
  • veins are thinner, less elastic, and carry blood rich in carbon dioxide back to the heart.

We could, as is often advised, simply turn these into why questions. And we can answer these on the basis of the connections we’ve already made:

Why are arteries elastic?

Because they need to accommodate changes in pressure

Why are arteries thick?

Because they need to accommodate high pressure

Why do arteries carry blood away from the heart?

Because blood coming from the heart comes out at high pressure and in spurts of variable pressure

Why do arteries carry blood that is rich in oxygen?

Because the blood coming from the heart is rich in oxygen

Why are veins less elastic?

Because the blood flows continuously and evenly

Why are veins less thick?

Because the blood flows at a lower pressure

Why do veins carry blood to the heart?

Because blood going to the heart flows continuously and evenly

Why do veins carry blood that is rich in CO2?

Because the blood going to the heart is rich in CO2

What’s missing? Connections between these facts. The facts have become more meaningful, but to be really understood you need to make the connections between the facts explicit.

Look again at our original questions. See how they relate the facts to each other? They don’t ask: why are arteries elastic? They ask: Why do arteries need to be more elastic than veins? They don’t ask: why do arteries carry blood that is rich in oxygen? They ask: why do vessels carrying blood from the heart need to be rich in oxygen?

By answering these questions, we have built up an understanding of the facts that ties them together in a multi-connected cluster:

pictorial representation of this information

For simplicity, I’ve just focused on the arteries. See how the four facts about arteries are connected together. Meaningfully connected. In a perfect world we’d be able to close the circle with a direct connection between the facts “Arteries carry blood rich in oxygen” and “Arteries are thick”, but as far as I know, the only connection between them is indirect, through the fact that “Arteries carry blood from the heart”.

So … the world isn’t perfect, and information doesn’t come in neatly wrapped bundles where every fact connects directly to every other fact. But the more connections you can make between related facts — the stronger a cluster you can make — the more deeply you will understand the information, and the more accessible it will be. That is, you will remember it more easily and for longer.

If it’s well enough connected

If it’s connected to strong anchor points

You will simply 'know' it.

You’re never going to forget that you breathe in oxygen and that your heart pumps out blood. These are strong anchor points. If the facts about arteries are strongly connected to these anchor points, you will never forget them either.

Asking questions is one of the best ways of making connections,

but

Bad questions can be worse than no questions at all.

Rote questions that direct your attention to unimportant details are better not asked.

Effective questions prepare you to pay attention to the important details in the text.

The best questions not only direct your attention appropriately, but also require you to integrate the details in the text. Ask yourself:

  • Is this helping me to select the important information?
  • Is it helping me make connections?

When the subject is new to you

When you don’t have enough prior knowledge about a subject to ask effective questions, you are better off forming connections using mnemonics — either through verbal elaboration, as in our sentence about “Art (ery) being thick around the middle so he wore trousers with an elastic waistband” or by creating interactive images.

However, mnemonics such as these — while perfectly effective — are only good for rote learning. Sometimes that’s all you want, of course. But if you’re going to be learning more information that relates to these facts, then you’re making a rod for your own back.

When you learn something by rote, it never gets easier. When you learn by building connections, every new fact is acquired more easily. And it’s progressive. An expert on a subject can hear a new fact in her area of expertise, and it’s there. Remembered. Without effort. Because she’s an expert. And what makes her an expert? Simply the fact that she’s built up a network of information that is so tightly connected, and that has so many strong anchor points, that the information is always retrievable.

Why questions, like any questions, are only effective to the extent that they direct attention to appropriate information.

Research confirms that it is better to search for consistent relations than inconsistent ones. In many cases your background knowledge may include information that is consistent with the new information, and information that is inconsistent.

By asking “Why is this true?” you focus on the consistent information.

 

References: 

  • Woloshyn, V.E., Willoughby, T., Wood, E., & Pressley, M. 1990. Elaborative interrogation facilitates adult learning of factual paragraphs. Journal of Educational Psychology, 82, 513-524.
  • Pressley, M. & El-Dinary, P.B. 1992. Memory strategy instruction that promotes good information processing. In D. Herrmann, H. Weingartner, A. Searleman & C. McEvoy (eds.) Memory Improvement: Implications for Memory Theory. New York: Springer-Verlag.

tags strategies: 

tags study: 

Subscribe to testing