Event boundaries and working memory capacity

In a recent news report, I talked about how walking through doorways creates event boundaries, requiring us to update our awareness of current events and making information about the previous location less available. I commented that we should be aware of the consequences of event boundaries for our memory, and how these contextual factors are important elements of our filing system. I want to talk a bit more about that.

One of the hardest, and most important, things to understand about memory is how various types of memory relate to each other. Of course, the biggest problem here is that we don’t really know! But we do have a much greater understanding than we used to do, so let’s see if I can pull out some salient points and draw a useful picture.

Let’s start with episodic memory. Now episodic memory is sometimes called memory for events, and that is reasonable enough, but it perhaps gives an inaccurate impression because of the common usage of the term ‘event’. The fact is, everything you experience is an event, or to put it another way, a lifetime is one long event, broken into many many episodes.

Similarly, we break continuous events into segments. This was demonstrated in a study ten years ago, that found that when people watched movies of everyday events, such as making the bed or ironing a shirt, brain activity showed that the event was automatically parsed into smaller segments. Moreover, changes in brain activity were larger at large boundaries (that is, the boundaries of large, superordinate units) and smaller at small boundaries (the boundaries of small, subordinate units).

Indeed, following research showing the same phenomenon when people merely read about everyday activities, this is thought to reflect a more general disposition to impose a segmented structure on events and activities (“event structure perception”).

Event Segmentation Theory postulates that perceptual systems segment activity as a side effect of trying to predict what’s going to happen. Changes in the activity make prediction more difficult and cause errors. So these are the points when we update our memory representations to keep them effective.

Such changes cover a wide gamut, from changes in movement to changes in goals.

If you’ve been following my blog, the term ‘updating’ will hopefully bring to mind another type of memory — working memory. In my article How working memory works: What you need to know, I talked about the updating component of working memory at some length. I mentioned that updating may be the crucial component behind the strong correlation between working memory capacity and intelligence, and that updating deficits might underlie poor comprehension. I distinguished between three components of updating (retrieval; transformation; substitution), and how transformation was the most important for deciding how accurately and how quickly you can update your contents in working memory. And I discussed how the most important element in determining your working memory ‘capacity’ seems to be your ability to keep irrelevant information out of your memory codes.

So this event segmentation research suggests that working memory updating occurs at event boundaries. This means that information before the boundary becomes less accessible (hence the findings from the walking through doorways studies). But event boundaries relate not only to working memory (keeping yourself updated to what’s going on) but also to long-term storage (we’re back to episodic memory now). This is because those boundaries are encoded particularly strongly — they are anchors.

Event boundaries are beginnings and endings, and we have always known that beginnings and endings are better remembered than middles. In psychology this is known formally as the primacy and recency effects. In a list of ten words (that favorite subject of psychology experiments), the first two or three items and the last two or three items are the best remembered. The idea of event boundaries gives us a new perspective on this well-established phenomenon.

Studies of reading have shown that readers slow down at event boundaries, when they are hypothesized to construct a new mental model. Such boundaries occur when the action moves to a new place, or a new time, or new characters enter the action, or a new causal sequence is begun. The most important of these is changes in characters and their goals, and changes in time.

As I’ve mentioned before, goals are thought to play a major role (probably the major role) in organizing our memories, particularly activities. Goals produce hierarchies — any task can be divided into progressively smaller elements. Research suggests that higher-order events (coarse-grained, to use the terminology of temporal grains) and lower-order events (fine-grained) are sensitive to different features. For example, in movie studies, coarse-grained events were found to focus on objects, using more precise nouns and less precise verbs. Finer-grained events, on the other hand, focused on actions on those objects, using more precise verbs but less precise nouns.

The idea that these are separate tasks is supported by the finding of selective impairments of coarse-grained segmentation in patients with frontal lobe lesions and patients with schizophrenia.

While we automatically organize events hierarchically (even infants seem to be sensitive to hierarchical organization of behavior), that doesn’t mean our organization is always effortlessly optimal. It’s been found that we can learn new procedures more easily if the hierarchical structure is laid out explicitly — contrariwise, we can make it harder to learn a new procedure by describing or constructing the wrong structure.

Using these hierarchical structures helps us because it helps us use knowledge we already have in memory. We can co-op chunks of other events/activities and plug them in. (You can see how this relates to transfer — the more chunks a new activity shares with a familiar one, the more quickly you can learn it.)

Support for the idea that event boundaries serve as anchors comes from several studies. For example, when people watched feature films with or without commercials, their recall of the film was better when there were no commercials or the commercials occurred at event boundaries. Similarly, when people watched movies of everyday events with or without bits removed, their recall was better if there were no deletions or the deletions occurred well within event segments, preserving the boundaries.

It’s also been found that we remember details better if we’ve segmented finely rather than coarsely, and some evidence supports the idea that people who segment effectively remember the activity better.

Segmentation, theory suggests, helps us anticipate what’s going to happen. This in turn helps us adaptively create memory codes that best reflect the structure of events, and by simplifying the event stream into a number of chunks of which many if not most are already encoded in your database, you save on processing resources while also improving your understanding of what’s going on (because those already-coded chunks have been processed).

All this emphasizes the importance of segmenting well, which means you need to be able to pinpoint the correct units of activity. One way we do that is by error monitoring. If we are monitoring our ongoing understanding of events, we will notice when that understanding begins to falter. We also need to pay attention to the ordering of events and the relationships between sequences of events.

The last type of memory I want to mention is semantic memory. Semantic memory refers to what we commonly think of as ‘knowledge’. This is our memory of facts, of language, of generic information that is untethered from specific events. But all this information first started out as episodic memory — before you ‘knew’ the word for cow, you had to experience it (repeatedly); before you ‘knew’ what happens when you go to the dentist, you had to (repeatedly) go to the dentist; before you ‘knew’ that the earth goes around the sun, there were a number of events in which you heard or read that fact. To get to episodic memory (your memory for specific events), you must pass through working memory (the place where you put incoming information together into some sort of meaningful chunk). To get to semantic memory, the information must pass through episodic memory.

How does information in episodic memory become information in semantic memory? Here we come to the process of reconstruction, of which I have often spoken (see my article on memory consolidation for some background on this). The crucial point here is that memories are rewritten every time they are retrieved.

Remember, too, that neurons are continually being reused — memories are held in patterns of activity, that is, networks of neurons, not individual neurons. Individual neurons may be involved in any number of networks. Here’s a new analogy for the brain: think of a manuscript, one of those old parchments, so precious that it must be re-used repeatedly. Modern technology can reveal those imperfectly erased hidden layers. So the neural networks that are memory codes may be thought of as imposed one on top of each other, none of them matching, as different patterns re-use the same individual neurons. The strongest patterns are the most accessible; patterns that are most similar (use more of the same neurons) will provide the most competition.

So, say you are told by your teacher that the earth goes around the sun. That’s the first episode, and there’ll be lots of contextual detail that relates to that particular event. Then on another occasion, you read a book showing how the earth goes around the sun. Again, lots of episodic detail, of which some will be shared with the first incident, and some will be different. Another episode, more detail, some shared, some not. And so on, again and again, until the extraneous details, irrelevant to the fact and always different, are lost, while those details that common to all the episodes will be strong, and form a new, tight chunk of information in semantic memory.

Event boundaries start off with an advantage — they are beginnings or endings, to which we are predisposed to attend (for obvious reasons). So they start off stronger than other bits of information, and by their nature are more likely to be vital elements, that will always co-occur with the event. So — if you have chosen your boundaries well (i.e., they truly are vital elements) they will become stronger with each episode, and will end up as vital parts of the chunk in semantic memory.

Let’s connect that thought back to my comment that the most important difference between those with ‘low’ working memory capacity and those with ‘high’ capacity is the ability to select the ‘right’ information and disregard the irrelevant. Segmenting your events well would seem to be another way of saying that you are good at selecting the changes that are most relevant, that will be common to any such events on other occasions.

And that, although some people are clearly ‘naturally’ better at it, is surely something that people can learn.