Let’s start with the biggest chunk, our memory. You might be familiar with the multistore model proposed by Richard Atkinson and Richard Shiffrin in 1968. This groundbreaking work has had an enormous influence on how memory is conceived and became the foundation for other theories to build upon. This model asserts that human memory could be separated in 3 components : sensory memory, short-term memory (later replaced by working memory) and long-term memory.

 

Figure memory sans fioritures

 

​

Evolution has provided us with sensory receptors able to receive data from the environment. Our eyes, our nose, our ears are organs that assure such functions. According to the model, the data they obtain is first stored in a special "storage" (the brain is not a hard drive but some of its parts seem to retain information similarly so it is a good enough term) called sensory memory. It has a very large capacity because it contains all the data that all our senses obtain from our environment without filtering. But it does so for a very, very short time, it is continuously renewed. This type of memory holds information for two main purpose, first, the data is used to construct perception (more details in the perception part), and secondly it holds information while more focused, deeper processing mechanisms filter out the relevant parts of the data set to grasp our attention. 

Later studies suggested that sensory memory is actually a category of multiple storages, the two most well documented ones being the iconic memory for visual information (where data is lost after less than a second) and the echoic memory for auditory information (stored for less than 3 seconds).

​

 

Let’s pretend that you are standing in a forest in autumn, may it be in real-life or a video game, you’re sensory memory is receiving data from your different senses and as a consequence you observe motionless trees, decaying leaves and branches on the ground. The only noise around is the one you attribute to the collisions of the wind with the crown of the plants.

Suddenly something changes, you turned your head in the direction of a tree on the left and you are now look with insistence. Right before, your sensory memory received varying amount of light from this direction which its  “immediate” processing mechanisms has 

interpreted as significant motion. In addition,  ‘cracking’ sounds have been received which could indicate that significant weight might have been applied on the decaying material on the ground, roughly in the same direction. Your behavior was a result of your brain forcing your attention on this unexpected data in order to identify, or not, danger by accumulating more data.  Almost every other task you were doing at that time, worrying or wandering has been postponed to focus on the eventual life or death situation.

​

​

Attention

Our brain cannot waste energy on processing all the data it receives, all the time. If it was it would probably be less open to errors, but the brain wants to keep itself alive, it needs quick and dynamic decision-making. It can’t know everything so it needs to remove unnecessary 

and misleading data to uncover trends. What is most likely to happen? What do can I do about it? I have 5 seconds before that wolf jumps me. In these situations there are only a few things that still exist, the wolf, the weapon, the topology of the terrain, the few objects between him and us.

Attention is the mechanism that mobilize the processing power of the working memory on certain elements (and is probably mostly regulated by the central executive component).

 

Attention can be bottom up and top-down

​

It works like a spotlight that enlightens the area it is directed to but keeps everything else in the dark. As a consequence, even though it is quite counterintuitive, we could say that we all have some sorts of built-in blinders.

To prove it to you we could just take a look at the “inattentional blindness” phenomenon. I suggest you watch this short video LINK (Simons & Levins 1998).

This is another illustration of the fact that our processing resources are easily saturable. We can be blind to salient changes in our environment if our attention is mobilized elsewhere.

 

II Working Memory

Originally called short-term memory, it described the type of memory that holded information after it caught our attention

It has been observed that it is limited in time and in space (space here meaning the amount of data it can hold, just like and hard drive.).

When playing RDR2, in order to obtain certain clothes or equipment upgrades for our character, we have to obtain certain ingredients in its world. As far as I know the only time the game shows you these ingredients is when you talk to the craftsman that make them for you. As a consequence once you are in the world you have to remember the long mental list you just made. As an attempt to keep them in mind you start to rehearse them regularly in thoughts (because you know through experience how limited you memory can be, actually without rehearsal most items would be lost to the void in less than a minute).

But how long before something in the game distracts you? A few shootouts later, you realize that you forgot most of the list and now you have to ride back to the craftsman.

Studies have shown that the elements you are most likely to remember are the first and last few of the list (these are called respectively primacy and recency effect).

 

As a guideline to this phenomenon, Miller in 1956 proposed the concept of “7+-2” which asserts that humans can recall 5 to 9 items (7 more or less 2) without errors immediately after encoding them. Items here are the largest meaningful units.

If you try to remember the digits 3, 4, 8, 7, 2 separately you will have 5 items to rehearse whereas if you were to group them in one or two number (34 872 or 34 and 872) you have created two meaningful items and are thus easier to remember.

Nowadays the concept is highly criticized, it mostly remains accurate in situations where we don’t have to process any other information than the items we try to remember, which is very rare in real life, and lately, its application has shown important limitations when it comes to guide the development of user interfaces, web-designs etc.

 

The term working memory (WM) was first used by Baddeley and Hitch in their multicomponent model of working memory of 1974 and is an important evolution.

The term suggest that the system it describes is holding and processing the relevant information required to accomplish executive functions and complex cognitive tasks.

That way it plays a central role to acting and thinking in the world.

However, as we seen earlier the amount of data it can handle is limited.

 

This model breaks down working memory into 4 sub-components:

• The central executive is the component that regulates the 3 others. It has the crucial responsibility to direct our limited attentional resources.

It is also the component that relate the 3 other components to long-term memory, plays a central role in recalling required information for the task at hand and transfering new information to the long term memory.

 

• The phonological loop deals with auditory information, processing of music and sound but is also used intensively for languages processing tasks.

This store will hold items for a few seconds unless we keep rehearsing them.

 

• The visuospatial sketchpad deals with visual information (colors, shapes etc), spatial information (orientation) and haptic information (touch, proprioception).

 

• The episodic buffer is a binding component and a sort of sequence maker. It is conceptualized by Baddeley as a sort of passive store that arrange data in time and allow us to create associations. It is constantly fed by data from the visuospatial sketchpad, the phonological loop and also directly from the other senses (smell, taste). It then constructs a kind of mental sequence of events from this data, which allows for associations.

In the game when event A, shooting a gun in a city is quickly followed by event B, half the town people shooting back at you, they are easier to associate as cause / consequence for future decision-making. That is why instant feedback is so important for interfaces and games, we rely on the time variable to understand our world.

Presented as such, this component’s role is crucial in pattern recognition in humans, and that is why it is often conceptualize as not limited to working memory.

 

It was added only later to the model since it was first thought that the central executive was the component accomplishing this task. But later, when different experiments designed to interfere with the central executive took place, the observations showed an overall decrease in performance for the task at hand but not in the ability of the subject to bind information together.

 

FIGURE

 

The different components more or less interact with each other according to the task demands.  Ex of words processing, reading, use of the 3 component quite the task that’s why we can’t do anything else than reading

​
 

Actually it is because of the study of the limitations of working memory that the multicomponent model is the way it is today. To highlight them, Baddeley and his colleagues designed a series of experiments in which subjects would be asked to accomplish multiple task simultaneously. In short, they studied multitasking. The different interferences they observed from certain type of tasks allowed them to define the different components.

First, they discovered that, even though we are not always aware of it, we are quite bad at multitasking. When we do, the overall performance is poorer (more errors observed, more time required) than if each task would have been attended to separately. Multitask divides our attention which means that each individual task is to be completed with less processing power available.

Secondly, attempting to complete two phonological or two visuospatial activities simultaneously result in a poorer performance for both task than if one phonological activity is executed simultaneously with one visuospatial task.

Let me give you a few examples, in the Simons & Levins video, the task “identifying your interlocutor” is overwhelmed by the task of “describing an itinerary”. The two tasks are very demanding for the visuospatial sketchpad which lead to the observation of “inattentional blindness”.

We all have experienced a moment in movies or games where multiple auditory sources interfere with each other. The music may be too high as characters are having a conversation or multiple characters throw their dialogues lines at the same time (in these cases the fault might also be partly because it is hard for our sensory mechanisms to distinguish the stimuli). Fortunately with enough attention we can enlighten one and let the other in the dark, thus reducing interference (“cocktail party effect”, Cherry 1953). The problem is attempting to follow both. Another common example of phonological saturation is found in libraries. They are always quiet and this is partly because it would be very hard for us to follow a conversation while reading.

However, singing and drawing, listening to music while working with your hands, or in games, playing an intense shootout with dramatic music or following a conversation on the way to get to your next objective (which is very Rockstaresque thing I would say huhu) are much less negatively impactful. This observation is the main support for the separation of the 

phonological and the visuospatial component. It is also why visuals and sounds have been associated forever in entertainment, interfaces, and almost everything else we do.

 

Still, whatever the case, multitasking always decreases performance.

Recent fMRI (functional magnetic resonance imaging) studies have highlighted one phenomenon called “underaddivity” which confirms it : “the sum of brain activation in the regions associated with the described tasks (in that case it was language processing and mental rotation) is substantially less when both tasks are performed concurrently as compared to the sum of the brain activation in each region when the task are performed separately (The Gamer’s brain p53; Just et al 2001).

For language processing tasks in particular, the decrease in brain activity can be very important.

Additionally, when multitasking the brain actually switches quickly from one task to the other , filling the gaps with what sounds the most plausible or logical (It does it so fast that it is hard to notice, plus, the resources that would allow us to notice are already mobilized). It has been suggested that in this case, a certain amount of processing power had to be allocated for the central executive in order to coordinate this switch and everything else it implies.

 

So that is a lot of details but to resume, it is always better to give our attention to one unique task if we care to perform well. If we have to multitask, the only way that our performance can reach our expectations is if one task is visual or spatial and the other phonological.

 

CTL

The quality of learning is hindered if people feel overwhelmed or, in other words, if cognitive overload occurs. The cognitive load theory ot CLT (John Sweller 1988) has produced powerful tools if you wish to manage the quantity of information people’s working memory is exposed to when they are exploring your content. It asserts that some tasks are more demanding than others to our attentional resources, the more demanding the task, the more disruptive and irritating a distraction will be (Lavie 2005). Intentionally having to remove distractors or irrelevant information increases cognitive load. An example of that is the Stroop effect that you probably have already seen without knowing its name (you can find example on google rapidly)

.

The “amount” of attentional resources available depend on many variables :

• • Stress and anxiety have been shown to reduce WM capacity (Eysenck et al 2007)

  • Sustained attention increase our level of fatigue, which in turn reduce the potential amount of resources available.

  • The age and health of the subject, children, old people, sick people cannot focus as much and as long.

  • Expertise, as a result of the brain’s on going self-optimisation. The cognitive load of a given task is decreased as it is mastered. Experts are more efficient at removing distractor and staying focused on relevant information (Marozeau et al 2010)

​

You can find books and articles that have described CLT extensively in the further reading & reference section below.

 

In the case of entertainment though I would like to argue that you don’t want to keep your players or viewers working memory in balance all the time. We often provoke saturation voluntarily, for example, for people to feel intensity. Dramatic musics, sound effects of explosions, shots, particle systems, enemies coming from multiple directions, characters screaming etc. Being overwhelmed is sometimes part of the fun, and part of learning in games. It encourages people to push further what they already learned. If you want your player to not feel in control in certain parts of your game, asking a lot from them is good way to start. Most often yes, we will want to find the most efficient feedback. Not too discrete to attract people’s attention and not too present to avoid causing unnecessary load. Players want to perform the right activities to stay in the “zone”, in a state of flow.

But a game is also an emotional experience, if it never tries to surprise, push the player where he doesn’t necessarily want to go, it will probably not be a game to be remembered.

 

Talking of surprises there is one last thing to remember about attention, it can either be passive or active. When active, it is directed in a top-down fashion, meaning that we direct our attention on specific data. When passive, it is a bottom-up process where a stimuli from the environment grasps it.

To illustrate the difference between the two let’s examine a hunter / hunted situation in RDR2. When your plan is to hunt an animal you focus voluntarily on the elements that could lead to complete your objective. The animals’ position in space, the location of your aim and so on. That is active attention.

Sometimes when roaming around, you will hear a roar indicating that a nearby predator may be looking to make of you his next meal. In order to avoid the consequences of its character’s death, your brain focuses immediately on finding the location of the threat (notice how I talk of your brain sometimes just to make you tilt).

 

CHECK GO COGNITIVE VIDEOS

(phonological important for working but not much for long term, whaddup)

early baddeley videos I think

 

Figure

 

III Long-term memory

 

Contrary to the previous two, Long-term memory has no known limits in space and time. There are a few mathematical

 

no known limits in time and space. No idea to the limit of quantity of information that can be stored .

But in reality we forget information all time

 

https://www.youtube.com/watch?v=wkM35n70XjY

​

https://www.simplypsychology.org/long-term-memory.html

 

Work in schemas so might be why meaningful items >

 

Explicit memory (or declarative memory) info that can be described and explicitly know you know, capital of countries, books you read or not...Memory of facts (semantics) and events (episodic). (search internet)

 

Implicit memory (procedural memory) all non-declarative information tied to action, cannot be easily described How we do things, different brain areas.

 

Priming effect implicit memory

 

Conditioning implicit memory

 

P32 Mental representation and rotation

P90-91 Idriss objet mental, tranches superficielles, economie

But not for LTM images

https://www.youtube.com/watch?v=XRma8PPD1oE

 

The deeper the information is processed in WM the better the long-term retention (Craik and Lockhart 1972) -important information can be taught in situation where cognitive resources required are high and information must be be processed deeply . Will be better hold

Learning by doing > learning by reading

 

Forget

Memory lapse Forgetting curve. For non-meaningful content and no mnemonic

Sapcing. new disuse theory Bjork

 

https://www.youtube.com/watch?v=Hv6Vye1JCjo&t=1s

https://www.psychologyinaction.org/psychology-in-action-1/2011/09/21/need-to-remember-something-try-forgetting-it

 

Figure.

​

Recent information better preserved if associated with well known info

simple info > complex info

organized > messy

images > words

meaningful > non

 

Repetition crucial to learning, after each rehearsal forgetting curve become less steep. So better to distribute rehearsal over time than to teach too much at one given time

 

Distortion

What we recall is distorted (especially for declarative memory)

can recall memory that did not occur (false memory bias)

Ex Loftus & Palmer (1974) eyewitness testimony?

​

​

Perception

that detect the energy level of photons reflecting their way to us. Since objects do not influence photons the same way, we can obtain information about our surroundings and ultimately visualize it.

​

https://en.wikipedia.org/wiki/Perception

Classically we all think we have 5 senses but the list might be bigger. Sense is physiological capacity that provide data for perception. Organs in our body that are able to receive light, sound waves, temperature, etc…

Think of it as graphs of data changing in real time, brain take that and construct images with it  

Perception is constructed, fovea only sees colors but not what we perceive

P67 spontané vs évoqué

P73 idriss tache de mariotte

Idriss P66+ neuf des dix connexions retine descendante

Humains animal d’approximations

Bruner perceptions discoveries

 

Do not perceive world as it is, instead our brains construct mental images of it. Perception subjective, and it is influenced personal experience, expectations, and even our culture in somes cases.    

 

Gestalt theories law of perception

Figure/ground

Multistability

Closure

Visual acuity

Symmetry

Similarity

Proximity

 

Weber-Fechner bias (Logarithmic)

P68 Idriss

​

​

​

​

​

​

​

Cognition is the sum of mental processes that are involved in the treatment of information. Perception, memory, motivation, learning and decision-making are all subjects of study.

​

There is still plenty to discover about the brain but the little we know is already extremely useful.

​

The brain is a product of Darwinism. It is most efficient when it comes to keeping humans alive with limited resources available and it does not work like a computer. 

​

The brain is very expensive organ, it consumes up to 20% of the energy we produce, even though it generally represents about only 2% of our mass.

​

​

​

​