On the exhibit floor of The Strong National Museum of Play, somewhere between the Pinball Playfield and Sesame Street exhibits, there is a quote by Diane Ackerman: “Play is our brain’s favorite way of learning.” This quote resonates deeply with me as a Cognitive Neuroscientist interested in the relationships between brain and behaviour, as well as the numerous ways in which games and science interact. For one very special week in October 2023, I was fortunate to visit The Strong on a Valentine-Cosman Research Fellowship with an idea: are principles of how the brain learns baked into game design?
We know that the ease with which objects are perceived also predicts positive engagement (known as perceptual fluency). So if game components are to be fun, these objects must be designed in the most fluent way possible. A nice example of perceptual fluency exists in the various iterations of the board game Candy Land. Given the young age of the target audience, movement is communicated with cards containing 1 or 2 colored squares rather than the more complex interpretations involved in dice throwing.
We also know that the brain processes certain combinations of information more effectively due to things called “correspondences.” For example, we must build a physically large musical instrument if we want to produce low pitches (compare a violin with a double bass). From this, we predict that large objects correspond to low pitches. By spending time at a piano, we learn that lower pitches come from hitting leftward keys, but higher notes come from hitting rightward keys. As such, we predict low tones should come from locations on the left and high tones from locations on the right. As a final example, as we walk outside (perhaps to take a stroll around the Hasbro Game Park at The Strong) we see that the grass is beneath us and the sky is above us. For this reason, we might regularly expect the color blue to be above the color green.
As part of my fellowship, I asked whether any of these correspondences related to color, location, and sound existed in the classic electronic game Simon. In the basic game, the player must reproduce an ever-increasing tonal sequence by pressing one of four colored quadrants in the correct order. If the player successfully reproduces the pattern, Simon adds one more element to the sequence.
The first observation that took me by surprise was that Simon was actually inspired by an earlier arcade unit by Atari. As described in the notes of game developer Ralph Baer: “Touch-Me was in a waist-high cabinet with four large dark buttons facing the player . . . during the game, the buttons lit up in random sequences. . . . It was the player’s job to follow the light sequence of the buttons . . . nice game idea, terrible execution . . . visually lousy, miserable sounds!”
While The Strong has one of these Touch-Me cabinets in the museum’s collection, it was in the process of repair and I never got to experience the “‘lousy” sounds and colors of this inspirational design. Happily, a hand-held version of the Atari game was available, enabling comparison with an incredible range of hand-held Simon-style units.
Just comparing the ways in which colors corresponded to location, there were lots of variation in design choice. For example, in the original Touch Me unit, blue is top-left and green is bottom-right. So this particular design contains one correspondence that we have learnt to expect: grass below, sky above. What is intriguing then is the reversal of this correspondence of most Simon variants where green is now above blue. The exception here is very cute Pocket Simon (1980), which reinstates green on the bottom and blue on the top.
Thanks to a healthy supply of variously sized batteries, I was able to make these units shine and sing again. The sounds were not as strident or as confident as you might have expected from an electronic game. The tones were often frail or uncertain, reminding us all that there is a delicacy in maturity. While listening to the sounds, I continued to pore over the notes of Ralph Baer that tracked the development of the initial Simon game. Indeed, much of the writing looked quite similar to my own laboratory notebooks: ideas sketched, hypotheses generated, data collected, and fed back into the next iteration of ideas. A quite beautiful document was the sketching of the four “bugle” sounds (and the error or “razz” noise) that made it into the final version of the game. But how did these sounds correspond to locations?
Sound-location mappings were inconsistent across units, and not in line with my predictions that lower sounds should be associated with lower and leftward locations. For example, the original 1979 version of Simon (and the more contemporary Simon Swipe from 2014) gets the vertical mapping “correct” in that lower tones are associated with lower quadrants (yellow and blue), but gets the horizontal mapping “incorrect” in that lower tones are associated with more rightward locations (red and blue).
Further surprising correspondences were also waiting for me on the exhibit floor of The Strong. In particular, I was delighted to find a huge, aerobic version of Simon outside in the Hasbro Game Park. As you can see from the picture, color-location is again in the canonical, but curious configuration where green is above blue. On playing this large-scale version however, I noticed that the sounds had been rearranged from the hand-held version—low tones now came from the high locations and high tones came from the low locations!
In reviewing Simon games with a scientific lens, I was struck by the disconnection between psychological principles and gamification principles. I predicted that the organization of color-location-sound elements in the Simon game would have been set up in a way that reflected how the brain processes information: play is—after all—our brain’s favorite way of learning. But is it appropriate to call these mappings “correct” or ”incorrect?” From a player point-of-view, perhaps there is additional fun in the incongruency between the elements in this game. Indeed, play might be the very best environment in which to experience the world not as we expect it but as it could be.
As a scientist, the “perfect” design of a particular game can be approached empirically. Processing fluency can be measured using speed and accuracy, and much of my laboratory’s future work will be in the use of games as scientific paradigms. Simon is essentially a measure of working memory capacity, and we will directly study the effects that correspondences have on game performance. It is my hope that in the same way games may reveal something novel about science, the application of science may provide us with novel opportunities for play.
By, Ben Dyson, 2023 Valentine-Cosman Research Fellow at The Strong