Neuroscientific techniques applied to audiovisual stimuli: cognitive-emotional analysis of Heineken commercials during COVID-19

Main Article Content

Mario Rajas Fernandez, Dr.
Lucía Sutil Martín, Dra.
Héctor Canorea Tiralaso, D.

Abstract

Using neuroscience techniques applied to the analysis of audiovisual content, four Heineken advertisements produced during the COVID-19 pandemic are studied. The main objective is to detect what cognitive and emotional responses the different stimuli elicit in thirty subjects and to evaluate whether the narrative construction of the advertisements works when conveying their advertising messages in images and sounds. For this purpose, after a preliminary textual analysis, tools for recording facial microexpressions and eye tracking are used, and the research is completed with a recall test. The results indicate the predominance of the emotions of joy and surprise, as well as the presence of other emotions (disgust, contempt, sadness) at specific moments. At the same time, it was found that the types of images that most captured the viewer's attention were those that included eye-catching, exciting, illustrative actions of the pandemic (mask, distance greeting, etc.) or that introduced written text on the screen. Likewise, it is observed that the Heineken product in its various materialisations (bottle, tap, logo, etc.) is only perceived prominently in the frame if there is a dramatic action that accompanies or underlines it. In contrast, the human figure (especially the eyes) is the visual component on which the subjects' gaze is predominantly focused. Finally, the results of the memory test coincide with the rest of the analyses with respect to the intensity of the emotions produced by the stimuli.

Downloads

Download data is not yet available.

Article Details

Section

COGNITIVE SCIENCE AND NEUROMARKETING

Author Biographies

Mario Rajas Fernandez, Dr., Rey Juan Carlos University

Professor of Audiovisual Script and Audiovisual Language and Technologies at the Department of Communication and Sociology of the Faculty of Communication Sciences at the Universidad Rey Juan Carlos. His field of research is the narrative, aesthetics and technology of emerging audiovisual media in educational and scientific communication.

Coordinator of the URJC's Ciberimaginario Consolidated Research Group.
(http://ciberimaginario.es)

He is currently Vice-Dean of Technological Infrastructures and Professional Sectors at the Faculty of Communication Sciences (2018-present). Previously, he was coordinator of the Academic Content Production Unit of the Centre for Innovation in Digital Education (CIED) of the URJC (2014-2017) in which more than a thousand educational and scientific videos were made. He has also held positions at the URJC as teaching manager of the Erasmus-Munde Academic Exchange (2011-2014), secretary of the Department of Communication 2 (2014) and integral tutor (2011-2015).

Previously, at the European University Miguel de Cervantes he held the position of coordinator of the Virtual Television Set (2007-2008).

He has been the director of the Contemporary Film Seminar at the URJC (2010-2016) and currently directs the Spanish Audiovisual Seminar (2018-present).
He is co-editor at Icono14 Editorial and director of the collections Estudios de Narrativa, Estudios de Comunicación, Estudios Visuales and Colección de Literatura (2011-present) and director of the collection Comunicación y Narrativa Audiovisual de la Editorial Sindéresis (2020-).

He has been recognised by the CNEAI for 1 six-year research period (2009-2015) and 1 six-year transfer period (2018). He has an h-index of 11 and an i13 index of 10 in Google Scholar, with a total of 386 citations, of which 298 have been produced since 2015.

He has published 21 research articles in journals indexed in selective databases such as JCR, SCOPUS, FECYT, Ulrich's or Latindex such as El Profesional de la Información or Revista Latina de Comunicación Social; he has also published 37 book chapters in publishers included in the SPI such as Springer, McGraw-Hill, Tirant lo Blanc, Síntesis or Comunicación y Sociedad and has coordinated 7 collective books in publishers such as Icono14, Fragua or Sindéresis.
He has participated in 7 national (1 as Principal Investigator) and 2 international competitive research projects, and in 9 educational innovation projects. He has carried out research stays in centres such as the Instituto Tecnológico de Monterrey (Mexico) (2009), the University of California Los Angeles (USA) (2011), the Escuela Internacional de Cine y Televisión de San Antonio de los Baños (Cuba) (2017) or the Universidade Católica Portuguesa (2021), as well as participating in more than 60 academic activities as a speaker at conferences, seminars and workshops related to his field of study.

As a researcher trainer, he has supervised 6 doctoral theses, and is currently supervising another 3. He has managed 2 pre-doctoral grants, 1 from the call for Aid for the recruitment of pre-doctoral researchers (research assistant) of the Community of Madrid (2018) and 1 from the Aid for the Training of Pre-doctoral Researchers of the Rey Juan Carlos University (2020).

Lucía Sutil Martín, Dra., Rey Juan Carlos University

Associate Professor, Department of Business Economics. Area of Market Commercialisation. Director of the Master's Degree in Development of Neurocognitive Techniques in Business Organisations: Neuromanagement. Director of the Brain Research Lab. Avda. de los Artilleros S/N, 28032 (Madrid).

The lines of research I follow are business neuroscience in the area of unconscious information processing, mainly cognitive-emotional processes.  Emotional processes in decision-making, neuromanagement, neuromarketing, consumer neuroscience, neuroeconomics, neurocommunication. Publications:

Baños-González, M., Rajas-Fernández, M., & Sutil-Martín, D. L. (2021). Analysis of Emotion and Recall in COVID-19 Advertisements: A Neuroscientific Study. International Journal of Environmental Research and Public Health, 18(16), 8721.
Sutil-Martín, D. L., & Otamendi, F. J. (2021). Soft Skills Training Program Based on Serious Games. Sustainability, 13(15), 8582.
Otamendi FJ. & Sutil DL. "The Emotional Effectiveness of Advertisement". Frontiers in Psychology, 2020.
Sutil DL. & Rienda JJ. "The Influence of Unconscious Perceptual Processing on Decision-Making: A New Perspective From Cognitive Neuroscience Applied to Generation Z". Frontiers in Psichology, 2020.
Chichi, A; Pravettoni, G., Sutil, DL., Parra, E. & Alcañiz, M. "A Novel Integrating Virtual Reality Approach for the Assessment of the Attachment Behavioral System. Frontiers in Psychology, 2017.
Projects:
Principal PI La Caixa Foundation Project: "Measurement and development of soft skills to favour the employability of young people using neuroscience techniques" 2019.
Principal PI Project (Art. 83): "The emotional impact of the Scotch Brite(TM) 60th anniversary campaign video" At 3M Company (2019).
Principal PI Project (Art. 83): "Study and Analysis of Work Climate and Training in Neuromanagement". In the company Supracafé (2017).
Principal PI of the Project (Art. 83) "Training in "Neuromanagement and experimentation with key tools and processes in leadership" In the company Red Eléctrica Española (2014-2015).
Principal PI of the Project (Art.83) "Design of instruments for the selection and training of volunteers" (2010-2012)

Héctor Canorea Tiralaso, D., Rey Juan Carlos University

Pre-doctoral fellow at the Universidad Rey Juan Carlos (URJC) in the Department of Communication and Sociology. Headquarters: Fuenlabrada Campus. Address: Camino del Molino s/n . 28943 - Fuenlabrada - Madrid.

Research interests: neuromarketing and video clips, having published in journals such as Revista Latina de Comunicación Social.

How to Cite

Rajas Fernandez, M., Sutil Martín, L., & Canorea Tiralaso, H. (2022). Neuroscientific techniques applied to audiovisual stimuli: cognitive-emotional analysis of Heineken commercials during COVID-19. Journal ICONO 14, 20(2). https://doi.org/10.7195/ri14.v20i2.1836

References

Ares, G., Giménez, A., Bruzzone, F., Vidal, L., Antúnez, L. y Maiche, A. (2013). Consumer Visual Processing of Food Labels: Results from an Eye-Tracking Study. Journal of Sensory Studies, 28(2), 138-153. http://dx.doi.org/10.1111/joss.12031

Bakalash, T. y Riemer, H. (2013). Exploring Ad-Elicited Emotional Arousal and Memory for the Ad Using fMRI. Journal of Advertising, 42(4), 275-291. https://doi.org/10.1080/00913367.2013.768065

Baraybar-Fernández, A., Baños-González, M., Barquero-Pérez, O., Goya-Esteban, R. y De-la-Morena-Gómez, A. (2017). Evaluación de las respuestas emocionales a la publicidad televisiva desde el Neuromarketing. Comunicar, 52, 19-28. https://doi.org/10.3916/C52-2017-02

Baschnagel, J. S. (2013). Using Mobile Eye-tracking to Assess Attention to Smoking Cues in a Naturalized Environment. Addictive Behaviors, 38(12), 2837-2840. http://dx.doi.org/10.1016/j.addbeh.2013.08.005

Bercea, M. D. (2012). Anatomy of Methodologies for Measuring Consumer Behavior in Neuromarketing Research. Proceedings of the Lupcon Center for Business Research European Marketing Conference. Munich, Alemania. https://bit.ly/3Hx5rQ8

Bigné, E., Llinares, C. y Torrecilla, C. (2016). Elapsed Time on First Buying Triggers Brand Choices within a Category: A Virtual Reality-Based Study. Journal of Business Research, 69(4), 1423-1427. https://doi.org/10.1016/j.jbusres.2015.10.119

Bix, L., Seo, W. y Sundar, R. P. (2013). The Effect of Colour Contrast on Consumers’ Attentive Behaviours and Perception of Fresh Produce. Packaging Technology and Science, 26(2), 96-104. http://dx.doi.org/10.1002/pts.1972

Bizer, G. Y. y Schindler, R. M. (2005). Direct Evidence of Ending-Digit Drop-off in Price Information Processing. Psychology & Marketing, 22(10), 771-802. http://dx.doi.org/10.1002/mar.20084

Brasel, S. A. y Gips, J. (2008). Breaking Through Fast-Forwarding: Brand Information and Visual Attention. Journal of Marketing, 72(6), 31-48. https://doi.org/10.1509/jmkg.72.6.031

Chae, S. W. y Lee, K. C. (2013). Exploring the Effect of the Human Brand on Consumers’ Decision Quality in Online Shopping: An Eye-Tracking Approach. Online Information Review, 37(1), 83-100. https://doi.org/10.1108/14684521311311649

Chandon, P., Hutchinson, W., Bradlow, E. y Young, S. (2009). Does In-Store Marketing Work? Effects of the Number and Position of Shelf Facings on Brand Attention and Evaluation at the Point of Purchase. Journal of Marketing, 73(6), 1-17. https://doi.org/10.1509/jmkg.73.6.1

Cherubino, P., Martinez-Levy, A.C., Caratù, M., Cartocci, G., Di Flumeri, G., Modica, E., Rossi, D., Mancini, M. y Trettel, A. (2019). Consumer Behaviour Through the Eyes of Neurophysiological Measures: State-of-the-Art and Future Trends. Computational Intelligence and Neuroscience, 1, 1-41. https://doi.org/10.1155/2019/1976847

Couwenberg, L. E., Boksem, M. A., Dietvorst, R. C., Worm, L., Verbeke, W. J. y Smidts, A. (2017). Neural Responses to Functional and Experiential Ad Appeals: Explaining Ad Effectiveness. International Journal of Research in Marketing, 34, 355-366. https://doi.org/10.1016/j.ijresmar.2016.10.005

Cuesta, U., Martínez-Martínez, L. y Niño, J. I. (2018). A Case Study in Neuromarketing: Analysis of the Influence of Music on Advertising Effectiveness Through Eye-Tracking, Facial Emotion and GSR. European Journal of Social Sciences Education and Research, 5(2), 73-82. https://doi.org/10.26417/ejser.v5i2.p84-92

Cullen, W., Gulati, G. y Kelly, B. D. (2020). Mental Health in the Covid-19 Pandemic. QJM: An International Journal of Medicine, 113(5), 311-312. https://doi.org/10.1093/qjmed/hcaa110

Cyr, D., Head, M. y Larios, H. (2010). Colour Appeal in Website Design within and Across Cultures: A Multi-method Evaluation. International Journal of Human-Computer Studies, 68(1-2), 1-21. http://dx.doi.org/10.1016/j.ijhcs.2009.08.005

De Oliveira, R., Dos Santos, R., Rocha, J. y Giraldi, J. (2015). Eye Tracking in Neuromarketing: A Research Agenda for Marketing Studies. International Journal of Psychological Studies, 7(1), 32-42. https://doi.org/10.5539/ijps.v7n1p32

Duchowski, A. T. (2003). Eye Tracking Methodology: Theory and Practice. Springer.

Ekman, P. y Friesen, W. V. (1978). Facial Action Coding System. Consulting Psychologists Press.

Fenko, A., Schifferstein, H. N. J. y Hekkert, P. (2010). Shifts in Sensory Dominance between Various Stages of User-Product Interactions. Applied Ergonomics, 41(1), 34-40. http://dx.doi.org/10.1016/j.apergo.2009.03.007

Fiorillo, A. y Gorwood, P. (2020). The Consequences of the COVID-19 Pandemic on Mental Health and Implications for Clinical Practice. European Psychiatry, 63(1), 1-4 https://doi.org/10.1192/j.eurpsy.2019.3

Fiszman, B. P., Velasco, C., Salgado-Montejo, A. y Spence, C. (2013). Using Combined Eye Tracking and Word Association in Order to Assess Novel Packaging Solutions: A Case Study Involving Jam Jars. Food Quality and Preference, 28(1), 328-338. http://dx.doi.org/10.1016/j.foodqual.2012.10.006

Grewal, D., Ailawadi, K. L., Gauri, D., Hall, K., Kopalle, P. y Robertson, J. R. (2011). Innovations in Retail Pricing and Promotions. Journal of Retailing, 87(1), S43-S52. http://dx.doi.org/10.1016/j.jretai.2011.04.008

Grier, S. y Bryant, C. A. (2005). Social Marketing in Public Health. Annual Reviews of Public Health, 26, 319-339. http://dx.doi.org/10.1146/annurev.publhealth.26.021304.144610

Halls, B. F. (2002). A New Model for Measuring Advertising Effectiveness. Journal of Advertising Research, 42(2), 23-31. https://doi.org/10.2501/JAR-42-2-23-31

Hamelin, N., El Moujahid, O. y Thih, P. (2017). Emotion and Advertising Effectiveness: A Novel Facial Expression Analysis Approach. Journal of Retailing and Consumer Services, 36, 103-111. https://doi.org/10.1016/j.jretconser.2017.01.001

Hawkins, D. I., Mothersbaugh, D. L. y Best, R. J. (2009). Consumer Behavior: Building Marketing Strategy. McGraw-Hill.

Hoffman, J. y Subramaniam, B. (1995). The Role of Visual Attention in Saccadic Eye Movements. Perception & Psychophysics, 57, 787-795. https://doi.org/10.3758/BF03206794

Holmes, E. A., O’Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L. y Bullmore, E. (2020). Multidisciplinary Research Priorities for the COVID-19 Pandemic: A Call for Action for Mental Health Science. Lancet Psychiatry, 7(6), 547-560. https://doi.org/10.1016/S2215-0366(20)30168-1

Hurley, R. A., Ouzts, A., Fischer, J. y Gomes, T. (2013). Effects of Private and Public Label Packaging on Consumer Purchase Patterns. Packaging Technology and Science, 26(7), 399-412. http://dx.doi.org/10.1002/pts.2012

Khushaba, R. N., Wise, C., Kodagoda, S., Louviere, J., Kahn, B. E. y Townsend, C. (2013). Consumer Neuroscience: Assessing the Brain Response to Marketing Stimuli Using Electroencephalogram (EEG) and Eye Tracking. Expert Systems with Applications, 40(9), 3803-3812. http://dx.doi.org/10.1016/j.eswa.2012.12.095

Kotler, P. y Keller, K. L. (2009). Marketing Management. Pearson Prentice Hall.

Kulke L., Feyerabend, D. y Schacht, A. (2020). A Comparison of the Affectiva iMotions Facial Expression Analysis Software with EMG for Identifying Facial Expressions of Emotion. Frontiers in Psychology, 11, 329. doi: https://doi.org/10.3389/fpsyg.2020.00329

Le Blanc, V. R., McConnell, M. M. y Monteiro, S. D. (2014). Predictable Chaos: A Review of the Effects of Emotions on Attention, Memory and Decision Making. Advances in Health Sciences Education, 20, 265-282. https://doi.org/10.1007/s10459-014-9516-6

Lee, J. y Ahn, J. H. (2012). Attention to Banner Ads and Their Effectiveness: An Eye-Tracking Approach. International Journal of Electronic Commerce, 17(1), 119-137. http://dx.doi.org/10.2753/JEC1086-4415170105

Lewinski, P., Fransen, M. L. y Tan Ed S. H. (2014). Predicting Advertising Effectiveness by Facial Expressions in Response to Amusing Persuasive Stimuli. Journal of Neuroscience, Psychology, and Economics, 7(1), 1-14. https://doi.org/10.1037/npe0000012

Magdin, M., Benko, Ľ., & Koprda, Š. (2019). A Case Study of Facial Emotion Classification Using Affdex. Sensors, 19(9), 2140. https://doi.org/10.3390/s19092140

Mallik, A. y Azharshaheen, S. S. (2021). Neuromarketing Managemental Applications in Marketing. Empirical Economics Letters, 20(3), 125-144. https://bit.ly/3mVywNv

Martínez, P. (2011). The Consumer Mind: Brand Perception and the Implication for Marketers. Kogan Page.

Maynard, O. M., Munafò, M. R. y Leonards, U. (2013). Visual Attention to Health Warnings on Plain Tobacco Packaging in Adolescent Smokers and Non-Smokers. Addiction, 108(2), 413-419. https://doi.org/10.1111/j.1360-0443.2012.04028.x

Page, T. J., Thorson, E. y Heide, M. P. (1990). The Memory Impact of Commercials Varying in Emotional Appeal and Product Involvement. Emotion in Advertising. Quorum Books.

Pfefferbaum, B. y North, C. S. (2020). Mental Health and the Covid-19 Pandemic. New England Journal of Medicine, 383, 510-512. https://doi.org/10.1056/NEJMp2008017

Poels, K. y Dewitte, S. (2006). How to Capture the Heart? Reviewing 20 Years of Emotion Measurement in Advertising. Journal of Advertising Research, 46(1), 18-37. https://doi.org/10.2501/S0021849906060041

Ramsøy, T. Z., Friis-Olivarius, M., Jacobsen, C., Jensen, S. B. y Skov, M. (2012). Effects of Perceptual Uncertainty on Arousal and Preference Across Different Visual Domains. Journal of Neuroscience, Psychology, and Economics, 5(4), 212-226. https://doi.org/10.1037/a0030198

Rowe, A. y Burridge, L. (2012). Ten Inbox Secrets: What Eye Tracking Reveals About Designing Better Emails. Journal of Direct, Data and Digital Marketing Practice, 14, 46-65. http://dx.doi.org/10.1057/dddmp.2012.23

Russo, J. E. (1978). Eye Fixations Can Save the World: A Critical Evaluation and a Comparison Between Eye Fixations and other Information Processing Methodologies. Advances in Consumer Research, 5, 561-570.

Singh, N. y Jain, S. (2018). Neuromarketing in Action-Towards a New Model of Persuasion. AADYA-Journal of Management and Technology, 8(1), 101-110.

Spence, C. (2016). Neuroscience-Inspired Design: From Academic Neuromarketing to Commercially Relevant Research. Organizational Research Methods, 22(1), 275-298. https://doi.org/10.1177/1094428116672003

Stöckli, S., Schulte-Mecklenbeck, M. y Borer, S. (2018). Facial Expression Analysis with AFFDEX and FACET: A Validation Study. Behavior Research Methods, 50(4), 1446-1460. https://doi.org/10.3758/s13428-017-0996-1

Taggart, R. W., Dressler, M., Kumar, P., Khan, S., Coppola, J. F. (2016). Determining Emotions Via Facial Expression Analysis Software. Proceedings of Student-Faculty Research Day, CSIS, Pace University, Nueva York, Estados Unidos.

Teixeira, T., Wedel, M. y Pieters, R. (2012). Emotion-Induced Engagement in Internet Video Advertisements. Journal of Marketing Research, 49(2), 144-159. http://dx.doi.org/10.1509/jmr.10.0207

Thomsen, S. R. P. D. y Fulton, K. B. A. (2007). Adolescents’ Attention to Responsibility Messages in Magazine Alcohol Advertisements: An Eye-Tracking Approach. Journal of Adolescent Health, 41(1), 27-34. http://dx.doi.org/10.1016/j.jadohealth.2007.02.014

Varan, D., Lang, A., Barwise, P., Weber, R. y Bellman, S. (2015). How Reliable are Neuromarketers Measures of Advertising Effectiveness? Journal Advertising Research, 55(2), 176-191. https://doi.org/10.2501/JAR-55-2-176-191

Venkatraman, V., Dimoka, A., Pavlou, P., Võ, K., Hampton, W., Bollinger, B., Hershfield, H., Ishihara, M. y Winer, R. (2015). Predicting Advertising Success Beyond Traditional Measures: New Insights from Neurophysiological Methods and Market Response Modeling. Journal of Marketing Research, 52(4). https://doi.org/10.1509/jmr.13.0593

Wedel, M. y Pieters, R. (2008). Eye-Tracking for Visual Marketing. Foundations and Trends in Marketing, 1(4), 231-320. http://dx.doi.org/10.1561/1700000011

Wedel, M. y Pieters, R. (2014). Looking at Vision: Eye/Face/Head Tracking of Consumers for Improved Marketing Decisions. En L. Moutinho, E. Bigné y A. K. Manrai (Eds.), The Routledge Companion to the Future of Marketing (pp. 15-43). Routledge.

Winer, R. S. (2011). Prólogo. En M. Álvarez del Blanco, Fusión Perfecta: Neuromarketing (pp. 11-14). Prentice Hall.

Zurawicki, L. (2010). Neuromarketing: Exploring the Brain of the Consumer. Springer.