Research

The automatic detection of persuasive strategies in advertisements presents a uniquely multimodal challenge at the intersection of vision, language, and social cognition. While recent advances in Large Vision-Language Models (LVLMs) offer promising capabilities for such tasks, current approaches often rely on restrictive evaluation schemes that do not reflect the inherently multi-label nature of persuasive messaging. In this work, we reframe persuasion strategy detection as a genuine multi-label classification problem and propose a principled evaluation framework to enhance interpretability and robustness. We apply this approach to both image and video datasets, examine their characteristics, and introduce novel input-agnostic baselines that achieve macro F1 scores of 0.082 and 0.289 on the respective test sets. As part of this analysis, we study label co-occurrence patterns and dataset ambiguities, providing insights that inform both model interpretation and future dataset design. To assess the native capabilities of LVLMs, we benchmark three open source models—PaliGemma, PaliGemma2, and Qwen2.5-VL—on the image persuasion dataset. Under zero-shot conditions, we demonstrate that querying each strategy individually with a logit-based decision threshold outperforms guided text generation. The best-performing zero-shot model, Qwen2.5-VL, achieves a macro F1 score of 0.227 and a sample F1 score of 0.234 on the image test set, and 0.381 and 0.400 respectively on the video test set. We further explore and compare two lightweight fine-tuning strategies that update only small subsets of model parameters while keeping the remaining weights frozen: fine-tuning of the image-to-text tokens linear projection and Low Rank Adaptation (LoRA) of the language model. Linear projector fine-tuning yields a top macro F1 score of 0.396 on the image test set, marking a substantial improvement over zero-shot performance. To evaluate cross-modal generalization, we apply fine-tuned image models to the video dataset. Our experiments reveal that, while projection-based fine-tuning enables partial knowledge transfer from image to video (macro F1 = 0.416 on a testing subset), LoRA adaptation severely disrupts cross-modal performance (macro F1 = 0.189). Finally, we perform a per-strategy performance analysis, looking into annotator- and data-centric factors that may influence LVLM performance. These findings highlight the viability of open-weight LVLMs for fine-grained persuasion analysis and suggest efficient pathways for domain-specific adaptation under realistic resource constraints.

Video memorability prediction has emerged as a key challenge for improving information retrieval, content design, and user engagement. Prior work has shown that semantic cues play a crucial role in determining memorability, with recent studies leveraging Contrastive Language-Image Pre-training (CLIP) encoders to incorporate semantic information. However, the specific improvements attributable to CLIP models remain unclear, as few studies systematically compare their performance against equivalent unimodal encoders or explore fine-tuning strategies. This work addresses that gap through a comprehensive, controlled evaluation of CLIP-based and unimodal encoders for video memorability prediction. We propose FCLIP, a domain-adapted extension of CLIP that undergoes additional contrastive pre-training on memorability-specific image-text pairs. Our experiments assess both feature extraction and supervised fine-tuning, ensuring fair comparisons across models with matched architecture and parameter count. Results show that FCLIP image encoders achieve a Spearman Rank Correlation Coefficient (SRCC) of 0.672 on the Memento10k dataset, significantly outperforming unimodal Vision Transformers. FCLIP text encoders similarly outperform unimodal baselines, reaching an SRCC of 0.632. These findings demonstrate that contrastive learning and domain adaptation substantially improve memorability prediction, highlighting the importance of semantic and multimodal pre-training in developing advanced content analysis systems.

Large Vision-Language Models (LVLMs) demonstrate impressive capabilities across multimodal tasks, yet their inner workings remain poorly understood, particularly in subjective domains that involve human perception. In this paper, we examine how attention patterns in LVLMs shift with task-specific adaptation, using the task of detecting persuasion strategies in advertisements as a case study. This task attempts to model 16 different techniques used to influence behavior or decision-making in marketing. In particular, we leverage PaliGemma to classify persuasion strategies by applying a score-based logit postprocessing approach. Under this setting, we compare zero-shot performance with fine-tuning of the linear projector that maps image features to the language embedding space, achieving 19.6% and 66.0% accuracy, respectively, on the test set of the Persuasion Strategies in Advertisements corpus. We perform an exhaustive model interpretability analysis to understand how this lightweight adaptation method influences downstream performance, showing that fine-tuning sharpens attention to image regions and reduces noise. Fine-tuning makes image token representations more task-specific, evidenced by distinct attention patterns across persuasion strategies. These findings motivate deeper exploration into LVLM interpretability.

This paper presents the 2026 edition of the MediaEval Predicting Movie and Commercial Memorability Task. Much like the previous edition, the benchmark is structured around four different subtasks: (1.1) predicting long-term memorability using a movie clip, (1.2) classifying successful recognition using participants EEG signals, (2.1) predicting long-term commercial video memorability and (2.2) predicting the brand memorability associated with a video clip. This edition builds upon previous years, introducing an expanded feature set that integrates colour palettes and movie metadata to enable modelling strategies that go beyond video characteristics.

This paper describes the THAU-UPM submissions to the MediaEval 2026 Predicting Movie and Commercial Memorability Task, focusing on EEG-based prediction of movie recall. We compare lightweight convolutional architectures, EEGNet, and pre-trained SignalJEPA representations under subject-independent and subject-dependent training regimes. Our results show that deep learning models can capture weak but meaningful signals from EEG, although performance remains below that of previous feature-based approaches. The best official test result is obtained with subject-dependent adaptation of pre-trained SignalJEPA features, reaching an AUC of 0.582 and suggesting that subject variability remains a major bottleneck for EEG-based recall prediction.

One of the powers of visual media lies in its ability to create a lasting impression on the viewer's memory. In this digital age, where media is abundant and attention spans are fleeting, the task of predicting which content will stick in the viewer's mind has become a critical challenge in computer vision. Computational memorability seeks to address this by developing models that estimate how memorable a piece of media is likely to be. In this review we focus on the MediaEval Predicting Video Memorability benchmark, a recurring evaluation task that has run annually since 2018. This benchmark provides a unique and consistent framework for researchers to compare and refine their memorability prediction techniques using standardised datasets and metrics. Its reproducible framework has proven invaluable for tracking progress and fostering innovation in this rapidly evolving domain. We analyse the evolution of the benchmark across its 2018–2023 editions, discussing the challenges that still remain, such as the need for more interpretability in models and the difficulty of predicting subjective and context-dependent memorability. By analysing and synthesising the collective insights gained from this task, we endeavour to inspire new avenues of inquiry and drive progress towards a more comprehensive understanding of this topic.

This paper introduces TESSCCo (TV-control EEG-based Silent Speech Command Corpus), a new dataset including electroencephalography (EEG) signals during Overt Speech (OS) and Covert Speech (CS) in different languages. The dataset comprises repetitions of five different commands pronounced covertly and overtly in English and Spanish from 21 healthy native Spanish speakers (13 male, 8 female, 23 ± 2 years old), while EEG and audio were recorded. In addition, 3 non-native healthy Spanish speakers were recorded under the same circumstances (3 male, 23.3 ± 0.6 years old). A total of 7936 available epochs (i.e., 11.02 hours of data) were recorded with a 32 channel, 256 Hz sampling rate Water based EEG device. The database was designed to maximize the number of different analysis involving EEG signals. The final number of epochs, as well as the statistical analysis (showing significance in Broca's and Wernicke's areas) and machine learning experiments (with subjects exceeding the chance level with basic machine learning models), show that this material is a valuable resource for research on future ways of communication.

While the influence of visual factors on movie memorability has been extensively studied, the roles of film recognizability, genre, and popularity remain underexplored. In this Quest for Insight paper, we investigate these factors using the additional metadata provided in this edition of the MediaEval Memorability Task. We use the "neutral" and "typical" labels as indicators of whether a film can be recognized from a given clip. To approximate film popularity, we leverage user ratings from the MovieLens dataset, while genre-level analyses are conducted using the provided genre annotations. Our analysis reveals substantial differences between the memorability distributions of neutral and typical clips, suggesting that recognizability plays an important role in memorability prediction. In contrast, we find little to no correlation between film popularity and clip memorability. Similarly, memorability scores exhibit only limited variation across film genres. These findings suggest that neither popularity nor genre substantially influences movie-clip memorability, whereas the extent to which a film is recognizable from a clip may be a key factor in understanding what makes content memorable.

The accurate modelling of video memorability, or the intrinsic properties that render a piece of audiovisual content more likely to be remembered, will facilitate the development of automatic systems that are more efficient in retrieving, classifying and generating impactful media. Recent studies have indicated a strong correlation between the visual semantics of video and its memorability. This underscores the importance of developing advanced visual comprehension abilities to enhance model performance. It has been demonstrated that Large Vision–Language Models (LVLMs) demonstrate exceptional proficiency in generalist, high-level semantic comprehension of images and video, due to their extensive multimodal pre-training on a vast scale. This work makes use of the vast generalist knowledge of LVLMs and explores efficient adaptation techniques with a view to utilising them as memorability predictors. In particular, the Quantized Low-Rank Adaptation (QLoRA) technique is employed to fine-tune the Qwen-VL model with memorability-related data extracted from the Memento10k dataset. In light of existing research, we propose a particular methodology that transforms Qwen-VL from a language model to a memorability score regressor. Furthermore, we consider the influence of selecting appropriate LoRA hyperparameters, a design aspect that has been insufficiently studied. We validate the LoRA rank and alpha hyperparameters using 5-Fold Cross-Validation and evaluate our best configuration on the official testing portion of the Memento10k dataset, obtaining a state-of-the-art Spearman Rank Correlation Coefficient (SRCC) of 0.744. Consequently, this work represents a significant advancement in modelling video memorability through high-level semantic understanding.

Designing reliable automatic models for social perception can contribute to a better understanding of human behavior, enabling more trustworthy experiences in the multimedia on-line communication environment. However, predicting social attributes from video data remains challenging due to the complex interplay of visual, auditory, and linguistic cues. In this paper, we address this challenge by investigating the effectiveness of Multimodal Large Language Models (MM-LLMs) for feature extraction in the MuSe-Perception challenge. Firstly, our analysis of the novel LMU-ELP dataset has revealed high correlations between certain perceptual dimensions, motivating using a single regression model for all 16 social attributes to be predicted for a set of speakers appearing in recorded video clips. We demonstrate that dimensionality reduction through Principal Component Analysis (PCA) can be applied to the label space without a relevant performance loss. Secondly, by employing frozen MM-LLMs as feature extractors, we explore their ability to capture perception-related information. We extract sequence embeddings from the Qwen-VL and Qwen-Audio models and train a Multi-Layer Perceptron over the attention-pooled vectors for each one of the encoders, obtaining a mean Pearson correlation of 0.22 using the average predictions for both models. Our best result of 0.31 is achieved by training the same architecture over the baseline vit-ver and w2v-msp features, which motivates further exploration on how to effectively leverage advanced MM-LLMs as feature extractors. Lastly, a post hoc analysis of our results highlights the limitations of Pearson correlation for evaluating regression performance in this context. In particular, a similar Pearson coefficient can be obtained with two very different prediction sets displaying different levels of variability. We take this result as a call to action in exploring alternative metrics to assess the regression performance for the task.

This paper summarises THAU-UPM's approach and results from the MediaEval 2023 Predicting Video Memorability task. Focused on the generalisation subtask, our work leverages a pre-trained Video Vision Transformer (ViViT), fine-tuned on memorability-related data, to model temporal and spatial relationships in videos. We propose novel, annotator-independent automatic segment selection methods grounded in visual saliency. These methods identify the most relevant video frames prior to conducting memorability score estimation. This selection process is implemented during both training and evaluation phases. Our study demonstrates the effectiveness of fine-tuning the ViViT model compared to a scratch-trained baseline, emphasising the importance of pre-training for predicting memorability. However, the model shows comparable sensitivity to both saliency-based and naive segment selection methods, suggesting that fine-tuning may harness similar benefits from various video segments. These results underscore the robustness of our approach but also signal the need for ongoing research.

The memorability of a video is defined as an intrinsic property of its visual features that dictates the fraction of people who recall having watched it on a second viewing within a memory game. Still, unravelling what are the key features to predict memorability remains an obscure matter. This challenge is addressed here by fine-tuning text and image encoders using a cross-modal strategy known as Contrastive Language-Image Pre-training (CLIP). The resulting video-level data representations learned include semantics and topic-descriptive information as observed from both modalities, hence enhancing the predictive power of our algorithms. Our proposal achieves in the text domain a significantly greater Spearman Rank Correlation Coefficient (SRCC) than a default pre-trained text encoder (0.575 ± 0.007 and 0.538 ± 0.007, respectively) over the Memento10K dataset. A similar trend, although less pronounced, can be noticed in the visual domain. We believe these findings signal the potential benefits that cross-modal predictive systems can extract from being fine-tuned to the specific issue of media memorability.