Psychological Factors Consumer Decision Making e.g., Impulsiveness, openness e.g., Buying choices Personalization 1. 2. 3. Increase click-through rate predictions Enhance recommendation quality Improve user experience
Personality in Computational Advertising A Benchmark Giorgio Roffo 1 & Alessandro Vinciarelli 2 1 University of Verona, IT 2 University of Glasgow, UK Vision, Image Processing & Sound Lab 4th Workshop on Emotions and Personality in Personalized Systems (EMPIRE)
Outline Ø Introduction Ø Participant Data Big Five Personality Users Favourite Ø Experimental Setup Ad Click Prediction Ad Rating Prediction Ø 3
Introduction Introduction Ø Shopping online plays an increasingly significant role in our lives Identify a need Collect Information Consumers decision making process Evaluate Alternatives Make a purchase decision Ø Personality influences people s behavior and interests [3,4] Ø Personality is increasingly attracting attention User Profiling [1] Recommender Systems [2] [1] R. Gao et Al, Improving user profile with personality traits predicted from social media content, In Conf. RecSys, 2013. [2] M. Tkalcic and Li Chen, Personality and Recommender Systems, In Recommender Systems Handbook, 2015. [3] C. A. Turkyilmaz et Al., The effect of personality traits and website quality on online impulse buying, In Conf. ICSIM 2015. [4] M Lauriola et Al., Personality traits and risky decision-making in a controlled experimental task: An exploratory study, In Personality and Individual Differences, 2001. 4
Introduction Ø A new dataset for Ads Recommendation Ø A collection of 300 real advertisements Ø Display formats: Rich-Media Ads, Image Ads, Text Ads 5
Introduction (2) Ø ADS is fully annotated Ø 120 participants Ø Ads voted according to if users would or not click on them Ø Click / No Click Labels Clicked = rating greater or equal to 4 No-Clicked = rating less than 4 6
Introduction (3) Ø Ads belong to 20 product/service categories Ø 15 Ads for each category 7
Introduction Participant data Ø User Preferences (from a predefined list of categories) Ø Most visited websites Ø Most watched films Ø Most listened music Ø Ø Favourite sports Ø Hobbies Ø Travel destinations Ø Demographic information (from a predefined list of choices) Ø Age, gender, nationality, home-town Ø job, type of job, monetary well-being 8
Introduction Big Five Personality Ø ADS is enriched with users psychological factors [1] Openness to experience (O) Conscientiousness (C) HIGH LOW Creative, Curious Conservative, Traditional High Responsible, Unreliable, Organized Easy-going Extroversion (E) Assertive, Talkative Timid, Reserved Agreeableness (A) Co-operative, Trusting Cold, Antagonist Neuroticism (N) Nervous, Insecure Calm, Secure [1] B. Rammstedt, Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German Journal of Research in Personality, 2007. 9
Introduction Big Five Personality (2) Ø The effect of B5 traits on attitudes toward advertisements [2] Openness to experience (O) Conscientiousness (C) HIGH Creative, Curious High Responsible, Organized ADS Supports technological innovation Information gathering and detailed processing Extroversion (E) Assertive, Talkative Attitude toward transformational ads Agreeableness (A) Co-operative, Trusting Attitude for non-comparative ads Nervous, Attitude for Neuroticism (N) Insecure comparative ads [2] D. Myers et al, The moderating effect of personality traits on attitudes toward advertisements: a contingency framework, Management & Marketing Challenges for Knowledge Society, 2010 10
Introduction Users Favourite Ø ADS contains more than 1,200 personal pictures (labelled as positive/negative) Negative Positive Ø Sentiment Analysis Inferring personality from favourite users pictures [1] Incorporating affective-like features coming from users favourite pictures [2] [1] Pietro Lovato et Al, Faved! Biometrics: Tell Me Which Image You Like and I'll Tell You Who You Are, Transactions on Information Forensics 2014 [2] C. Segalin et Al, The we Like are our Image: Continuous Mapping of Favorite into Self-Assessed and Attributed Personality, Transactions on Affective Computing, 2016 11
Introduction Experimental Setup (1) Ø X set of observations X = x 1 x 2 x N Ø N = 120 subjects Ø x i feature vector for subject i Ø 1-of-K representation f 1 = Favorite websites, f 8 [0,1] f 2 = Favorite films, f @ [0,1] f 3 = Favorite music, f D [0,1] x i = f 1, f 2, f 3, 12
Introduction Experimental Setup (2) Ø Experiments were performed at Category Level Ø Balanced click distribution over the 20 categories 1,229 clicked instances 1,171 not clicked instances Click/No-Click Ratings 1-5 x i = f 1, f 2, f 3, y i = x i w i Regression f B5 13
Introduction Ad Click Prediction Training Testing Ø Task: Recommend advert on which a user most probably clicks U I8 1 0 0 0 1 0 1 1 0 1 0 0 1 1 1 0 1 0 1 1 U I88J 0 1 1 0 1 0 0 0 1 0 1 1 0 1 0 1 1 0 1 1 U I888 0 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 0 1 1 0 U I8@J 0 0 1 1 1 1 1 0 0 1 0 1 0 0 0 0 1 1 0 1 10-fold cross-validation Ø 120 users: 12 folds of 10 users each Ø Training on 11 folds (110 users) Ø Testing on 1 (10 users) 14
Introduction Ad Click Prediction Ø Accuracy Measure: Ø Area Under the ROC Curve (AUC) tpr AUC ROC-Curve fpr Ø Logistic Regression (without-big5 features): 51.9% Ø Logistic Regression (+Big5 features): 53.4% +1.5% 15
Introduction Ad Rating Prediction Ø Task: Predict the feedback a user would give to an advert (from 1 star to 5 stars) Training U I8 U I88J 2.5 3 2 2.3 3.5 2.4 5 4.5 4.2 3.3 2.5 4.6 2.1 3.8 3.1 2.1 1.1 2.3 1.1 1.1 3.5 3.6 2.2 2.4 4.5 5.6 3.2 4.2 2.9 4.8 3.8 2.7 4.9 5 3.3 2.8 1.5 2.8 3.2 1.9 Testing U I888 U I8@J 2.5 3 3.8 3.1 2.1 1.1 2.3 3.6 2.2 2.4 4.5 5.6 3.6 4.2 2.3 3.5 2.4 2.3 3.5 2.4 5 4.5 4.2 3.3 2.5 4.6 2.1 4.2 2.9 4.8 3.8 2.7 4.9 4.5 5.6 3.2 4.2 4.5 3.2 1.9 10-fold cross-validation 16
Introduction Ad Rating Prediction r Q,8 r Q,@J R u I888 u I8@J 2.5 3 3.8 3.1 2.1 1.1 2.3 3.6 2.2 2.4 4.5 5.6 3.6 4.2 2.3 3.5 2.4 2.3 3.5 2.4 5 4.5 4.2 3.3 2.5 4.6 2.1 4.2 2.9 4.8 3.8 2.7 4.9 4.5 5.6 3.2 4.2 4.5 3.2 1.9 u I888 2.5 3 3.8 3.1 2.1 1.1 2.3 3.6 2.2 2.4 4.5 5.6 3.6 4.2 2.3 3.5 2.4 2.3 3.5 2.4 RX u I8@J 5 4.5 4.2 3.3 2.5 4.6 2.1 4.2 2.9 4.8 3.8 2.7 4.9 4.5 5.6 3.2 4.2 4.5 3.2 1.9 r Q,8 r Q,@J Ø Accuracy Measure: Root Mean Square Error RMSE = 8 ( L (Q,R) L r Q,R r Q,R ) @ Ø Logistic Regression (without-big5 features): 0.20 Ø Logistic Regression (+Big5 features): 0.16 Ø T-test (p-value < 0.01, Lilliefors Test H=0) 17
Introduction Ø A new dataset for Ads Recommendation Ø A collection of 300 real advertisements Ø 120 unacquainted users Ø Users Preferences Ø Demographic information Ø Personality traits (Big-Five Factor Model) Ø Users Favourite (traits inference) vdownload page: http://giorgioroffo.it/datasets/ads16-main-download.zip Password: EMPIRE2016 Size: 751MB 18
Thank you for your attention v Download page: http://giorgioroffo.it/datasets/ads16-main-download.zip Password: EMPIRE2016 Size: 751MB Vision, Image Processing & Sound Lab
20
Extra Slides Within-Subject User Study Ø Clustering the Big-Five traits by Affinity Propagation Figure 2. Spider-Diagrams for O-C-E-A-N Big-Five traits 21
Extra Slides Within-Subject User Study (2) Supports technological innovation Strong willingness to buy products online Attitude toward transformational ads 22
Evaluation Methodology Evaluation Methodology Ø Ad click Prediction: Predict adverts on which a user most probably click Recommended Not recommended Clicked True-Positive (tp) False-Negative (fn) Not clicked False-Positive (fp) True-Negative (tn) YZ Precision = YZ[\Z Recall (True Positive Rate) = False Positive Rate = \Z \Z[Y] YZ YZ[\] Area Under the ROC Curve (AUC) tpr AUC ROC-Curve fpr 23
Evaluation Methodology Evaluation Methodology Ø Ad Rating Prediction: Predict the feedback a user would give to an advert (from 1 star to 5 stars) RMSE = 8 L (Q,R) L ( r Q,R r Q,R ) @ (Root Mean Square Error) MSE = 8 L (Q,R) L ( r Q,R r Q,R ) @ (Mean Square Error) MAE = 8 L (Q,R) L r Q,R r Q,R (Mean Absolute Error) 24
Experiments and Results Ad Click Prediction Ø Results show the effect of B5 on Ads recommendation Table 5: Performance for ad click prediction (F-Measure). Big-Five features systematically contribute to the overall performance. Ø Fraction of recommended Ads that are clicked (Precision) Ø Fraction of clicked Ads that are recommended (Recall) 25
Experiments and Results Ad Rating Prediction 26
Extra Slides Ad Rating Prediction (2) 27