Personalising Messages to User Characteristics

We decided to exclude reciprocity and scarcity from the follow-on studies. Only 2 reciprocity messages were validated with Kappa ≥ 0.4, and these were positive and negative framings of different message contents, making them hard to use for comparison in follow-up studies. On reflection, we also felt that reciprocity is hard to apply in a system, as it requires a plausible favour (a message that validated was “We have spent a lot of effort and money in organising this “How to Eat a Healthy Breakfast” workshop. We will be disheartened if you don’t eat fresh foods for breakfast.”). Whilst 4 scarcity messages were validated with the reasonable agreement (Kappa ≥ 0.4), none validated with Kappa ≥ 0.7. Additionally, scarcity is also hard to use by an interactive healthy eating coach in a persuasive message, as messages such as - This is your last chance to eat your cereal with fruit and nuts today and replenish your body with important nutrients, may not be plausible in real life1.

Study 2: Adaptive Message Selection

Our next study investigates the relationship between message properties (Cialdini principle and framing) and people’s personality on the one hand and message persuasiveness on the other. We used a mixture of a within and between study design. Each participant saw all messages, using the four Cialdini principles and the two framing types. Personality was used as a between subject variable.

Participants. Participants were recruited through Amazon Mechanical Turk. As before, they had to be based in the US, have an acceptance rate of 90%, and have passed a Cloze test for English fluency. Participants were paid $1.5. 152 participants took part: 89 males (22 aged 18–25; 48: 26–40; 19: 41–65); 62 females (8: 18–25; 35: 26–40; 18: 41–65; 1 over 65); and 1 undisclosed (aged 26–40).

Procedure. Participants provided their gender and age (which were optional). In addition, their attitude and behaviour towards food were examined by two questions. Next, we used a brief personality test applying personality sliders intended for the Five-Factor Model. The participants were displayed stories created by Dennis et al. (2012) that portray two extreme personalities (low and high) for each trait (see Figure 1). The participants specified how similar they were to these personalities by moving the slider. This produced a score for each trait out of 180. This personality measuring tool was previously validated as correlating with the 40-item mini markers test for the Five-Factor Model as previously used by Smith et al. (2016, 2019).

Next, participants imagined a fictional person ‘Alex’ who resembled their personality. They rated the 16 messages in random order on how persuasive they would be for Alex using 4 criteria: motivational, effectiveness, appropriateness, and convincing which were averaged for analysis (See Figure 2). They were told that the messages would be used to encourage Alex to eat healthy breakfasts.

Effect of Cialdini’s Principles on Perceived Persuasiveness. AUT was the highest-rated principle while LIK was the lowest. A one-way repeated measures ANOVA on principles showed a significant effect, F(2.76, 416.10) = 158.75, p < 0.001. AUT was significantly more persuasive than the other principles (p < 0.001) and LIK was significantly less persuasive (p < 0.001). There was no significant difference between COM and CON. This supports the hypothesis (H1) that perceived persuasiveness ratings differ for the different principles.

Effect of Framing on Perceived Persuasiveness. To test H2, we compared the persuasiveness of both framings of all the message pairs. On average, positively framed messages were rated higher (M = 3.03, SD = 1.15) than negatively framed ones (M = 2.56, SD = 1.17). This difference was significant, t(1215) = 17.78, two-tailed p < 0.001. This supports the hypothesis, i.e., H2 that peoples’ ratings of perceived persuasiveness differ for different framing.

Effect of Personality on Perceived Persuasiveness of Cialdini Principles. A one-way repeated measures ANOVA with principles as the within-subjects variable and personality traits as between-subjects variables showed that there is a significant effect of trait level for Conscientiousness (F(1, 147) = 6.73, p = 0.01), with higher persuasiveness for the high conscientious group than for the low conscientious group. There was no interaction effect with principles, though participants with high Conscientiousness seemed to rate COM messages higher. There were no significant main effects of trait level for the other traits. However, there was a trend for Openness (p = 0.07) and Extraversion (p = 0.06), with the low openness group and the high Extraversion group having higher persuasiveness ratings. There was a significant interaction effect of Cialdini’s principles with the Openness trait level on persuasiveness (F(2.79, 400.10), p = 0.047). Participants with low Openness rated both CON and LIK messages higher than participants with high Openness, as illustrated in Fig. 5(d). This provides some support for hypothesis H1a that perceived persuasiveness ratings for different principles vary depending on personality.

We also conducted a correlation analysis using the original numerical values for the traits (see Table 3). It shows many significant but small correlations between personality traits and Cialdini’s principles. This further supports H1a.

For each trait, for the low and high values, a one-way repeated measures ANOVA showed a significant main effect of Cialdini’s principles (p < 0.001). Pairwise comparisons showed that AUT was significantly more persuasive than the other principles (p < 0.001) and LIK significantly less (p < 0.001). There was no difference between COM and CON. This indicates that the results of the analyses of the whole dataset (related to H1) are still valid independent of personality.

Effect of Personality on Perceived Persuasiveness of Framing. A one-way repeated measures ANOVA with framing as the within-subject variable and trait level as the between-subjects effects showed that there is a significant effect for Conscientiousness (F(1, 147) = 6.73, p = 0.01). Participants with high Conscientiousness rated both the framed messages much higher than those with low Conscientiousness. This is similar to what we found for Conscientiousness above, so highly conscientious people are more easily persuaded.

For both, low and high trait values, a t-test showed a significant effect of framing (two-tailed, p < 0.001). Participants tended to rate the persuasiveness of positively framed messages higher than negatively framed ones. This supports our earlier findings for H2.

It is interesting to note that high Emotional Stability participants rated negatively framed messages a little lower than low stability ones, but overall, both groups rated positively framed messages higher than negative ones.

There were significant interaction effects between personality and framing for the trait level of Conscientiousness (F(1, 147) = 5.30, p = 0.02) and Emotional Stability (F(1, 148) = 5.12, p = 0.03), see Fig. 6. There was a trend for participants with low Openness rating both positively and negatively framed messages higher than participants with high Openness, however, this was not significant. This supports hypothesis (H2a) that peoples’ ratings of perceived persuasiveness for framings vary depending on the personality of participants.

Interaction Effects between Personality, Cialdini’s Principles and Framing. All positively framed messages for all Cialdini’s principles were rated higher than the corresponding negatively framed ones. A two-way repeated measures ANOVA on Cialdini’s principles and framing showed that there is a significant interaction effect between Cialdini’s principles and framing, F(2.85, 429.91) = 4.24, p < 0.01. This supports hypothesis H3a.

A two-way repeated measures ANOVA on Cialdini’s principles and framings with personality traits as Between-Subjects Effects showed that there is a significant 3-way interaction effect between Emotional Stability, Cialdini’s principles and framing, F(3, 444) = 2.74, p = 0.04. Figure 5(c) shows hardly any difference in the messages rated by participants with high and low Emotional Stability on Cialdini’s principles, but the statistics indicate that there is a difference when framing is also taken into account. This supports hypothesis H3b.

Best message type. Authority (AUT)messages were favoured over Commitment and Consistency (COM), Consensus (CON) and Liking (LIK). A chi- square goodness-of-fit test showed there was a significant effect for the best message type, x² (2)= 132.74, p < 0.05. This supports hypothesis H4.

Best message framing. Positively framed messages were favoured over negatively framed ones. A chi-square goodness-of-fit test showed there was a significant effect for the best message type, x² (2)= 44.11, p < 0.05. This supports hypothesis H4.

Interaction between best message type and best message framing. A Fisher’s exact probability test resulted in a borderline significant relationship between best message type and the selection of framing, two-tailed Fisher’s exact yielded p = 0.05. This supports hypothesis H6.

Impact of personality on best message type and best message framing. A Fisher’s exact probability test resulted in a significant overall effect of personality on best message type for Agreeableness, two-tailed Fisher’s exact p < 0.05. This supports hypothesis H4.

Impact of gender on best message type and best message framing. A Fisher’s exact probability test resulted in no significant overall impact of gender on best message type, two-tailed Fisher’s exact p > 0.05. In addition, a chi-square test showed that males and females did not differ on preference for the best message framing, x² (2)= 1.71, p > 0.05. This does not support hypothesis H5.

Impact of age on best message type and best message framing. A Fisher’s exact probability test resulted in no significant overall impact of age on best message type, two-tailed Fisher’s exact p > 0.05. In addition, a chi-square test showed that males and females did not differ on preference for the best message positive or negative framing, x² (2)=0.76, p > 0.05. This does not support hypothesis H5.

Interaction between age and gender. A chi-square test resulted in a significant effect of gender on best message framing for the ages between 26 to 40, x² (2)= 6.97, p <0.05. This supports hypothesis H6a.

Interaction between personality and age. A Fisher’s exact probability test resulted in a significant effect of age on best message type for participants with high Extraversion (see figure 5) and high Openness, two-tailed Fisher’s exact p < 0.05. This supports hypothesis H6c.

Interaction between personality and gender. A Fisher’s exact probability test resulted in a significant effect of gender on the best message type for the highly agreeable participants, two-tailed Fisher’s exact p < 0.05 (see figure 4). While there was a significant effect of gender on best message framing for the low conscientious and low emotionally stable participants, two-tailed Fisher’s exact p < 0.05. This supports hypothesis H6b.

Interaction between personality, age, and gender. Overall, there were no significant effects between personality with best message type and best message framing for gender and age. This does not support hypothesis H6d.