Since ‘hiring’ my male and female bots as greeters at Basilique, I’ve been curious to see how they compared, and now – after a 2-month serial A/B test – I have the answer: Juliet is better than Paul at converting visitors into group members – but just barely.
I have to admit, I predicted that Juliet would beat Paul by a higher margin (about 2 to 1) – but the difference was much less significant than I expected.
On one hand, you might ask: Why would there be a difference? They’re bots – they’re not even really male or female, they’re only software that appears different. The same, of course, could be said for any avatar after only a few moments of textual exchange. One can never really know an avatar driver’s gender, regardless of their appearance inworld. Our brains just fill in the gaps, and quickly too. When it comes to avatar appearance, if it looks like a duck, and sounds like a duck, then it’s probably a duck.
On the other hand, you might have expected – as I did – that a female would outperform a male in this job – based on gender alone. After all, isn’t this why most people in (non-managerial / non-professional) sales and customer service roles are women? In the UK, women account for
- 93% of receptionists
- 73% of food servers
- 69% of sales assistants and retail cashiers
- 63% of telephone salespeople
Is this because women are naturally better at these jobs than men? Well, that might be a contributing factor (and there is evidence to suggest that there is a hint of truth in that theory), but there’s a lot more to it than that – including occupational preferences, difference in availability for work, family caregiving demands, opportunity, and bias in hiring and career development.
I think it’s also likely that there are many of us – no matter how progressive we might imagine ourselves to be – that prefer men and women in stereotypical roles. We might feel more comfortable with women in low-level service positions, because that is what we’re accustomed to in real life.
In both worlds, I would predict that people of either gender (and sexual orientation) are more open to invitations offered by females than they are to those offered by males. When it comes to responding positively to Second Life group invitations, this data appears to marginally support that hypothesis.
As I shared in my previous post on the subject of using bots for region task management, I was astonished by Paul’s early results in converting visitors into group members:
One exciting result over the past week has been his effect on new memberships. Over the past 4 days since I first got him inviting visitors to join Basilique, he’s sent out 357 invitations to visitors landing at Basilique (that were not yet members) and has converted 134 people into new members. That’s a conversion rate of 37.5%, which any sales person (in any world) will tell you is very high for cold calling! No one, including me, has ever been able to get as many new Basilique members involved as Paul has!
In retrospect, Paul outperformed my human host by a country mile. Why? First of all, Paul works 24/7, without pausing even for a nature break. Second, he invites everyone that visits (that isn’t already a member), not just those in his line of sight or those that look approachable to him. Paul believes in 100% equal opportunity, and he does not fear rejection. Lastly, he can carry on dozens of conversations in IM at once without missing a beat, so he never fails to give every visitor equal attention.
These early results demonstrated – at least in this case – that a robot was much more productive as a group inviter than a real human. Paul was able to lift Basilique membership from roughly 600 to 1270 in about 30 days. Conversely, my human host managed to generate about 200 net members over a span of roughly 4 months.
After Paul’s brilliant success (and the many positive comments I got from his many new fans), I decided to hire his friend Juliet.
During the test period, I didn’t do anything significantly differently at Basilique. We held similar events, in similar venues, attended by similar people. Yet, there was a difference in visitor conversion.
Juliet is different to Paul in many ways. Obviously, she’s a youthful and attractive female. What you can’t see from these pictures is that has very sexy AO that brings the best out of a very shapely body.
Paul staffs Harvey’s Bar, whilst Juliet works at Silky’s Café. Both locations are very close to the landing point, and both are clearly visible from the main square. During the study period, Paul dressed in a suit, while Juliet dressed in casual clothes that accentuate her curvaceous figure.
Apart from appearances and location, that is where the differences end. They used exactly the same invitation IM message, delivered exactly 2 minutes after a visitor first landed, and responded to guests by using exactly the same chat AI.
Over the study period, Juliet converted 699 of 3,453 visitors into members (20%) while Paul converted 731 of 3,892 visitors to members (19%).
Indeed, Paul generated more net members from visitors, but there were more visitors during the time in which I measured his performance (Feb 23 – March 23) compared to the time I measured Juliet’s performance (March 24 – April 23).
At the end of the test, Juliet performed 8% better than Paul.
After a few calculations, I am 94% confident that having Juliet invite my visitors to become members will result in a higher conversion rate. It’s questionable, however, whether these results are statistically significant.
How can we interpret these results?
Could conversion numbers be swayed by a bot’s appearance alone? If not, how else can one explain a difference in performance?
One could argue that implied gender and appearance was the difference that made the difference. Perhaps an implied female presence is less threatening than a male one? Perhaps she is generally more attractive than Paul, which gives her an edge?
Advertising research into male and female voice-overs in ads tells us that “half of Americans (48%) believe a male voice is more forceful while 46% believe a female voice is more soothing”. With that said, only 7% of people say that a female voice is more likely to sell them a car or a computer (compared to 23% saying a male voice is more likely to do so).
Still, research into tipping confirms what most of us already know: “that more attractive waitresses get higher tips than less attractive waitresses, regardless of the level of service. Waitresses with larger breasts get higher tips. Waitresses with blond hair get higher tips. Tips increase as a waitress’ body size decreases. Waitresses who wear makeup receive higher tips from male customers but not from female customers.” (source)
I’d love to know the division of genders that visited Basilique – and whether Paul or Juliet had any greater success with either group. Names and real life identities being as cryptic as they are, however, would render any insights I could draw from those findings to be speculative at best.
There are many more areas that I’m keen to explore. For my next tests, I might experiment with different styles of dress, different locations, different ages, and different message content. I’m also keen to do content analysis on their IMs for trends and keyword phrases.
Hey, I’m a research geek – it’s what I do. 🙂
Are you surprised by these results? Did you, like me, imagine they’d be different? Why or why not? What further things would you suggest I test?