The Sun’s Facebook chatbot drove nearly half users back to its site

“We were keen to test on small niche audience,”said Emma Fulton, director of digital strategy and operations at The Sun. “It proved what we thought was correct: Bots will work better when they are more targeted.”

The bot sent a daily round-up of five of the most important stories at around 5 p.m. to catch the rush hour. Readers could then subscribe to receive updates from their club, of which 95 percent of people did. During the four-week window, The Sun sent 263 notifications about specific clubs, mostly relating to the big three — Arsenal, Chelsea and Manchester United — which linked through to the story on the site and were published based on the news cycle. Successful pieces included reporting on Lucas Perez signing a four-year deal with Arsenal and this one on Marcelo Brozovic rejecting Chelsea’s bid in favor for Inter Milan. The number of notifications The Sun sent out increased in frequency as the transfer deadline drew closer.

The number of people who subscribed to updates from the chatbot was in the low thousands; The Sun’s football Facebook page, by contrast, has 2.6 million followers. But The Sun was shooting for engagement rather than mass scale. On the transfer deadline day, when the number of notifications reached its peak at 41 team-specific stories, 43 percent of chatbot subscribers clicked through to the main site. On an average day during that month, this was closer to 23 percent.

“It underlines the fact that publishing more messages was not putting people off, instead it was driving much more engagement,” added Fulton.

Although, as Josh Robinson, head of digital at Synergy points out, people are opting in. “Naturally, engagement is high because an individual is choosing to engage with it and the experience is (or at least feels) personal. The traction of this personal experience carries through into high conversion.” While the creative potential is there, publishers will be under pressure to deliver more than automated feeds.

Facebook analytics are limited; publishers can’t see when a message is opened or whether people click through. Instead, analytics are inferred by the number of people visiting the site from the messenger platform. The mobile click-through rate from the number of subscribers was 87 percent, dropping to 8 percent for tablet and 5 percent for desktop.

Return rate was also a key metric for The Sun, which found that 61 percent of people returned to the site within seven days of their first visit. Overall 3 percent of the subscriber base unsubscribed, Fulton explained this happened early on in their experience after their first round-up piece of news; virtually no one unsubscribed after the second week.

The Sun pre-loaded the bot with just a few hundred possible responses, so it’s not capable of high levels of interaction with users. Instead, the messages are functional, explaining how users can subscribe and manage subscriptions. For a bot on celebrity gossip, Fulton believes artificial intelligence will help create a more conversational tone.

In the coming weeks, the bot will begin publishing general football news, like match stories or player injuries. It will have to be more selective in the number of notifications so it doesn’t spam users, either by introducing more levels of personalization, like allowing people to follow news on a particular player, or by publishing fewer stories. For some big match days, sending five messages will be acceptable; on an average Wednesday, sending five is too many.

“It’s been brilliant to prove it’s a platform we can use to engage with an audience,” added Fulton. “The payoff for the effort for the social team compared to the engagement we’re getting is really working; the bot really makes sense as an ongoing part of our business.”