To grow subscribers, The Wall Street Journal focuses on product testing

When the team launched with two people in 2015, it was focused purely on improving subscriber conversions at the Journal and the following year at Barron’s. By 2017, it had started working on other outcomes like increasing consumption. After the team scored several subscription wins, it started working this year on improving time spent on site, subscriber retention and event attendance. It runs dozens of tests a month on tasks like getting more email newsletter registrations.

“Digital experimentation is something you can do anywhere,” said Peter Gray, the Journal’s vp of optimization.

The optimization team looks for ways to make small changes that impact the largest possible audience. That strategy is girded by simple math: A product change that’s seen by 5 percent of a publication’s audience needs to improve outcomes by 200 percent to get the same results as a change seen by 100 percent of an audience that improves outcomes by 10 percent.

While personalization has its merits, Gray said, his team operates under the assumption that there are more 10 percent wins out there than 200 percent ones. “It’s a distraction to solve for a smaller piece of your audience,” Gray said.

While the Journal has weighed radical changes for subscribers, such as getting rid of ads, Gray’s team focuses on product changes that can be implemented as quickly and cheaply as possible, like the copy in existing marketing messages, their placement and design. Of the adjustments that improved outcomes by more than 4 percent, over two-thirds involved text or design.

“If you’re just playing around with words and copy and design, no one from the business side is going to have trouble with that,” Gray said.

For example, the team improved newsletter signups 24 percent by replacing a tile-based format with a list-focused one. It also improved app signups by 23 percent by getting rid of a flashy welcome screen that didn’t communicate the value of the Journal.

Adopting a data-driven approach doesn’t mean abandoning the creative processes of the past, Gray emphasized.

“Suddenly, all these people who had to rely exclusively on their gut, they’re worried that [their creative] agency will go away when in fact there’s more creativity required,” Gray said. “You have to create four solutions, and you let them speak.”

Other publishers, including The Economist to The Atlantic to the Seattle Times, have adopted collaboration and a data-driven approach to grow subscriptions, a trend that is poised to continue.

“Historically, there’s been this separation of church and state between the editorial side and the product-marketing side,” said Matt Lindsay, the founder of Mather Economics. “But as subscriber revenue becomes the primary revenue stream, the product will be driven a lot more by the preferences of the audience.”

Update: An earlier version of this story said the Wall Street Journal had a goal of reaching 3 million subscribers. It was Dow Jones & Co. that had the goal.