August 23, 2017
Among the best at this has been Netflix. Their service reflects the results of various A/B user tests that have been conducted. There are plenty of links to reports on this but some of the most notable tests helped Netflix identify strategies around content image styles, initial main pages before subscription, content detail pages with trailers’ playing and thumbs up/thumbs down ratings.
When
it comes to pay TV, we believe that A/B testing increases the value of cloud-based
products like our Ambient TV by minimizing
the need for operators to build test-beds and recruit test users. Like Netflix,
we understand that the transparency to consumers of tests conducted within the
actual service can yield more accurate results. The service environment and
system infrastructure where Alticast’s Cloud UI is built is not much different
from that of Netflix, so what are some of the lessons that are helping us match
Netflix’s quality of A/B testing? Here are a few:
·
First, it’s important to prevent technical obstacles to
distract from the desired test. To avoid issues of slow response times, Netflix
optimized the loading speed of content lists on both TV and mobile screens. To
be exact, it increased the loading speed on mobile screens. In case of TV, Netflix
uses the need for the remote control to buy time to load lists across the lower
pane of the screen. That strategy wouldn’t work on mobile screens, which
require faster loading speeds as page scrolling is rapidly done by simple flicks
of viewers’ finger tips.
·
Second, it’s about test participants. Traditionally, less
than 0.1% of service users have been recruited and limited tests have been performed.Cloud UI liberates operators from those restrictions so they can make the size
of user groups and the types of tests as flexible as required. Moreover, the
operator does not need to take pre-cautions in categorizing before the testing
starts. The users can be classified during the course of the testing process
when certain characteristics are discovered. Such information can be critical
in producing insights.
·
Lastly, more users mean more analyzable data. In the past, the analysis could be done manually because
of the small numbers within the test groups. With results
data now being generated by thousands of users, it is essential now that data
platforms are automated to enable faster and more precise data aggregation and
analysis as the result data are generated by thousands of users. Without
automated data analysis, high-quality, real-time A/B testing would be impossible.
To
sum up, A/B testing is highly valuable when it is able to run diverse test
cases flexibly and simultaneously, as it is in cloud-based platforms such as
Alticast’s Ambient TV. Cloud UIs that are integrated with automated data
platforms ensure rapid testing, accuracy in handling collected data and the
ability to generate and apply testing materials for immediate use in optimizing
the value of the product for the viewer.
Posted by Jill OToole at 1:40 AM