The road to strategic website design

Business white paper
The road to
strategic website
design
The Optimost Web Optimization Maturity Model

Business white paper | HP Web Optimization Maturity Model
Digital marketers spend large sums attracting website traffic.
Much of that investment is wasted. Traffic continues to convert
at woefully low rates. Visitors repeatedly fail to engage.
Effective marketing in the digital age is no longer based on gut feel, accumulated experience,
or the highest paid person’s opinion; it’s based on customer data. It requires companies to put
optimization and testing at the center of web design and development processes.
Many organizations have already taken the first steps to truly understanding what works best
for their customers. They have realized the value of listening to customer data, leveraging inhouse
or basic self-service testing tools to identify the website design changes most likely to
improve lift, revenue, or engagement.
Yet unfulfilled promises of much higher, sustained return on investment from basic optimization
have led these organizations to conclude that on its own this is not enough. They need a
comprehensive optimization program that not only identifies the most valuable changes, but
also delivers the necessary know-how, secures the essential budget, and provides the leverage
to enable them to act effectively.
The Optimization Maturity Model
We believe that in the next few years, every single website change will be driven by customer
behavior through a conversion rate optimization program. Every release will be tested against
a hypothesis, all content will be personalized to the visitor, and optimization teams will be profit
centers in their organizations.
Assess your
Web Optimization
Maturity with our
interactive tool at
optimost.com
As they work towards reaching this goal, organizations are asking how mature they are
compared to the highest performing peers of their industry. They need to know if they are
doing the right things, if they are doing them right, and the route they need to take to climb
up the ladder.
With no clear answers to those questions, we developed our Optimization Maturity Model.
Whether you have no optimization experience, do a small amount of optimization in-house
using your own technology, or are currently leveraging a self-service software package for
conversion optimization, the maturity model is designed to help you quickly benchmark your
organization against your industry peers. By understanding where your organization stands you
can prioritize the issues you need to address.
Our Optimization Maturity Model contains four dimensions that together describe how
organizations mature. Based on the longest experience in the conversion optimization industry,
they indicate the areas an organization should focus on to help it succeed in generating a higher
return on their website investments:
• Type of testing
• Content tested
• Segmentation
• Organization.
Understanding testing types
Effective optimization starts with effective testing. The number and types of test an
organization runs gives an indication of an organization’s optimization effectiveness.
As an organization matures it increases not only the velocity of its testing, but also the
quality and complexity.
A/B testing is less advanced than multivariate testing (MVT). MVT can pack multiple different
variations into a single test, such as whether button A in combination with banner C or button
B in combination with banner A gives a better lift. You are going to find out far more than by
running the button and banner tests individually. The more permutations you can test, the
more likely you are to find the perfect combination.
2

Business white paper | HP Web Optimization Maturity Model
So if one customer is running ten A/B tests and one MVT and another is running one A/B test
and seven MVTs, even though the first has a higher volume of tests, the second is probably
doing more advanced optimization. This clear sign of maturity requires not only a greater
volume of traffic to distinguish across the different variations, but also sophisticated
optimization tools able to generate the data required and expert analysts to interpret it.
Types of content tested reflects maturity
At their basic level tests simply compare text or image variations. Should a button say ‘Buy’
or ‘Buy now’? Does a banner work best in green or blue? Do customers respond better to this
copy or that? Immature organizations focus most of their testing on simple comparisons such
as these. The investment required is minimal. The insight is valuable – but there is a lot more
to learn.
As an organization matures it focuses more of its optimization effort on interpreting customer
data that provides a deeper understanding of visitor behavior and desires. It is able to make
more strategic design decisions – decisions around page layout, page flow, dynamic content,
and the critical pages that are direct drivers for revenue.
Curiosity or a usability study, for example, might drive organizations to invest in tests that are
more complicated to set up, such as tests that examine page layouts. Does the navigation bar
work best on the left or at the top of the page? Should we size page elements so you don’t need
to scroll?
But that insight isn’t always enough. For example, a company considering eliminating unpopular
products from its pages to avoid distracting potential buyers needs to figure out the impact
of doing that on order sizes, propensity to buy, and – ultimately – revenue. For this it needs to
track and test metrics across multiple pages. This complex task is extremely valuable when you
get it right.
More generally, testing across multiple pages provides valuable insight into how a visitor arrived
at a page, where they are going and the impact of that. Testing the flow of an entire checkout
funnel, for example, illuminates the various paths people take and where they’re most likely to
become distracted and leave the checkout funnel. It can help answer business questions such
as ‘How could we recapture lost sales?’
Testing dynamic content is about testing pages where the content isn’t fixed. Product
recommendations, for example, might reflect what the visitor has just clicked on or put in their
cart. Backend systems decipher what best to show them. If Peter likes Manchester United and
Jane likes Detroit Tigers, you want to promote football merchandise to Peter and baseball
merchandise to Jane.
Testing that can be complicated, requiring testers to consider every possible template, every
possible use case, and every possible scenario. Getting it right requires sophisticated tools and
expert analysis. On a static page, you would simply tag the page to pull up the relevant items.
On a dynamic page, however, getting the right recommendations displayed next to the right
product – and knowing that the test is running correctly – is much more challenging, because
you can’t see it directly. You have to ensure Manchester United shorts are promoted when
Peter looks at a Manchester United shirt. You have to ensure a Detroit Tigers cap is promoted
when Jane looks at Detroit Tigers sunglasses.
Finally, testing pages directly associated with revenue poses the highest risk, but offers the
highest reward. Raising conversion rates on the last page of a checkout funnel, for example, by
50 percent will increase revenue by 50 percent because 50 percent more people are completing
their purchase. It is a very valuable page to improve. If you get it wrong, however, you could
equally lose 50 percent of your revenue. More mature optimization organizations are better
equipped to test pages such as this.
3

Business white paper | HP Web Optimization Maturity Model
Segmentation enables a more personal experience
Less mature organizations don’t personalize the visitor experience. They serve the same
generic content to all of their visitors and, likewise, don’t distinguish between visitors when they
are testing.
As they mature they first identify the visitor segments that are of interest to them. They
may decide to target content differently based on context (referrer, keyword, device type),
location, demographics (income, age), customer data (loyalty status), or behavior (new or
repeat visitor, category affinity) for example. With segments defined, they test behaviors
segment by segment.
The most mature organizations are exploring behavioral targeting, which uses complex
machine learning algorithms to discover the most profitable segments in the data without
defining them first.
A segmentation strategy such as this demands tools able to provide more granular testing.
The more mature the organization, the more granular the testing and ultimate personalization,
and the more complex the content segmented – from basic to dynamic content.
Organizational approaches that elevate maturity
Effective marketing in the digital age requires organizations to make decisions based on
customer data, rather than internal politics. To understand maturity from an organizational
standpoint, we first examine how much optimization is valued. Has it won the organizational
support that allows decisions to be made purely on test results?
The percentage of releases tested also gives an indication of how much an organization values
testing. As an organization matures ad-hoc testing reflecting the whims of the business is
superseded by a closely followed testing roadmap. All website changes are verified before
time and money is invested. After all, running a test is vastly more cost effective than spending
$500,000 implementing a change that hurts conversions and has to be rolled back.
Mature organizations also recognize the importance of testing everything at every level.
After core teams have tested changes on primary websites, those changes should be verified
on marginal sites so these can be optimized for site-specific tastes or behavior.
Well enough placed to minimize politics
We’ve already discussed how mature organizations follow an optimization roadmap, but for
optimization to be effective that roadmap must be derived from an organization’s strategic
goals. If, for example, search drives the largest revenue volumes, testing how search results
work should be higher up the agenda than testing the navigation bar.
In less mature organizations, optimization is often buried deep in IT or marketing where it is
at the mercy of agendas higher up in the organization. Web design and testing decisions will
be made on the highest paid person’s opinion rather than the data. If somebody high up in the
organization wants a blue button, they get it. If they want something tested, it’s tested. Any
plans are quickly thrown out of the window.
In mature organizations, where optimization sits close to the CMO, politics have been minimized
and decisions, even prioritization decisions, are based on data. Optimization is an important
strategic tool that influences business decision-making.
Going where the data leads you means sometimes taking paths that lack political support. A
tight link to senior management ensures the viability of optimization successes; their buy-in
helps clear roadblocks when politics get in the way of doing the right thing for the business.
4

Business white paper | HP Web Optimization Maturity Model
Attributing the value of optimization to move towards a profit center
Optimization is one of the most powerful levers for increasing profit or revenue. It should not
be subject to the short term whims of middle management; its budget should be based on how
much value it delivers. The ultimate goal here is for optimization to be a profit center where it is
measured, like any business unit, on how much revenue and profit it delivers.
But while less mature organizations struggle to keep track of test results, even more mature
organizations find it difficult to attribute hard numbers to the tests they run. Two examples:
Understanding the value of each click is difficult if the purchase is completed offline. A customer
might get a car insurance quote on the website, but then contact a call center to complete the
purchase. It is hard to know definitively that 17 percent of the people that clicked converted and
that revenue added up to $5 million. The link between how many people clicked and the ultimate
revenue is tenuous.
The week after implementing a very positive test result, a competitor announces a
promotion, impacting your conversion rate. If everything else stayed the same, revenue
gains might have added up to millions. But nothing ever stays the same for long, so it is
hard to pin down the true value of that test over a longer period.
The ideal of optimization as a profit center is always going to difficult, but not impossible,
to attain.
Leveraging insight to develop best practices
A high maturity organization is on a journey of continuous learning and improvement.
It has a history of running many tests, captures what has been tested, what has worked, and
what hasn’t. Different business units share it to develop best practices. Those best practices
become hypothesis for future tests and further improvement, creating momentum for the
optimization program.
Naturally, since every situation is different and what works in one situation may not work in
another, organizations need to keep testing their hypotheses.
Best practices can also be used as tools for internal publicity, for getting buy-in from senior
management and moving towards a profit center. A mature organization with many success
stories to tell is better placed to make that argument.
Efficient use of data
See our “Enhancing the
value of optimization
investments” whitepaper
for more on the
value of connecting
optimization and web
analytics data
It is one thing to collect data; it is another to analyze and take effective action.
A growing number of digital marketers are feeding their optimization data into third-party
web analytics platforms to derive further insight from their optimization data. The combined
data is extremely powerful; it can help them paint a picture of a visitor’s complete journey and
show how different test iterations are performing against the custom analytics reports that
marketers read day in and day out. Having this information will ensure marketers can effectively
make key business decisions on how to move forward with the learnings.
Moving quickly from insight to action
Marketing teams within less mature organizations often have to wait weeks or even months
for IT to implement the changes they require based on what they have learned from the data.
They have neither the authority nor the knowledge to log into the website editor and implement
the changes themselves. If changes are related to a specific campaign, they may not be able to
respond during the time the campaign is running. Bigger changes may not be implemented until
long into the future, with the organization giving up a small fortune each month while it waits.
Very mature organizations have empowered their optimization team; those in IT responsible for
implementing the changes report to directly to them. IT follows optimization’s lead rather than
optimization having to join in IT’s queue. Optimization can be more successful, rapidly delivering
value to the organization in line with its strategic goals.
5

Business white paper | HP Web Optimization Maturity Model
Benchmark against peers
This chart, based on a representative sample of our customers, shows that we work
with enterprises at all levels of maturity to improve their optimization program, and also
demonstrates that more organized teams out execute their peers.
Organization Score
Execution Score
See how you compare with our
free interactive Maturity
Model at optimost.com
You can find our free interactive Optimization Maturity Model at optimost.com. Use our tool to
assess your own organization’s web optimization maturity, or get in touch and we can guide you
through the process. The tool will show you where you fall on the chart above and provide much
more detail to help you make the internal case for optimization. We’ve also included a short
interactive web marketing quiz on the site to help you convince any skeptics in your organization
of the value of reaching optimization maturity. Once you’ve identified your organization’s current
level of maturity, we can work with you to recommend the next steps that will help you progress
up the maturity curve.
Zoltan Liu
Head of Customer Strategy and Solutions
New York office
Robert Brennan
Managed Services Manager
New York office
Uri Kogan
Worldwide Head of Marketing
Silicon Valley office
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only
warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein
should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
April 2015