The Web as a Living Laboratory: How MECLABS uses Internet ...

The Web as a Living Laboratory 

The Three Most Important Discoveries from 

over a Decade of Experimentation 

Dr. Flint McGlaughlin 

Managing Director (CEO) 

MECLABS

Experiment #1: Background 

Experiment ID: REGOnline Homepage Test 

Location: MarketingExperiments Research Library 

Test Protocol Number: TP1427 

Research Notes: 

Background: REGOnline is event management software that lets users create 

online registration forms and event websites to manage their events. 

Goal: To increase number of completed leads on home page. 

Primary research question: Which page will achieve the greatest addressable 

lead rate? 

Approach: A/B multifactor split test 

2

Experiment #1: Control 

Control ‐ Homepage 

• Our researchers 

hypothesized that we could 

increase the appeal 

associated with the value 

proposition of this offer by 

focusing more on the 

product and its specific 

features and benefits. 

3

Experiment #1: Treatment 

Treatment ‐ Homepage 

• Headline was written to focus 

more on the product. 

• Specific features and benefits 

are utilized to express the 

value. 

• The page emphasizes “Free 

Access.” 

• Also, ensured that this value 

was being communicated in 

subsequent steps. 

4

Experiment #1: Side‐by‐side 

Control 

Treatment 

5

Experiment #1: Results 

24.5% Decrease in Conversion 

The Treatment generated 24.5% less completed leads 

Versions 

Conversion 

Rate 

Rel. diff 

Control –Two‐step homepage 2.3% ‐ 

Treatment –Three‐step homepage 1.7% ‐24.5% 

 

What you need to understand: In spite of having clearer value and reducing 

the amount of form fields in the first step, the control still outperformed 

the treatment. 

6

HOW can we improve this page? 

Control ‐ Homepage 

7

What you need to understand 

Key Principles 

1. Asking “how” leads to information; asking “why” leads to wisdom. 

Yet marketers are all too busy asking how. 

8

Key “Why” Questions 

WHY did the treatment 

decrease conversion by 25%? 

WHY did the more people say 

“yes” to the control 

9

The Problem: Few View the Internet as a Lab 

Which of the following ways does your organization use the Internet? 

Marketing Channel 

64% 

Informational Resource 

37% 

Marketing Hub 

30% 

Customer Research Lab 

10% 

Brand Research Lab 

6% 

Source: ©2012 MarketingSherpa Website Optimization Benchmark Survey 

Methodology: Fielded April 2012, N=2,677 

10

Experiment #2: Background and Test Design 

Experiment ID: REGonline SEO landing page test 




Background: A technology and media company specializing in online registration and 

event management software. 

Goal: To increase the amount of leads generated online. 

Primary research question: Which online capture process will generate the highest 

addressable lead rate? 

Approach: A/B multifactor split test 

11

Experiment #2: Background 

Homepage from 

Previous Test 

• Before we could get a lift, we 

needed to learn more about 

the prospects coming to this 

site. 

24.5% 

Decrease in Conversion 

• We decided to use one of 

their SEO pages as a research 

window into the cognitive 

psychology of the customer’s 

motivation. 

u = 2q + t + m + 2v + i © 

12

Experiment #2: Control 

SEO Landing Page 

• This landing page was 

offering the same 

product as the home 

page but dealt with a 

smaller subset of 

visitors that matched 

the profile of those 

coming to the 

homepage. 

• Our researchers could 

test here without the 

negative consequences 

of hurting conversion 

on the homepage. 

13

Experiment #2: Treatment 

Treatment SEO Landing Page 

• For our first test on this 

page, we tested 

focusing on how this 

product enabled 

creating registration 

forms easier and could 

cut their time in half… 

• …and yet it still had a 

robust functionality. 

14

Experiment #2: Side‐by‐side 

Control 

Treatment 

Do you think the new emphasis will out perform the control? 

If so, by how much? 

15

Experiment #2: Results 

548% Increase in Complete Leads 

The new page’s conversion rate increased by 548.46% 

Design 

Conversion Rate 

(%) 

Relative 

Difference 

Statistical Level 

of Confidence 

Original Page 0.7% ‐ ‐ 

Treatment 4.8% 548% 

99% 

What you need to understand: By applying key optimization 

methodologies to the lead capture process, the treatment was able to 

improve step‐level clickthrough rates by 1,312%, and completed leads 

captured by 548%. 

16

Experiment #2: Learning Applied 

SEO Page Test 

Original Homepage 

New Home Page Test 

548% Learning Learning 

90% 

• We were able to take what we learned about the motivations of their 

customers from testing on the SEO landing page and apply it to the 

homepage, which generated a 90% increase in leads captured. 

17



1. Asking “how” leads to information; asking “why” leads to wisdom. Yet 

marketers are all too busy asking how. 

2. Sometimes we need to slow down in order to go fast. Action is 

overrated; action should be grounded in contemplation. Admittedly, 

contemplation without action is anemic, but then action without 

contemplation is dangerous. 

18



1. Asking “how” leads to information; asking “why” leads to wisdom. Yet 

marketers are all too busy asking how. 

2. Sometimes we need to slow down in order to go fast. Action is 

overrated; action should be grounded in contemplation. Admittedly, 

contemplation without action is anemic, but then action without 

contemplation is dangerous. 

3. Indeed, the marketer should be the philosopher of the organization ‐‐ 

for the vigorous action of sales needs to be grounded in the rigorous 

contemplation of marketing. 

19

The Marketer as Philosopher 

Three Critical Observations

Receive Advance Notice of Dr. McGlaughlin's New Book 

The Marketer 

as Philosopher 

By Dr. Flint McGlaughlin’s 

• To receive advance notice of 

the publication of Dr. 

McGlaughlin’s upcoming book, 

The Marketer as Philosopher. 

…and… 

• To get a key excerpt that 

explains the Inversion of the 

Funnel with the Micro‐Yes Map. 

Visit the following webpage: 

www.meclabs.com/philosophy

OBSERVATION #1: 

All Marketing is Comprised of 4 Elements 

22


The Four Elements of Marketing 

The Message 


The Market 

The Means 

23


Marketing Dangers 

THE DANGER OF COMPANY LOGIC 

24


Experiment: Background 

Experiment ID: (Protected) 




Background: A company offering dedicated business hosting services 

Goal: To increase the amount of leads 

Primary research question: Which page design will generate the greatest 

amount of leads? 

Approach: A/B multi‐factor split test (radical redesign) 

25


Experiment: Original 

26



• The banner is the first thing that people see when arriving at this page, but what does it 

mean? It might mean something to the company, but it conveys little value to the 

prospect. 

27



• This company only truly needs 4 fields in order to obtain a qualified lead, and yet 20 

different fields must be completed on this page. 

• Call to action “Request a Quote” is impersonal and implies commitment and cost. 

28


Experiment: Original vs. Treatment 

29


Experiment: Treatment 

• Immediately lets the visitor know where 

the are and what they can do on this 

page. 

• Provides organized content that the 

prospect could navigate through based 

on their own needs 

• Uses images with easily understandable 

meaning 

• Uses the more personal/ lower 

commitment call‐to‐action language of 

“Call to Discussing Your Needs” 

• Moves all unnecessary forms fields to a 

secondary step. 

30


Experiment: Original vs. Treatment 

31


Experiment: Result 

189% Increase in Overall Conversion 

The new page’s conversion rate increased by 188.46% 

Design Conversion Rate (%) 

Relative 

Difference 

Statistical Level 

of Confidence 

Original Page 2.00% ‐ ‐ 

Treatment 5.77% 188.46% 

95% 

32


The Goal of the Marketer is to Move the Prospect 

Up, Not Down, the Funnel 

33


The Inverted Funnel 

1. The funnel is often presented as 

one potentially useful analogy for 

marketing. It is in fact, the 

primary analogy. All marketing 

should influence a decision. 

2. The funnel analogy distorts 

reality. People are not falling into 

your funnel, they are falling out. 

The funnel must be inverted. 

g= 9.81 m/s 2 

3. People don’t travel down the 

center of the funnel. People are 

climbing up the sides. 

34


The Inverted Funnel 

(Ma)YES 

Y 

Sales Call 

(Mi)YES 

(Mi)YES 

Landing Page 

Business Software Suite 

#1 On‐Demand. 6459+ World Clients 

Award‐Winning Solution. Free Trial 

www.XXXXXXXXXX.com/Business 

PPC Ad 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 


35



THE DANGER OF CONFLATED OBJECTIVES 

36



Experiment ID: Email Messaging Test 



Background: Largest physician‐only social network which offers its knowledge 

base to pharmaceutical companies. 

Goal: To increase email clickthrough from rented list to landing page. 

Primary research question: Which email will generate the highest 

clickthrough rate? 

Approach: Multifactor sequential (year‐to‐year) test. 

37


Experiment: Email control 

Company 

• This page, though above 

average in many 

respects, still has 

conflated the objective 

of the email with the 

objective of the landing 

page. 

• This email is selling and 

asking for too much at 

this stage in the 

conversion process. 

38


Experiment: Email control 

Email 

Subject Line 

Landing Page 

Headline 

Company 

Headline 

First Two 

Inches 

First Two 

Inches 

Body 

Body 

CTA 

CTA 

39


Experiment: Email treatment 

• Each section of the 

page guides the reader 

through a logical series 

of micro‐conversions. 

Company Name 

Optimized Email 

Name 

Number 

• Call to action asks for 

just the right amount 

of commitment at 

this stage of the 

conversion process. 

Name 

Name 

Name 

Name 

40


Experiment: Side‐by‐side 

Original 

Optimized 

Company 

Company Name 

Number 

Nam 

e 

Name 

Name 

41


Experiment: Results 

104% increase in conversion 

The treatment generated a 104% higher clickthrough rate. 

Versions Clickthrough Rate Relative difference 

Control 1.55% ‐ 

Treatment 3.16% 104% 

 

What you need to understand: By changing the messaging of the email to 

reflect the company’s value proposition, we were able to generate a 104% 

lift in clickthrough rate. 

42


The Value Proposition is the Force Which Compels 

the Prospect Up the Funnel 

43


The Force of the Value Proposition 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Ma)YES 

Value Proposition 



Y 

1. The answer to the question, 

“What is your value proposition?” 

can only be discovered by 

answering yet another question: If 

I am your ideal prospect, why 

should I buy from you rather than 

any of your competitors? 

44



Value Proposition Question: If I am your ideal prospect, why should I 

buy from you rather than your competitors? 

1. You are fundamentally answering a first‐person question posed in the mind of your 

customers. 

2. A value proposition focuses on a specific customer segment. This requires you to 

consider who you are not going to serve and the associated tradeoffs. 

3. A value proposition has a specific action in mind. It is seeking to answer “why” for a 

specific “what.” 

4. A value proposition must differentiate you from your competitors. In at least one 

way, you must have an “only” factor. 

45



Perceived 

Cost 

Perceived 

Value 

46



(Ma)YES 

Y 

(Mi)YES 

For the Primary, Product, Prospect, and Process‐ 

Level Value Propositions, the objective is to tip 

the perceived value (Pv) to be greater than the 

perceived cost (Pc). 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




47



(Ma)YES 

Y 

(Mi)YES 

For the Primary, Product, Prospect, and Process‐ 

Level Value Propositions, the objective is to tip 

the perceived value (Pv) to be greater than the 

perceived cost (Pc). 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




48



The force of a value proposition can be measured by four essential 

elements of the offer: 

Appeal – How much do I desire this offer? 

Exclusivity – Where else can I get this offer? 

Credibility – Can I trust your claims? 

Clarity – What are you actually offering? 

49



THE DANGER OF ASSUMED VALUE 

50



Experiment ID: (Protected) 


Test Protocol Number: 1214 


Background: A leading software provider. 

Goal: To increase total leads. 

Primary research question: Which process will generate the most leads? 

Approach: Radical redesign of the complete lead‐generation process. 

51


Experiment: Original PPC ad 

Original PPC Ad 

{Keyword XXXXXXXXX Software} 

Award‐Winning XXXXXXXX Software. 

Fully Integrated. Free Trial 

www.XXXXXXXXXX.com/XXXXXXXX 

(Mi)YES 

(Ma)YES 

Y 

X 





(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




52


Experiment: PPC Ads Side‐by‐Side 

Original 





• Original ad uses only vague 

qualitative statements like 

“Award Winning” and “Fully 

integrated” 

Optimized 

XXXXXXXXX Software Suite 




• The optimized uses specific 

quantitative statements like 

“#1 On Demand” and 

“6459+ World Clients” to 

communicate the value. 

53


Experiment: Optimize PPC Ad 

Optimized PPC Ad 

XXXXXXXXX Software Suite 




(Mi)YES 

(Ma)YES 

Y 

 





(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




54




The optimized PPC Ad conversion rate increased by 20.9% 

PPC Ad CR Rel. Diff. 


Treatment 1.08% 20.9% 

55


Experiment: Original Landing Page 

Original Landing Page 

(Ma)YES 

Y 

(Mi)YES 

(Mi)YES 

X 

(Mi)YES 

(Mi)YES 

(Mi)YES 




56


Experiment: Original Landing Page 

Original Landing Page 

• Again, the original landing 

page from the ad is using 

vague language to 

communicate the value. 

• Also, there is a significant 

disconnect between the 

value communicated in the 

PPC ad and the value in 

landing page. Where is the 

“Award winning”? Where is 

the “Fully Integrated”? 

57


Experiment: Optimized Landing Page 

Optimized Landing Page 

• The optimized version 

immediately connects the 

PPC ad to the landing page 

maintaining strong 

continuity. 

• As in the PPC ad, clear 

quantitative language is 

used. 

• Awards are shown 

prominently. 

• Testimonials and CTA both 

add value. 

58


Experiment: Optimized Landing Page 

Optimized Landing Page 

(Ma)YES 

Y 

(Mi)YES 

 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




59




The optimized Landing Page conversion rate increased by 54.26% 

Landing Page CR Rel. Diff. 


Treatment 11.06% 54.26% 

60


Experiment: Original Form Page 

(Ma)YES 

Y 

(Mi)YES 

X 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




61


Experiment: Original Form Page 

Original Form Page 

• This form page is not really 

communicating any value. 

They have stopped trying to 

sell and assuming that the 

customer is convinced to 

complete the form. 

• Also, there is little 

connection that this page 

has to either the landing 

page or PPC ad. 

62


Experiment: Optimized Form Page 

Optimized Form Page 

• The optimized page, 

continues to communicate 

the value of the offer even 

on the registration page. 

• The message is directly 

connected to both the 

landing page and the PPC 

campaign. 

63


Experiment: Optimized Form Page 

Optimized Form Page 

(Ma)YES 

Y 

 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 

(Mi)YES 




64




The optimized Landing Page conversion rate increased by 54.26% 

Landing Page CR Rel. Diff. 


Treatment 31.25% 97.27% 

65


Experiment: Overall Results 


The optimized path’s conversion rate increased by 272.2% 

Metric Control Treatment 

Relative 

Difference 

PPC Advertisement 0.89% 1.08 20.9% 

Landing Page Clickthrough 7.17% 11.06% 54.26% 

Form Completion 15.84% 31.25% 97.27% 

Impression‐to‐lead Conversion .009% .033% 272.2% 

What you need to understand: In this experiment, a 272% increase in 

conversion led to 268% more projected revenue and, when combined with 

the corresponding 66% reduction in cost‐per‐acquisition, the optimized 

path produced more than 4 times the monthly profit (302% increase). 

66

Receive Advance Notice of Dr. McGlaughlin's New Book 


as Philosopher 

By Dr. Flint McGlaughlin’s 

• To receive advance notice of 

the publication of Dr. 

McGlaughlin’s upcoming book, 

The Marketer as Philosopher. 

…and… 

• To get a key excerpt that 

explains the Inversion of the 

Funnel with the Micro‐Yes Map. 

Visit the following webpage: 

www.meclabs.com/philosophy

LIVE OPTIMIZATION (FROM AUDIENCE) 

68

Live Optimization: CenturyLink 

69

Live Optimization: OSCON 

70

Live Optimization: Electric Fireplaces Direct 

71

The Web as a Living Laboratory 

The Three Most Important Discoveries over a 

Decade of Experimentation 

Dr. Flint McGlaughlin 

Managing Director (CEO) 

MECLABS

The Web as a Living Laboratory: How MECLABS uses Internet ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?