Wilhelm Hofmann & Paresh V. Patel - Faculty

Wilhelm Hofmann & Paresh V. Patel

Overview
• Use and popularity of experience sampling
• Different approaches to implementation
– PDA approach
– Local app approach
– SMS survey distribution approach
• Introducing SurveySignal
– Core features
– How it works
– User experiences
– Security/Ethical issues
– Resources

Experience Sampling
(Bolger, Davis, & Rafaeli, 2003; Hektner, Schmidt, & Csikszentmihalyi, 2006; Mehl & Conner, 2012)
• Repeated, context-sensitive measurement of thoughts,
feelings, judgments, behaviors over given period of time (via
self-report/informant report)
• Examples: health behaviors, self-regulation, relationship
processes, time use, purchase behavior, emotions, happiness
etc.
• High ecological validity: Capturing psychological (and other
variables) where the “action takes place”
• High statistical power (multilevel analyses)
• Can easily be combined with laboratory assessments, other
sources of data (e.g., physiological measures, location data),
and psychological interventions

Common experience sampling approaches
1. The PDA approach
Criteria (various costs and efforts)
Device costs High
(Programming) effort involved in setting up
customized protocol
Typically high, although some software
exists (e.g., ESP)
Online access/data plan Not needed (local storage), unless
results need to be fed back in real-time
Compatibility issues None (only one operating system)
Need to meet before and after experience
sampling phase?
Special participant burden Need to carry extra device and learn
how to use it
Yes

Common experience sampling approaches
2. The local-app approach
Criteria (various costs and efforts)
Device and other costs No device costs (if using participant’s
smartphone)
(Programming) effort involved in setting up
customized protocol
Moderate to high, depending on
application
Online access/data plan Not needed (local storage), unless
results need to be fed back in real-time
Compatibility issues Yes (e.g. Android vs. iOS vs. Blackberry
vs. Windows Mobile)
Need to meet before and after experience
sampling phase?
Not necessarily, if people download and
install app on their own
Special participant burden Need to download and install the app

2012 Operating Systems Market Shares (US)
33.4
9.5
3.6 0.8
52.2
Google (Android)
Apple (iOS)
Blackberry (RIM)
Microsoft
Symbian

Common experience sampling approaches
3. The SMS survey distribution approach
Criteria (various costs and efforts)
Device and other costs No device costs;
Text messaging costs
(Programming) effort involved in setting up
customized protocol
Online access/data plan Required
None (when user-friendly survey
software is used)
Compatibility issues Minimal (mobile surveys display well
on most smartphone browsers)
Need to meet before and after experience
sampling phase?
Special participant burden None
Not necessarily

combines
(1) A simple and fast signup and smartphone
verification system
(2) Text messaging as a signalling and reminder device,
according to fixed, random, or mixed schedules
(3) Seamless integration with mobile-compatible
survey software such as Qualtrics ,
SurveyMonkey, or SurveyGizmo

Create mobilecompatible
online surveys
Feed in
Links
Embedded Data
•Participant ID
•Day/Signal Number
•Send/Response Time
How it works
SurveySignal
Create Survey
Project
Phone
Registration
Page
Study Panel
Sampling
Schedule
Verification
Consent
Response
Registration
Monitoring
Signaling
Reminding
Response
Recruitment/
Screening
Text messages
with Link

Researcher Account
• Registering an account and exploring the
system is free
• Purchase SMS credit to send signals as needed
at a rate of 10 cents per signal

Researcher Account Interface
Create an unlimited number of survey
projects within a free online account

Participant Signup Page

Customize the Experience Sampling Schedule

Edit Survey Links
Specify which daily signal is
connected with which individual
survey

Administration and Response
Monitoring Tools

Project 1: Hofmann, Finkel & Fitzsimons
• Completed Experience Sampling Project on Close
Relationships and Self-Regulation
• N = 230 (115 dyads)
• 7 days
• 6 daily random signals drawn from time segments
• plus one nightly diary @ 9pm
• Both partners signaled at the same time
• Signal timeout: 3 hours
• No reminder
• Average Response Rate: 70%

Response Delay (Relationships Study)
Frequency Percent
Cumulative
Percent
within 1 minute 792 11.7 11.7
within 5 min 1838 27.2 39.0
within 10 min 589 8.7 47.7
within 30 min 1216 18.0 65.7
within 60 min 1003 14.9 80.6
within 120 min 975 14.4 95.0
delayed 2 hrs or more 338 5.0 100.0
Total 6751 100.0
Median Response Time: 11.65 min
(Median Time Difference between partners’
responses to parallel signals: 15.98 min)

Project 2: Milyavskaya & Koestner
• Ongoing Experience Sampling Project at McGill
University
• 5 daily signals for 7 days
• Signal timeout: 5 hours
• Signal reminder after 10 min
• Current N = 145
• Average Response Rate: 77%

Response Delay (Milyavskaya & Koestner Study)
Frequency Percent
Cumulative
Percent
within 1 min 767 27.1 27.1
within 5 min 578 20.4 47.5
within 10 min 224 7.9 55.4
within 30 min 609 21.5 76.9
within 60 min 285 10.1 87.0
within 120 min 225 7.9 94.9
delayed 2 hrs. or more 143 5.1 100.0
Total 2831 100.0
Median Response Time: 6.33 min

Security/Ethical Issues
• Signup Security
– Wireless phone check (optional)
– Each phone number can be registered only once per study
– Opt-in procedure during registration
• SurveySignal Participant Information
– Protected within server architecture
– Phone numbers never fully displayed (e.g., 77****4326)
• Actual Survey Response Data
– Collected within secure survey environment of choice (e.g.,
Qualtrics, Survey Monkey)
• Terms and Conditions
– Purpose entirely research-based
– All researchers have to subscribe to APA Ethical Principles

Resources
• www.surveysignal.com
• Email Support (support@surveysignal.com)
• Online resources (within account)
– Manual
– FAQs
– Parameter cheat sheet
– Example survey project
Thank you very much for your attention!

Appendix 1: A typical Study Flow
Study
Welcome Page
•info about purpose, requirements,
compensation, etc.
Pre-Screening
•screening (e.g., according to demographics)
•test of mobile phone browser compatibility
Signup on
SurveySignal
•Phone number, Email
•informed consent
Intake Survey
•Demographics
•Personality measures
Mobile Phase
•Collection of Experience-Sampling
Data on Participant Smartphones
Follow-up
•Short- and Long-term Change
measures, etc.

Appendix II: Core features at a glance
• Works on all smartphones with text messaging (SMS) capability and internet
access
• no download/installation necessary
• Built-in phone registration and verification process
• Optional wireless number validation
• Supports signal-contingent sampling (via text messaging) according to fixed,
random, or mixed schedules
• Other applications: event-contingent sampling, one-shot cross-sectional
smartphone surveys
• Messaging works in more than 200 countries at constant rate of
10 cents per signal
• Signal Timeout and response reminders
• Possible to send simultaneous signals to dyad/group members
• Panel administration and response monitoring tools
• Seamless integration with prominent survey software applications such as
Qualtrics, SurveyMonkey, or SurveyGizmo