20 December 2004
AL World Project Coordinator
TutorHome has been available since February 2003, with all ALs required to use it since
January 2004. ALs are expected to log on at least once a week, depending on the level of
computing usage associated with the course. Student details have been delivered solely
online since March 2004.
The TutorHome team organised a survey to evaluate the site’s current usefulness and to
help set direction for the next round of developments to take place during 2005. The
survey was administered by IET.
A random sample of 2,148 ALs were contacted via their OU e-mail addresses and asked to
complete an online questionnaire published using IET’s electronic survey administration
system (ELSA). They had the option of requesting a paper version of the questionnaire.
The sample included the 13 ALs working on web-intensive courses and was stratified to
help ensure representation across ALs working on web-enhanced and web-focused
The survey remained open from 16 September 2004 through 18 October 2004. An e-mail
reminder was sent to non-respondents on 28 September 2004.
Evaluation of TutorHome, Autumn 2004 -- 2
The questionnaire explored ALs’ experience in using several of the key areas of
TutorHome. The areas covered:
working with student groups
working with course resources
support for working online
Respondents were asked to indicate their agreement with statements on a five-point
scale, where five equalled Strongly Agree and one equalled Strongly Disagree. Each
area included an overall question.
Possible developments were also explored:
providing more facilities for working with student groups
expanding online recruitment facilities
providing more opportunities for participating in community.
Respondents were asked to indicate interest in new developments using a five-point
scale, where five equalled Very Interested and one equalled Not at All Interested.
Each content-specific section and a final section on overall impressions also provided
opportunities to make suggestions for improvement through open-ended questions.
Evaluation of TutorHome, Autumn 2004 -- 3
Summary – Overview of
Of the 2,148 ALs invited to participate in the survey, 622 completed the questionnaire,
for a response rate of 29%. Only 6% of respondents completed the paper version of the
61% of respondents indicated agreement with the statement, All in all, I find TutorHome
easy-to-use, by giving it a rating of four or five. The average rating for the question was
In evaluating aspects of TutorHome usability, respondents generally agreed that
TutorHome gave a positive impression of the OU but indicated some difficulty navigating
and remembering where to find resources and facilities that they had used before.
When asked to identify what activities they found most valuable, respondents placed
Getting my students’ contact information first (20% of responses). Downloading student
information (14%) and Getting my students’ course results (13%) came next. Looking
up information related to working with the OU and Claiming for expenses ranked last,
with 5% each.
The most common arrangement for working online was to use a dial-up modem at
home (42% of responses). Most other respondents used a broadband connection at
home (33%) or a computer at another workplace (23%).
When asked to compare their approach to finding information on TutorHome with
their approach to finding information on the Internet, respondents indicated a greater
tendency to hunt for specific information on TutorHome, along with a tendency to return
to sources that have helped in the past.
In making suggestions for the front page of TutorHome, respondents most often
mentioned design improvements and access to job vacancies, the eTMA system, and
Evaluation of TutorHome, Autumn 2004 -- 4
Summary -- Exploration of
Among the overall questions for the TutorHome areas, Tutor Claims Online received the
highest rating (4.02), although it had been used by only 24% of respondents. Accessing
information related to my course and Faculty followed (3.88), having been used by 40% of
respondents. Working with my students received a rating of 3.74.
Regional TutorHome sites had been used by 38% of respondents and received the lowest
overall rating (3.32)
Responses indicated that, despite high overall satisfaction, Tutor Claims Online would
benefit from covering more types of expenses and claims. Comments also indicated that
its placement on TutorHome could be more prominent.
In answering questions about working with course resources, respondents indicated
potential room for improvement in communications with course teams, particularly in having
appropriate online connections. There was strong support for being able to access tutor
notes and marking guidelines online and for being able to access related courses’ websites
for background and ideas.
Their comments emphasised an interested in having more course information and contact
information available. While some respondents suggested improvements to FirstClass,
others preferred it over TutorHome.
In rating aspects of working with their student groups, respondents indicated that
changes to student details were clear and that information on students’ progress was
generally sufficient. There may be scope to improve pathways for support and referrals
and especially to provide more information on students’ history and circumstances along
with more information about communications being sent to students.
Comments included suggestions for improving student details, along with requests for more
assessment-related information and better support for handling queries.
Evaluation of TutorHome, Autumn 2004 -- 5
Summary -- Exploration of
TutorHome Areas (2)
In rating the University’s support for working online, respondents expressed high
confidence in responding to student e-mails and seemed relatively satisfied with the
support provided by the LTS Helpdesk. There is potential room for improvement in letting
students know what to expect from ALs and for providing stronger technical support in
using FirstClass and in developing general computing skills.
Suggestions emphasised providing more financial support for working online, with providing
equipment or support and providing more training also receiving frequent mention.
The 40% of respondents who had worked with the employment-related information on
TutorHome found it relatively easy to understand. The sub-site could do more to identify
how to get support with employment issues, and respondents do not yet seem to be relying
on it to find out about changes that may affect their appointments.
Comments clustered around improving the application process, providing more equipment
are support, and improving Tutor Claims Online.
Finally, respondents who had worked with the Regional information available through
TutorHome gave the highest rating to the sites’ role in alerting them to opportunities for
support and development. Their greatest difficulty lay in being able to tell when information
on the sites had changed.
Requests included indicating when changes had occurred, providing more contact
information, and providing more resources or facilities.
Evaluation of TutorHome, Autumn 2004 -- 6
Summary – Exploration of
In rating their interest in new facilities for working with student groups, respondents
expressed greatest interest in having links for referring student queries. They expressed
least interest in opportunities to let the University know about their experiences and in
having more resources related to induction.
There was considerable interest in having expanded facilities to retrieve and submit
application forms online, with 62% of respondents giving a rating of 5, Very Interested.
There was a similar expectation that such a facility would include all tutoring roles.
Respondents indicated moderate interest in having opportunities to participate in
community. Being able to obtain more information about University research had the
greatest appeal, and being able to contribute news stories to TutorHome and other
University news channels had the least.
When asked what would help them feel valued, respondents commented that existing
discussion facilities suffice, that the University should recognise financial and time
constraints, and that improving communications opportunities would help.
When asked what would help them feel more connected to others, respondents mentioned
having more opportunities to make contact with other ALs, improving TutorHome, and
having opportunities to tell others about themselves. Others mentioned that the existing
discussion facilities suffice.
In comments identifying the biggest benefit of TutorHome, access to student information
received by far the greatest mention, with ease of use, providing a single point of access,
speed, and access to conferencing and e-mail also ranking high.
In suggesting the single most important improvement to TutorHome, respondents
identified further improvements to student facilities and information, along with FirstClass
improvements, improved technical operations, design improvements, navigational
improvements, better access to information, and facilitating contact with others.
Evaluation of TutorHome, Autumn 2004 -- 7