Yu-hao Wu Tay-sheng Jeng Yi-shin Deng - IAFOR

iafor.org

Yu-hao Wu Tay-sheng Jeng Yi-shin Deng - IAFOR

Yu-hao Wu

Tay-sheng Jeng

Yi-shin Deng


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

A Refined Heuristic Principles for Instructional Media Sharing Platform Evaluation

Yu-hao Wu, Tay-sheng Jeng, Yi-shin Deng

0195

National Cheng Kung University, Taiwan

The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings 2012

Abstracts

This research aims to find new heuristic principles especially used as instructional media

on an internet sharing platform. Apart from Jakob Nielsen’s heuristic evaluation ten

original principles, this research would like to focus on developing and refining new

heuristic evaluation principles for instructional multimedia.

This research takes the examples of three similar functions of video sharing platforms

(but different style). Users are asked to take the usability test to finish the tasks. Heuristic

evaluation is considered as the method to evaluate the aforementioned platforms.

Arranging the existing literature from user experience and heuristic evaluation field, there

are some similar regulated principles are listed down. These new principles are added

into Jakob Nielsen’s ten original principles to evaluate the instructional media sharing

platforms. Experts from user experience domain are asked to complete the heuristic

evaluation form in order to find the designs which are against the usability. The results

from the method using of “heuristic evaluation” need to match the results from the

method using of “usability test”. Otherwise, the new principles should be reconsidered

again until it matches.

Through taking the usability test, the users are encouraged to voice what they think and

their observations about what they are doing and about their experience with the three

different platforms. The test was concluded with questions about general feelings and

observations about the whole exercise. The results from the usability test will be listed

down systematically and will be regarded as the final standard answer.

Key words: heuristic, evaluation, usability, instruction, video, media, sharing, platform

iafor

The International Academic Forum

www.iafor.org

429


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

CHAPTER 1 Introduction

1.1 Research background

TED (Technology, Entertainment and Design) is a global set of conferences owned by

the private non-profit Sapling Foundation, formed to disseminate "ideas worth

spreading."

“TEDTalks began as a simple attempt to share what happens at TED with the world.

Under the moniker "ideas worth spreading," talks were released online. They rapidly

attracted a global audience in the millions. Indeed, the reaction was so enthusiastic

that the entire TED website has been reengineered around TEDTalks, with the goal of

giving everyone on-demand access to the world's most inspiring voices.” (TED

website)

Take “TEDTalks” as a model, we could see that the ideas spreading is quite fast and

changes immediately in our lives. TEDTalks has provided a platform for people to

share their ideas not only in an entity lecture hall but also in a virtual internet. The

video is actually at the same time educating the people who were using the platform.

Through the TED and sort of the online social media has become an innovated and

creative central issue. In this research, it would like to take TEDTalks as the example

to transfer the innovation and creative thinking video on the prototype website

through the basic evaluation of Heuristics.

1.2 Research objectives and aims of research

This research objectives would like to focus on developing and refining a new

heuristic evaluation principles of educational multimedia. Take Jakob Nielsen’s

original ten heuristics(developed for evaluating software in general) as an example,

this research is to refine a new principles for instructional media sharing platform on

the internet.

1.3 Scope and limitations

Heuristic evaluation is a methodology for investigating the usability of software

originally developed by Jakob Nielsen (1993, 2000), a widely acknowledged usability

expert. According to Nielsen (1994), heuristic evaluation “involves having a small set

of evaluators examine the interface and judge its compliance with recognized

usability principles (the “heuristics”).”

In this study, Nielsen’s protocol was modified and refined for evaluating e-learning

programs. In order to find the usability of the platform for the instructional sites, the

designing of the platform will be considered as an issue and limitation in this research.

Because of the designing of the platform issue, heuristic evaluation will be involved

430


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

into the design process while designing the platform.

And the selection of the participants will be one of the limitations in this research as

well. Both positive and negative comments were received for each of the three sets of

heuristics - interface, educational design and content - and in some instances the

evaluators offered constructive suggestions for improvement. Examination of the

responses revealed that many of the problems had been noted by only one or two of

the evaluators. The identified problems were used to develop a list for further

investigation and remediation in the final version of the materials. (P. Albion 1999)

CHAPTER 2 Literature Review

2.1 Internet Video Sharing

The online videos have existed long before YouTube popped up on the internet. It

was very inconvenient for the users to upload videos, manage, share and watch them

with a lack of an easy-to-use integrated platform in that period of time. More

importantly, the videos distributed by traditional media servers and peer-to-peer file

downloads like BitTorrent were standalone units of content. There is no other related

video clips connect for each single video, for example other episodes of a show that

the user had just watched. And also, in the period of time, there was very little in the

way of content reviews or ratings.

Nowadays, the new changes of the video sharing sites, YouTube and its competitors

have overcome these problems in the new generation. They tried to let the content

suppliers upload their video effortlessly, automatically converting from many

different formats, and tag the keywords for the uploaded videos. Users could easily

share videos by mailing links to them, or embedding them on web pages or in blogs.

They could also rate and comment on videos, or even bringing the new social aspects

to the viewing of videos. Thus, popular videos can rise to the top.

2.1.1 The Social Network in Video Sharing Website – YouTube

The social network existing in YouTube further enables communities and groups.

Videos are no longer independent from each other, and neither are users. This has

substantially contributed to the success of YouTube and similar sites.

“YouTube” is one of the fastest-growing websites, and has become the 4th most

accessed site in the Internet. It has become the most successful Internet site providing

a new generation of short video sharing service. YouTube has a significant impact on

the Internet traffic distribution, and itself is suffering from severe scalability

constraints. Today, YouTube alone comprises approximately 20% of all HTTP traffic,

431


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

or nearly 10% of all traffic on the Internet. (Xu Cheng, C. D., Jiangchuan Liu. 2007)

Understanding the features of YouTube and similar video sharing sites is crucial to

network traffic engineering and to sustainable development of this new generation of

service.

2.2 E-learning

Defining e-learning as instruction delivered on a computer by way of CD-ROM,

Internet, or intranet with the following features:

Includes content relevant to the learning objective

Uses instructional methods such as examples and practice to help learning

Uses media elements such as words and pictures to deliver the content and

methods

May be instructor-led (synchronous e-learning) or designed for selfpaced

individual study (asynchronous e-learning)

Builds new knowledge and skills linked to individual learning goals or to

improved organizational performance

As you can see, this definition has several elements concerning the what, how, and

why of e-learning.

What. e-Learning courses include both content (that is, information) and instructional

methods (that is, techniques) that help people learn the content.

How. e-Learning courses are delivered via computer using words in the form of

spoken or printed text and pictures, such as illustrations, photos, animation, or video.

Some forms of e-learning (asynchronous) are designed for individual self-study. New

e-learning formats called virtual classrooms or synchronous e-learning are designed

for real-time instructor-led training. Both formats may support asynchronous

collaboration with others through tools such as wikis, discussion boards, and email.

Why. e-Learning courses are intended to help learners reach personal learning

objectives or perform their jobs in ways that improve the bottom-line goals of the

organization.

In short, the “e” in e-learning refers to the “how”: the course is digitized so it can be

stored in electronic form. The “learning” in e-learning refers to the “what”: the course

includes content and ways to help people learn it; and the “why” refers to the purpose:

to help individuals achieve educational goals or to help organizations build skills

related to improved job performance. (Mayer, R. C. C. R. E. 2008)

This definition brings out that the goal of e-learning is to build up a transferable

knowledge and skills linked to organizational performance or to help individuals

achieve personal learning goals. Although the guidelines were presented throughout

the books which do apply to lessons designed for educational or general interest

432


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

learning goals, the emphasis is on instructional programs that are built or purchased

for workforce learning.

2.3 Heuristic Evaluation

Heuristic Evaluation (HE) is a usability engineering method for finding the usability

problems in a user interface design so that they can be attended to as part of an

interactive design process. Heuristic evaluation involves having a small set of

evaluators examine the interface and judge its compliance with recognized usability

principles. Heuristic Evaluation is used to identify interaction problems with the

computerized applications at the early assessment stage.

2.3.1 Interface design heuristics (after Nielsen) (Albion, P. 1999)

Visibility of system status: The system should always keep users informed

about what is going on, through appropriate feedback within reasonable time.

Match between system and the real world: The system should speak the users'

language, with words, phrases, and concepts familiar to the user, rather than

system-oriented terms. Follow real-world conventions, making information

appear in a natural and logical order.

User control and freedom: Users often choose system functions by mistake and

will need a clearly marked "emergency exit" to leave the unwanted state without

having to go through an extended dialogue. Support undo and redo.

Consistency and standards: Users should not have to wonder whether different

words, situations, or actions mean the same thing. Follow platform conventions.

Error prevention: Even better than good error messages is a careful design

which prevents a problem from occurring in the first place.

Recognition rather than recall: Make objects, actions, and options visible. The

user should not have to remember information from one part of the dialogue to

another. Instructions for use of the system should be visible or easily retrievable

whenever appropriate.

Flexibility and efficiency of use: Accelerators-unseen by the novice user may

often speed up the interaction for the expert user to such an extent that the

system can cater to both inexperienced and experienced users. Allow users to

tailor frequent actions.

Aesthetic and minimalist design: Dialogues should not contain information

which is irrelevant or rarely needed. Every extra unit of information in a dialogue

competes with the relevant units of information and diminishes their relative

visibility.

Help users recognize, diagnose, and recover from errors: Error messages

433


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

should be expressed in plain language (no codes), precisely indicate the problem,

and constructively suggest a solution.

Help and documentation: Even though it is better if the system can be used

without documentation, it may be necessary to provide help and documentation.

Any such information should be easy to search, focused on the user's task, list

concrete steps to be carried out, and not be too large.

434


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

CHAPTER 3 Research Method

3.1 Framework

This research based on Heuristic Evaluation heavily in order to refine new principles

for instructional video sharing platform on the internet. Heuristic methods have been

shown to be cost-effective in the area of user interface evaluation. (P. Albion 1999)

What is this research? Why is this research deserved to be study? How does this

research operate?

In short, this research is trying to find the much more appropriate principles of

evaluating the interface for the instructional media sharing(e-learing) platform. To

improve the usability when using, method for evaluating the interface now has been

launched by Jakob Nielsen, which is developed for evaluating software in general.

This research is trying to refine the new principles especially for the way of

instructional media sharing.

Nowadays, in the education scope, teachers would like to share the digital

information (such as video, photos, curriculum slides…etc.) to the internet for the

students to have more discussion on the internet. But the interaction between teachers

and students were quite not common in the past experiences. This research assumes

that the interface design of the platform (eg. NCKU’s Moodle…) is the first important

reason for that. This research is to develop the new evaluation principles for

instructional media sharing interface in order to improve the usability between the

platform and the users (teachers and students). Therefore, it could then increase the

interaction on the internet between teachers and students or even more, the browsers.

435


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

3.2 Research Process

Taking into the consideration of the above, this research is operating as follow:

436


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

3.3 Methods

The experiment started base on usability testing and heuristic evaluation. Once the

results from usability testing are matched with the results from heuristic evaluation, it

should take consideration into the final principles. Three different website (NCTU

OCW, TED and MIT) will be taken as the examples for the participants to do the

experiments. In order to find out the appropriate considerations of the principles,

experiments should take several times to adjust the questions to reach the best one.

CHAPTER 4 Usability Testing Experiment

4.1 Introducion

The original intention for “OpenCourseWare(OCW)” is the unlock and share the

knowledge. But the translation for OpenCourseWare in Chinese “開放式課程” is

usually mistaking the meaning into “an entirely opened course”. But in fact, the OCW

is not a curriculum, it provides the sources and knowledge from the school, and the

self-study people set the study goal by themselves. They could decide the learning

ways and the content by themselves. The OCW learning system is totally different

form the school or the long distance learning. In brief, OCW is not a set of a traveling

package; it is more like a backpacking. The traveler could arrange their time and the

schedule by the information they got in a detail way or in an easy comfortable way.

NCTU has launched the website of NCTU OCW since 2007 with free of charge.

From 2007 till now (2012), NCTU has set up 127 courses (include 103 courses with

video). All the sources are posted on the OCW official website

(http://ocw.nctu.edu.tw). The courses are followed by the original taken courses. It

provides the entire content, outline, goal and video in order for self-learning.

For the learner, OCW could let you know the content before the class start, in order to

arrange your courses. Students could listen to the different teachers’ videos for the

cross-referencing and to clarify the concept of its subject and understand the content

of the courses. For the students who had already taken the course, self-learner, and the

working people, they could refresh and re-experience the courses essentials. And also,

for the teachers, they could pay more attention on its academic and teaching part.

Because the curriculum resources is shared on the internet, the new teachers could

take them as a reference for their preparation of teaching. It provides the students and

the self-learner multiple channels of learning in order to reach the goal of “Lifelong

Learning”.

A usability test is intended to determine the extent an interface facilitates a user’s

ability to complete routine tasks. Typically the test is conducted with a group of

potential users either in a usability lab, remotely (using e-meeting software and

telephone connection), or on-site with portable equipment. Users are asked to

437


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

complete a series of routine tasks. Sessions are recorded and analyzed to identify

potential areas for improvement to the web site.

This research conducted an onsite usability test using a live version of NCTU OCW

located on the test administrator’s laptop. The laptops using SCREEN2EXE software

captured the participant’s comments and the navigation choices. And also record the

whole testing time by Sony DV. The test administrator and participant were present in

the testing room. The session captured each participant’s navigational choices, task

completion rates, comments, overall satisfaction ratings, questions and feedback.

4.2 Executive Summary

This research conducted an onsite usability test during the October, 2012. The

purpose of the test was to assess the usability of the web interface design, information

flow, and information architecture.

Eight people participated in this usability test. Typically, a total of six to eight

participants are involved in a usability test to ensure stable results. Each individual

session lasted approximately one hour.

In general all participants found the NCTU OCW website to be clear, straightforward,

and 62.5% (5 of the 8 participants) thought the web site was not difficult but also not

easy to use. 2 of the 8 participants (25%) thought the website was easy to use.

The test identified only a few minor problems including:

• The lack of the video frame capture in the area of speech on each videos

• Lack of the link on each topic text of the course or lesson

• Lack of the hot recommendation page(should be an independent page, not just an

area on the top of the course page or one of the area of the front page.)

• Lack of a categorization page under the discussion page. / Discard the category of

“calculus, physics, and chemistry field” but set the category choices in the

questionnaire for the user to choose from the dynamic dropdown list.

• Lack of a categorization in the speech page. There are too many speeches

launched on its page.

• Confusion about the categorization of the college. (e.g., Institute of Architecture

is in the College of Social Science in NCTU, but in the College of Planning and

Design in NCKU.)

• Confusion about the search function bar. (e.g., what to type in the search bar)

• Lack of the discussion or feedback on the video pages.

• The place of the submenu is not directly. (e.g., the button of “curriculum front

page”, “curriculum videos”, “curriculum outlines” and “curriculum schedule”.)

This document contains the participant feedback, satisfactions ratings, task

completion rates, ease or difficulty of completion ratings, time on task, errors, and

438


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

recommendations for improvements. A copy of the scenarios and questionnaires are

included in the Attachments’ section.

4.3 Methodology

4.3.1 Sessions

The test administrator contacted and recruited participants via NCKU staff (includes

the assistant, students, teachers…etc.). The test administrator sent e-mails informing

them of the test logistics and requesting their availability and participation.

Participants responded with an appropriate date and time.

Each individual session lasted approximately one hour. During the session, the test

administrator explained the test session and asked the participant a brief introductory

questions (see Attachment A). Participants read the task scenarios and tried to find the

information on the website.

After each task, the administrator asked the participant some questions in order to get

the user’s impressions on the task. Post-task scenario subjective measures included:

• How easy it was to find the information from the home page.

• Ability to keep track of their location in the website.

• Accurateness of predicting which section of the website contained the

information.

After the last task was completed, the test administrator asked the participant the exit

questions to get the impressions of the website (see Attachment B). In addition, the

test administrator asked the participants the following overall website questions:

• What the participant liked most.

• What the participant liked least.

• Recommendations for improvement.

And asked them to rate the website overall by using a 5-point Likert scale (Strongly

Disagree to Strongly Agree) (see Attachment C) for eight subjective measures

including:

• Ease of use

• Frequency of use

• Difficulty to keep track of location in website

• Learn ability - how easy it would be for most users to learn to use the website

• Information facilitation – how quickly participant could find information

• Look & feel appeal – homepage’s content makes me want to explore the site

further

• Site content – site’s content would keep me coming back

• Site organization

439


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

4.3.2 Participants

The participants were from various types of field, it includes university student,

graduate school student, assistant from professor, junior high school teacher and

unemployed male.

8 participants were scheduled over the testing. All of the 8 participants completed the

test. Of the 8 participants, 4 were male and 4 were female.

Profession

University

Student

Graduate

School

Student

Assistant

from

Professor

Junior High

School

Teacher

1 4 1 1 1

4.3.3 Evaluation Tasks/Scenarios

Unemployed

Test participants attempted completion of the following tasks:

• Find the Introduction to OCW

• Find the Important Message

• Find the speech of “Crazy about the opera—Teaching you how to sing A

Cappella”

• Find the course video of “Outlines of Atchitecture -97—Green Architecture”

• Find the course outline content of “Outlines of Atchitecture -97—Green

Architecture”

• Find the Hot Recommendation Course “Liner Algebra I -- Department of Applied

Mathematics”

• Find the course handout of “Liner Algebra I -- Department of Applied

Mathematics”

• Search for—“統計學 Statistics 統計研究所 陳鄰安老師”

• Conduct a new “Chemistry” topic discussion

• Send the Feedback

4.4 Results

4.4.1 Task Completion Success Rate

All participants successfully completed Task 2, 3, 4, 6, 7, 8 and 10. Five of the eight

(62.5%) completed Task 1(find the introduction to OCW). Approximately whole

(87.5%) of participants were able to complete Task 5 (find the course outline content

of “Outlines of Architecture -97—Green Architecture”) and 75% were able to

complete Task 9 (conduct a new “Chemistry” topic discussion). Three of the

participants (37.5%) had more intention to find the additional task. But two of the

three (66.6%) completed the additional task.

440


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

Task Description

Task Test Description

1 Find the Introduction to OCW

2 Find the Important Message

3

Find the speech of “Crazy about the opera—Teaching you how to sing A

Cappella”

4 Find the course video of “Outlines of Atchitecture -97—Green Architecture”

5

6

7

Find the course outline content of “Outlines of Atchitecture -97—Green

Architecture”

Find the Hot Recommendation Course “Liner Algebra I -- Department of

Applied Mathematics”

Find the course handout of “Liner Algebra I -- Department of Applied

Mathematics”

8 Search for—“統計學 Statistics 統計研究所 陳鄰安老師”

9 Conduct a new “Chemistry” topic discussion

10 Send the Feedback

Task Completion Rates

Partici

pant

Task

1 2 3 4 5 6 7 8 9 10 11

1 √ √ √ √ √ √ √ √ √ √ na

2 √ √ √ √ √ √ √ √ √ √ na

3 √ √ √ √ √ √ √ √ √ √ na

4 √ √ √ √ √ √ √ √ √ √

5 √ √ √ √ √ √ √ √

6 √ √ √ √ √ √ √ √ √ na

7 √ √ √ √ √ √ √ √ √ √

8 √ √ √ √ √ √ √ √ √ na

Success 5 8 8 8 7 8 8 8 6 8 2

Compl

etion

Rates

62.5

%

4.4.2 Time on Task

100

%

100

%

100

%

87.5

%

The testing software recorded the time on task for each participant. Some tasks were

inherently more difficult to complete than others and is reflected by the average time

on task.

Task 4 required participants to find the lesson of Green Architecture video in the

441

100

%

100

%

100

%

75

%

100

%

66

%


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

course of Outlines of Architecture-97semester and took the longest time to complete

(mean = 350 seconds). However, completion times ranged from 60 (1 minute) to 350

seconds (more than 5 minutes and approximately 6 minutes) with most times less than

120 seconds (less than 2 minutes).

Time on Task

Partici

pant

Task

1 2 3 4 5 6 7 8 9 10 11

1 62 18 50 60 60 72 80 165 40 10 na

2 111 15 40 112 28 176 148 33 53 1 na

3 18 3 35 90 19 13 50 20 39 15 na

4 58 19 12 86 16 101 94 16 13 1 50

5 43 11 39 307 77 88 225 36 21 3 38

6 38 15 22 350 50 93 68 42 14 26 na

7 33 5 55 65 38 50 30 26 13 32 10

8 39 14 44 200 32 49 85 44 43 41 na

Avg.

TOT*

50.2

5

4.4.3 Summary of Data

12.5 37.1

25

158.

75

40

80.2

5

97.5 47.7

5

29.5 16.1

25

32.6

The table below displays a summary of the test data. Low completion rates and

satisfaction ratings and high errors and time on tasks are highlighted in red.

For example:

Summary of Completion, Errors, Time on Task

Task Task Completion Errors Time on Task

1 5 2 50

2 8 0 13

3 8 3 37

4 8 8 159

5 7 3 40

6 8 6 80

7 8 6 98

8 8 2 48

9 6 5 30

10 8 2 16

4.5 Overall Metrics

4.5.1 Overall Ratings

After task session completion, participants rated the site for eight overall measures

(See Attachment C). These measures include:

442

67


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

• Ease of use

• Frequency of use

• Difficulty of keeping track of where they were in the site

• How quickly most people would learn to use the site

• Getting information quickly

• Homepage’s content facilities exploration

• Relevancy of site content

• Site organization

Half of the participants (50%) agreed (i.e., agree or strongly agree) that they will use

this system frequently and that the site’s content would keep them coming back. Also

half of the participants (50%) agreed that the system is unnecessarily complex, and

agreed the system very cumbersome to use as well. Only 12.5% agreed that they felt

confident while using the system.

SUS Likert Scale (see Attachment C)

Task

1

2

3

4

5

would like to

use this

system

frequently

found the

system

unnecessarily

complex

thought the

system was

easy to use

need the

support of a

technical

person to be

able to use

this system

found

various

the

functions in

this system

were well

Strongly

Disagree

Disagree Neutral Agree Strongly

Agree

Percent

Agree

2 2 3 1 50%

1 3 4 50%

1 5 2 25%

4 4 0%

1 5 2 25%

443


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

Task

6

7

8

9

10

integrated

too much

inconsistency

in this system

most people

would learn

to use this

system

quickly

very

found the

system very

cumbersome

to use

confident

using the

system

needed to

learn a lot of

things before

I could get

going with

this system

Strongly

Disagree

Disagree Neutral Agree Strongly

Agree

Percent

Agree

3 2 3 37.5%

2 3 1 2 37.5%

1 1 2 4 50%

1 2 4 1 12.5%

3 3 2 25%

*Percent Agree (%) = Agree & Strongly Agree Responses combined

4.5.2 Likes, Dislikes, Participant Recommendations

Upon completion of the tasks, participants provided feedback for what they liked

most and least about the website, and recommendations for improving the website.

Liked Most

The following comments capture what the participants liked most:

The latest news

Homepage video timeline part

Speech content

Course schedule, video (chapter list)

Advice feedback

Discussion page

Homepage

Platform design (clear, beautiful, no ads, easy to understand)

444


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

Liked Least

The following comments capture what the participants liked the least:

Search function

Category of course and speech (should be detailed list down)

The category of the school college

Hot recommendation

The submenu placed in a indirectly area which is not obvious

Cheerless of the feedback

Font size too small

Too many videos in the page of speech page

Recommendations for Improvement

The link should be set on the topic. (e.g., hot recommendation, course and speech

topic)

The homepage’s gradient color could be more nature.

GUI could be much younger. (it’s too official)

The submenu placed in a indirectly area which is not obvious. It could put it just

under the video or just under the title of the course or speech topic.

The department category of the college is not clear. (Because there are different

categories in different schools)

More interactive dynamic dropdown list

The search function should be more obviously and more functional consistently.

CHAPTER 5 Conclusion

Most of the participants found NCTU OCW to be well-designed of it platform, clean

and uncluttered. But most of them thought that it is not so easy to use for the first time

user. It needs time to get familiar with it. In brief, user-centered design should take

more parts in this website design. Implementing the recommendations and continuing

to work with users will ensure a continued user-centered website.

445


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

CHAPTER 6 References

Albion, P. (1999). Heuristic evaluation of educational multimedia: from theory

to practice. Paper presented at the Australasian Society for Computers in

Learning in Tertiary Education (ASCILITE), Brisbane, Austrlia.

Govindasamy, T. (2001). Successful Implementation of E-Learning: Pedagogical

Considerations.

Jakob Nielsen, R. L. M. (1994). Usability Inspection Methods: Wiley; 1 edition

(April 25, 1994).

Mayer, R. C. C. R. E. (2008). e-Learning and the Science of Instruction (pp.

10-11).

Quinn, C. N. (1996). Pragmatic evaluation: lessons from usability. Paper

presented at the Making new connections : ASCILITE 1996 : proceedings of the

1996 annual conference, 1996.

Reeves, T. C. B., Lisa; Elliott, Dean; Grant, Michael; Holschuh, Doug; Kim,

Beaumie; Kim, Hyeonjin; Lauber, Erick; Loh, Sebastian. (2002). Usability and

Instructional Design Heuristics for E-Learning Evaluation.

Xu Cheng, C. D., Jiangchuan Liu. (2007). Understanding the Characteristics of

Internet Short Video Sharing: YouTube as a Case Study. Paper presented at the

CoRR.

Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F. (2004). Can e-learning

replace classroom learning? Communications of the ACM, 47(5), 75-79. doi:

10.1145/986213.986216

446


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

CHAPTER 7 Appendix

Attachment A-Brief Introductory Questions

447


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

Attachment B-Exit Questions

448


The Asian Conference on Media and Mass Communication 2012

Official Conference Proceedings Osaka, Japan

Attachment C-5-point Likert scale

449

More magazines by this user
Similar magazines