An Interactive and Team Approach to Multimedia Design Curriculum

An Interactive and Team Approach to Multimedia Design Curriculum

[dsp EDUCATION]Min Wu and K.J. Ray LiuAn Interactive and Team Approachto Multimedia Design CurriculumOver the past decade,increasingly powerfultechnologies have made iteasier to compress, distribute,and store multimediacontent. Many internationalmultimedia standards have been establishedto enable new technologies—HDTV, DVD, digital audio, and real-timeInternet games, just to name a few—to beenjoyed by millions of people around theglobe. Simultaneously, advances in wirelessand networking technologies, alongwith a significant decrease in cost for storagemedia, have led to the proliferation ofmultimedia data. This convergence of networking,computing, and multimediatechnologies has collapsed the distancebetween the ability to create content andthe ability to consume content.The merger of computing and communicationshas created a ubiquitousinfrastructure that brings digital multimediacloser to users, opening uptremendous educational and commercialopportunities in multimedia contentcreation, delivery, rendering, and archivingfor millions of users worldwide. Assuch, multimedia has become a basicskill demanded by an increasing numberof potential jobs for electrical engineering/computerscience (EE/CS) graduates.Yet multimedia has been widelyregarded by educators as a very difficultsubject to teach at the undergraduatelevel because it involves a wide spectrumof issues, including coding, compression,enhancement, communication,data mining, and rendering of speech,audio, image, video, and their combinations;it also involves interaction withdatabases, networking, and artificialintelligence. At the graduate level, oftenwe find each medium (e.g., image andspeech) is a subject taught separately,focusing on specialized techniques andadvanced theories. Given the limitedtheoretical foundations of undergraduatestudents, one can imagine that it iseven harder to teach them multimediatechnologies in an interesting, yet nonsuperficialway.Multimedia has many practicalaspects, with a variety of applicationsattractive to undergraduate students,especially members of the new digitalgeneration who are introduced to allsorts of digital media as children. A studentmust have some hands-on experiencein developing algorithms andperforming simulations before he/shecan have a level of true understandingand appreciation of the theoreticalaspects in advanced studies.How much can an undergraduate studentlearn and practice during onesemester? And what is the best way forthem to learn multimedia? During thefall semester of 2001, the University ofMaryland, College Park, embarked on thedevelopment of a multimedia CapstoneDesign course targeting senior-level students.The course has been offered to seniorsand first-year graduate studentsevery semester since. It serves as our firststep toward a new way of teaching undergraduatemultimedia. The central idea isto teach students the principal conceptsthrough lectures, then to reinforce theirunderstanding through hands-on labexercises and team projects.In this article, we intend to shareour experience and new ways of thinkingabout curriculum development. Wehope it will be beneficial for colleaguesin multimedia signal processing areasfor use in developing or revising curriculumto fit the needs and resourcesof their own programs. We would like tocordially invite interested colleagues tojoin us in building an internationalpartnership on multimedia research andeducation for a broader impact.VISION AND CONCEPTSOF INNOVATIVEMULTIMEDIA EDUCATIONThe multimedia curriculum at the seniorlevel has high potential impact in termsof the number of prospective students.This makes it very important to thinkcarefully about the vision and conceptson which we want to base our uniqueapproach. As it is unrealistic to expect anundergraduate student to master theextensive theoretical background in amatter of months and finish with a perfectproject, our vision is to■ offer strong intuitive componentsrevealing a solid scientific perspectivesuch that students with limited backgroundcan participate and contribute■ provide room for creative thinkingthrough open-ended problemformulations■ match students with various backgroundsto the corresponding handson,open-ended projects that willcapitalize on their existing skills andspecial talents■ utilize visualization, interactivetools, or computer-aided designfacilities to shorten lengthy learningand implementation time so thatthe focus can be on the appropriateengineering context.In addition to understanding the fundamentalaspects of engineering disciplines,students need skills beyond theirtechnical fields. The technology developmentparadigm in the IT era is being revolutionized;now someone with an idea andknow-how can pull together enoughresources—capital, talent, facilities, andmarket—to be able to realize his or herIEEE SIGNAL PROCESSING MAGAZINE [14] NOVEMBER 20051053-5888/05/$20.00©2005IEEE

[dsp EDUCATION] continued IEEE SIGNAL PROCESSING MAGAZINE [16] NOVEMBER 2005dream. This paradigm of students’ futuresis quite different from the traditional pathof finding engineering jobs in large corporateand research laboratories and fittingin. The entrepreneurial spirit can beessential to the success of engineers of thedigital generation. We try to incorporateelements of entrepreneurship in the proposedcurriculum through a term project.A FUN WAY OF LEARNINGMULTIMEDIABased on the vision and conceptsdescribed above, we developed and firstoffered the multimedia Capstone course(ENEE408G) in spring 2002. The experimenthas been quite encouraging, as evidencedfrom a report in an interview inthe summer 2003 issue of the UMD ECEConnections newsletter. Our course developmenthas employed a number ofmethodologies that have made ENEE408Ga unique and attractive course.WELL-DEVELOPED, HANDS-ONLAB ASSIGNMENTSAs shown in Table 1, one of the two paralleltracks of the course covers the fundamentalsof four basic media types(speech, audio, image, and video)through eight-week lectures and labassignments. We use PowerPoint presentationsand Flash demos to illustrate fundamentalconcepts and multimediaexamples that cannot be vividly conveyedusing traditional blackboard teaching.The lecture learning is complementedby four lab assignments on audio,speech, image, and video. Each lab consistsof tasks of increasing difficultyand creativity levels. For example, inthe “Image Processing and DigitalPhotography” assignment, we first providestudents with conceptual understandingthrough playing with apowerful image editing tool (e.g., PaintShop Pro or Adobe Photoshop) to accomplishseveral image processing andenhancement tasks. We then give studentsa few MATLAB building blocks andask them to integrate and build a simpleJPEG-like image coder. Next, studentsare asked to study a sample program inthe Pocket PC multimedia programmingmanual that we have developed for themand extend it to design a small PocketPCimage processing program usingMicrosoft’s embedded Visual C++/Basic.Finally, each student team is provided ahigh-end digital camera to learn digitalphotography first hand and create an artpiece using the image processing techniquesthat they have learned.INNOVATIVE TERM PROJECTWITH ENTREPRENEURIAL SPIRITThe second track shown in Table 1 is aninnovative term project in which studentswork in teams to develop a creativeproduct on a desktop platform or amobile platform such as PocketPC. Sucha learning process is described in thecourse syllabus as follows:Term Design Project: This is ateam-based final project on designand implementation of multimediasignal processing systems usinghandheld devices or desktop computers.Each student team will emulatea high-tech company that will:1) develop a product idea anddecide on system specifications of amultimedia product,2) partition and coordinate thedesign tasks,3) implement, test, and documentthe design, and4) demonstrate and market theproduct.[TABLE 1] ENEE408G COURSE SCHEDULE CHART.BUILDING A TEAM CULTURE WITHSOUND ORGANIZATIONAL SKILLSWe recognize that it is unrealistic toexpect every student to master an extensivetechnical background and finishwith a perfect project in a matter ofmonths. Therefore, teamwork andsound team spirit are key to success. Weencourage students to use their imaginationand creativity to find an interestingidea that they are excited to develop;then we guide them to complete a proofof-conceptprototype within four to sixweeks through well-organized teamwork.We experimented with various ways offorming student teams. For example, studentsare assigned at random to formteams for the first design project, and theyare assigned to work with different partnerson the second project. From the thirdproject on, the students are asked to teamup voluntarily and stay with one team forthe remainder of the semester. This way“forces” all students to learn to work withpeople they are unfamiliar with, yet allowsfor some stability and enough time todevelop expertise and team culture for thefinal big project. Students have incentivesto behave well and contribute to the teamwork from the first project; otherwise,they may be embarrassed if no groupwants them as a member.A peer evaluation form is collectedafter each project to give us an ideaWEEK LECTURE LAB ASSIGNMENT TEAM PROJECT1 INTRODUCTION WARM UP ONMATLAB ANDPOCKETPC2 IMAGE PROCESSING IMAGE TEAM UP34 VIDEO CODING VIDEO5 IDEA DEVELOPMENT6 SPEECH PROCESSING SPEECHAND RECOGNITION7 DEFINE SYSTEM SPEC8 AUDIO PROCESSING AUDIO SCHEDULINGAND RIGHTSMANAGEMENT9 INSTRUCTOR’S APPROVAL OFDEVELOPMENT PLAN10 PROJECT DEVELOPMENT:1) JOB PARTITION, 2) PROGRESSREVIEW, 3) VERIFICATION,4) INTEGRATION AND TESTING,AND 5) DOCUMENTATION11121314 PROJECT PRESENTATION AND DEMO15 FINAL REPORT DUEIEEE SIGNAL PROCESSING MAGAZINE [16] NOVEMBER 2005

about team dynamics and help resolveany problems. The evaluation form alsoasks each student to briefly summarizehow the job is divided, processed, andintegrated by the team. One intent ofthis question is to ensure that a studentunderstands both his or her own roleand other teammates' roles in the project,regardless of whether the studentwas in a leader position or was being led.INCORPORATE UP-TO-DATEDEVELOPMENT TOOLSAND DEVICESTo prepare students to meet the demandsof the IT workforce of the 21st century, itis important to equip them with behindthe-scenesknowledge as well as algorithmdevelopment and implementationskills. We incorporate the most up-todate,state-of-the-art tools and devicesinto the course, such as PocketPC, digitalphoto and video cameras, multimediaeditors, and speech processing software.Recognizing that many employers expectthat students have already mastered cutting-edgesoftware tools and platforms,we have adopted in our course the latestplatforms on desktop PC and PocketPC,as well as major prototyping and developmenttools such as MATLAB, VisualStudio, and Embedded Visual Studio.Given today’s rapidly advancing pace inthe technology field, the ability to learnto use references and examples to quicklymaster new concepts and tools hasbecome an important skill for EE/CS students.This is why we have provided studentswith examples and referencematerial to help them learn new toolsfor new platforms (such as programmingon PocketPC), which is mostlyconducted on their own. Through labassignments and projects, students masterthe tools with first-hand experienceand extensive practice.SYSTEMATIC ASSESSMENTOF STUDENTS’ LEARNINGWe have constructed a systematic assessmentmechanism for student projectsand teamwork. For each lab and project,we have developed a grading sheet thatoutlines major tasks, requirements, andgrading criteria. At the beginning of theproject, we clearly convey to the studentsthe expectations and rationale behind thegrading in terms of developing key skillsuseful for their future careers. Alongwith our department undergraduateoffice, we have developed assessmentrubric on oral communications, writtenreport, and engineering designs, to quantitativelyassess students’ performanceand provide timely feedback that helpsstudents improve.For team-oriented projects, we havedeveloped the teamwork evaluation questionnaireto solicit students’ peer assessment.Handed out before the start of theproject, the questionnaire is intended toprovide constructive guidelines andincentives for students to develop effectivemechanisms for working together. Forexample, the importance of timely communicationfor effective teamwork isemphasized on the questionnaire througha multiple-choice question about how[FIG1] Screen shots of the student project “Kanji Tutor.”IEEE SIGNAL PROCESSING MAGAZINE [17] NOVEMBER 2005

[dsp EDUCATION] continued IEEE SIGNAL PROCESSING MAGAZINE [18] NOVEMBER 2005soon students communicate and respondto teammates, whether it is promptly(within a day), moderately (within two tothree days), or rarely. We complement thepeer evaluation with close observationand interaction with the student teamduring the lifespan of the project. Thisallows us to provide timely feedback andguidance for students to resolve issues inteam coordination and improve the effectivenessof teamwork.EXAMPLES OF TERM PROJECTS“Kanji Tutor” is one example of capitalizingon students’ special talents and creativity.As a beginner learning Japanese,one student led her team to develop anovel PocketPC tool to facilitate learningto read and write sophisticatedJapanese/Chinese characters (“Kanji”).As illustrated in Figure 1, the studentteam developed an algorithm to verifythe correctness of Kanji writing in differentstrictness levels. The team designed afriendly user interface and createdunique animations and quizzes to helpusers memorize Kanjis.Inspired by a TV remote controllersoftware coming with PocketPC, anotherstudent team took just six weeks to completethe creation of “PocketPoint”shown in Figure 2. This prototypedemonstrates the concept of usingPocketPC to wirelessly control the flowof a desktop PowerPoint presentation; itprovides a mobile visual aid such asthumbnails or a preview of slides. Withthis unique course, the creativity andproof-of-concept capability achieved bythe undergraduate students are often farabove expectation. For example, an ideasimilar to PocketPoint was independentlyinvestigated as a multiyear researchproject at another institution.A student team led by an amateurguitar player developed “PocketInteractive Chords,” a PocketPC applicationthat allows users to tune a guitarand learn to play basic guitar chords.Capturing guitar sound from the built-inmicrophone of PocketPC, the studentsused their signal processing knowledgeto develop estimators for tuning guitarstrings and identifying notes and chordsbeing played. They designed other funmodules and integrated them into acomplete program with a rich set of features,including tuning, tutoring, andinteractive quiz, as shown in Figure 3.We have also exposed students to topicsfrom the latest research and haveencouraged them to develop projectskeyed to emerging technologies. Securemultimedia multicast, content-basedaudio fingerprinting, and digital watermarkingare just a few examples. Forthese research-oriented projects, weinvited graduate students to serve asmentors for the undergraduate teams.These efforts have provided an effectivevehicle for fostering interaction betweenundergraduate and graduate studentsand integrating research and undergraduateeducation.(a)[FIG2] Screen shots of the student project “PocketPoint.”(a) (b) (c)[FIG3] Screen shots of the student project “Pocket Interactive Chords.”(b)SHARING AND COLLABORATIONOF MULTIMEDIA RESEARCHAND EDUCATIONMultimedia has become a vital componentof the digital era, and it has fundamentallychanged the way we work andlive. Research related to multimedia hasgrown to encompass a number of areas,such as multimedia coding, communications,synthesis, indexing, search,retrieval, recognition, understanding,security, and forensics. The global natureof information technology has nurtureda number of strong research groups withunique strengths worldwide. Multimediaresearch has evolved to a stage where asingle institution often specializes in asubset of active areas or particularmethodologies. Collaboration amongmultiple institutions across the globeIEEE SIGNAL PROCESSING MAGAZINE [18] NOVEMBER 2005

will provide a breeding ground for cooperativeresearch and education for multimediatechnologies with much broaderimpact. This will, in turn, allow peopleall over the world to overcome physicaldistances and access a huge amount oftext, image, audio, and video informationthrough networks.To respond to the global demand andthe technical skills required by the newgeneration of the workforce, we aresharing our development experienceand material. We intend to build aninternational partnership for a combinedresearch and education programon multimedia. The program will offermultimedia education at the undergraduatelevel with a hands-on, team-orientedapproach, and it will aid incooperatively developing and transferringthe latest results from multimediarelatedresearch to education in a timelyfashion. Close collaboration in developingand sharing research results andresources can expedite the transfer ofnew research into curriculum for theglobal, high-tech multimedia workforce.A set of course modules, includingpresentation material, lab assignments,and sample codes, will form a repositoryto enable multimedia educators worldwideto assemble suitable material forbuilding a multimedia course tailoredto their specific needs and facilities. Ifany reader is interested in teaming upand participating in this program,please feel free to contact the authors.FINAL REMARKSA number of students have found themultimedia course added much to theirpersonal and professional interests.Student comments include:“This course was my first fullfledgedexposure to multimedia signalprocessing. Now, I hope toinvestigate multimedia signal processingin work or in graduateschool at least on some level. Thishas become one of my favoriteinterdisciplinary topics.”“I had a lot of fun in yourcourse. I learned a lot, which Ihope to use very shortly. I have aproduct idea that I am going tostart designing . . .”“The reason I applied for jobsinstead of graduate schools last fallis that I had not yet found a problemthat I was passionate about. . . .However, last semester, I finallyfound what I was looking for. I wantto devote myself to signal processingchallenges . . . Taking 408G openedmy eyes to all sorts of signal processingproblems I would love to tackle.”Dr. Teddy Kumar, technical director ofMedia Vision Laboratory of SarnoffCorporation, came to the University ofMaryland in early March to recruit students.A number of former and current ENEE408Gstudents were invited for interviews. In an e-mail to the instructors, Dr. Kumar wrote,“Almost all students said that the best coursethey had taken at the University of Marylandwas the multimedia signal processing course.They were all excited about their work init and the hands-on project experience.”For more information, please visit theonline showcase at or contact the authors at kjrliu@umd.eduand Wu is an assistant professor of theDepartment of Electrical and ComputerEngineering and the Institute ofAdvanced Computer Studies at theUniversity of Maryland, College Park.K.J. Ray Liu is a professor in theElectrical and Computer EngineeringDepartment and the Institute of SystemsResearch at the University of Maryland,College Park.[SP]IEEE SIGNAL PROCESSING MAGAZINE [19] NOVEMBER 2005

More magazines by this user
Similar magazines