Spring / Summer 2013 - Rochester Institute of Technology

Focus Area | Accessibility in Computing
Integrated Video to Improve Math Access: Roger Gaborski, Anurag
Agarwal, and Richard Zanibbi (from left to right) are the co-principal
investigators on the project “AccessMath: Improving Math Lectures
for Low Vision Students through Integrated Video, Note-Taking
and Search.” For the project, Agarwal is creating course notes
and content for lectures on linear algebra. Zanibbi and Gaborski,
experts in computer vision and computational acoustics, are enabling
the system to recognize the mathematical algorithms in the lectures.
Prototype Could Equal
Ease of Use in Math Lecture
Ludi also recently received a grant from
the National Science Foundation for a
project titled “AccessMath: Improving
Math Lectures for Low Vision Students
through Integrated Video, Note-Taking
and Search.”
Anurag Agarwal, professor in the
School of Mathematical Sciences, and
Roger Gaborski and Richard Zanibbi,
professors in the Golisano College’s
department of computer science, are
co-principal investigators on the grant.
The goal is to make math lectures
more accessible to students with low
vision. Initially, it will be tested on
university-level students but the
application could be used at any
grade level. Here’s how it works:
The commercially sold product Mimio
captures text as it’s written on a white
board that’s connected to a magnetic bar.
The pens are encased in sleeves that act
as transmitters. When the pen is pushed
against the whiteboard, the Mimio tracks
the location of the pen and records a
digital “stroke.” The pen strokes will
be captured on a server along with
the video and audio from each lecture.
Simultaneously, the lecture is broadcast
in real time on the students’ iPads.
“It’s a very exciting project to be part
of,” says Zanibbi. “For students with low
vision it’s not easy to search through large
amounts of visual information. We are
trying to develop algorithms to search
through on recorded videos and pen
strokes as well as course notes using
images from lecture videos and
handwritten sketches as queries to better
support the students’ participation both
during and outside of lecture.”
Zanibbi’s and Gaborski’s expertise
in computer vision and computational
acoustics is needed to develop algorithms
that index and search the video data.
Agarwal, who teaches linear algebra
courses to first- and second-year students,
is creating course notes and content for
lectures on linear algebra. For example,
if a student were to search for the Pythagorean
theorem, a 2 + b 2 = c 2 , the theorem
could be retrieved from the database of
archived lectures.
The next phase of the prototype is to
roll out some mock-up lectures to low-
18
Spring/Summer 2013 Report