I. OBJECTIVES, OUTCOMES and ASSESSMENT A ... - pace university
I. OBJECTIVES, OUTCOMES and ASSESSMENT A ... - pace university
I. OBJECTIVES, OUTCOMES and ASSESSMENT A ... - pace university
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
I. <strong>OBJECTIVES</strong>, <strong>OUTCOMES</strong> <strong>and</strong> <strong>ASSESSMENT</strong><br />
A. Background Information<br />
A.1. Degree Title:<br />
Bachelor of Science in Computer Science<br />
A.2. Program Modes:<br />
This is a single BS in Computer Science program with day, late afternoon <strong>and</strong><br />
evening classes offered on the Pleasantville <strong>and</strong> New York City campuses.<br />
CS271, "Fundamentals of the UNIX Operating System," a one-credit, required<br />
class, has been presented as an online class since before Fall 2004. (For Fall<br />
2006 CS271 was changed to “Fundamentals of UNIX <strong>and</strong> C Programming” <strong>and</strong><br />
made two credits.)<br />
A.3. Actions to Correct Previous Concerns:<br />
The previous visit reported no concerns.<br />
A.4. Contact Information:<br />
Dr. Susan M. Merritt, Dean<br />
The Ivan G. Seidenberg School of Computer Science <strong>and</strong> Information Systems<br />
Goldstein Academic Center<br />
Pace University<br />
861 Bedford Road<br />
Pleasantville, NY 10570-2799<br />
Telephone: 914-773-3750 E-mail: smerritt@<strong>pace</strong>.edu<br />
B. Accreditation Summary<br />
B.1.A. Objectives <strong>and</strong> Outcomes<br />
The Seidenberg School has a long st<strong>and</strong>ing commitment to assessment. A<br />
review of the major assessment activities over the past ten years is in<br />
Attachment I-11.<br />
The last accreditation report from the Middle States Commission on Higher<br />
Education singled-out the Seidenberg School for the “numerous things [it has<br />
done] to maintain, assess <strong>and</strong> validate the quality of its offerings.” For the past<br />
six years a computer science faculty member has been provided a stipend to<br />
serve as Assessment Director. In Summer 2004, as assessment activities<br />
intensified, the School hired a full-time assessment/research analyst.<br />
Seidenberg representatives participate actively in the bimonthly meetings of the<br />
University-wide Assessment Committee <strong>and</strong> have taken the lead in various<br />
1
initiatives including authoring the frequently asked questions section of the<br />
University’s assessment web site.<br />
The School held its first full faculty Assessment Day on May 3, 2006 <strong>and</strong> has<br />
scheduled it as an annual event. Activities consist of occasional workshops but<br />
mostly focus on evaluating assessment data. In response to Assessment Day<br />
<strong>and</strong> other assessment initiatives this past year, the School received high acclaim<br />
from the Assistant Vice President of Planning, Assessment, Research <strong>and</strong><br />
Academic Budgeting (Attachment I-12). The most vigorous acclaim was the 15<br />
million dollar endowment to the School from Ivan G. Seidenberg, the CEO of<br />
Verizon, in recognition of the School’s effectiveness in preparing students.<br />
B.1.A1. Provide the institution's mission statement. Include any other<br />
mission statements that are relevant.<br />
Mission of Pace University<br />
Pace University is committed to providing the best possible private education to a<br />
diverse <strong>and</strong> talented student body at each Pace University location. The<br />
University’s continuous commitment to Opportunitas must be retained,<br />
strengthened <strong>and</strong>, when necessary, redefined in an uncertain world where many<br />
are excluded from the full benefits of higher education due to economic <strong>and</strong><br />
social factors beyond their control. Pace’s sustained emphasis on excellence will<br />
be a constant, which will guide decision-making regarding academic programs<br />
<strong>and</strong> other endeavors. In a world of increasing interdependence, Pace must<br />
strengthen its international efforts <strong>and</strong> continue its significant investment in<br />
technology. In addition, the University’s commitment to self-evaluation <strong>and</strong> civic<br />
engagement must remain as major goals. All of this must be accomplished<br />
during a time of considerable economic change <strong>and</strong> in the context of<br />
strengthening Pace’s own financial base.<br />
from: Pace University Second Century Strategic Plan 2003-2008 page 1<br />
Mission of the Ivan G. Seidenberg School of Computer Science <strong>and</strong> Information<br />
Systems<br />
The mission of the School is to prepare men <strong>and</strong> women for meaningful work,<br />
lifelong learning, <strong>and</strong> responsible participation in a new <strong>and</strong> dynamic information<br />
age.<br />
from: Pace University Undergraduate Catalog: 2004-2006, page 160<br />
2
B.1.A.2. Describe how your program's educational objectives align with<br />
your institution's mission.<br />
Opportunitas is the one word that describes the mission of Pace University since<br />
its founding one hundred years ago. It refers to delivering an education that<br />
makes professional success accessible to talented men <strong>and</strong> women. The<br />
<strong>university</strong> serves a highly diverse population. Attachment I-1 presents statistics<br />
documenting the success of the School’s diversity efforts.<br />
The BS in Computer Science program, accredited by the Computer Science<br />
Accreditation Commission of the Computing Sciences Accreditation Board in<br />
1986, emerged in this context. The dominant objective was <strong>and</strong> remains<br />
preparing graduates to be industrial professionals who could take the lead issues<br />
related to software <strong>and</strong> other areas of computer science. To this end, graduates<br />
are supplied with a durable knowledge structure that supports the development<br />
of expertise in contemporary computing environments <strong>and</strong> the ability to stay<br />
current as the field advances <strong>and</strong> changes. Graduates are also supplied with the<br />
verbal abilities <strong>and</strong> teamwork skills required to prosper as well as the know-how<br />
to be an ethical professional enhanced by an awareness of the social, legal,<br />
security <strong>and</strong> global issues surrounding applied computing.<br />
A professional education in computing, combined with the arts <strong>and</strong> sciences,<br />
always striving for diversity, is consistent with the University’s mission.<br />
3
B.1.A.3. List the program’s documented measurable educational objectives.<br />
Graduates of the B.S. in Computer Science program will:<br />
1. be equipped for responsible <strong>and</strong> ethical participation <strong>and</strong> leadership roles<br />
as computing professionals. This extends to a readiness for lifelong<br />
learning in many forms including graduate study.<br />
2. underst<strong>and</strong> the foundational ideas in computer science <strong>and</strong> be able to use<br />
this knowledge to guide their thinking in technical problem-solving.<br />
3. be effective in the design <strong>and</strong> construction of software applications.<br />
4. be able to work effectively on teams. This includes oral <strong>and</strong> written<br />
communication skills as well as collaborative skills.<br />
5. be informed on social issues that affect <strong>and</strong> are affected by the application<br />
of computing technology.<br />
B.1.A.4. Explain how the program's educational objectives align with the<br />
needs of its constituencies <strong>and</strong> include a list of the stakeholders.<br />
Stakeholders in Pace University's BS in CS Program<br />
• current computer science students<br />
• employers hiring new or recent graduates<br />
• alumni/ae<br />
• computer science faculty<br />
• the Seidenberg School’s Advisory Board<br />
The program's chief stakeholders are the computer science students, who need<br />
a program that will deliver readiness to begin <strong>and</strong> pursue progressive<br />
achievement in professional life. To this end, the objectives are formulated based<br />
on characteristics of a well qualified professional. Student input on the objectives<br />
is garnered by way of their academic advisors <strong>and</strong> through informal studentfaculty<br />
interaction. For example, students have spoken about developing the skill<br />
set for designing <strong>and</strong> building software applications (objective 3).<br />
Employers hiring new or recent graduates constitute another dependent<br />
constituency. Dependence resides in the investments that employers make in<br />
those they hire <strong>and</strong> in their need to entrust employees with critical tasks. The<br />
concrete needs of employers are determined mainly through Advisory Board<br />
meetings <strong>and</strong> consultations as well as supervisors of student interns. Over the<br />
past five years, collaborative skills <strong>and</strong> oral <strong>and</strong> written communication abilities<br />
have assumed central importance (objective 4). Another source of input are<br />
4
faculty members engaged with industrial projects who have evidence of<br />
employment needs.<br />
Program alumni/ae prosper to the extent that they have been provided with<br />
knowledge <strong>and</strong> abilities beyond entry level skills that will promote professional<br />
ascent. For example, life-long learning <strong>and</strong> being equipped for responsible <strong>and</strong><br />
ethical participation in professional roles is integral to positions of ever increasing<br />
responsibility (objective 1). Assessment taps into the experiences <strong>and</strong><br />
recommendations of recent alumni/ae with a survey, the latest of which was<br />
performed in November 2005. The results help to shape the objectives <strong>and</strong> the<br />
resulting curriculum.<br />
The faculty’s sense of professional accomplishment rests on the ability of the<br />
program to attract motivated students <strong>and</strong> prepare them for success. For<br />
example, when speaking about program worthiness, the faculty emphasizes a<br />
firm grasp of foundational ideas <strong>and</strong> the ability to use this knowledge in problemsolving<br />
(objective 2). Faculty input on the objectives comes primarily by way of<br />
their participation in the Curriculum Committee.<br />
Members of the Advisory Board extend themselves to see the School prosper in<br />
many ways. For example, the Advisory Board actively promotes the Annual<br />
Leadership in Service <strong>and</strong> Technology fund-raiser for scholarships. The stature<br />
of the School reflects upon their own. Beyond this, Advisory Board members are<br />
also employers, <strong>and</strong> as such they provide the employers’ opinions on the<br />
objectives. For example, informedness on social issues surrounding the<br />
application of computing technology is an objective they endorsed (objective 5).<br />
In the end, all of our stakeholders benefit when the program meets each of the<br />
five objectives <strong>and</strong> produces well-rounded, well-equipped graduates.<br />
B.1.A.5. For each program educational objective, indicate the following:<br />
• the mechanism(s) used to measure it.<br />
• when it is measured <strong>and</strong> who is responsible for obtaining the<br />
measurement.<br />
• the extent to which the educational objective is being met.<br />
• all improvements, if any, that have been identified as a result of<br />
the assessment data collected.<br />
• the implementation time-line for any improvements identified.<br />
• where the improvement has been documented.<br />
5
Mechanism 1: The Alumni/ae Survey<br />
Assesses Objectives 1 through 5<br />
To the faculty, the evidence was strong that objectives two, three, <strong>and</strong> four were<br />
being satisfied. The leadership <strong>and</strong> lifelong learning parts of objective one were<br />
being satisfied as well.<br />
The chief instrument for assessing success in fulfilling the program's objectives is<br />
the Alumni/ae Survey (Attachment I-2). Its most recent administration was in Fall<br />
2005. The current assessment schedule has it on a three year cycle for<br />
distribution to graduates three to five years out. Responsibility for its<br />
administration rests with the Seidenberg School's Assessment/Research Analyst.<br />
This mechanism measures objectives 1 through 5.<br />
Twelve computer science faculty members, on the Spring 2006 Assessment Day,<br />
studied the summarization of the completed questionnaires (Attachment I-2).<br />
The technique was to perform a content analysis of responses, looking for<br />
indications of objective realizations in the descriptions of the job responsibilities.<br />
For instance, a senior software engineer gave a job description citing "project<br />
management," "requirements gathering," <strong>and</strong> "system design." From this the<br />
faculty inferred the manifestation of leadership, collaborative skills,<br />
communication skills, an ability to apply ideas of computer science in problemsolving,<br />
<strong>and</strong> abilities integral to the design <strong>and</strong> construction of software systems.<br />
Another alumnus mentioned serving as the "interim chairman of an industry<br />
st<strong>and</strong>ards committee for systems management interoperability." This was clear<br />
evidence of leadership (<strong>and</strong> the correlative skills of collaboration <strong>and</strong><br />
communication) as well as knowing when technical knowledge applies, where it<br />
applied, <strong>and</strong> how it applies. The question explicitly asking about graduate school<br />
study or industrial training to keep up with technology showed that virtually<br />
everyone was doing this. Of the alumni/ae who graduated from 1996 to 2003<br />
who responded, 35% had completed or were in a masters degree program <strong>and</strong><br />
78% had completed some additional education.<br />
Only implied in the results of the survey was the disposition toward ethical<br />
conduct <strong>and</strong> a conversance with social issues (objective 5). The faculty was<br />
convinced that the limitation lay in the instrument used for the alumni/ae survey,<br />
not in the absence of these characteristics among our graduates. This is because<br />
evidence shows that the program’s learning outcome seeding these, outcome e<br />
(to acquire an underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social responsibilities),<br />
is being met. Objective 5 is supported by the presence of the international<br />
computing honors society Upsilon Pi Epsilon (UPE) <strong>and</strong> other honor societies<br />
present on campus including Alpha Chi, the national college honor scholarship<br />
society as well as by continuing programs for alumni/ae.<br />
The next alumni/ae survey, due in 2009, canvassing the classes of 2004 to 2006,<br />
will include an additional item asking alumni/ae to identify the two or three most<br />
6
significant social, ethical, legal or security issues that they have encountered <strong>and</strong><br />
about their ability to deal with them. The instrument will also include another item<br />
asking alumni/ae about the international study opportunities of which they availed<br />
themselves. Since 2004, Seidenberg faculty have offered four courses open to all<br />
Seidenberg students that have included travel to either India or to Great Britain<br />
<strong>and</strong> France. These courses would typically be electives in the general education<br />
part of the curriculum. The University has long offered a multitude of international<br />
opportunities through international field study courses <strong>and</strong> study abroad<br />
programs.<br />
These new questions addressing objective 5 will be pre-tested in Fall 2007 with a<br />
small scale alumni/ae survey. Hence, relative to the assessment of the program's<br />
educational objectives <strong>and</strong> their evaluation with this mechanism, the central<br />
improvement is a change in the primary assessment instrument. The new<br />
instrument will be available for display along with the other materials for the<br />
visiting team in the Goldstein Academic Center, room 314.<br />
Mechanism 2: Employment of New Graduates<br />
Assesses Objectives 2, 3, <strong>and</strong> 4<br />
A successful job search following graduation is evidence of being equipped for<br />
professional participation. The employment data collected by the Cooperative<br />
Education <strong>and</strong> Career Services Department supply substantiation. For each<br />
graduating class, Cooperative Education <strong>and</strong> Career Services furnishes the<br />
Seidenberg Assessment/Research Analyst with information on the organizations,<br />
positions, <strong>and</strong>, when available, the salaries of graduates who obtained full-time<br />
jobs through their auspices (or located jobs without their assistance but were<br />
good enough to respond to an employment inquiry). With respect to the class of<br />
2004 (data for the class of 2005 was incomplete), the highest paid graduate from<br />
the BS in CS program took a job as a programmer/analyst with the SG<br />
Constellation Financial Management Company, LLC, for $85,000. The second<br />
highest paid graduate took a job as a Web Application Developer for CIBC World<br />
Markets for $75,000. Other placements included a programmer/analyst at MBIA,<br />
a programmer/analyst at Nextel Communications, <strong>and</strong> a software developer at<br />
Epic Systems Corporation. This comes from the table in Attachment I-3.<br />
The job placements shown in Attachment I-3 go back through 2001. They<br />
substantiate that graduating students are judged by employers to have the<br />
abilities, skills, <strong>and</strong> attitudes needed to add value to their organizations in the<br />
arena of designing <strong>and</strong> building software (objective 3). Owing to the nature of<br />
software construction, this necessarily carries with it fulfillment in the areas of<br />
objectives 2 <strong>and</strong> 4. The personal intangibles that make for confidence in a<br />
c<strong>and</strong>idate being interviewed are tied-up with objective 1.<br />
7
The improvement identified by the faculty is, once again, not in the program but<br />
in the assessment. Creating a survey asking whether students are getting the<br />
job offers they had expected or most wanted would be useful.<br />
Mechanism 3: Performance Reviews of Interns in Cooperative Education<br />
Assesses Objectives 2, 3 <strong>and</strong> 4<br />
The Cooperative Education <strong>and</strong> Career Services Department queries those<br />
responsible for supervising interns toward the end of each internship assignment.<br />
Supervisors are asked about “academic preparation”, “quality of work”, <strong>and</strong> other<br />
aspects of work performance.<br />
The Computer Science Department has access to <strong>and</strong> annually examines the<br />
performance reviews of undergraduates on cooperative education internships.<br />
Although undergraduates are not alumni/ae with three to five years of<br />
professional experience, interns are evaluated on criteria that converge upon the<br />
program's objectives <strong>and</strong> relate closely to long term professional success.<br />
Moreover, direct behavioral observation by external examiners is always<br />
informative. The following comments, culled from the data in Attachment I-8,<br />
speak of good capabilities <strong>and</strong> developmental progress.<br />
work attitude:<br />
• "was a great asset to our company; he was a hard<br />
worker <strong>and</strong> appreciated everything he learned"<br />
• "always asks for more work; very competent <strong>and</strong><br />
helpful; a pleasure to work with"<br />
• "great work ethic; always looking to do more"<br />
work quality<br />
• "demonstrates unusually high technical aptitude<br />
for a second year student"<br />
• "programs are well written <strong>and</strong> tested"<br />
• "overall he presents very good quality; occasional<br />
omission of details has been discussed with the<br />
student, <strong>and</strong> solid improvement has been observed"<br />
academic preparation<br />
• "demonstrates a good underst<strong>and</strong>ing of the W2K<br />
server environment"<br />
8
• "very good with h<strong>and</strong>s-on work, but we have not<br />
evaluated him on his theoretical skills"<br />
• "did not come in with all the skills, but was able<br />
to pick up everything very quickly"<br />
verbal <strong>and</strong> writing ability<br />
• "clear <strong>and</strong> concise"<br />
• "programs <strong>and</strong> documentation were well written"<br />
• "great writing skills"<br />
The Cooperative Education <strong>and</strong> Career Services Department reports that, "40%<br />
of graduating Pace co-op interns were offered full-time positions from their co-op<br />
employers upon graduation" (http://appserv.<strong>pace</strong>.edu/execute/page.cfm?doc_id=8008).<br />
Historically, this value is slightly higher for students in the BS in CS. Thus, since<br />
the program's objectives correspond to career readiness, this is a good indicator<br />
that they are being met.<br />
The multiple indicators above enable the conclusion that our program objectives<br />
en masse are being fulfilled.<br />
Metamechanism: Assessment Day<br />
Once a year, at the end of the Spring semester, the Seidenberg School holds an<br />
"Assessment Day" for the faculty hosted by the assessment staff for the purpose<br />
of "closing the loop.” "Closing the loop," consists of workshops <strong>and</strong> general<br />
discussions:<br />
• interpreting recent assessment data relative to evaluating how well<br />
program objectives <strong>and</strong> student outcomes are being met<br />
• examining the efficacy of past improvements<br />
• revamping past improvements that are not working as hoped <strong>and</strong> to plan<br />
new improvements (brainstorm, prioritize, formulate practical strategies)<br />
• considering the adequacy of assessment instruments <strong>and</strong> analyses<br />
Assessment Day was held on May 3, 2006. The program <strong>and</strong> the notes<br />
recording the events of the day are in Attachment I-4.<br />
B.1.A.6. List the program's student outcomes.<br />
The program's student outcomes (also referred to as the program's learning<br />
outcomes), specifying what students are expected to know by the time of<br />
graduation, are those promulgated by ABET for computer science programs<br />
9
(coming out of Engineering Criteria 2000: Criteria for Accrediting Programs in<br />
Engineering in the United States by the Engineering Accreditation Commission).<br />
a) An ability to apply knowledge of computing <strong>and</strong> mathematics appropriate to<br />
the discipline.<br />
b) An ability to analyze a problem <strong>and</strong> to identify <strong>and</strong> define the computing<br />
requirements appropriate to its solution.<br />
c) An ability to design, implement <strong>and</strong> evaluate a computer-based system,<br />
process, component, or program to meet desired needs.<br />
d) An ability to function effectively on teams to accomplish a common goal.<br />
e) An underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social responsibilities.<br />
f) An ability to communicate effectively with a range of audiences.<br />
g) An ability to analyze the impact of computing on individuals, organizations <strong>and</strong><br />
society, including ethical, legal, security <strong>and</strong> global policy issues.<br />
h) Recognition of the need for, <strong>and</strong> an ability to engage in, continuing<br />
professional development.<br />
i) An ability to use current techniques, skills, <strong>and</strong> tools necessary for computing<br />
practices.<br />
j) An underst<strong>and</strong>ing of the processes that support the delivery <strong>and</strong> management<br />
of information systems within a specific application environment.<br />
k) An ability to apply design <strong>and</strong> development principles in the construction of<br />
software systems of varying complexity.<br />
B.1.A.7. For each outcome, indicate the following:<br />
• the mechanism(s) used to measure it.<br />
• when it is measured <strong>and</strong> who is responsible for obtaining the<br />
measurement.<br />
• the extent to which the educational outcome is being met.<br />
• all improvements, if any, that have been identified as a result of<br />
the assessment data collected.<br />
• the implementation time-line for any improvements identified.<br />
• where the improvement has been documented.<br />
10
Computer Science<br />
Program<br />
Outcomes<br />
a) An ability to apply<br />
knowledge of<br />
computing <strong>and</strong><br />
mathematics<br />
appropriate to the<br />
discipline.<br />
b) An ability to<br />
analyze a problem,<br />
<strong>and</strong> identify <strong>and</strong> define<br />
the computing<br />
requirements<br />
appropriate to its<br />
solution.<br />
c) An ability to design,<br />
implement <strong>and</strong><br />
evaluate a computerbased<br />
system,<br />
process, component,<br />
or program to meet<br />
desired needs.<br />
d) An ability to<br />
function effectively on<br />
teams to accomplish a<br />
common goal.<br />
e) An underst<strong>and</strong>ing<br />
of professional, ethical<br />
<strong>and</strong> social<br />
responsibilities.<br />
f) An ability to<br />
communicate<br />
effectively with a range<br />
of audiences.<br />
g) An ability to<br />
analyze the impact of<br />
computing on<br />
individuals,<br />
organizations <strong>and</strong><br />
society, including<br />
ethical, legal, security<br />
<strong>and</strong> global issues.<br />
h) Recognition of the<br />
need for, <strong>and</strong> an ability<br />
to engage in,<br />
continuing professional<br />
development.<br />
i) An ability to use<br />
current techniques,<br />
skills, <strong>and</strong> tools<br />
necessary for<br />
computing practice.<br />
Table 1.1 shows which outcomes are informed by each of the mechanisms. Input on<br />
each outcome comes from multiple sources.<br />
Course<br />
Displays<br />
Table 1.1 Program Outcomes by Assessment Mechanisms<br />
Student<br />
Opinion<br />
Survey<br />
Summative<br />
Assessment<br />
Course<br />
Opinion<br />
Survey<br />
11<br />
Co-op <strong>and</strong><br />
Career<br />
Services’<br />
Survey of<br />
Internship<br />
Employers<br />
NSSE<br />
Advisory<br />
Board<br />
Curriculum<br />
Committee<br />
Academic<br />
Program<br />
Review<br />
X X X X X X X<br />
X X X X X X<br />
X X X X X X X X<br />
X X X X X X X X<br />
X X X X X X X X<br />
X X X X X X X X<br />
X X X X X X X<br />
X X X X X X X<br />
X X X X X X X X
Computer Science<br />
Program<br />
Outcomes<br />
j) An ability to apply<br />
mathematical<br />
foundations,<br />
algorithmic principles,<br />
<strong>and</strong> computer science<br />
theory in the modeling<br />
<strong>and</strong> design of<br />
computer-based<br />
systems in a way that<br />
demonstrates<br />
comprehension of the<br />
tradeoffs involved in<br />
the design choices.<br />
k) An ability to apply<br />
design <strong>and</strong><br />
development principles<br />
in the construction of<br />
software systems of<br />
varying complexity.<br />
Course<br />
Displays<br />
Student<br />
Opinion<br />
Survey<br />
Summative<br />
Assessment<br />
Course<br />
Opinion<br />
Survey<br />
12<br />
Co-op <strong>and</strong><br />
Career<br />
Services’<br />
Survey of<br />
Internship<br />
Employers<br />
NSSE<br />
Advisory<br />
Board<br />
Curriculum<br />
Committee<br />
Academic<br />
Program<br />
Review<br />
X X X X X X X<br />
X X X X X X
The nine mechanisms yielding information that is integrated to form assessments<br />
of each outcome are shown <strong>and</strong> described in the following table, Table 1.2:<br />
Table 1.2 Mechanisms for Assessing Program Outcomes<br />
Mechanism Description Frequency<br />
Course Displays<br />
(on display - G314)<br />
Student Opinion Survey<br />
(Attachment I-5)<br />
Summative Assessment<br />
(Attachment I-6)<br />
Course Opinion Survey<br />
(Attachment I-7)<br />
Co-op <strong>and</strong> Career<br />
Services' Survey of<br />
Internship Employers<br />
(Attachment I-8)<br />
National Survey of<br />
Student Engagement<br />
(NSSE)<br />
(Attachment I-9)<br />
Advisory Board<br />
(on display - G314)<br />
Curriculum Committee<br />
(on display - G314)<br />
Academic Program<br />
Review<br />
(on display - G314)<br />
Documents class<br />
activities <strong>and</strong><br />
evidence of<br />
attainment including<br />
samples of major<br />
assignments, projects,<br />
<strong>and</strong> exams<br />
Elicits self reports on<br />
achievement of eight<br />
of the program's<br />
learning outcome<br />
Directly assesses<br />
students’ knowledge<br />
<strong>and</strong> skills on program<br />
outcomes a, c, i, <strong>and</strong> j<br />
Queries students on<br />
clarity <strong>and</strong> fulfillment<br />
of course objectives<br />
Evaluates<br />
performance on many<br />
aspects of work<br />
readiness<br />
Directly assesses<br />
what students put into<br />
their education (class<br />
presentations, papers,<br />
teamwork)<br />
Advises on the<br />
abilities <strong>and</strong> skills<br />
sought by industry to<br />
insure that the<br />
curriculum supports<br />
the outcomes<br />
Oversees the content<br />
of individual courses<br />
<strong>and</strong> the coherence of<br />
the curriculum relative<br />
to the program<br />
outcomes<br />
A comprehensive<br />
evaluation of CS<br />
programs at Pace led<br />
by an external<br />
reviewer.<br />
As needed to<br />
review specific<br />
courses for<br />
currency <strong>and</strong><br />
prospective<br />
update<br />
13<br />
Last<br />
administered<br />
Each course is<br />
reviewed at least<br />
every three years<br />
Annually Summer 2006<br />
Annually<br />
Each semester for<br />
all courses<br />
For each<br />
internship<br />
Yearly for Pace<br />
first year students<br />
<strong>and</strong> seniors<br />
Advisory Board<br />
meets four times a<br />
year;<br />
CS representative<br />
is present at least<br />
once a year<br />
Monthly<br />
throughout the Fall<br />
<strong>and</strong> Spring<br />
semesters<br />
Approximately<br />
every five years;<br />
scheduled by the<br />
Provost’s Office<br />
Spring 2006<br />
Summer 2006<br />
Summer 2006<br />
Spring 2006<br />
Last major formal<br />
input from the<br />
Advisory Board<br />
was in November<br />
2005<br />
May 2006<br />
Spring 2001<br />
Responsibility<br />
Curriculum Committee<br />
Office of the Seidenberg<br />
Associate Dean<br />
Curriculum Committee<br />
Seidenberg Assessment/<br />
Research Office<br />
Cooperative Education <strong>and</strong><br />
Career Services<br />
Department<br />
Office of Planning,<br />
Assessment, Research, <strong>and</strong><br />
Academic Budgeting<br />
(OPARAB)<br />
The Dean of the Seidenberg<br />
School<br />
Operates under the aegis of<br />
the Computer Science<br />
Department<br />
The Dean of the Seidenberg<br />
School
Each mechanism presented in Table 1.2 for assessing program outcomes will<br />
now be discussed individually.<br />
Course Displays<br />
This mechanism provides documentation of class activities <strong>and</strong> evidence of<br />
attainment. The Curriculum Committee believes there is no surer way to see<br />
what students are doing in a course <strong>and</strong> how they are performing than through<br />
samples of work, projects, <strong>and</strong> exams. Products are direct indicators of<br />
achievement.<br />
Course displays assess every outcome because it is the constellation of courses<br />
within the program through which the outcomes are developed. Table 1.3 shows<br />
the outcomes addressed by each course.<br />
Course displays are maintained by individual faculty <strong>and</strong> presented for review<br />
whenever the Curriculum Committee needs to reconsider the content <strong>and</strong> the<br />
presentation of a particular course.<br />
Table 1.3 How Student Learning Outcomes Are Fulfilled by Required Courses<br />
(An X Indicates that the Course Contributes to the Outcome's Fulfillment.)<br />
Outcome CS121 CS122 CS232 CS241 CS242 CS271 CS312 CS361 CS371 CS389 CS488<br />
a X X X X X<br />
b X X X X X X<br />
c X X X X X<br />
d X X X<br />
e X X<br />
f X X X<br />
g X X<br />
h X X X X<br />
i X X X X X X X X X<br />
j X X X X X<br />
k X X X X X<br />
Course Titles<br />
CS121 Computer Programming I (4 credits)<br />
CS122 Computer Programming II (4 credits)<br />
CS232 Computer Organization (4 credits)<br />
CS241 Data Structures <strong>and</strong> Algorithms I (4 credits)<br />
CS242 Data Structures <strong>and</strong> Algorithms II (4 credits)<br />
CS271 Fundamentals of UNIX <strong>and</strong> C Programming (2 credits)<br />
CS312 Research Methods in Computers <strong>and</strong> Society (3 credits)<br />
CS361 Programming Languages <strong>and</strong> Implementation (4 credits)<br />
CS371 Operating Systems <strong>and</strong> Architecture (4 credits)<br />
CS389 Software Engineering (4 credits)<br />
CS488 Computer Networks <strong>and</strong> the Internet (4 credits)<br />
14
Student Opinion Survey<br />
The "Student Opinion Survey," a mechanism created <strong>and</strong> managed by the<br />
Seidenberg School, is administered annually to students shortly after graduation<br />
(Attachment I-5). Among other things, students are asked about achievement on<br />
program outcomes a through k. While self-reports are only an indirect measure<br />
(i.e. they do not require demonstrations of the ad hoc abilities <strong>and</strong> skills), "…the<br />
research literature indicates a moderate to strong correspondence between<br />
students' self-reports <strong>and</strong> more objective measures of learning outcomes." [Linda<br />
C. Strauss <strong>and</strong> Patrick T. Terenzini; "Assessing Student Performance on EC2000<br />
Criterion 3.a-k"; Proceedings of the 2005 American Society for Engineering<br />
Education; June 2005, page 8 -- available on display in the Goldstein Academic<br />
Center, room 314].<br />
The instrument’s stability has afforded before/after insight into whether program<br />
improvements seem to be having the intended effects. The only changes to the<br />
instrument, since it was first administered in 1997, were made in 2005. These<br />
included rewordings of three items (5, 16, <strong>and</strong> 17) <strong>and</strong> three new questions<br />
added at the end (46, 47, <strong>and</strong> 48). The next student opinion survey will include<br />
an additional item asking graduates about the international study opportunities of<br />
which they availed themselves to better assess the global aspect of outcome g.<br />
A major change in the curriculum, to be discussed, was implemented for Fall<br />
2002 when software engineering was made the required capstone. Table 1.4<br />
below contrasts the data from 2002 with the data from 2005 to assess this<br />
change on the outcomes effected. The tabulated percentages have no intrinsic<br />
meaning. The reason for two sets of values is to establish a baseline <strong>and</strong> show a<br />
change of direction.<br />
Table 1.4 Indirect Measures of Outcomes a through k<br />
from the Student Opinion Surveys (New Graduates) in 2002 <strong>and</strong> 2005<br />
Outcome Applicable Items<br />
2002<br />
Reported Fulfillment<br />
2005<br />
Reported Fulfillment<br />
a 1, 7, 9 51% 56%<br />
b 3, 4, 5, 8 45% 62%<br />
c 5, 7 37% 50%<br />
d 14 60% 67%<br />
e 15, 16, 17 62% 64%<br />
f 11, 12, 13 67% 50%<br />
g 15, 16, 17 62% 64%<br />
h 2, 18, 19 69% 78%<br />
i 4, 6, 10 49% 64%<br />
j 8, 9 43% 57%<br />
k 5, 6 34% 53%<br />
15
Here is how the statistics reported above were derived. Firstly, items were<br />
mapped onto learning outcomes. For instance, items 1, 7, <strong>and</strong> 9 apply to<br />
outcome a:<br />
1. Coursework at Pace helped me to underst<strong>and</strong> the fundamental principles of<br />
computing.<br />
7. Coursework at Pace helped me to integrate concepts from various areas of<br />
my discipline to solve computing problems.<br />
9. Coursework at Pace helped me to develop facility with a set of quantitative<br />
techniques for analyzing technical <strong>and</strong> organizational problems.<br />
For each item respondents select one of six responses on the Likert scale:<br />
strongly agree, agree, slightly agree, slightly disagree, disagree,<br />
strongly disagree.<br />
Outcome a is "An ability to apply knowledge of computing <strong>and</strong> mathematics<br />
appropriate to the discipline."<br />
For any particular year, say 2005, the percentage of graduates who "selected<br />
strongly agree" or "agree" with each applicable item are averaged. This gives a<br />
composite:<br />
item 1: strongly agree = 40.0%<br />
+ agree = 26.7%<br />
66.7%<br />
item 7: strongly agree = 26.7%<br />
+ agree = 26.7%<br />
53.4%<br />
item 9: strongly agree = 26.7%<br />
+ agree = 20.0%<br />
46.7%<br />
Thus (66.7% + 53.4% + 46.7%) / 3 = 55.6% ≈ 56% of the program's 2005<br />
graduates may be said to concur that outcome a was fulfilled. In contrast, this<br />
value for graduates in 2002, the last year before software engineering was<br />
instituted as the required capstone, was 50%. Changes in all the outcomes<br />
conceivably bolstered by CS389 are in the right direction, whatever the causal<br />
pathways. This qualification about the causal pathways is needed because other<br />
significant changes were made to the program with the intent of strengthening it<br />
during the same time interval. The data from the Student Opinion Surveys from<br />
2001 through 2005 are on file in the Goldstein Academic Center, room 314.<br />
16
Summative Assessment<br />
The Summative Assessment is an achievement test for seniors created <strong>and</strong><br />
administered by the Curriculum Committee. This test is a direct measure of the<br />
technical program outcomes (a, c, i, <strong>and</strong> j) but not the "soft" ones. The beta<br />
version was used for the first time in Spring 2006. This beta version is in<br />
Attachment I-6 with the results <strong>and</strong> a mapping of items to program outcomes. Of<br />
the 80 items, around 35 are either true/false or multiple choice, <strong>and</strong> around 35<br />
are fill-in-the-blank. Ten require chunks of code to be written, <strong>and</strong> two require<br />
prose.<br />
Thirty students took the test. Most were given approximately two hours, but those<br />
who arrived late had less.<br />
The beta version was too long. Students rushed, eschewing items requiring<br />
more concentration <strong>and</strong> problem-solving in favor of the ones that were easier to<br />
grasp <strong>and</strong> answer (e.g. true/false <strong>and</strong> multiple choice). Even with this bias,<br />
different students completed different sets of items. Consequentially, the test<br />
could not be scored with the same kind of comprehensiveness, uniformity, <strong>and</strong><br />
objectiveness applied to evaluative classroom tests. Items requiring code,<br />
problem-solving, or prose were evaluated on a three-point scale of competence:<br />
“strong,” “fair,” <strong>and</strong> “lacking.” When numbers were required, these ratings were<br />
treated as 2, 1, <strong>and</strong> 0 respectively.<br />
Program<br />
Learning<br />
Outcome<br />
a<br />
c<br />
i<br />
j<br />
Table 1.5 Direct Measures of Outcomes a, c, i, <strong>and</strong> j<br />
from the Summative Assessment of April 2006<br />
Section 2: #4: 70%<br />
#6: 70%<br />
#7: 83%<br />
Indicative Items<br />
<strong>and</strong> Class Average on Each One<br />
Section 3: #2: 87%<br />
#4: 53%<br />
#5: 67%<br />
rating of 2 rating of 1 rating of 0<br />
Section 1: #4a (coding): 23 2 5<br />
#4b (coding): 15 4 11<br />
Section 4: #3 (coding): 14 1 15<br />
Section 2: #1: 83%<br />
#3: 73%<br />
Section 5: #9: 76%<br />
17<br />
Score<br />
74%<br />
69%<br />
1.23 out of 2<br />
which is<br />
62%<br />
The score on outcome i is somewhat lower than the others. If we knew these<br />
scales were valid <strong>and</strong> cross-calibrated, this could be the logical place for applying<br />
improvement efforts. However, we know no such thing. The assessment of<br />
77%
outcome i via the Student Opinion Surveys of 2002 <strong>and</strong> 2005, back in Table 1.4,<br />
shows no relative weakness. Data on the corresponding outcomes from the<br />
Student Opinion Surveys <strong>and</strong> the Summative Assessment are plotted in Table<br />
1.6 <strong>and</strong> depicted in Figure 1.1 beneath it.<br />
Table 1.6 Comparing Fulfillment of Program Learning Outcomes a, c, i, <strong>and</strong> j<br />
as Indicated by Student Opinion Surveys <strong>and</strong> by the Summative Assessment<br />
Program<br />
Learning<br />
Outcome<br />
Student Opinion Survey<br />
2002<br />
Student Opinion Survey<br />
2005<br />
18<br />
Summative Assessment<br />
April 2006<br />
a 51% 56% 74%<br />
c 37% 50% 69%<br />
i 49% 64% 62%<br />
j 43% 57% 77%<br />
90%<br />
80%<br />
70%<br />
60%<br />
50%<br />
40%<br />
30%<br />
20%<br />
10%<br />
0%<br />
Figure 1.1 Visualization of Table 1.6<br />
Student Opinion<br />
Survey<br />
2002<br />
Student Opinion<br />
Survey<br />
2005<br />
Summative<br />
Assessment<br />
The summative assessment is to be an annual occurrence, each April, managed<br />
by the Computer Science Curriculum Committee. The 2007 test will be an<br />
abridgement of 2006's, consisting of the nine items selected as outcomes'<br />
indicators as well as some experimental items for future year-to-year usage.<br />
Course Opinion Survey<br />
A "Course Opinion Survey" is administered toward the end of each course<br />
(Attachment I-7). Students are asked, using a Likert scale, whether the course<br />
objectives were clear <strong>and</strong> if the course satisfied these. Routinely, semester<br />
a<br />
c<br />
i<br />
j
averages run between 80% <strong>and</strong> 85%. Social scientists concur that class<br />
averages on items such as these are meaningful. Our interest in this as an<br />
assessment mechanism resides in the fact that every course has a set of<br />
st<strong>and</strong>ard objectives which cumulatively develop the program outcomes (a<br />
through k).<br />
The Seidenberg Assessment Research Office is responsible for administering<br />
<strong>and</strong> tabulating the results. Reports are distributed to instructors, department<br />
chairpeople, <strong>and</strong> the Dean.<br />
Cooperative Education (Co-op) <strong>and</strong> Career Services’ Survey of Internship<br />
Employers<br />
The Cooperative Education <strong>and</strong> Career Services Department queries those<br />
responsible for supervising interns toward the end of each internship assignment.<br />
Supervisors are asked about “academic preparation”, “quality of work”, <strong>and</strong> other<br />
aspects of work performance. This mechanism is a direct measure of those skills<br />
needed for success “on the job,” including: all outcomes associated with the<br />
ability to use current techniques, skills, <strong>and</strong> tools applied in computing practice<br />
(outcome i), those associated with teamwork <strong>and</strong> effective communication<br />
(outcomes d <strong>and</strong> f) <strong>and</strong> the other outcomes indicated as applicable in Table 1.2.<br />
The Computer Science Department has access to <strong>and</strong> annually examines the<br />
performance reviews of undergraduates on cooperative education internships.<br />
Data, both numerical summarizations <strong>and</strong> comments are in Attachment I-8.<br />
Employer comments were quoted earlier in this report. One of the statistics is<br />
that academic preparation is rated excellent or good in greater than 95% of the<br />
internships.<br />
The National Survey of Student Engagement (NSSE)<br />
The National Survey of Student Engagement (NSSE) has been administered at<br />
Pace since 2003. Administration, data analysis <strong>and</strong> the distribution of results are<br />
managed by the Office of Planning, Assessment, Research, <strong>and</strong> Academic<br />
Budgeting (OPARAB). Participants are first year students <strong>and</strong> seniors. This tool<br />
assesses oral <strong>and</strong> written communication experiences, experiences relating to<br />
collaborative work, <strong>and</strong> the development of professional, ethical <strong>and</strong> social<br />
responsibilities (outcomes d, e, <strong>and</strong> f). The NSSE statistics are based on a<br />
University-wide sample of first year students <strong>and</strong> seniors. The results are<br />
interesting because they are professional-quality measures that supply a contrast<br />
to benchmark institutions, <strong>and</strong> demonstrate the commitment of the University to<br />
these outcomes.<br />
19
NSSE results "…are a direct indicator of what students put into their education<br />
<strong>and</strong> an indirect indicator of what they get out of it" ("Introduction <strong>and</strong> Rationale for<br />
Using NSSE in Accreditation,'" page 1).<br />
The following are results from the Spring 2005 NSSE administration bearing<br />
upon outcome f, “An ability to communicate effectively with a range of<br />
audiences"<br />
In your experience at your institution during the current school year, about how<br />
often have you:<br />
1.b. Made a presentation in class?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.40 2.23 2.22<br />
Seniors 2.97 2.73 2.83<br />
To what extent has your experience at this institution contributed to your<br />
knowledge, skills, <strong>and</strong> personal development in:<br />
11.d. Speaking clearly <strong>and</strong> effectively?<br />
Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.99 2.70 2.69<br />
Seniors 2.99 2.89 2.93<br />
In your experience at your institution during the current school year, about how<br />
often have you:<br />
1.d. Worked on a paper or project that required integrating ideas or information<br />
from various sources?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 3.19 2.78 3.06<br />
Seniors 3.28 2.56 3.34<br />
20
During the current school year, about how many papers or reports have<br />
you written that are:<br />
3.c. 20 or more pages long?<br />
None = 1 Between 1 <strong>and</strong> 4 = 2 Between 5 <strong>and</strong> 10 = 3<br />
Between 11 <strong>and</strong> 20 = 4 More than 20 = 5<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 1.59 1.24 1.24<br />
Seniors 1.89 1.64 1.69<br />
3.d. between 5 <strong>and</strong> 19 pages long?<br />
None = 1 Between 1 <strong>and</strong> 4 = 2 Between 5 <strong>and</strong> 10 = 3<br />
Between 11 <strong>and</strong> 20 = 4 More than 20 = 5<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.63 2.38 2.36<br />
Seniors 2.86 2.56 2.62<br />
3.e. fewer than pages long?<br />
None = 1 Between 1 <strong>and</strong> 4 = 2 Between 5 <strong>and</strong> 10 = 3<br />
Between 11 <strong>and</strong> 20 = 4 More than 20 = 5<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 3.09 3.02 3.14<br />
Seniors 3.20 2.92 3.10<br />
To what extent has your experience at this institution contributed to your<br />
knowledge, skills, <strong>and</strong> personal development in:<br />
11.c. Writing clearly <strong>and</strong> effectively?<br />
Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 3.02 2.95 2.90<br />
Seniors 3.00 3.03 3.05<br />
The following are results bearing loosely on outcome e, “An underst<strong>and</strong>ing of<br />
professional, ethical <strong>and</strong> social responsibilities."<br />
21
During the current school year, about how often have you:<br />
6.d. Examined the strengths <strong>and</strong> weaknesses of your own views on a topic or<br />
issue?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.34 2.54 2.54<br />
Seniors 2.56 2.66 2.65<br />
6.e. Tried to better underst<strong>and</strong> some else's views by imagining how an issue<br />
looks from his or her perspective?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.56 2.79 2.76<br />
Seniors 2.69 2.82 2.80<br />
6.f. Learned something that changed the way you underst<strong>and</strong> an issue or<br />
concept?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.59 2.80 2.76<br />
Seniors 2.78 2.83 2.83<br />
To what extent has your experience at this institution contributed to your<br />
knowledge, skills, <strong>and</strong> personal development in:<br />
11.n. Developing a personal code of values <strong>and</strong> ethics?<br />
Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.66 2.44 2.48<br />
Seniors 2.59 2.47 2.59<br />
The National Study of Student Engagement also presents data bearing upon outcome d,<br />
"An ability to function effectively on teams to accomplish a common goal":<br />
22
In your experience at your institution during the current school year, about how<br />
often have you:<br />
1.g. Worked with other students on projects during class?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.48 2.52 2.46<br />
Seniors 2.76 2.53 2.59<br />
1.h. Worked with classmates outside of class to prepare class assignments?<br />
Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.38 2.17 2.36<br />
Seniors 2.81 2.56 2.83<br />
To what extent has your experience at this institution contributed to your<br />
knowledge, skills, <strong>and</strong> personal development in:<br />
11.h. Working effectively with others?<br />
Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.91 2.85 2.88<br />
Seniors 3.12 3.02 3.14<br />
Finally, since learning effectively on one's own is considered to be supportive of<br />
an ability to engage in continuing professional development, then NSSE<br />
contributes data bearing upon outcome h:<br />
To what extent has your experience at this institution contributed to your<br />
knowledge, skills, <strong>and</strong> personal development in:<br />
11.j. Learning effectively on your own?<br />
Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />
Pace University Urban Consortium Doctoral Intensive<br />
First Year 2.87 2.85 2.83<br />
Seniors 3.00 2.96 2.98<br />
The NSSE questionnaire is in Attachment I-9 along with the means <strong>and</strong> st<strong>and</strong>ard<br />
deviations from the 2005 administration on all items for Pace, the Urban<br />
Consortium, <strong>and</strong> Doctoral Intensive institutions. The overall conclusion from the<br />
23
tables presented above is that Pace is doing as well as or better than its<br />
benchmark schools in writing, speaking, <strong>and</strong> collaborative skills. Although we<br />
look mildly weaker in the realm of viewing <strong>and</strong> underst<strong>and</strong>ing social issues (items<br />
6.d., 6.e. <strong>and</strong> 6.f), we are not weaker in supplying experiences tending to<br />
cultivate personal values <strong>and</strong> a code of ethics.<br />
Advisory Board<br />
The Seidenberg School’s Advisory Board consists of twenty five industrial<br />
practitioners from major corporations including IBM Research, Verizon <strong>and</strong><br />
PepsiCo. A full membership roster with organizational affiliations <strong>and</strong> minutes of<br />
meetings are available on display in the Goldstein Academic Center, room 314.<br />
The Advisory Board meets four times a year. It provides important guidance on<br />
curriculum issues, specifically the abilities <strong>and</strong> skills sought by industry when<br />
hiring new graduates.<br />
For example, collaborative skills were being strongly advocated by the Advisory<br />
Board since January 2001. As a result of this, <strong>and</strong> other assessments, software<br />
engineering (CS 389) was enriched with a semester-long team project <strong>and</strong> was<br />
made a requirement. This enhances the fulfillment of outcome d, “an ability to<br />
function effectively on teams to accomplish a common goal.” Another<br />
improvement motivated by the Advisory Board was to make the Data<br />
Communications course (CS 388) a requirement. This enhances the fulfillment of<br />
outcome i, “an ability to use current techniques, skills <strong>and</strong> tools necessary for<br />
computing practice.”<br />
Dr. Frances E. Allen; an Advisory Board member from IBM, a former Board<br />
member of CRA, <strong>and</strong> an IBM <strong>and</strong> an ACM fellow; is an advocate of fundamental<br />
underst<strong>and</strong>ings <strong>and</strong> constantly warns of fad technology. "Whether or not<br />
theoretical details are used directly, they influence one's thinking." Dr. Allen’s<br />
input has encouraged rigor in the algorithms <strong>and</strong> the programming languages<br />
courses leading to strengthening of outcome a, “an ability to apply knowledge of<br />
computing <strong>and</strong> mathematics appropriate to the discipline.” In the same vein, a<br />
different Advisory Board member, Henry Marzullo, emphasizes the soft skills. "I<br />
remember very little of the facts from my college days. Every day, however, I<br />
use the skills I learned in college – to write clearly, to analyze objectively, <strong>and</strong> to<br />
think creatively." His influence led to a public speaking assignment required of<br />
every student in CS 242, Data Structures <strong>and</strong> Algorithms II, bolstering the<br />
program’s strength relative to outcome f, “an ability to communicate effectively<br />
with a range of audiences.”<br />
Dean Merritt arranges for faculty to be present at Advisory Board meetings,<br />
encourages Advisory Board members to cooperate with faculty on assessment<br />
projects, <strong>and</strong> passes information relative to the educational milieu to the<br />
Curriculum Committee <strong>and</strong> to individual faculty members. This information<br />
24
includes items such as Gartner Group reports, pertinent articles from<br />
professional publications, observations from meetings she attends such as the<br />
Fall 2005 National Science Foundation workshop on the future of computer<br />
science, <strong>and</strong> insights from her participation as a CAC/ABET Commissioner.<br />
Keeping the curriculum in step with industrial practice will occupy our attention for<br />
as long as technology continues to advance. Advisory Board consultations will<br />
continue to be a primary informational resource.<br />
Curriculum Committee<br />
The CS Curriculum Committee, meeting monthly throughout the Fall <strong>and</strong> Spring<br />
semesters, integrates assessment inputs from all sources to make<br />
determinations relative to course content. Also, it is a proactive mechanism that<br />
originates improvements rooted in observations of how students learn. The CS<br />
Curriculum Committee takes responsibility for initiatives on the curriculum<br />
including interfacing with the Advisory Board, for monitoring course content <strong>and</strong><br />
activities, for constructing <strong>and</strong> evaluating the summative assessment, <strong>and</strong> other<br />
inquires, evaluations, <strong>and</strong> controls directly affecting coursework <strong>and</strong> students'<br />
experiences. For instance, outcome d was strengthened when the Curriculum<br />
Committee acted on the Advisory Board’s recommendation for an increased<br />
focus on collaborative skills. The Curriculum Committee acted to do this based<br />
not only on the Advisory Board but also with respect to other assessment<br />
sources (e.g. Academic Program Review).<br />
Academic Program Review<br />
The Academic Program Review is an assessment mechanism introduced by the<br />
Provost’s Office in 2000. The term “program review” is slightly misleading<br />
because the review might focus on a single program but it could focus on all<br />
programs within a department. Regardless of its scope, all program reviews<br />
include a self study, one or more external reviewers, a formal report on the<br />
findings by the external reviewer, <strong>and</strong> a response to this report by faculty (the<br />
Curriculum Committee or a designated subcommittee). When a program review<br />
is scheduled, the responsibility for carrying it forward is with the Dean’s Office.<br />
The latest CS Academic Program Review took place is Spring 2001 <strong>and</strong><br />
encompassed the BS in CS as well as the other computer science programs in<br />
the Seidenberg School, including the MS in CS. The two-day on-site visit was<br />
conducted by Dr. Rachelle Heller, a computer science professor from George<br />
Washington University who spent two days on site.<br />
Relative to the BS in CS curriculum, the report advised that students should<br />
receive more experience with professional, GUI development tools <strong>and</strong> software<br />
25
testing tools. As a result, JBuilder was adopted for the Java programming<br />
classes (CS121 <strong>and</strong> CS122) <strong>and</strong> for the data structures <strong>and</strong> algorithms classes<br />
(CS241 <strong>and</strong> CS242). We have since switched to Eclipse. The capstone class in<br />
software engineering (CS389) introduces JUnit <strong>and</strong> Subversion. Another<br />
recommendation was for each student to experience the collaborative analysis,<br />
design, construction, <strong>and</strong> testing of a piece of software for solving a problem with<br />
no direct solution. This was fulfilled in September 2002 when CS389, the<br />
project-based software engineering seminar, became a requirement of new<br />
matriculants. These recommendations worked together to contribute to the<br />
achievement of outcomes a, b, c, d, h, i, j, <strong>and</strong> k.<br />
The report is on display in the Goldstein Academic Center, room 314.<br />
Curriculum Improvements Resulting from Assessment<br />
There have been a number of assessment-based changes made to the<br />
curriculum <strong>and</strong> the student environment since the program's accreditation was<br />
last renewed. The first-order improvements, <strong>and</strong> most palpable, have been to<br />
the computer science courses required for graduation. Annual changes are<br />
documented in the students' worksheets. Worksheets enumerate the graduation<br />
requirements for that year's matriculants (entering freshman <strong>and</strong> transfer<br />
students). The Undergraduate Catalog shows these changes as well but with<br />
less chronological granularity because the Catalog is published only every other<br />
year. Table 1.4, below, summarizes these changes.<br />
Table 1.7 Assessment Based Curriculum Improvements<br />
Assessment Action<br />
Change<br />
Effective<br />
Academic Program Review;<br />
Advisory Board<br />
CS389<br />
Software Engineering was changed from an elective to a degree requirement<br />
Fall 2002<br />
Advisory Board; Curriculum CS388<br />
Committee; Student Opinion<br />
Survey<br />
Data Communications was changed from an elective to a degree requirement<br />
CS362<br />
Fall 2002<br />
Advisory Board; Curriculum<br />
Committee<br />
Programming Languages <strong>and</strong> Implementation II (compiler) was removed as a<br />
requirement to allow for the addition of CS389; course was discontinued; topics<br />
from compiler construction were integrated into CS361<br />
CS231<br />
Fall 2002<br />
Advisory Board; Curriculum<br />
Committee<br />
Computer Organization I (assembler) was removed as a requirement to allow<br />
for the addition of CS388; course was discontinued; PHY109 was made a<br />
requirement<br />
PHY109<br />
Fall 2002<br />
CS Curriculum Committee<br />
Digital Electronics was changed from a possible science electives to a course<br />
requirement to supply coverage of gates, adders, registers, <strong>and</strong> instruction<br />
decoders that was lost with the removal of CS231<br />
CS488<br />
Fall 2002<br />
Advisory Board Computer Networks <strong>and</strong> the Internet replaced CS388 as a degree requirement;<br />
CS388 remains available as an elective<br />
Fall 2005<br />
Advisory Board; Course CS271<br />
Opinion Survey; Curriculum UNIX (re-titled: UNIX <strong>and</strong> C) was changed from a<br />
Fall 2006<br />
Committee<br />
one-credit class to two credits; coverage will include C<br />
26
B.1.A.8. Indicate how your student outcomes map to your program<br />
objectives.<br />
As a matter of terminology, "program learning outcomes" <strong>and</strong> "student learning<br />
outcomes" refer to the same 11 statements. The key word is outcomes. (These<br />
same statements constitute our program's "student attributes" which are the<br />
focus of B.1.B.)<br />
Table 1.8<br />
How Program Objectives Are Supported by Student Learning Outcomes<br />
(Table extends to the following page.)<br />
Program's Educational Objectives Supporting Learning Outcomes<br />
Be equipped for responsible <strong>and</strong><br />
ethical participation <strong>and</strong> leadership<br />
roles as computing professionals<br />
<strong>and</strong><br />
Be equipped for lifelong learning<br />
Underst<strong>and</strong> the foundational ideas in<br />
computer science <strong>and</strong> be able to use<br />
this knowledge for guidance in<br />
technical problem-solving<br />
Be effective in the design <strong>and</strong><br />
construction of software applications<br />
d) An ability to function effectively on teams to<br />
accomplish a common goal<br />
e) An underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social<br />
responsibilities.<br />
f) An ability to communicate effectively with a range of<br />
audiences.<br />
h) Recognition of the need for, <strong>and</strong> an ability to engage<br />
in, continuing professional development.<br />
a) An ability to apply knowledge of computing <strong>and</strong><br />
mathematics appropriate to the discipline.<br />
b) An ability to analyze a problem <strong>and</strong> to identify <strong>and</strong><br />
define the computing requirements appropriate to its<br />
solution.<br />
a) An ability to apply knowledge of computing <strong>and</strong><br />
mathematics appropriate to the discipline.<br />
b) An ability to analyze a problem <strong>and</strong> to identify <strong>and</strong><br />
define the computing requirements appropriate to its<br />
solution.<br />
c) An ability to design, implement <strong>and</strong> evaluate a<br />
computer-based system, process, component, or<br />
program to meet desired needs.<br />
i) An ability to use current techniques, skills, <strong>and</strong> tools<br />
necessary for computing practices.<br />
j) An ability to apply mathematical foundations,<br />
algorithmic principles, <strong>and</strong> computer science theory in the<br />
modeling <strong>and</strong> design of computer-based systems in a wa<br />
that demonstrates comprehension of the tradeoffs<br />
involved in design choices.<br />
27
Program's Educational Objectives Supporting Learning Outcomes<br />
k) An ability to apply design <strong>and</strong> development<br />
principles in the construction of software systems of<br />
varying complexity.<br />
Be able to work effectively on teams<br />
<strong>and</strong><br />
Have oral <strong>and</strong> written communication<br />
skills<br />
Be informed on social issues that affect<br />
<strong>and</strong> are affected by the application of<br />
computing technology<br />
d) An ability to function effectively on teams to<br />
accomplish a common goal.<br />
f) An ability to communicate effectively with a range of<br />
audiences.<br />
e) An underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social<br />
responsibilities.<br />
g) An ability to analyze the impact of computing on indivi<br />
organizations <strong>and</strong> society, including ethical, legal, security<br />
global policy issues.<br />
To bring the mapping of the program's objectives down to the coursework level,<br />
see Table 1.3 showing the specifics on how student learning outcomes are fulfilled<br />
by required courses.<br />
28
B.1.B. Student Attributes<br />
Our student attributes <strong>and</strong> our program's outcomes are one <strong>and</strong> the same.<br />
B.1.C. Assessments <strong>and</strong> Program Improvement<br />
B.1.C.1. Describe your procedure for periodically assessing the extent to<br />
which each of the above objectives is being met by your program. Include:<br />
• Frequency <strong>and</strong> timing of assessments<br />
• What data are collected (should include information on initial<br />
student placement <strong>and</strong> subsequent professional development)<br />
• How data are collected<br />
• From whom data are collected (should include students <strong>and</strong><br />
computing professionals)<br />
• How assessment results are used <strong>and</strong> by whom<br />
The "above objectives" (in the item) refer to our program objectives (i.e. the<br />
career <strong>and</strong> professional accomplishments for which the program strives to<br />
prepare graduates), <strong>and</strong> the specified points have been addressed in responding<br />
to item B.1.A.5. We respectfully refer this section for narratives that supplement<br />
Table 1.9, below.<br />
Mechanism<br />
Alumni/ae Survey<br />
conducted by the<br />
Seidenberg<br />
Assessment/Research<br />
Analyst<br />
Table 1.9 Mechanisms for Assessing Program Objectives<br />
What data is<br />
collected<br />
descriptions of<br />
current job<br />
responsibilities<br />
<strong>and</strong> educational<br />
activities since<br />
graduation (or<br />
plans for<br />
additional<br />
education) by<br />
graduates out<br />
for two to five<br />
years<br />
How the<br />
data is<br />
collected<br />
mailed<br />
questionnaire<br />
that may be<br />
answered via<br />
email, fax,<br />
telephone, or<br />
U.S. Mail<br />
29<br />
Frequency<br />
<strong>and</strong> timing<br />
of data<br />
collection<br />
every three<br />
years; last<br />
administration<br />
was Fall 2005<br />
How the results are<br />
used <strong>and</strong> by whom<br />
Results are summarized by<br />
the Seidenberg<br />
Assessment/Research<br />
Analyst who forwards them<br />
to the CS Curriculum<br />
Committee <strong>and</strong> the<br />
Associate Dean. They are<br />
reviewed by the faculty, for<br />
interpretation <strong>and</strong><br />
discussion, at an annual<br />
Assessment Day event.
Mechanism<br />
Employment of New<br />
Graduates - the Survey<br />
conducted by the<br />
Cooperative Education <strong>and</strong><br />
Career Services<br />
Department<br />
Survey of Internship<br />
Employers conducted by<br />
the Cooperative Education<br />
<strong>and</strong> Career Services<br />
Department<br />
What data is<br />
collected<br />
statistics for<br />
each graduating<br />
class on the<br />
organizations,<br />
positions, <strong>and</strong><br />
(when available)<br />
salaries of the<br />
full-time jobs<br />
located by new<br />
graduates<br />
performance<br />
reviews by<br />
supervisors of<br />
inters on each<br />
intern's work<br />
quality,<br />
academic<br />
preparation,<br />
verbal <strong>and</strong><br />
writing ability,<br />
work attitude,<br />
etc<br />
How the<br />
data is<br />
collected<br />
The<br />
Cooperative<br />
Education <strong>and</strong><br />
Career Services<br />
Department has<br />
employment<br />
data on h<strong>and</strong><br />
for graduates<br />
who located<br />
full-time jobs<br />
through their<br />
auspices <strong>and</strong>,<br />
for those who<br />
located jobs<br />
independently,<br />
pursues similar<br />
information via<br />
questionnaire.<br />
traditionally a<br />
mailed<br />
questionnaire;<br />
the Cooperative<br />
Education <strong>and</strong><br />
Career Services<br />
Department is<br />
in the midst of<br />
migrating to a<br />
Web-based<br />
survey<br />
administration<br />
30<br />
Frequency<br />
<strong>and</strong> timing<br />
of data<br />
collection<br />
annually,<br />
during the<br />
summer <strong>and</strong><br />
fall following a<br />
class's<br />
graduation.<br />
upon the<br />
completion of<br />
each individual<br />
internship<br />
How the results are<br />
used <strong>and</strong> by whom<br />
Results get sent to the<br />
Seidenberg School's<br />
Assessment/Research<br />
Analyst who forwards them<br />
to the CS Curriculum<br />
Committee, the Associate<br />
Dean, <strong>and</strong> the Dean. They<br />
are reviewed by the full<br />
faculty, for interpretation<br />
<strong>and</strong> discussion, at an<br />
annual Assessment Day<br />
event.<br />
Results get sent to the<br />
Seidenberg School's<br />
Assessment/Research<br />
Analyst who forwards them<br />
to the CS Curriculum<br />
Committee, the Associate<br />
Dean, <strong>and</strong> the Dean. They<br />
are reviewed by the full<br />
faculty, for interpretation<br />
<strong>and</strong> discussion, at an<br />
annual Assessment Day<br />
event.
B.1.C.2. Attach as an appendix copies of the actual documentation that was<br />
generated by your data collection <strong>and</strong> assessment process since the last<br />
accreditation visit or for the past three years if this is the first visit. Include<br />
survey instruments, data summaries, analysis results, etc.<br />
Below is a list of the attachments referenced in the narrative for this<br />
section, B.1.<br />
Attachment Number Description Section<br />
Attachment I-1<br />
Diversity in the Seidenberg School of Computer<br />
Science <strong>and</strong> Information Systems<br />
Alumni/ae Survey<br />
B.1.A.2.<br />
Attachment I-2<br />
Cover Letter<br />
Instrument<br />
Results<br />
B.1.A.5.<br />
Attachment I-3<br />
Full Time Employment Information for New<br />
Graduates 2001-2005<br />
Assessment Day<br />
B.1.A.5.<br />
Attachment I-4<br />
Agenda<br />
Assessment Exercises<br />
Minutes<br />
Student Opinion Survey 2001-2005<br />
Instrument<br />
B.1.A.5.<br />
Attachment I-5<br />
Distribution <strong>and</strong> Returns Statistics<br />
Data Analyses<br />
Student Comments - 2005<br />
Summative Assessment<br />
B.1.A.7.<br />
Attachment I-6<br />
Instrument – Spring 2006<br />
Results – Spring 2006<br />
Mappings between Items <strong>and</strong> Outcomes<br />
B.1.A.7.<br />
Attachment I-7<br />
Course Opinion Survey<br />
Instrument<br />
Student Internship Information<br />
B.1.A.7.<br />
Attachment I-8<br />
Organizations, positions, salaries<br />
Employer’s Form for Intern Evaluation<br />
Summary of Evaluations<br />
National Survey of Student Engagement (NSSE)<br />
B.1.A.5.<br />
B.1.A.7.<br />
Attachment I-9<br />
Instrument<br />
Results - 2005<br />
B.1.A.7.<br />
Attachment I-10 Evolution of Assessment in the BS in CS B.1.A.<br />
Attachment I-11 Commendation on 2006 Assessment Initiatives B.1.A.<br />
31
B.1.C.3. Describe your use of the results of the program's assessments to<br />
identify program improvements <strong>and</strong> modifications to program educational<br />
objectives <strong>and</strong> outcomes.<br />
Include:<br />
• Any major changes within the last five years<br />
• Any significant future program improvement plans based upon<br />
recent assessments<br />
• Any changes in program educational objectives or outcomes<br />
For objectives see section B.1.A.5. For outcomes see section B.1.A.7.<br />
No significant changes to the program are in the offing for the near future.<br />
Improvements are being planned, but all relate to assessment instruments<br />
<strong>and</strong> procedures. Relative to instruments, the Summative Assessment is<br />
being abbreviated for the sake of validity <strong>and</strong> usability. The Alumni/ae<br />
Survey is being extended to assess the program objectives related to<br />
social responsibilities <strong>and</strong> ethical conduct.<br />
32
B.1.D. Documentation<br />
B.1.D1. Describe the documentation that exists for each of the following:<br />
• Measurable program educational objectives<br />
• Expected outcomes<br />
• Assessment process<br />
• Results of assessment<br />
Indicate in your response who has access to each documentation item.<br />
Documentation's Location <strong>and</strong> Access<br />
Documentary Item Location Access<br />
Program's Educational Objectives Ivan G Seidenberg School of CSIS Website:<br />
>Home<br />
>Academics<br />
>Undergraduate<br />
>Computer Science, BS<br />
Public<br />
Minutes of the CS Curriculum Committee<br />
Expected Outcomes Ivan G Seidenberg School of CSIS Website:<br />
>Home<br />
>Academics<br />
>Undergraduate<br />
>Computer Science, BS<br />
Assessment h<strong>and</strong>out to CS <strong>and</strong> IS faculty at<br />
the:<br />
Westchester CSIS Faculty Meeting on 9/6/05<br />
<strong>and</strong> the<br />
New York CSIS Faculty Meeting on 9/8/05<br />
Assessment Day H<strong>and</strong>out (5/3/06)<br />
Assessment Process Learning Outcomes Assessment Plan for the<br />
School of Computer Science <strong>and</strong> Information<br />
Systems – 9/27/2004<br />
Results of Assessment Reports from the CSIS Assessment/Research<br />
Analyst<br />
Assessment Day H<strong>and</strong>outs<br />
Assorted internal documents (e.g. from Co-op<br />
<strong>and</strong> Career Services, the Office of Planning,<br />
Assessment, Research <strong>and</strong> Academic<br />
Budgeting, etc.)<br />
33<br />
Faculty,<br />
Associate Dean,<br />
Dean<br />
Public<br />
Faculty,<br />
Associate Dean<br />
Dean<br />
Faculty,<br />
Associate Dean<br />
Dean<br />
Faculty,<br />
Associate Dean,<br />
Dean