19.08.2013 Views

I. OBJECTIVES, OUTCOMES and ASSESSMENT A ... - pace university

I. OBJECTIVES, OUTCOMES and ASSESSMENT A ... - pace university

I. OBJECTIVES, OUTCOMES and ASSESSMENT A ... - pace university

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

I. <strong>OBJECTIVES</strong>, <strong>OUTCOMES</strong> <strong>and</strong> <strong>ASSESSMENT</strong><br />

A. Background Information<br />

A.1. Degree Title:<br />

Bachelor of Science in Computer Science<br />

A.2. Program Modes:<br />

This is a single BS in Computer Science program with day, late afternoon <strong>and</strong><br />

evening classes offered on the Pleasantville <strong>and</strong> New York City campuses.<br />

CS271, "Fundamentals of the UNIX Operating System," a one-credit, required<br />

class, has been presented as an online class since before Fall 2004. (For Fall<br />

2006 CS271 was changed to “Fundamentals of UNIX <strong>and</strong> C Programming” <strong>and</strong><br />

made two credits.)<br />

A.3. Actions to Correct Previous Concerns:<br />

The previous visit reported no concerns.<br />

A.4. Contact Information:<br />

Dr. Susan M. Merritt, Dean<br />

The Ivan G. Seidenberg School of Computer Science <strong>and</strong> Information Systems<br />

Goldstein Academic Center<br />

Pace University<br />

861 Bedford Road<br />

Pleasantville, NY 10570-2799<br />

Telephone: 914-773-3750 E-mail: smerritt@<strong>pace</strong>.edu<br />

B. Accreditation Summary<br />

B.1.A. Objectives <strong>and</strong> Outcomes<br />

The Seidenberg School has a long st<strong>and</strong>ing commitment to assessment. A<br />

review of the major assessment activities over the past ten years is in<br />

Attachment I-11.<br />

The last accreditation report from the Middle States Commission on Higher<br />

Education singled-out the Seidenberg School for the “numerous things [it has<br />

done] to maintain, assess <strong>and</strong> validate the quality of its offerings.” For the past<br />

six years a computer science faculty member has been provided a stipend to<br />

serve as Assessment Director. In Summer 2004, as assessment activities<br />

intensified, the School hired a full-time assessment/research analyst.<br />

Seidenberg representatives participate actively in the bimonthly meetings of the<br />

University-wide Assessment Committee <strong>and</strong> have taken the lead in various<br />

1


initiatives including authoring the frequently asked questions section of the<br />

University’s assessment web site.<br />

The School held its first full faculty Assessment Day on May 3, 2006 <strong>and</strong> has<br />

scheduled it as an annual event. Activities consist of occasional workshops but<br />

mostly focus on evaluating assessment data. In response to Assessment Day<br />

<strong>and</strong> other assessment initiatives this past year, the School received high acclaim<br />

from the Assistant Vice President of Planning, Assessment, Research <strong>and</strong><br />

Academic Budgeting (Attachment I-12). The most vigorous acclaim was the 15<br />

million dollar endowment to the School from Ivan G. Seidenberg, the CEO of<br />

Verizon, in recognition of the School’s effectiveness in preparing students.<br />

B.1.A1. Provide the institution's mission statement. Include any other<br />

mission statements that are relevant.<br />

Mission of Pace University<br />

Pace University is committed to providing the best possible private education to a<br />

diverse <strong>and</strong> talented student body at each Pace University location. The<br />

University’s continuous commitment to Opportunitas must be retained,<br />

strengthened <strong>and</strong>, when necessary, redefined in an uncertain world where many<br />

are excluded from the full benefits of higher education due to economic <strong>and</strong><br />

social factors beyond their control. Pace’s sustained emphasis on excellence will<br />

be a constant, which will guide decision-making regarding academic programs<br />

<strong>and</strong> other endeavors. In a world of increasing interdependence, Pace must<br />

strengthen its international efforts <strong>and</strong> continue its significant investment in<br />

technology. In addition, the University’s commitment to self-evaluation <strong>and</strong> civic<br />

engagement must remain as major goals. All of this must be accomplished<br />

during a time of considerable economic change <strong>and</strong> in the context of<br />

strengthening Pace’s own financial base.<br />

from: Pace University Second Century Strategic Plan 2003-2008 page 1<br />

Mission of the Ivan G. Seidenberg School of Computer Science <strong>and</strong> Information<br />

Systems<br />

The mission of the School is to prepare men <strong>and</strong> women for meaningful work,<br />

lifelong learning, <strong>and</strong> responsible participation in a new <strong>and</strong> dynamic information<br />

age.<br />

from: Pace University Undergraduate Catalog: 2004-2006, page 160<br />

2


B.1.A.2. Describe how your program's educational objectives align with<br />

your institution's mission.<br />

Opportunitas is the one word that describes the mission of Pace University since<br />

its founding one hundred years ago. It refers to delivering an education that<br />

makes professional success accessible to talented men <strong>and</strong> women. The<br />

<strong>university</strong> serves a highly diverse population. Attachment I-1 presents statistics<br />

documenting the success of the School’s diversity efforts.<br />

The BS in Computer Science program, accredited by the Computer Science<br />

Accreditation Commission of the Computing Sciences Accreditation Board in<br />

1986, emerged in this context. The dominant objective was <strong>and</strong> remains<br />

preparing graduates to be industrial professionals who could take the lead issues<br />

related to software <strong>and</strong> other areas of computer science. To this end, graduates<br />

are supplied with a durable knowledge structure that supports the development<br />

of expertise in contemporary computing environments <strong>and</strong> the ability to stay<br />

current as the field advances <strong>and</strong> changes. Graduates are also supplied with the<br />

verbal abilities <strong>and</strong> teamwork skills required to prosper as well as the know-how<br />

to be an ethical professional enhanced by an awareness of the social, legal,<br />

security <strong>and</strong> global issues surrounding applied computing.<br />

A professional education in computing, combined with the arts <strong>and</strong> sciences,<br />

always striving for diversity, is consistent with the University’s mission.<br />

3


B.1.A.3. List the program’s documented measurable educational objectives.<br />

Graduates of the B.S. in Computer Science program will:<br />

1. be equipped for responsible <strong>and</strong> ethical participation <strong>and</strong> leadership roles<br />

as computing professionals. This extends to a readiness for lifelong<br />

learning in many forms including graduate study.<br />

2. underst<strong>and</strong> the foundational ideas in computer science <strong>and</strong> be able to use<br />

this knowledge to guide their thinking in technical problem-solving.<br />

3. be effective in the design <strong>and</strong> construction of software applications.<br />

4. be able to work effectively on teams. This includes oral <strong>and</strong> written<br />

communication skills as well as collaborative skills.<br />

5. be informed on social issues that affect <strong>and</strong> are affected by the application<br />

of computing technology.<br />

B.1.A.4. Explain how the program's educational objectives align with the<br />

needs of its constituencies <strong>and</strong> include a list of the stakeholders.<br />

Stakeholders in Pace University's BS in CS Program<br />

• current computer science students<br />

• employers hiring new or recent graduates<br />

• alumni/ae<br />

• computer science faculty<br />

• the Seidenberg School’s Advisory Board<br />

The program's chief stakeholders are the computer science students, who need<br />

a program that will deliver readiness to begin <strong>and</strong> pursue progressive<br />

achievement in professional life. To this end, the objectives are formulated based<br />

on characteristics of a well qualified professional. Student input on the objectives<br />

is garnered by way of their academic advisors <strong>and</strong> through informal studentfaculty<br />

interaction. For example, students have spoken about developing the skill<br />

set for designing <strong>and</strong> building software applications (objective 3).<br />

Employers hiring new or recent graduates constitute another dependent<br />

constituency. Dependence resides in the investments that employers make in<br />

those they hire <strong>and</strong> in their need to entrust employees with critical tasks. The<br />

concrete needs of employers are determined mainly through Advisory Board<br />

meetings <strong>and</strong> consultations as well as supervisors of student interns. Over the<br />

past five years, collaborative skills <strong>and</strong> oral <strong>and</strong> written communication abilities<br />

have assumed central importance (objective 4). Another source of input are<br />

4


faculty members engaged with industrial projects who have evidence of<br />

employment needs.<br />

Program alumni/ae prosper to the extent that they have been provided with<br />

knowledge <strong>and</strong> abilities beyond entry level skills that will promote professional<br />

ascent. For example, life-long learning <strong>and</strong> being equipped for responsible <strong>and</strong><br />

ethical participation in professional roles is integral to positions of ever increasing<br />

responsibility (objective 1). Assessment taps into the experiences <strong>and</strong><br />

recommendations of recent alumni/ae with a survey, the latest of which was<br />

performed in November 2005. The results help to shape the objectives <strong>and</strong> the<br />

resulting curriculum.<br />

The faculty’s sense of professional accomplishment rests on the ability of the<br />

program to attract motivated students <strong>and</strong> prepare them for success. For<br />

example, when speaking about program worthiness, the faculty emphasizes a<br />

firm grasp of foundational ideas <strong>and</strong> the ability to use this knowledge in problemsolving<br />

(objective 2). Faculty input on the objectives comes primarily by way of<br />

their participation in the Curriculum Committee.<br />

Members of the Advisory Board extend themselves to see the School prosper in<br />

many ways. For example, the Advisory Board actively promotes the Annual<br />

Leadership in Service <strong>and</strong> Technology fund-raiser for scholarships. The stature<br />

of the School reflects upon their own. Beyond this, Advisory Board members are<br />

also employers, <strong>and</strong> as such they provide the employers’ opinions on the<br />

objectives. For example, informedness on social issues surrounding the<br />

application of computing technology is an objective they endorsed (objective 5).<br />

In the end, all of our stakeholders benefit when the program meets each of the<br />

five objectives <strong>and</strong> produces well-rounded, well-equipped graduates.<br />

B.1.A.5. For each program educational objective, indicate the following:<br />

• the mechanism(s) used to measure it.<br />

• when it is measured <strong>and</strong> who is responsible for obtaining the<br />

measurement.<br />

• the extent to which the educational objective is being met.<br />

• all improvements, if any, that have been identified as a result of<br />

the assessment data collected.<br />

• the implementation time-line for any improvements identified.<br />

• where the improvement has been documented.<br />

5


Mechanism 1: The Alumni/ae Survey<br />

Assesses Objectives 1 through 5<br />

To the faculty, the evidence was strong that objectives two, three, <strong>and</strong> four were<br />

being satisfied. The leadership <strong>and</strong> lifelong learning parts of objective one were<br />

being satisfied as well.<br />

The chief instrument for assessing success in fulfilling the program's objectives is<br />

the Alumni/ae Survey (Attachment I-2). Its most recent administration was in Fall<br />

2005. The current assessment schedule has it on a three year cycle for<br />

distribution to graduates three to five years out. Responsibility for its<br />

administration rests with the Seidenberg School's Assessment/Research Analyst.<br />

This mechanism measures objectives 1 through 5.<br />

Twelve computer science faculty members, on the Spring 2006 Assessment Day,<br />

studied the summarization of the completed questionnaires (Attachment I-2).<br />

The technique was to perform a content analysis of responses, looking for<br />

indications of objective realizations in the descriptions of the job responsibilities.<br />

For instance, a senior software engineer gave a job description citing "project<br />

management," "requirements gathering," <strong>and</strong> "system design." From this the<br />

faculty inferred the manifestation of leadership, collaborative skills,<br />

communication skills, an ability to apply ideas of computer science in problemsolving,<br />

<strong>and</strong> abilities integral to the design <strong>and</strong> construction of software systems.<br />

Another alumnus mentioned serving as the "interim chairman of an industry<br />

st<strong>and</strong>ards committee for systems management interoperability." This was clear<br />

evidence of leadership (<strong>and</strong> the correlative skills of collaboration <strong>and</strong><br />

communication) as well as knowing when technical knowledge applies, where it<br />

applied, <strong>and</strong> how it applies. The question explicitly asking about graduate school<br />

study or industrial training to keep up with technology showed that virtually<br />

everyone was doing this. Of the alumni/ae who graduated from 1996 to 2003<br />

who responded, 35% had completed or were in a masters degree program <strong>and</strong><br />

78% had completed some additional education.<br />

Only implied in the results of the survey was the disposition toward ethical<br />

conduct <strong>and</strong> a conversance with social issues (objective 5). The faculty was<br />

convinced that the limitation lay in the instrument used for the alumni/ae survey,<br />

not in the absence of these characteristics among our graduates. This is because<br />

evidence shows that the program’s learning outcome seeding these, outcome e<br />

(to acquire an underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social responsibilities),<br />

is being met. Objective 5 is supported by the presence of the international<br />

computing honors society Upsilon Pi Epsilon (UPE) <strong>and</strong> other honor societies<br />

present on campus including Alpha Chi, the national college honor scholarship<br />

society as well as by continuing programs for alumni/ae.<br />

The next alumni/ae survey, due in 2009, canvassing the classes of 2004 to 2006,<br />

will include an additional item asking alumni/ae to identify the two or three most<br />

6


significant social, ethical, legal or security issues that they have encountered <strong>and</strong><br />

about their ability to deal with them. The instrument will also include another item<br />

asking alumni/ae about the international study opportunities of which they availed<br />

themselves. Since 2004, Seidenberg faculty have offered four courses open to all<br />

Seidenberg students that have included travel to either India or to Great Britain<br />

<strong>and</strong> France. These courses would typically be electives in the general education<br />

part of the curriculum. The University has long offered a multitude of international<br />

opportunities through international field study courses <strong>and</strong> study abroad<br />

programs.<br />

These new questions addressing objective 5 will be pre-tested in Fall 2007 with a<br />

small scale alumni/ae survey. Hence, relative to the assessment of the program's<br />

educational objectives <strong>and</strong> their evaluation with this mechanism, the central<br />

improvement is a change in the primary assessment instrument. The new<br />

instrument will be available for display along with the other materials for the<br />

visiting team in the Goldstein Academic Center, room 314.<br />

Mechanism 2: Employment of New Graduates<br />

Assesses Objectives 2, 3, <strong>and</strong> 4<br />

A successful job search following graduation is evidence of being equipped for<br />

professional participation. The employment data collected by the Cooperative<br />

Education <strong>and</strong> Career Services Department supply substantiation. For each<br />

graduating class, Cooperative Education <strong>and</strong> Career Services furnishes the<br />

Seidenberg Assessment/Research Analyst with information on the organizations,<br />

positions, <strong>and</strong>, when available, the salaries of graduates who obtained full-time<br />

jobs through their auspices (or located jobs without their assistance but were<br />

good enough to respond to an employment inquiry). With respect to the class of<br />

2004 (data for the class of 2005 was incomplete), the highest paid graduate from<br />

the BS in CS program took a job as a programmer/analyst with the SG<br />

Constellation Financial Management Company, LLC, for $85,000. The second<br />

highest paid graduate took a job as a Web Application Developer for CIBC World<br />

Markets for $75,000. Other placements included a programmer/analyst at MBIA,<br />

a programmer/analyst at Nextel Communications, <strong>and</strong> a software developer at<br />

Epic Systems Corporation. This comes from the table in Attachment I-3.<br />

The job placements shown in Attachment I-3 go back through 2001. They<br />

substantiate that graduating students are judged by employers to have the<br />

abilities, skills, <strong>and</strong> attitudes needed to add value to their organizations in the<br />

arena of designing <strong>and</strong> building software (objective 3). Owing to the nature of<br />

software construction, this necessarily carries with it fulfillment in the areas of<br />

objectives 2 <strong>and</strong> 4. The personal intangibles that make for confidence in a<br />

c<strong>and</strong>idate being interviewed are tied-up with objective 1.<br />

7


The improvement identified by the faculty is, once again, not in the program but<br />

in the assessment. Creating a survey asking whether students are getting the<br />

job offers they had expected or most wanted would be useful.<br />

Mechanism 3: Performance Reviews of Interns in Cooperative Education<br />

Assesses Objectives 2, 3 <strong>and</strong> 4<br />

The Cooperative Education <strong>and</strong> Career Services Department queries those<br />

responsible for supervising interns toward the end of each internship assignment.<br />

Supervisors are asked about “academic preparation”, “quality of work”, <strong>and</strong> other<br />

aspects of work performance.<br />

The Computer Science Department has access to <strong>and</strong> annually examines the<br />

performance reviews of undergraduates on cooperative education internships.<br />

Although undergraduates are not alumni/ae with three to five years of<br />

professional experience, interns are evaluated on criteria that converge upon the<br />

program's objectives <strong>and</strong> relate closely to long term professional success.<br />

Moreover, direct behavioral observation by external examiners is always<br />

informative. The following comments, culled from the data in Attachment I-8,<br />

speak of good capabilities <strong>and</strong> developmental progress.<br />

work attitude:<br />

• "was a great asset to our company; he was a hard<br />

worker <strong>and</strong> appreciated everything he learned"<br />

• "always asks for more work; very competent <strong>and</strong><br />

helpful; a pleasure to work with"<br />

• "great work ethic; always looking to do more"<br />

work quality<br />

• "demonstrates unusually high technical aptitude<br />

for a second year student"<br />

• "programs are well written <strong>and</strong> tested"<br />

• "overall he presents very good quality; occasional<br />

omission of details has been discussed with the<br />

student, <strong>and</strong> solid improvement has been observed"<br />

academic preparation<br />

• "demonstrates a good underst<strong>and</strong>ing of the W2K<br />

server environment"<br />

8


• "very good with h<strong>and</strong>s-on work, but we have not<br />

evaluated him on his theoretical skills"<br />

• "did not come in with all the skills, but was able<br />

to pick up everything very quickly"<br />

verbal <strong>and</strong> writing ability<br />

• "clear <strong>and</strong> concise"<br />

• "programs <strong>and</strong> documentation were well written"<br />

• "great writing skills"<br />

The Cooperative Education <strong>and</strong> Career Services Department reports that, "40%<br />

of graduating Pace co-op interns were offered full-time positions from their co-op<br />

employers upon graduation" (http://appserv.<strong>pace</strong>.edu/execute/page.cfm?doc_id=8008).<br />

Historically, this value is slightly higher for students in the BS in CS. Thus, since<br />

the program's objectives correspond to career readiness, this is a good indicator<br />

that they are being met.<br />

The multiple indicators above enable the conclusion that our program objectives<br />

en masse are being fulfilled.<br />

Metamechanism: Assessment Day<br />

Once a year, at the end of the Spring semester, the Seidenberg School holds an<br />

"Assessment Day" for the faculty hosted by the assessment staff for the purpose<br />

of "closing the loop.” "Closing the loop," consists of workshops <strong>and</strong> general<br />

discussions:<br />

• interpreting recent assessment data relative to evaluating how well<br />

program objectives <strong>and</strong> student outcomes are being met<br />

• examining the efficacy of past improvements<br />

• revamping past improvements that are not working as hoped <strong>and</strong> to plan<br />

new improvements (brainstorm, prioritize, formulate practical strategies)<br />

• considering the adequacy of assessment instruments <strong>and</strong> analyses<br />

Assessment Day was held on May 3, 2006. The program <strong>and</strong> the notes<br />

recording the events of the day are in Attachment I-4.<br />

B.1.A.6. List the program's student outcomes.<br />

The program's student outcomes (also referred to as the program's learning<br />

outcomes), specifying what students are expected to know by the time of<br />

graduation, are those promulgated by ABET for computer science programs<br />

9


(coming out of Engineering Criteria 2000: Criteria for Accrediting Programs in<br />

Engineering in the United States by the Engineering Accreditation Commission).<br />

a) An ability to apply knowledge of computing <strong>and</strong> mathematics appropriate to<br />

the discipline.<br />

b) An ability to analyze a problem <strong>and</strong> to identify <strong>and</strong> define the computing<br />

requirements appropriate to its solution.<br />

c) An ability to design, implement <strong>and</strong> evaluate a computer-based system,<br />

process, component, or program to meet desired needs.<br />

d) An ability to function effectively on teams to accomplish a common goal.<br />

e) An underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social responsibilities.<br />

f) An ability to communicate effectively with a range of audiences.<br />

g) An ability to analyze the impact of computing on individuals, organizations <strong>and</strong><br />

society, including ethical, legal, security <strong>and</strong> global policy issues.<br />

h) Recognition of the need for, <strong>and</strong> an ability to engage in, continuing<br />

professional development.<br />

i) An ability to use current techniques, skills, <strong>and</strong> tools necessary for computing<br />

practices.<br />

j) An underst<strong>and</strong>ing of the processes that support the delivery <strong>and</strong> management<br />

of information systems within a specific application environment.<br />

k) An ability to apply design <strong>and</strong> development principles in the construction of<br />

software systems of varying complexity.<br />

B.1.A.7. For each outcome, indicate the following:<br />

• the mechanism(s) used to measure it.<br />

• when it is measured <strong>and</strong> who is responsible for obtaining the<br />

measurement.<br />

• the extent to which the educational outcome is being met.<br />

• all improvements, if any, that have been identified as a result of<br />

the assessment data collected.<br />

• the implementation time-line for any improvements identified.<br />

• where the improvement has been documented.<br />

10


Computer Science<br />

Program<br />

Outcomes<br />

a) An ability to apply<br />

knowledge of<br />

computing <strong>and</strong><br />

mathematics<br />

appropriate to the<br />

discipline.<br />

b) An ability to<br />

analyze a problem,<br />

<strong>and</strong> identify <strong>and</strong> define<br />

the computing<br />

requirements<br />

appropriate to its<br />

solution.<br />

c) An ability to design,<br />

implement <strong>and</strong><br />

evaluate a computerbased<br />

system,<br />

process, component,<br />

or program to meet<br />

desired needs.<br />

d) An ability to<br />

function effectively on<br />

teams to accomplish a<br />

common goal.<br />

e) An underst<strong>and</strong>ing<br />

of professional, ethical<br />

<strong>and</strong> social<br />

responsibilities.<br />

f) An ability to<br />

communicate<br />

effectively with a range<br />

of audiences.<br />

g) An ability to<br />

analyze the impact of<br />

computing on<br />

individuals,<br />

organizations <strong>and</strong><br />

society, including<br />

ethical, legal, security<br />

<strong>and</strong> global issues.<br />

h) Recognition of the<br />

need for, <strong>and</strong> an ability<br />

to engage in,<br />

continuing professional<br />

development.<br />

i) An ability to use<br />

current techniques,<br />

skills, <strong>and</strong> tools<br />

necessary for<br />

computing practice.<br />

Table 1.1 shows which outcomes are informed by each of the mechanisms. Input on<br />

each outcome comes from multiple sources.<br />

Course<br />

Displays<br />

Table 1.1 Program Outcomes by Assessment Mechanisms<br />

Student<br />

Opinion<br />

Survey<br />

Summative<br />

Assessment<br />

Course<br />

Opinion<br />

Survey<br />

11<br />

Co-op <strong>and</strong><br />

Career<br />

Services’<br />

Survey of<br />

Internship<br />

Employers<br />

NSSE<br />

Advisory<br />

Board<br />

Curriculum<br />

Committee<br />

Academic<br />

Program<br />

Review<br />

X X X X X X X<br />

X X X X X X<br />

X X X X X X X X<br />

X X X X X X X X<br />

X X X X X X X X<br />

X X X X X X X X<br />

X X X X X X X<br />

X X X X X X X<br />

X X X X X X X X


Computer Science<br />

Program<br />

Outcomes<br />

j) An ability to apply<br />

mathematical<br />

foundations,<br />

algorithmic principles,<br />

<strong>and</strong> computer science<br />

theory in the modeling<br />

<strong>and</strong> design of<br />

computer-based<br />

systems in a way that<br />

demonstrates<br />

comprehension of the<br />

tradeoffs involved in<br />

the design choices.<br />

k) An ability to apply<br />

design <strong>and</strong><br />

development principles<br />

in the construction of<br />

software systems of<br />

varying complexity.<br />

Course<br />

Displays<br />

Student<br />

Opinion<br />

Survey<br />

Summative<br />

Assessment<br />

Course<br />

Opinion<br />

Survey<br />

12<br />

Co-op <strong>and</strong><br />

Career<br />

Services’<br />

Survey of<br />

Internship<br />

Employers<br />

NSSE<br />

Advisory<br />

Board<br />

Curriculum<br />

Committee<br />

Academic<br />

Program<br />

Review<br />

X X X X X X X<br />

X X X X X X


The nine mechanisms yielding information that is integrated to form assessments<br />

of each outcome are shown <strong>and</strong> described in the following table, Table 1.2:<br />

Table 1.2 Mechanisms for Assessing Program Outcomes<br />

Mechanism Description Frequency<br />

Course Displays<br />

(on display - G314)<br />

Student Opinion Survey<br />

(Attachment I-5)<br />

Summative Assessment<br />

(Attachment I-6)<br />

Course Opinion Survey<br />

(Attachment I-7)<br />

Co-op <strong>and</strong> Career<br />

Services' Survey of<br />

Internship Employers<br />

(Attachment I-8)<br />

National Survey of<br />

Student Engagement<br />

(NSSE)<br />

(Attachment I-9)<br />

Advisory Board<br />

(on display - G314)<br />

Curriculum Committee<br />

(on display - G314)<br />

Academic Program<br />

Review<br />

(on display - G314)<br />

Documents class<br />

activities <strong>and</strong><br />

evidence of<br />

attainment including<br />

samples of major<br />

assignments, projects,<br />

<strong>and</strong> exams<br />

Elicits self reports on<br />

achievement of eight<br />

of the program's<br />

learning outcome<br />

Directly assesses<br />

students’ knowledge<br />

<strong>and</strong> skills on program<br />

outcomes a, c, i, <strong>and</strong> j<br />

Queries students on<br />

clarity <strong>and</strong> fulfillment<br />

of course objectives<br />

Evaluates<br />

performance on many<br />

aspects of work<br />

readiness<br />

Directly assesses<br />

what students put into<br />

their education (class<br />

presentations, papers,<br />

teamwork)<br />

Advises on the<br />

abilities <strong>and</strong> skills<br />

sought by industry to<br />

insure that the<br />

curriculum supports<br />

the outcomes<br />

Oversees the content<br />

of individual courses<br />

<strong>and</strong> the coherence of<br />

the curriculum relative<br />

to the program<br />

outcomes<br />

A comprehensive<br />

evaluation of CS<br />

programs at Pace led<br />

by an external<br />

reviewer.<br />

As needed to<br />

review specific<br />

courses for<br />

currency <strong>and</strong><br />

prospective<br />

update<br />

13<br />

Last<br />

administered<br />

Each course is<br />

reviewed at least<br />

every three years<br />

Annually Summer 2006<br />

Annually<br />

Each semester for<br />

all courses<br />

For each<br />

internship<br />

Yearly for Pace<br />

first year students<br />

<strong>and</strong> seniors<br />

Advisory Board<br />

meets four times a<br />

year;<br />

CS representative<br />

is present at least<br />

once a year<br />

Monthly<br />

throughout the Fall<br />

<strong>and</strong> Spring<br />

semesters<br />

Approximately<br />

every five years;<br />

scheduled by the<br />

Provost’s Office<br />

Spring 2006<br />

Summer 2006<br />

Summer 2006<br />

Spring 2006<br />

Last major formal<br />

input from the<br />

Advisory Board<br />

was in November<br />

2005<br />

May 2006<br />

Spring 2001<br />

Responsibility<br />

Curriculum Committee<br />

Office of the Seidenberg<br />

Associate Dean<br />

Curriculum Committee<br />

Seidenberg Assessment/<br />

Research Office<br />

Cooperative Education <strong>and</strong><br />

Career Services<br />

Department<br />

Office of Planning,<br />

Assessment, Research, <strong>and</strong><br />

Academic Budgeting<br />

(OPARAB)<br />

The Dean of the Seidenberg<br />

School<br />

Operates under the aegis of<br />

the Computer Science<br />

Department<br />

The Dean of the Seidenberg<br />

School


Each mechanism presented in Table 1.2 for assessing program outcomes will<br />

now be discussed individually.<br />

Course Displays<br />

This mechanism provides documentation of class activities <strong>and</strong> evidence of<br />

attainment. The Curriculum Committee believes there is no surer way to see<br />

what students are doing in a course <strong>and</strong> how they are performing than through<br />

samples of work, projects, <strong>and</strong> exams. Products are direct indicators of<br />

achievement.<br />

Course displays assess every outcome because it is the constellation of courses<br />

within the program through which the outcomes are developed. Table 1.3 shows<br />

the outcomes addressed by each course.<br />

Course displays are maintained by individual faculty <strong>and</strong> presented for review<br />

whenever the Curriculum Committee needs to reconsider the content <strong>and</strong> the<br />

presentation of a particular course.<br />

Table 1.3 How Student Learning Outcomes Are Fulfilled by Required Courses<br />

(An X Indicates that the Course Contributes to the Outcome's Fulfillment.)<br />

Outcome CS121 CS122 CS232 CS241 CS242 CS271 CS312 CS361 CS371 CS389 CS488<br />

a X X X X X<br />

b X X X X X X<br />

c X X X X X<br />

d X X X<br />

e X X<br />

f X X X<br />

g X X<br />

h X X X X<br />

i X X X X X X X X X<br />

j X X X X X<br />

k X X X X X<br />

Course Titles<br />

CS121 Computer Programming I (4 credits)<br />

CS122 Computer Programming II (4 credits)<br />

CS232 Computer Organization (4 credits)<br />

CS241 Data Structures <strong>and</strong> Algorithms I (4 credits)<br />

CS242 Data Structures <strong>and</strong> Algorithms II (4 credits)<br />

CS271 Fundamentals of UNIX <strong>and</strong> C Programming (2 credits)<br />

CS312 Research Methods in Computers <strong>and</strong> Society (3 credits)<br />

CS361 Programming Languages <strong>and</strong> Implementation (4 credits)<br />

CS371 Operating Systems <strong>and</strong> Architecture (4 credits)<br />

CS389 Software Engineering (4 credits)<br />

CS488 Computer Networks <strong>and</strong> the Internet (4 credits)<br />

14


Student Opinion Survey<br />

The "Student Opinion Survey," a mechanism created <strong>and</strong> managed by the<br />

Seidenberg School, is administered annually to students shortly after graduation<br />

(Attachment I-5). Among other things, students are asked about achievement on<br />

program outcomes a through k. While self-reports are only an indirect measure<br />

(i.e. they do not require demonstrations of the ad hoc abilities <strong>and</strong> skills), "…the<br />

research literature indicates a moderate to strong correspondence between<br />

students' self-reports <strong>and</strong> more objective measures of learning outcomes." [Linda<br />

C. Strauss <strong>and</strong> Patrick T. Terenzini; "Assessing Student Performance on EC2000<br />

Criterion 3.a-k"; Proceedings of the 2005 American Society for Engineering<br />

Education; June 2005, page 8 -- available on display in the Goldstein Academic<br />

Center, room 314].<br />

The instrument’s stability has afforded before/after insight into whether program<br />

improvements seem to be having the intended effects. The only changes to the<br />

instrument, since it was first administered in 1997, were made in 2005. These<br />

included rewordings of three items (5, 16, <strong>and</strong> 17) <strong>and</strong> three new questions<br />

added at the end (46, 47, <strong>and</strong> 48). The next student opinion survey will include<br />

an additional item asking graduates about the international study opportunities of<br />

which they availed themselves to better assess the global aspect of outcome g.<br />

A major change in the curriculum, to be discussed, was implemented for Fall<br />

2002 when software engineering was made the required capstone. Table 1.4<br />

below contrasts the data from 2002 with the data from 2005 to assess this<br />

change on the outcomes effected. The tabulated percentages have no intrinsic<br />

meaning. The reason for two sets of values is to establish a baseline <strong>and</strong> show a<br />

change of direction.<br />

Table 1.4 Indirect Measures of Outcomes a through k<br />

from the Student Opinion Surveys (New Graduates) in 2002 <strong>and</strong> 2005<br />

Outcome Applicable Items<br />

2002<br />

Reported Fulfillment<br />

2005<br />

Reported Fulfillment<br />

a 1, 7, 9 51% 56%<br />

b 3, 4, 5, 8 45% 62%<br />

c 5, 7 37% 50%<br />

d 14 60% 67%<br />

e 15, 16, 17 62% 64%<br />

f 11, 12, 13 67% 50%<br />

g 15, 16, 17 62% 64%<br />

h 2, 18, 19 69% 78%<br />

i 4, 6, 10 49% 64%<br />

j 8, 9 43% 57%<br />

k 5, 6 34% 53%<br />

15


Here is how the statistics reported above were derived. Firstly, items were<br />

mapped onto learning outcomes. For instance, items 1, 7, <strong>and</strong> 9 apply to<br />

outcome a:<br />

1. Coursework at Pace helped me to underst<strong>and</strong> the fundamental principles of<br />

computing.<br />

7. Coursework at Pace helped me to integrate concepts from various areas of<br />

my discipline to solve computing problems.<br />

9. Coursework at Pace helped me to develop facility with a set of quantitative<br />

techniques for analyzing technical <strong>and</strong> organizational problems.<br />

For each item respondents select one of six responses on the Likert scale:<br />

strongly agree, agree, slightly agree, slightly disagree, disagree,<br />

strongly disagree.<br />

Outcome a is "An ability to apply knowledge of computing <strong>and</strong> mathematics<br />

appropriate to the discipline."<br />

For any particular year, say 2005, the percentage of graduates who "selected<br />

strongly agree" or "agree" with each applicable item are averaged. This gives a<br />

composite:<br />

item 1: strongly agree = 40.0%<br />

+ agree = 26.7%<br />

66.7%<br />

item 7: strongly agree = 26.7%<br />

+ agree = 26.7%<br />

53.4%<br />

item 9: strongly agree = 26.7%<br />

+ agree = 20.0%<br />

46.7%<br />

Thus (66.7% + 53.4% + 46.7%) / 3 = 55.6% ≈ 56% of the program's 2005<br />

graduates may be said to concur that outcome a was fulfilled. In contrast, this<br />

value for graduates in 2002, the last year before software engineering was<br />

instituted as the required capstone, was 50%. Changes in all the outcomes<br />

conceivably bolstered by CS389 are in the right direction, whatever the causal<br />

pathways. This qualification about the causal pathways is needed because other<br />

significant changes were made to the program with the intent of strengthening it<br />

during the same time interval. The data from the Student Opinion Surveys from<br />

2001 through 2005 are on file in the Goldstein Academic Center, room 314.<br />

16


Summative Assessment<br />

The Summative Assessment is an achievement test for seniors created <strong>and</strong><br />

administered by the Curriculum Committee. This test is a direct measure of the<br />

technical program outcomes (a, c, i, <strong>and</strong> j) but not the "soft" ones. The beta<br />

version was used for the first time in Spring 2006. This beta version is in<br />

Attachment I-6 with the results <strong>and</strong> a mapping of items to program outcomes. Of<br />

the 80 items, around 35 are either true/false or multiple choice, <strong>and</strong> around 35<br />

are fill-in-the-blank. Ten require chunks of code to be written, <strong>and</strong> two require<br />

prose.<br />

Thirty students took the test. Most were given approximately two hours, but those<br />

who arrived late had less.<br />

The beta version was too long. Students rushed, eschewing items requiring<br />

more concentration <strong>and</strong> problem-solving in favor of the ones that were easier to<br />

grasp <strong>and</strong> answer (e.g. true/false <strong>and</strong> multiple choice). Even with this bias,<br />

different students completed different sets of items. Consequentially, the test<br />

could not be scored with the same kind of comprehensiveness, uniformity, <strong>and</strong><br />

objectiveness applied to evaluative classroom tests. Items requiring code,<br />

problem-solving, or prose were evaluated on a three-point scale of competence:<br />

“strong,” “fair,” <strong>and</strong> “lacking.” When numbers were required, these ratings were<br />

treated as 2, 1, <strong>and</strong> 0 respectively.<br />

Program<br />

Learning<br />

Outcome<br />

a<br />

c<br />

i<br />

j<br />

Table 1.5 Direct Measures of Outcomes a, c, i, <strong>and</strong> j<br />

from the Summative Assessment of April 2006<br />

Section 2: #4: 70%<br />

#6: 70%<br />

#7: 83%<br />

Indicative Items<br />

<strong>and</strong> Class Average on Each One<br />

Section 3: #2: 87%<br />

#4: 53%<br />

#5: 67%<br />

rating of 2 rating of 1 rating of 0<br />

Section 1: #4a (coding): 23 2 5<br />

#4b (coding): 15 4 11<br />

Section 4: #3 (coding): 14 1 15<br />

Section 2: #1: 83%<br />

#3: 73%<br />

Section 5: #9: 76%<br />

17<br />

Score<br />

74%<br />

69%<br />

1.23 out of 2<br />

which is<br />

62%<br />

The score on outcome i is somewhat lower than the others. If we knew these<br />

scales were valid <strong>and</strong> cross-calibrated, this could be the logical place for applying<br />

improvement efforts. However, we know no such thing. The assessment of<br />

77%


outcome i via the Student Opinion Surveys of 2002 <strong>and</strong> 2005, back in Table 1.4,<br />

shows no relative weakness. Data on the corresponding outcomes from the<br />

Student Opinion Surveys <strong>and</strong> the Summative Assessment are plotted in Table<br />

1.6 <strong>and</strong> depicted in Figure 1.1 beneath it.<br />

Table 1.6 Comparing Fulfillment of Program Learning Outcomes a, c, i, <strong>and</strong> j<br />

as Indicated by Student Opinion Surveys <strong>and</strong> by the Summative Assessment<br />

Program<br />

Learning<br />

Outcome<br />

Student Opinion Survey<br />

2002<br />

Student Opinion Survey<br />

2005<br />

18<br />

Summative Assessment<br />

April 2006<br />

a 51% 56% 74%<br />

c 37% 50% 69%<br />

i 49% 64% 62%<br />

j 43% 57% 77%<br />

90%<br />

80%<br />

70%<br />

60%<br />

50%<br />

40%<br />

30%<br />

20%<br />

10%<br />

0%<br />

Figure 1.1 Visualization of Table 1.6<br />

Student Opinion<br />

Survey<br />

2002<br />

Student Opinion<br />

Survey<br />

2005<br />

Summative<br />

Assessment<br />

The summative assessment is to be an annual occurrence, each April, managed<br />

by the Computer Science Curriculum Committee. The 2007 test will be an<br />

abridgement of 2006's, consisting of the nine items selected as outcomes'<br />

indicators as well as some experimental items for future year-to-year usage.<br />

Course Opinion Survey<br />

A "Course Opinion Survey" is administered toward the end of each course<br />

(Attachment I-7). Students are asked, using a Likert scale, whether the course<br />

objectives were clear <strong>and</strong> if the course satisfied these. Routinely, semester<br />

a<br />

c<br />

i<br />

j


averages run between 80% <strong>and</strong> 85%. Social scientists concur that class<br />

averages on items such as these are meaningful. Our interest in this as an<br />

assessment mechanism resides in the fact that every course has a set of<br />

st<strong>and</strong>ard objectives which cumulatively develop the program outcomes (a<br />

through k).<br />

The Seidenberg Assessment Research Office is responsible for administering<br />

<strong>and</strong> tabulating the results. Reports are distributed to instructors, department<br />

chairpeople, <strong>and</strong> the Dean.<br />

Cooperative Education (Co-op) <strong>and</strong> Career Services’ Survey of Internship<br />

Employers<br />

The Cooperative Education <strong>and</strong> Career Services Department queries those<br />

responsible for supervising interns toward the end of each internship assignment.<br />

Supervisors are asked about “academic preparation”, “quality of work”, <strong>and</strong> other<br />

aspects of work performance. This mechanism is a direct measure of those skills<br />

needed for success “on the job,” including: all outcomes associated with the<br />

ability to use current techniques, skills, <strong>and</strong> tools applied in computing practice<br />

(outcome i), those associated with teamwork <strong>and</strong> effective communication<br />

(outcomes d <strong>and</strong> f) <strong>and</strong> the other outcomes indicated as applicable in Table 1.2.<br />

The Computer Science Department has access to <strong>and</strong> annually examines the<br />

performance reviews of undergraduates on cooperative education internships.<br />

Data, both numerical summarizations <strong>and</strong> comments are in Attachment I-8.<br />

Employer comments were quoted earlier in this report. One of the statistics is<br />

that academic preparation is rated excellent or good in greater than 95% of the<br />

internships.<br />

The National Survey of Student Engagement (NSSE)<br />

The National Survey of Student Engagement (NSSE) has been administered at<br />

Pace since 2003. Administration, data analysis <strong>and</strong> the distribution of results are<br />

managed by the Office of Planning, Assessment, Research, <strong>and</strong> Academic<br />

Budgeting (OPARAB). Participants are first year students <strong>and</strong> seniors. This tool<br />

assesses oral <strong>and</strong> written communication experiences, experiences relating to<br />

collaborative work, <strong>and</strong> the development of professional, ethical <strong>and</strong> social<br />

responsibilities (outcomes d, e, <strong>and</strong> f). The NSSE statistics are based on a<br />

University-wide sample of first year students <strong>and</strong> seniors. The results are<br />

interesting because they are professional-quality measures that supply a contrast<br />

to benchmark institutions, <strong>and</strong> demonstrate the commitment of the University to<br />

these outcomes.<br />

19


NSSE results "…are a direct indicator of what students put into their education<br />

<strong>and</strong> an indirect indicator of what they get out of it" ("Introduction <strong>and</strong> Rationale for<br />

Using NSSE in Accreditation,'" page 1).<br />

The following are results from the Spring 2005 NSSE administration bearing<br />

upon outcome f, “An ability to communicate effectively with a range of<br />

audiences"<br />

In your experience at your institution during the current school year, about how<br />

often have you:<br />

1.b. Made a presentation in class?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.40 2.23 2.22<br />

Seniors 2.97 2.73 2.83<br />

To what extent has your experience at this institution contributed to your<br />

knowledge, skills, <strong>and</strong> personal development in:<br />

11.d. Speaking clearly <strong>and</strong> effectively?<br />

Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.99 2.70 2.69<br />

Seniors 2.99 2.89 2.93<br />

In your experience at your institution during the current school year, about how<br />

often have you:<br />

1.d. Worked on a paper or project that required integrating ideas or information<br />

from various sources?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 3.19 2.78 3.06<br />

Seniors 3.28 2.56 3.34<br />

20


During the current school year, about how many papers or reports have<br />

you written that are:<br />

3.c. 20 or more pages long?<br />

None = 1 Between 1 <strong>and</strong> 4 = 2 Between 5 <strong>and</strong> 10 = 3<br />

Between 11 <strong>and</strong> 20 = 4 More than 20 = 5<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 1.59 1.24 1.24<br />

Seniors 1.89 1.64 1.69<br />

3.d. between 5 <strong>and</strong> 19 pages long?<br />

None = 1 Between 1 <strong>and</strong> 4 = 2 Between 5 <strong>and</strong> 10 = 3<br />

Between 11 <strong>and</strong> 20 = 4 More than 20 = 5<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.63 2.38 2.36<br />

Seniors 2.86 2.56 2.62<br />

3.e. fewer than pages long?<br />

None = 1 Between 1 <strong>and</strong> 4 = 2 Between 5 <strong>and</strong> 10 = 3<br />

Between 11 <strong>and</strong> 20 = 4 More than 20 = 5<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 3.09 3.02 3.14<br />

Seniors 3.20 2.92 3.10<br />

To what extent has your experience at this institution contributed to your<br />

knowledge, skills, <strong>and</strong> personal development in:<br />

11.c. Writing clearly <strong>and</strong> effectively?<br />

Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 3.02 2.95 2.90<br />

Seniors 3.00 3.03 3.05<br />

The following are results bearing loosely on outcome e, “An underst<strong>and</strong>ing of<br />

professional, ethical <strong>and</strong> social responsibilities."<br />

21


During the current school year, about how often have you:<br />

6.d. Examined the strengths <strong>and</strong> weaknesses of your own views on a topic or<br />

issue?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.34 2.54 2.54<br />

Seniors 2.56 2.66 2.65<br />

6.e. Tried to better underst<strong>and</strong> some else's views by imagining how an issue<br />

looks from his or her perspective?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.56 2.79 2.76<br />

Seniors 2.69 2.82 2.80<br />

6.f. Learned something that changed the way you underst<strong>and</strong> an issue or<br />

concept?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.59 2.80 2.76<br />

Seniors 2.78 2.83 2.83<br />

To what extent has your experience at this institution contributed to your<br />

knowledge, skills, <strong>and</strong> personal development in:<br />

11.n. Developing a personal code of values <strong>and</strong> ethics?<br />

Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.66 2.44 2.48<br />

Seniors 2.59 2.47 2.59<br />

The National Study of Student Engagement also presents data bearing upon outcome d,<br />

"An ability to function effectively on teams to accomplish a common goal":<br />

22


In your experience at your institution during the current school year, about how<br />

often have you:<br />

1.g. Worked with other students on projects during class?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.48 2.52 2.46<br />

Seniors 2.76 2.53 2.59<br />

1.h. Worked with classmates outside of class to prepare class assignments?<br />

Very often = 4 Often = 3 Sometimes = 2 Never = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.38 2.17 2.36<br />

Seniors 2.81 2.56 2.83<br />

To what extent has your experience at this institution contributed to your<br />

knowledge, skills, <strong>and</strong> personal development in:<br />

11.h. Working effectively with others?<br />

Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.91 2.85 2.88<br />

Seniors 3.12 3.02 3.14<br />

Finally, since learning effectively on one's own is considered to be supportive of<br />

an ability to engage in continuing professional development, then NSSE<br />

contributes data bearing upon outcome h:<br />

To what extent has your experience at this institution contributed to your<br />

knowledge, skills, <strong>and</strong> personal development in:<br />

11.j. Learning effectively on your own?<br />

Very much = 4 Quite a bit = 3 Some = 2 Very little = 1<br />

Pace University Urban Consortium Doctoral Intensive<br />

First Year 2.87 2.85 2.83<br />

Seniors 3.00 2.96 2.98<br />

The NSSE questionnaire is in Attachment I-9 along with the means <strong>and</strong> st<strong>and</strong>ard<br />

deviations from the 2005 administration on all items for Pace, the Urban<br />

Consortium, <strong>and</strong> Doctoral Intensive institutions. The overall conclusion from the<br />

23


tables presented above is that Pace is doing as well as or better than its<br />

benchmark schools in writing, speaking, <strong>and</strong> collaborative skills. Although we<br />

look mildly weaker in the realm of viewing <strong>and</strong> underst<strong>and</strong>ing social issues (items<br />

6.d., 6.e. <strong>and</strong> 6.f), we are not weaker in supplying experiences tending to<br />

cultivate personal values <strong>and</strong> a code of ethics.<br />

Advisory Board<br />

The Seidenberg School’s Advisory Board consists of twenty five industrial<br />

practitioners from major corporations including IBM Research, Verizon <strong>and</strong><br />

PepsiCo. A full membership roster with organizational affiliations <strong>and</strong> minutes of<br />

meetings are available on display in the Goldstein Academic Center, room 314.<br />

The Advisory Board meets four times a year. It provides important guidance on<br />

curriculum issues, specifically the abilities <strong>and</strong> skills sought by industry when<br />

hiring new graduates.<br />

For example, collaborative skills were being strongly advocated by the Advisory<br />

Board since January 2001. As a result of this, <strong>and</strong> other assessments, software<br />

engineering (CS 389) was enriched with a semester-long team project <strong>and</strong> was<br />

made a requirement. This enhances the fulfillment of outcome d, “an ability to<br />

function effectively on teams to accomplish a common goal.” Another<br />

improvement motivated by the Advisory Board was to make the Data<br />

Communications course (CS 388) a requirement. This enhances the fulfillment of<br />

outcome i, “an ability to use current techniques, skills <strong>and</strong> tools necessary for<br />

computing practice.”<br />

Dr. Frances E. Allen; an Advisory Board member from IBM, a former Board<br />

member of CRA, <strong>and</strong> an IBM <strong>and</strong> an ACM fellow; is an advocate of fundamental<br />

underst<strong>and</strong>ings <strong>and</strong> constantly warns of fad technology. "Whether or not<br />

theoretical details are used directly, they influence one's thinking." Dr. Allen’s<br />

input has encouraged rigor in the algorithms <strong>and</strong> the programming languages<br />

courses leading to strengthening of outcome a, “an ability to apply knowledge of<br />

computing <strong>and</strong> mathematics appropriate to the discipline.” In the same vein, a<br />

different Advisory Board member, Henry Marzullo, emphasizes the soft skills. "I<br />

remember very little of the facts from my college days. Every day, however, I<br />

use the skills I learned in college – to write clearly, to analyze objectively, <strong>and</strong> to<br />

think creatively." His influence led to a public speaking assignment required of<br />

every student in CS 242, Data Structures <strong>and</strong> Algorithms II, bolstering the<br />

program’s strength relative to outcome f, “an ability to communicate effectively<br />

with a range of audiences.”<br />

Dean Merritt arranges for faculty to be present at Advisory Board meetings,<br />

encourages Advisory Board members to cooperate with faculty on assessment<br />

projects, <strong>and</strong> passes information relative to the educational milieu to the<br />

Curriculum Committee <strong>and</strong> to individual faculty members. This information<br />

24


includes items such as Gartner Group reports, pertinent articles from<br />

professional publications, observations from meetings she attends such as the<br />

Fall 2005 National Science Foundation workshop on the future of computer<br />

science, <strong>and</strong> insights from her participation as a CAC/ABET Commissioner.<br />

Keeping the curriculum in step with industrial practice will occupy our attention for<br />

as long as technology continues to advance. Advisory Board consultations will<br />

continue to be a primary informational resource.<br />

Curriculum Committee<br />

The CS Curriculum Committee, meeting monthly throughout the Fall <strong>and</strong> Spring<br />

semesters, integrates assessment inputs from all sources to make<br />

determinations relative to course content. Also, it is a proactive mechanism that<br />

originates improvements rooted in observations of how students learn. The CS<br />

Curriculum Committee takes responsibility for initiatives on the curriculum<br />

including interfacing with the Advisory Board, for monitoring course content <strong>and</strong><br />

activities, for constructing <strong>and</strong> evaluating the summative assessment, <strong>and</strong> other<br />

inquires, evaluations, <strong>and</strong> controls directly affecting coursework <strong>and</strong> students'<br />

experiences. For instance, outcome d was strengthened when the Curriculum<br />

Committee acted on the Advisory Board’s recommendation for an increased<br />

focus on collaborative skills. The Curriculum Committee acted to do this based<br />

not only on the Advisory Board but also with respect to other assessment<br />

sources (e.g. Academic Program Review).<br />

Academic Program Review<br />

The Academic Program Review is an assessment mechanism introduced by the<br />

Provost’s Office in 2000. The term “program review” is slightly misleading<br />

because the review might focus on a single program but it could focus on all<br />

programs within a department. Regardless of its scope, all program reviews<br />

include a self study, one or more external reviewers, a formal report on the<br />

findings by the external reviewer, <strong>and</strong> a response to this report by faculty (the<br />

Curriculum Committee or a designated subcommittee). When a program review<br />

is scheduled, the responsibility for carrying it forward is with the Dean’s Office.<br />

The latest CS Academic Program Review took place is Spring 2001 <strong>and</strong><br />

encompassed the BS in CS as well as the other computer science programs in<br />

the Seidenberg School, including the MS in CS. The two-day on-site visit was<br />

conducted by Dr. Rachelle Heller, a computer science professor from George<br />

Washington University who spent two days on site.<br />

Relative to the BS in CS curriculum, the report advised that students should<br />

receive more experience with professional, GUI development tools <strong>and</strong> software<br />

25


testing tools. As a result, JBuilder was adopted for the Java programming<br />

classes (CS121 <strong>and</strong> CS122) <strong>and</strong> for the data structures <strong>and</strong> algorithms classes<br />

(CS241 <strong>and</strong> CS242). We have since switched to Eclipse. The capstone class in<br />

software engineering (CS389) introduces JUnit <strong>and</strong> Subversion. Another<br />

recommendation was for each student to experience the collaborative analysis,<br />

design, construction, <strong>and</strong> testing of a piece of software for solving a problem with<br />

no direct solution. This was fulfilled in September 2002 when CS389, the<br />

project-based software engineering seminar, became a requirement of new<br />

matriculants. These recommendations worked together to contribute to the<br />

achievement of outcomes a, b, c, d, h, i, j, <strong>and</strong> k.<br />

The report is on display in the Goldstein Academic Center, room 314.<br />

Curriculum Improvements Resulting from Assessment<br />

There have been a number of assessment-based changes made to the<br />

curriculum <strong>and</strong> the student environment since the program's accreditation was<br />

last renewed. The first-order improvements, <strong>and</strong> most palpable, have been to<br />

the computer science courses required for graduation. Annual changes are<br />

documented in the students' worksheets. Worksheets enumerate the graduation<br />

requirements for that year's matriculants (entering freshman <strong>and</strong> transfer<br />

students). The Undergraduate Catalog shows these changes as well but with<br />

less chronological granularity because the Catalog is published only every other<br />

year. Table 1.4, below, summarizes these changes.<br />

Table 1.7 Assessment Based Curriculum Improvements<br />

Assessment Action<br />

Change<br />

Effective<br />

Academic Program Review;<br />

Advisory Board<br />

CS389<br />

Software Engineering was changed from an elective to a degree requirement<br />

Fall 2002<br />

Advisory Board; Curriculum CS388<br />

Committee; Student Opinion<br />

Survey<br />

Data Communications was changed from an elective to a degree requirement<br />

CS362<br />

Fall 2002<br />

Advisory Board; Curriculum<br />

Committee<br />

Programming Languages <strong>and</strong> Implementation II (compiler) was removed as a<br />

requirement to allow for the addition of CS389; course was discontinued; topics<br />

from compiler construction were integrated into CS361<br />

CS231<br />

Fall 2002<br />

Advisory Board; Curriculum<br />

Committee<br />

Computer Organization I (assembler) was removed as a requirement to allow<br />

for the addition of CS388; course was discontinued; PHY109 was made a<br />

requirement<br />

PHY109<br />

Fall 2002<br />

CS Curriculum Committee<br />

Digital Electronics was changed from a possible science electives to a course<br />

requirement to supply coverage of gates, adders, registers, <strong>and</strong> instruction<br />

decoders that was lost with the removal of CS231<br />

CS488<br />

Fall 2002<br />

Advisory Board Computer Networks <strong>and</strong> the Internet replaced CS388 as a degree requirement;<br />

CS388 remains available as an elective<br />

Fall 2005<br />

Advisory Board; Course CS271<br />

Opinion Survey; Curriculum UNIX (re-titled: UNIX <strong>and</strong> C) was changed from a<br />

Fall 2006<br />

Committee<br />

one-credit class to two credits; coverage will include C<br />

26


B.1.A.8. Indicate how your student outcomes map to your program<br />

objectives.<br />

As a matter of terminology, "program learning outcomes" <strong>and</strong> "student learning<br />

outcomes" refer to the same 11 statements. The key word is outcomes. (These<br />

same statements constitute our program's "student attributes" which are the<br />

focus of B.1.B.)<br />

Table 1.8<br />

How Program Objectives Are Supported by Student Learning Outcomes<br />

(Table extends to the following page.)<br />

Program's Educational Objectives Supporting Learning Outcomes<br />

Be equipped for responsible <strong>and</strong><br />

ethical participation <strong>and</strong> leadership<br />

roles as computing professionals<br />

<strong>and</strong><br />

Be equipped for lifelong learning<br />

Underst<strong>and</strong> the foundational ideas in<br />

computer science <strong>and</strong> be able to use<br />

this knowledge for guidance in<br />

technical problem-solving<br />

Be effective in the design <strong>and</strong><br />

construction of software applications<br />

d) An ability to function effectively on teams to<br />

accomplish a common goal<br />

e) An underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social<br />

responsibilities.<br />

f) An ability to communicate effectively with a range of<br />

audiences.<br />

h) Recognition of the need for, <strong>and</strong> an ability to engage<br />

in, continuing professional development.<br />

a) An ability to apply knowledge of computing <strong>and</strong><br />

mathematics appropriate to the discipline.<br />

b) An ability to analyze a problem <strong>and</strong> to identify <strong>and</strong><br />

define the computing requirements appropriate to its<br />

solution.<br />

a) An ability to apply knowledge of computing <strong>and</strong><br />

mathematics appropriate to the discipline.<br />

b) An ability to analyze a problem <strong>and</strong> to identify <strong>and</strong><br />

define the computing requirements appropriate to its<br />

solution.<br />

c) An ability to design, implement <strong>and</strong> evaluate a<br />

computer-based system, process, component, or<br />

program to meet desired needs.<br />

i) An ability to use current techniques, skills, <strong>and</strong> tools<br />

necessary for computing practices.<br />

j) An ability to apply mathematical foundations,<br />

algorithmic principles, <strong>and</strong> computer science theory in the<br />

modeling <strong>and</strong> design of computer-based systems in a wa<br />

that demonstrates comprehension of the tradeoffs<br />

involved in design choices.<br />

27


Program's Educational Objectives Supporting Learning Outcomes<br />

k) An ability to apply design <strong>and</strong> development<br />

principles in the construction of software systems of<br />

varying complexity.<br />

Be able to work effectively on teams<br />

<strong>and</strong><br />

Have oral <strong>and</strong> written communication<br />

skills<br />

Be informed on social issues that affect<br />

<strong>and</strong> are affected by the application of<br />

computing technology<br />

d) An ability to function effectively on teams to<br />

accomplish a common goal.<br />

f) An ability to communicate effectively with a range of<br />

audiences.<br />

e) An underst<strong>and</strong>ing of professional, ethical <strong>and</strong> social<br />

responsibilities.<br />

g) An ability to analyze the impact of computing on indivi<br />

organizations <strong>and</strong> society, including ethical, legal, security<br />

global policy issues.<br />

To bring the mapping of the program's objectives down to the coursework level,<br />

see Table 1.3 showing the specifics on how student learning outcomes are fulfilled<br />

by required courses.<br />

28


B.1.B. Student Attributes<br />

Our student attributes <strong>and</strong> our program's outcomes are one <strong>and</strong> the same.<br />

B.1.C. Assessments <strong>and</strong> Program Improvement<br />

B.1.C.1. Describe your procedure for periodically assessing the extent to<br />

which each of the above objectives is being met by your program. Include:<br />

• Frequency <strong>and</strong> timing of assessments<br />

• What data are collected (should include information on initial<br />

student placement <strong>and</strong> subsequent professional development)<br />

• How data are collected<br />

• From whom data are collected (should include students <strong>and</strong><br />

computing professionals)<br />

• How assessment results are used <strong>and</strong> by whom<br />

The "above objectives" (in the item) refer to our program objectives (i.e. the<br />

career <strong>and</strong> professional accomplishments for which the program strives to<br />

prepare graduates), <strong>and</strong> the specified points have been addressed in responding<br />

to item B.1.A.5. We respectfully refer this section for narratives that supplement<br />

Table 1.9, below.<br />

Mechanism<br />

Alumni/ae Survey<br />

conducted by the<br />

Seidenberg<br />

Assessment/Research<br />

Analyst<br />

Table 1.9 Mechanisms for Assessing Program Objectives<br />

What data is<br />

collected<br />

descriptions of<br />

current job<br />

responsibilities<br />

<strong>and</strong> educational<br />

activities since<br />

graduation (or<br />

plans for<br />

additional<br />

education) by<br />

graduates out<br />

for two to five<br />

years<br />

How the<br />

data is<br />

collected<br />

mailed<br />

questionnaire<br />

that may be<br />

answered via<br />

email, fax,<br />

telephone, or<br />

U.S. Mail<br />

29<br />

Frequency<br />

<strong>and</strong> timing<br />

of data<br />

collection<br />

every three<br />

years; last<br />

administration<br />

was Fall 2005<br />

How the results are<br />

used <strong>and</strong> by whom<br />

Results are summarized by<br />

the Seidenberg<br />

Assessment/Research<br />

Analyst who forwards them<br />

to the CS Curriculum<br />

Committee <strong>and</strong> the<br />

Associate Dean. They are<br />

reviewed by the faculty, for<br />

interpretation <strong>and</strong><br />

discussion, at an annual<br />

Assessment Day event.


Mechanism<br />

Employment of New<br />

Graduates - the Survey<br />

conducted by the<br />

Cooperative Education <strong>and</strong><br />

Career Services<br />

Department<br />

Survey of Internship<br />

Employers conducted by<br />

the Cooperative Education<br />

<strong>and</strong> Career Services<br />

Department<br />

What data is<br />

collected<br />

statistics for<br />

each graduating<br />

class on the<br />

organizations,<br />

positions, <strong>and</strong><br />

(when available)<br />

salaries of the<br />

full-time jobs<br />

located by new<br />

graduates<br />

performance<br />

reviews by<br />

supervisors of<br />

inters on each<br />

intern's work<br />

quality,<br />

academic<br />

preparation,<br />

verbal <strong>and</strong><br />

writing ability,<br />

work attitude,<br />

etc<br />

How the<br />

data is<br />

collected<br />

The<br />

Cooperative<br />

Education <strong>and</strong><br />

Career Services<br />

Department has<br />

employment<br />

data on h<strong>and</strong><br />

for graduates<br />

who located<br />

full-time jobs<br />

through their<br />

auspices <strong>and</strong>,<br />

for those who<br />

located jobs<br />

independently,<br />

pursues similar<br />

information via<br />

questionnaire.<br />

traditionally a<br />

mailed<br />

questionnaire;<br />

the Cooperative<br />

Education <strong>and</strong><br />

Career Services<br />

Department is<br />

in the midst of<br />

migrating to a<br />

Web-based<br />

survey<br />

administration<br />

30<br />

Frequency<br />

<strong>and</strong> timing<br />

of data<br />

collection<br />

annually,<br />

during the<br />

summer <strong>and</strong><br />

fall following a<br />

class's<br />

graduation.<br />

upon the<br />

completion of<br />

each individual<br />

internship<br />

How the results are<br />

used <strong>and</strong> by whom<br />

Results get sent to the<br />

Seidenberg School's<br />

Assessment/Research<br />

Analyst who forwards them<br />

to the CS Curriculum<br />

Committee, the Associate<br />

Dean, <strong>and</strong> the Dean. They<br />

are reviewed by the full<br />

faculty, for interpretation<br />

<strong>and</strong> discussion, at an<br />

annual Assessment Day<br />

event.<br />

Results get sent to the<br />

Seidenberg School's<br />

Assessment/Research<br />

Analyst who forwards them<br />

to the CS Curriculum<br />

Committee, the Associate<br />

Dean, <strong>and</strong> the Dean. They<br />

are reviewed by the full<br />

faculty, for interpretation<br />

<strong>and</strong> discussion, at an<br />

annual Assessment Day<br />

event.


B.1.C.2. Attach as an appendix copies of the actual documentation that was<br />

generated by your data collection <strong>and</strong> assessment process since the last<br />

accreditation visit or for the past three years if this is the first visit. Include<br />

survey instruments, data summaries, analysis results, etc.<br />

Below is a list of the attachments referenced in the narrative for this<br />

section, B.1.<br />

Attachment Number Description Section<br />

Attachment I-1<br />

Diversity in the Seidenberg School of Computer<br />

Science <strong>and</strong> Information Systems<br />

Alumni/ae Survey<br />

B.1.A.2.<br />

Attachment I-2<br />

Cover Letter<br />

Instrument<br />

Results<br />

B.1.A.5.<br />

Attachment I-3<br />

Full Time Employment Information for New<br />

Graduates 2001-2005<br />

Assessment Day<br />

B.1.A.5.<br />

Attachment I-4<br />

Agenda<br />

Assessment Exercises<br />

Minutes<br />

Student Opinion Survey 2001-2005<br />

Instrument<br />

B.1.A.5.<br />

Attachment I-5<br />

Distribution <strong>and</strong> Returns Statistics<br />

Data Analyses<br />

Student Comments - 2005<br />

Summative Assessment<br />

B.1.A.7.<br />

Attachment I-6<br />

Instrument – Spring 2006<br />

Results – Spring 2006<br />

Mappings between Items <strong>and</strong> Outcomes<br />

B.1.A.7.<br />

Attachment I-7<br />

Course Opinion Survey<br />

Instrument<br />

Student Internship Information<br />

B.1.A.7.<br />

Attachment I-8<br />

Organizations, positions, salaries<br />

Employer’s Form for Intern Evaluation<br />

Summary of Evaluations<br />

National Survey of Student Engagement (NSSE)<br />

B.1.A.5.<br />

B.1.A.7.<br />

Attachment I-9<br />

Instrument<br />

Results - 2005<br />

B.1.A.7.<br />

Attachment I-10 Evolution of Assessment in the BS in CS B.1.A.<br />

Attachment I-11 Commendation on 2006 Assessment Initiatives B.1.A.<br />

31


B.1.C.3. Describe your use of the results of the program's assessments to<br />

identify program improvements <strong>and</strong> modifications to program educational<br />

objectives <strong>and</strong> outcomes.<br />

Include:<br />

• Any major changes within the last five years<br />

• Any significant future program improvement plans based upon<br />

recent assessments<br />

• Any changes in program educational objectives or outcomes<br />

For objectives see section B.1.A.5. For outcomes see section B.1.A.7.<br />

No significant changes to the program are in the offing for the near future.<br />

Improvements are being planned, but all relate to assessment instruments<br />

<strong>and</strong> procedures. Relative to instruments, the Summative Assessment is<br />

being abbreviated for the sake of validity <strong>and</strong> usability. The Alumni/ae<br />

Survey is being extended to assess the program objectives related to<br />

social responsibilities <strong>and</strong> ethical conduct.<br />

32


B.1.D. Documentation<br />

B.1.D1. Describe the documentation that exists for each of the following:<br />

• Measurable program educational objectives<br />

• Expected outcomes<br />

• Assessment process<br />

• Results of assessment<br />

Indicate in your response who has access to each documentation item.<br />

Documentation's Location <strong>and</strong> Access<br />

Documentary Item Location Access<br />

Program's Educational Objectives Ivan G Seidenberg School of CSIS Website:<br />

>Home<br />

>Academics<br />

>Undergraduate<br />

>Computer Science, BS<br />

Public<br />

Minutes of the CS Curriculum Committee<br />

Expected Outcomes Ivan G Seidenberg School of CSIS Website:<br />

>Home<br />

>Academics<br />

>Undergraduate<br />

>Computer Science, BS<br />

Assessment h<strong>and</strong>out to CS <strong>and</strong> IS faculty at<br />

the:<br />

Westchester CSIS Faculty Meeting on 9/6/05<br />

<strong>and</strong> the<br />

New York CSIS Faculty Meeting on 9/8/05<br />

Assessment Day H<strong>and</strong>out (5/3/06)<br />

Assessment Process Learning Outcomes Assessment Plan for the<br />

School of Computer Science <strong>and</strong> Information<br />

Systems – 9/27/2004<br />

Results of Assessment Reports from the CSIS Assessment/Research<br />

Analyst<br />

Assessment Day H<strong>and</strong>outs<br />

Assorted internal documents (e.g. from Co-op<br />

<strong>and</strong> Career Services, the Office of Planning,<br />

Assessment, Research <strong>and</strong> Academic<br />

Budgeting, etc.)<br />

33<br />

Faculty,<br />

Associate Dean,<br />

Dean<br />

Public<br />

Faculty,<br />

Associate Dean<br />

Dean<br />

Faculty,<br />

Associate Dean<br />

Dean<br />

Faculty,<br />

Associate Dean,<br />

Dean

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!