City College of San Francisco - California Competes

THEME II
While indirect measures can provide useful information about the effectiveness of services, they are also
fraught with challenges that need to be addressed. For example, “a decrease in the number of students
being denied graduation due to incomplete coursework” may not be a direct result of students “reading
and comprehending the CCSF graduation requirements and utilizing the [soon-to-be-operational] Degree
Audit [program].” It might be the result of more effective use of the degree audit program by counselors
and other staff or some other intervention provided by an office other than Admissions and Records.
Similarly, DSPS faces significant challenges in developing and assessing students’ self-advocacy. If counselors
and staff develop strategies to promote more timely requests for accommodations (e.g., better
publicity of deadlines, direct requests to students, etc.), does this undermine the students’ development
of independence and self-advocacy While this question is a classic example of one of the underlying
struggles that DSPS professionals grapple with in the design and implementation of student development
programs, it is also a significant question to be addressed by these professionals as they attempt to assess
the effectiveness of their programs and services.
Clearly, direct measures of student learning are the most effective means for assessing the effectiveness
of programs and services in developing SLOs. Several of the departments in Student Development have
programs that are consistent with traditional learning assessment tools used in other instructional departments
(e.g., tests, essays, etc.). For example, the Learning Assistance Department, the Career Development
and Placement Center, and DSPS offer courses and workshops that can use testing and other traditional
assessment tools to determine the extent to which these offerings are producing specific student outcomes.
A few departments propose tracking specific student behaviors that are directly related to the desired
developmental outcomes as a means for directly assessing program effectiveness. For example, Student
Health Services suggests a review of students’ medical history documents to determine the extent to which
students followed medical advice or implemented suggested healthy activities.
Continuing Student Counseling proposes the use of a classic “input-treatment-outcomes” model for assessing
some of its SLOs. They suggest that a random sample of students could be surveyed at the time they
become eligible for CSCD services, i.e., completed one year of college or more than 24 units of college
credit. Then students could be surveyed again at the time they are leaving the institution. These surveys
could include information about shifts in students’ attitudes such as academic confidence, goals, and
achievements, as well as information about the amount and type of services that students used during
their time within the CSCD’s period of responsibility for advising the student. This “input-output” data
might produce a useful assessment of the “value added” by Continuing Counseling. However, the CSCD’s
response to the initial inventory questionnaire contains significant cautionary notes about the validity
of this type of assessment. Since students served by Continuing Student Counseling have already been
affected by other student development programs and services, it is difficult to distinguish between the
direct impact of CSCD and the delayed effect of other services. In addition, factors such as age and maturity
must be considered. Therefore, the assessments of these outcomes will require fairly sophisticated
research and analyses.
Such cautionary statements reflect a Division that is not only aware of the need to develop direct measures
of effectiveness, but is also sensitive to the research complexities of confounding variables. This is the type
of thinking, planning, and development that the new accreditation standards seek to encourage. By directly
confronting the challenges of assessing student learning outcomes in student development the Division
is operationalizing its mission statement and defining its programs in terms of effectiveness. As the process
continues it will also reap the benefits of these assessments.
274 CITY COLLEGE OF SAN FRANCISCO