This is the seventh annual report on Assessment of Student Learning for Pueblo Community College.  This report summarizes the work of the Assessment of Student Learning (ASL) committee during the past academic year and includes recommendations for the next year’s assessment plans.

The desire of the committee is to move toward program-level assessment.  This means that departments will be doing assessment activities designed to measure their effectiveness in producing qualified graduates able to begin pursuing a career or to transfer to a four-year institution.

At PCC, department chairs are in charge of the assessment process for their individual departments.  Each fall, the department chairs submit an assessment plan to the ASL committee.  The committee reviews all plans and makes suggestions.  In spring, the departments implement their plans and gather assessment data.  The data are analyzed by the departments and submitted to the ASL committee as their assessment results.  These results are published in the appendix of this annual report. 

The PCC faculty is involved in the assessment process in several ways. In addition to writing or working on their department assessment report, faculty members attend workshops or conferences, participate in professional development activities, and participate in on-campus workshops on assessment topics.

Students are encouraged to take an active role in assessment since the quality of education is influence by the activities that take place as part of assessment.  Assessment activities are publicized by faculty in the classroom and the Chair of the Assessment of Student Learning Committee met with Student Senate during the past year.

Local constituencies, such as employers, parents, and civic leaders have a stake in assessment and are being informed about assessment through messages on cable channel 18 and through the assessment web page on the PCC web site.

One area being reviewed is the “closing of the loop” where the assessment process reveals something that needs to be strengthened, modified, or changed.  If the proposed change to the program can be measured, that change should become part of the assessment plan for the following year.  The Assessment Reporting form includes a section for describing changes to the program that follow the assessment results.

During the past year, the ASL committee presented two assessment days for faculty to work on assessment plans and three different workshops which faculty members participated in for professional development on assessment.

For the 2006-07 Academic Year all 23 departments submitted Assessment Reports. Fourteen of those reports involved measurements in the area of Program Assessments.  All five of the PCC core competencies were measured by at least one department. 

The ASL Committee recommends the continuation of the twice yearly Assessment Days. The committee liked the effectiveness of this type of meeting.  Because of the success of the March 2007 meeting in the Ballroom this meeting format is being suggested for future Assessment Day meetings.

The committee also urges broader faculty attendance at conferences and workshops.  These have helped to prepare a number of faculty members with better assessment skills.  In addition the committee recommends that the short assessment workshops that were scheduled the past academic year be continued and expanded.

Introduction

Background

The Pueblo Community College (PCC) Assessment of Student Learning (ASL) committee submits this Seventh Annual Assessment Report to the administration, faculty, students, and staff of the college and to the Pueblo community.

During the past year, the ASL committee has been encouraging departments to move towards program level assessment.  The committee has also been actively working to increase the number of professional development opportunities related to assessment of student learning, and to improve communication with outside entities through the PCC web site. 

Program-level assessment looks at the entire program rather than individual classes within the program.  This assessment can be done in one of several ways.  Certain programs require students to pass a national accreditation test before they are eligible for work.  Other programs have students engage in internships or occupation experiences where industry professionals can assess the work of the students.  Some programs make use of a capstone class to evaluate the effectiveness of their graduates. Transfer programs can be evaluated by monitoring the success of students once they transfer to the four-year institution.

Assessment Process

The assessment process at PCC involves a formal process of evaluating the effectiveness of student learning.  PCC departments determine outcomes that are most meaningful for their students, employers and the community in general.  The assessment of student learning committee facilitates this process by helping develop department plans and providing opportunities for professional development that relate directly to assessment.  This report summarizes the results of assessment for the academic year 2006-2007.

Some departments engage in classroom assessment, and some departments engage in program level assessment.  Classroom assessment takes measurements of student success in a single class or group of classes and determines how the students measure up against a predetermined benchmark.

Program Level assessment looks at the results of the entire program and how the outcomes of student success can be measured.  Several departments engage in program level assessment.  Examples of program level assessment include measurements on standardized tests that are administered for all students within a specific degree program.  Examples of this are Nursing, Dental Hygiene and Law Enforcement among other programs.  Other examples are measurements on student performance making use of a capstone course or with an evaluation given to students as part of an internship or occupational experience.

Faculty Involvement

Once again this year most departments had strong faculty involvement in the assessment process.  Numerous faculty members participated in professional development assessment activities throughout the year.

• Most faculty members were involved in two Assessment Work Days in October and March where departments met to work on their assessment plans.
  
  
• Nine faculty members (Lana Carter, Laura Solano, Delia Lechtenberg, Bill Lewis, Robert Malwitz, Melinda Milkowski, Joe Easton, Jennifer Sherman, and  Boyd Rodman ) attended the Fall Assessment Conference, “Assessment 2006,” held at Community College of Aurora.
  
  
• Three ASL committee members (Boyd Rodman, Duane Garrett, and Dawn Mathis) and one non-member (Tim Gama) attended the New Mexico Higher Education Assessment and Retention Conference in February, in Albuquerque, New Mexico. Representatives from PCC have attended this conference for the last four years.
  
  
• The Assessment Coordinator, Boyd Rodman, attended the Higher Learning Commission Conference the end of April 2007 in Chicago, IL.
  
  
• Boyd Rodman worked actively with the committee for the October 2006 Assessment conference in Aurora.
  
  
• Three assessment workshops were held during the Spring semester and several faculty members were in attendance at each.
  
  
• The ASL committee participated in an AQIP action project devoted to tying assessment of student learning results to the annual planning/budgeting process.  Current status of the project is still in the research stage and a draft report is currently being written.

Student Involvement

The ASL committee recognized the importance of students’ understanding and appreciation of the assessment process.  With this in mind the assessment coordinator met with PCC’s Student Senate prior to Assessment Week to outline the stakes students have in the assessment process and to highlight the value of assessment to students.

Stakeholder Involvement

Information about Assessment of Student Learning will be communicated to the public in general through this report, and through the posting of this material on the PCC web site.

Methodology

The assessment process is determined by the individual departments.  Each department makes a commitment to an assessment plan during the first semester of the academic year. The departments follow through with data collection and analysis during the second semester.

Successes in Assessment
A number of good department Assessment Reports were submitted and all 23 departments were active in the assessment process. The Criminal Justice Department provided an exemplary assessment plan and analysis. This plan showed program changes and adjustments based on the data collected; it also used pre and post exams to establish student competency in each program goal. 

The Dental Assisting department was one of 18 departments that Assessed the PCC Core Competencies.  Dental Assisting chose Interpersonal skills.  The assessment showed 85% of students met the criteria in DEA 181.  The Math Department developed a tool to diagnose student problem areas as part of their Assessment Project.  The tool was used by all the math faculty in appropriate classes.

Core Competency Assessment Highlights
When departments do classroom assessment they are asked to evaluate the core competencies that have been defined for the college.  These five core competencies are described in the following statement.

Upon successful completion of the general education requirements, our students will be able to do the following:

• Read, write, and speak effectively.
• Interpret, evaluate, and synthesize information.
• Use technology to achieve educational objectives.
• Use interpersonal skills essential for their chosen fields.
• Apply global and cultural perspectives.

We can identify these as Communication, Interpersonal, Critical Thinking, Technology and Diversity. Here is a summary table showing how each department accounted for evaluating these criteria.

Summary of Assessments Activities
Including Assessment of Core Competencies

	Assessment Types		Core Competencies
	Program	Classroom	Communication	Inter-	Critical Thinking	Technology	Diversity
				personal
HOS	Capstone		x
LEA	POST Exam
ENT	Capstone				x
CRJ		Student Competencies			x		x
BCMT			x
REA		Classroom success				x
MAT		Historical Success rates			x
SCI		Biology success	x
SSELT	Employment survey				x
ENG			x		x
ECE		Curriculum Concepts
HUM			x
PTA	Employer survey			x	x
MIT	National Exam Employer Survey
OTA	Field Work Evaluation, Employer Survey, National Exam				x
NUR	National Exam		x		x
DEA	Mock Certification Exam			x	x		x
RES	Student Eval, Proctor Eval			x
DEH	National Exam, Student Exit Questionnaire, Employer Survey			x
EMT	National Exam				x
WEL		Comparison of different groups of student
MAC	Master CAM certification		x
AUT	Tested to NATEF standards,
	Student satisfaction survey

Program Level Assessments

Fourteen of the twenty-three departments this year engaged in Program Level Assessment.  These Assessments are looking more at the overall effectiveness of the total curriculum with respect to student success.

Business & Technology Division

Hospitality Studies

Hospitality Studies students completed a Capstone project as a Program Level Assessment.  The evaluations of the department’s Grand Buffet showed 100% of students score 90% or above on this project.  The department plan is to develop a more detailed rubric for evaluating the Capstone project in the future.

Law Enforcement Academy

The Law Enforcement Academy goal of passing state-level POST exam has not been completed because the test has not yet been administered.

Engineering Technology and Industrial Electronics Program

Engineering Technology and Industrial Electronics were measuring critical thinking and troubleshooting skills with the use of fluid power principles in a Capstone course required of all students.  A rubric was used as the assessment tool.  100% of the students were successful in meeting the criteria.  The plan is to continue and expand the criteria to enhance the students’ ability to troubleshoot hydraulic systems.

Automotive

The automotive department tested a group of their students to the NATEF standards with an average score of 88% achieved by the group.  The Automotive Department also did a student satisfaction survey to determine student’s perception of the program.  Students showed satisfaction with several classes.

Machining

The Machining department utilized the Master CAM certification test as a means of evaluating their students.  Two out of six students passed the certification test on the first attempt.  Program adjustments will include the addition of a one credit certification class and better entry level testing of skills.

Arts and Sciences Division

Social Sciences, Education and Library Technician

The Library Technician program conducted a survey of program graduates to identify their satisfaction with the program.  Any dissatisfaction with the overall program will be investigated to see what changes should be made as part of a future report.

Health Division

Physical Therapist Assistant Program

A random survey of employers indicated employer satisfaction with PTA graduates meeting the needs of the physical therapy community.

Medical Imaging Technology

89 per cent of Radiologic technology graduates of the 2006 class passed the ARRT certification exam in Radiography.  This percentage was higher than the anticipated 85% pass rate.  To improve future pass rates the department will increase the number of Web-VISTA computer based practice exams. 

In an employer survey on satisfaction with program graduates, all employers surveyed expressed that both Radiologic Technology and Sonography graduates possessed the skills and abilities that make them valuable employees.  Employer and graduate feedback indicate a need to strengthen skills in the area of C-arm mobile fluoroscopic and operating room/surgical procedures.  The plan is to involve the Radiologic Technology Advisory committee to form a subcommittee to address these concerns.

Occupational Therapy Assistant

The Occupational Therapy Assistant Program set a goal to meet the OTA community needs. The level II field work supervisors were asked to rate the students on their field work.  Out of 32 fieldwork supervisors all felt the students had adequate background for the practice.  A survey of employers was conducted to evaluate graduates’ preparation for employment; 90% of employers were satisfied with graduate skills.  To prepare for entry level employment as OTAs, students took the certification exam by the National board of Certification for Occupational Therapy.  Of the 12 students taking the exam in 2006, all 12 passed.

Nursing

The Nursing department exceeded in all categories the national pass rates on their LPN exams.  The RN program is actively working on ways to increase student success on national NCLEX-RN exams through the use of ERI standardized testing.  One hundred percent of the PN and ADN students received satisfactory clinical evaluations.

Dental Assisting

The Dental Assisting department met or exceeded goals in areas such as critical thinking, patient care, radiology, and job seeking skills.  The department gives their students a mock certification exam to measure the overall effectiveness of the program.

Dental Hygiene

The Dental Hygiene department exceeded its goals by having 100% pass rate on the national Board Dental Hygiene Exam and ultimately on the Central Regional Dental Testing Service exam.

Although new information and areas were noted as future areas of focus, 100% of students and employers surveyed felt the students were adequately prepared in the majority of related areas.

Respiratory Care

Emergency Medical Services

Emergency Medical Services will be adding more scenario based questions and critical thinking exercises to the curriculum in order to increase the first time pass rate on the National Registry exam at the Paramedic and EMT Intermediate levels. 

At the last national exam 87.5% of paramedics and 70% of intermediates passed on the first attempt.  In the skills exam 100% of paramedics and 72% of intermediates passed on the first attempt.   Additional remediation will be offered to those students that do not pass the National Registry exam on the first try. The staff will also discuss additional skills training for the EMT Basic National Registry Exam.

Classroom Level Assessments

The following departments assessed students at the classroom or course level.  Some of these assessments were in addition to Program Level Assessments by the same departments.

Business & Technology Division

Hospitality Studies

Hospitality Studies assessed effective communication and exceeded their goals in all categories.  The plan is to incorporate more effective communication and emphasize the importance of speaking and listening skills.

Law Enforcement Academy

The Law Enforcement Academy assessed students’ skills necessary to evaluate a crime scene.   A rubric was designed in regard to a mock crime scene.  All criteria were exceeded. 

Engineering Technology and Industrial Electronics Program

Engineering Technology and Industrial Electronics were measuring critical thinking and troubleshooting skills with the use of fluid power principles.  A rubric was used as the assessment tool.  100% of the students were successful in meeting the criteria. 

Criminal Justice

The Criminal Justice Department effectively used pre and post examinations to assess their program reading comprehension, critical thinking skills, professional ethics and empathetic awareness of socio-economic prejudices.  They used these assessments to adjust their program scenarios.

Business Computers and Media Technology

The Business, Computers and Media Technology Department measured student presentation skills across a number of classes using a common rubric.  Students were evaluated on responsiveness, appropriateness, verbal effectiveness, and non-verbal effectiveness.  91% of students scored in either the developing or advanced category.  The data was taken across several classes and evaluated by five different faculty members.

Machining

The Machining Department tested students’ communications skills with a presentation in MAC 245.  All students participated in the presentation assignment and the department is looking for other ways to incorporate communication skills in their classes.

Welding

The Welding Department assessed the students at the three high school programs against the students on the PCC campus.  They utilized a competition among the four sections of the program.  The results showed little difference between the different sections which reinforced the consistency across the program.  It was decided to increase the difficulty of the competition when it is next held.

Arts and Sciences Division

Reading

Reading students in REA030, 060, and 090 completed a survey to self-assess computer skills at the beginning and end of semester.  REA 030 students self reported less confidence in their computer skills then students in REA 060 and 090 levels.  Over the half of the students enrolled in REA 060 and 090 rated their computer skills as “good.”  33.4% of all students surveyed reported that they had neither completed nor were currently taking an introductory computer course.  Recommendations based on results are to add a CIS 110 as a pre or co-requisite to REA 030.  

Instructors in REA 060 and 090 tried specific vocabulary strategies to determine if the strategies improved student outcomes.  Student proficiency appears to be improved on final exams when students are required to participate in a specific study strategy such as making vocabulary cards.

Math

The Math department developed a tool to prepare students for a second opportunity to test on various math concepts. The tool diagnoses student problem areas, and it was anticipated the tool would allow students to improve their average test scores.  100% of the faculty accepted the tool as a common practice, and 70% of the students used the tool to assess their readiness for a second opportunity.  Of the students taking a second opportunity, 50% raised their test score by an average of 20%, and 50% remained about the same.  The tool will continue to be under review and revision for more widespread and standardized use.

Science

The Science department integrated a written research paper and oral PowerPoint presentation to their BIO 204 and CHE 112 courses.  One hundred percent of the students achieved satisfactory scores on both their written and oral assignments.  In the future the science department will expand the requirement for a research paper and oral PowerPoint presentation to other courses.  They also exceeded their goal of 70% of BIO 105 students obtaining at least a 72% on their comprehensive finals.

Social Sciences Education and Library Technician
The Social Sciences, Education and Library Technician program assessed critical thinking using a department-wide rubric.  The rubric was applied to written assignments in PSY 101 and HIS 201.  Fall and spring evaluations showed students performing better than the 75% target level that was set in the criteria.

English

The English Assessment showed an increase in skills in ENG 060 but lower levels in 090 and 121.  They sited various reasons including lack of part-time instructor professional development. They plan to improve communication with Adjunct instructors and establish and communicate common standards.  A project is in the works to design a web based MLA tutorial for all students. 

Early Childhood Education

Early Childhood Education measured students on their knowledge of Jean Piaget’s stages of cognitive development and Erik Erikson’s psychosocial stages.  25 students were tested and 22 of them (88%) scored at the 85% or higher level on these tests.  Additional theory material will be added to the curriculum for the fall semester.

Fine Arts and Humanities

The Fine Arts Department evaluated the students acceptance of diverse “voices” in our society.  This was measured through a reflection paper, on which 81% of Jewelry studio students did acceptable work.

Another measure was communication at the appropriate level for the course they are taking.  Measurements using an oral final indicated that 83% was able to demonstrate the appropriate level of communication at the completion of the course.

Health Division

Physical Therapist Assistant Program

One of PTA’s goals was to demonstrate competency in PTA theory and skills which resulted in students exceeding the criteria on the comprehensive lab final. On the outcome of obtaining entry level in clinical performance PTA students did not meet the criteria set, possibly due to inconsistencies in the use of the tool employed to evaluate students on clinical performance.

Occupational Therapy Assistant

Students were measured on case studies in OTA 216.  The measurements were conducted with a unit test.  94% of the students in OTA 216 passed that test with a grade of C or better.

Nursing

The Nursing department set goals of measuring student communication and critical thinking skills at the end of their program.  The results of these measurements are still being evaluated.

Dental Assisting

The Dental Assisting department met or exceeded goals in areas such as critical thinking, patient care, radiology, and job seeking skills.  Dental Assisting give their students a mock certification exam to measure the overall effectiveness of the program.

Respiratory Care

The Respiratory Care department measured interpersonal skills of their students..  All students measured received a satisfactory or better rating on their interpersonal skills.  All students also scored a 3 or higher on their self-evaluation and peer evaluations on interpersonal skills.

Dental Hygiene

The Dental Hygiene department measured students on resolving ethical dilemmas making use of their interpersonal skills. 100% of the students making use of the interpersonal skill sheet successfully completed the process.

Emergency Medical Services

Emergency Medical Services will be adding more scenario based questions and critical thinking exercises to the curriculum in order to increase the first time pass rate on the National Registry exam at the Paramedic and EMT Intermediate levels.

Closing The Loop

A major purpose of Assessment is to identify opportunities for improvement in the quality and effectiveness of the academic experience for students.  With this goal in mind, any assessment activity should have a goal of discovering areas of concern and making changes in the methods of delivery or curriculum taught to overcome these concerns.  We often refer to this as “closing the loop”.  To do this we have added a column to the Assessment Report which identifies the need for change and what those proposed changes are.

The goal for the ASL committee for next year is to encourage the measurement of the proposed changes to see if they do in deed improve the quality of the academic experience.  The vision is to test, improve, and test again, and continue this process until the quality has significantly improved.

This closing of the loop is only one way that possible assessment activities can be identified.  Many of the career and tech departments have advisory committees who see their graduates in action out in the workplace.  Input from these advisory committees can be quite helpful in coming up with new curriculum areas to test.