WVSU | West Virginia State University

West Virginia State University
Social Work Program

2004-2008 Assessment Report

Assessment Plan

Assessment of educational outcomes is an important component of the Social Work Program at WVSU. Program objectives identify educational outcomes by defining what graduates should know and be able to do. Program objectives specify the requisite skills, capabilities, and competencies needed for effective professional practice. Therefore, program assessment is the activity of investigating the extent to which the Social Work Program is succeeding in accomplishing the following objectives:

1. Apply critical thinking skills within the context of professional social work practice.

2. Practice within the values and ethics of the social work profession and with an understanding of and respect for the positive value of diversity.

3. Demonstrate the professional use of self.

4. Understand the forms and mechanisms of oppression and discrimination and the strategies of advocacy and change that advance social and economic justice.

5. Understand the history of the social work profession and its current structures and issues.

6. Apply the knowledge and skills of generalist social work to practice with systems of all sizes.

7. Apply knowledge of the bio-psycho-social variables that affect the individual development and behavior, and use theoretical frameworks to understand the interactions among individuals and between individuals and social systems (i.e., families, groups, organizations and communities).

8. Analyze the impact of social policies on client systems, workers, and agencies.

9. Evaluate research studies and apply findings to practice, and under supervision, to evaluate their own practice interventions and those of relevant systems.

10. Use communication skills differentially with a variety of client populations, colleagues, and members of the community.

11. Use supervision and consultation appropriate to generalist practice.

12. Function within the structure of organizations and service delivery systems, and under supervision, seek necessary organizational change.

The Social Work Program has an assessment plan and procedures in place to evaluate each of these program objectives. Because they are discrete, measurable, and comprehensive, program objectives form the basis for the assessment plan, which is a thoughtful and organized system for collecting information to determine the extent to which program graduates demonstrate educational outcomes.

The assessment plan includes the following components:
1. A set of quantitative and qualitative measures;
2. A system for administering each measure, including the method and frequency;
3. A benchmark identifying the level of student achievement needed for success;
4. A system for aggregating, reviewing and analyzing student outcomes; and
5. A mechanism for presenting results in a way that will be easily understood.

The program uses the following four summative measures to provide information about student competencies:

Field Evaluation – All field supervisors are asked to rate students on items designed to assess program objective competencies. The instrument is provided to field supervisors by the Director of Field Instruction and is completed at the end of Social Work Practice II (S Wk 404) and Social Work Practice III (S Wk 406). The instrument asks supervisors to rate students on discrete competencies using a Likert scale ranging from 0 (low) to 4 (high). Students are also asked separately to rate themselves on the same competencies. The benchmark for success for each item is a mean of 2. The instrument is viewed as having high face validity as determined by field supervisors and the Field Advisory Committee. The instruments are returned to the Director of Field Instruction who cleans the data and forwards the forms to a faculty member in the Department of Social Work (most recently, Dr. Raphael Mutepa) who enters the data into a SPSS database for analysis. This faculty member is provided a supplemental contract during the month of June to write a Summary Report of findings for review by various stakeholders.

Employer Survey – This survey is administered every two years to a cross section of employers that hire our BSW graduates in the Charleston and surrounding areas. The survey asks employers to rate our graduates on a scale of 1 (unprepared) to 5 (fully prepared) on items that fall into the categories of values, knowledge, and skills. Specific items are linked to various program objectives. The benchmark for success is when 75% of employers rate students as being “prepared” (rating of 3 or higher) for the item being measured. The instrument, which has been endorsed by the Field Advisory Committee and Community Advisory Committee as a convincing measure of educational outcomes (face validity), is mailed to between 30 – 35 employers by the Director of Field Instruction in April/May of alternating years. The completed surveys are returned to the Director of Field Instruction who compiles the results into a Summary Report that is presented to various stakeholders the following semester (see section below titled Structure for Using Findings for Program Improvement, p. 55). The Employer Survey was last conducted in May 2008.

Graduate Exit Survey – This survey is administered each semester with graduating seniors by the Director of Field Instruction. While primarily a formative measure, there are several items that ask students to assess their perceived preparation for generalist practice. These items are linked to Program Objective 6. The benchmark for success is when 80% of students rate themselves as being prepared for practice. Results of the Graduate Exit Survey are compiled each year by the Director of Field Instruction in a Summary Report.

Licensure Pass/Fail Rates – Each year, the Social Work Program provides the names of BSW graduates to the West Virginia Board of Social Work Examiners who generates a pass/fail report of our students who took the WV social work licensing exam in the previous 12 months. While we cannot link the results to most competencies, we believe it is an overall measure of knowledge and skills for generalist practice (Program Objective 6). The benchmark for success is when 70% of students pass the exam. The report is requested each summer by the Chair of the Department of Social Work who works with the Director of Field Instruction and an outside consultant to generate a Summary Report.

A graphic presentation of the Assessment Plan is provided below.

ASSESSMENT PLAN

Program Objective Measure(s) Procedure for
Implementation Benchmark
(Success) Analysis
Procedure
Apply critical thinking skills within the context of professional social work practice. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Practice within the values and ethics of the social work profession and with an understanding of and respect for the positive value of diversity. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Demonstrate the professional use of self. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Understand the forms and mechanisms of oppression and discrimination and the strategies of advocacy and change that advance social and economic justice. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Understand the history of the social work profession and its current structures and issues. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Program Objective Measure(s) Procedure for
Implementation Benchmark
(Success) Analysis
Procedure
Apply the knowledge and skills of generalist social work to practice with systems of all sizes. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Graduate Exit Survey Survey administered each semester with candidates for graduation 80% of students rate themselves ready for practice Descriptive data aggregated annually
Licensing Exam Annual Pass/Fail report received from WV Bd. of SW Examiners 70% of graduates will pass the exam Trend data aggregated annually
Apply knowledge of the bio-psycho-social variables that affect the individual development and behavior, and use theoretical frameworks to understand the interactions among individuals and between individuals and social systems (i.e., families, groups, organizations and communities). Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Analyze the impact of social policies on client systems, workers, and agencies. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Evaluate research studies and apply findings to practice, and under supervision, to evaluate their own practice interventions and those of relevant systems. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Use communication skills differentially with a variety of client populations, colleagues, and members of the community. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Use supervision and consultation appropriate to generalist practice. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis
Function within the structure of organizations and service delivery systems, and under supervision, seek necessary organizational change. Field Evaluation Survey administered
by mail each semester M = 2
(Scale 0-4) SPSS data analysis by year
Employer Survey Survey administered by mail every 2 years 75% of employers rate students as being “prepared” Cross sectional descriptive data analysis

In addition to summative measures of program objectives, the Social Work Program also uses numerous formative measures to provide information about how students progress through their educational experiences. Examples of these include course evaluations, mid-point evaluations of field experience, graduate exit surveys (described above), advising evaluations, student grades, and focus group data. Because the kind if educational assessment required by AS 8 relies primarily on summative measures, formative measures are mentioned here but not described in detail.
Summary of Data Collected for Each Program Objective

Objective 1: Apply critical thinking skills within the context of professional social work
practice.

Over the past five years on the Field Evaluation, field supervisors consistently rated students as satisfactorily meeting this objective. Mean scores range from a low of 2.86 to a high of 4.00 (on a scale of 0 to 4) on an item labeled “critical thinking skills and problem solving abilities”. Likewise, students’ self-ratings range from a mean score of 2.83 to 3.71. Mean scores for most years fall above 3; scores for all years exceed the benchmark score of 2.00

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to this item (n=31) reported that our graduates were prepared with the skills needed to think critically and solve problems (benchmark = 75% of employers).

Objective 2: Practice within the values and ethics of the social work profession and with
an understanding of and respect for the positive value of diversity.

Over the past five years, field supervisors consistently rated students as satisfactorily practicing within the values and ethics of the social work profession. Mean scores range from a low of 2.95 to a high of 4.00 (on a scale of 0 to 4) on an item labeled “professional values and ethical behavior”. Likewise, students’ self-ratings range from a mean score of 3.11 to 3.71. Regarding sensitivity to and respect for human diversity, mean scores from supervisors range from 3.05 to 3.67, and mean scores from students range from 2.94 to 3.50 on an item labeled “human diversity”. Scores for all years exceed the benchmark score of 2.00

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to values and ethics survey items (n=31) reported that our graduates started employment prepared to practice within the values and ethics of the profession and with understanding of and respect for human diversity (benchmark = 75% of employers).

Objective 3: Demonstrate the professional use of self.

Annual mean scores from field supervisors range from 3.09 to 3.89; mean scores from students range from 3.11 to 4.00 on an item labeled “professional development”. Another item labeled “relations with clients” is designed to assess the student’s ability to establish and maintain professional relationships and to handle difficult situations. On this item, mean scores from supervisors over the five-year period range from 2.77 to 3.78; and from students, mean scores range from 2.74 to 3.50. A third item labeled “professional decorum” measures professional behavior and appearance. On this item, mean scores from supervisors range from 2.64 to 3.89; and from students, mean scores range from 2.83 to 3.86. Scores for all years exceed the benchmark score of 2.00.

Objective 4: Understand the forms and mechanisms of oppression and discrimination
and the strategies of advocacy and change that advance social and
economic justice.

Over the past five years, field supervisors consistently rated students as satisfactorily understanding oppression and discrimination and strategies for advancing social change efforts. Mean scores on an item labeled “social economic justice” range from a low of 2.76 to a high of 3.31 (on a scale of 0 to 4). Likewise, students’ self-ratings range from a mean score of 2.87 to 3.46. Scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 100% of employers who responded to a populations-at-risk item (n=31) reported that our graduates are prepared to practice in this area (benchmark = 75% of employers).

Objective 5: Understand the history of the social work profession and its current
structures and issues.

Mean scores from an item labeled “knowledge” range from 2.77 to 3.67 from field supervisors, and from 2.74 to 3.46 from students. Scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 100% of employers who responded to a social service systems item (n=31) reported that our graduates are prepared in this area (benchmark = 75% of employers).

Objective 6: Apply the knowledge and skills of generalist social work to practice with
systems of all sizes.

Field supervisors’ and students’ assessment of students’ ability to apply knowledge and skill of generalist social work to practice over a five-year period is summarized as follows:
________________________________________________________________________
*Range of Mean Scores
Item Supervisors Students_____
Generalist Roles 2.77 – 3.78 2.94 – 3.62
Service to Individuals & Families 2.68 – 3.89 2.78 – 3.57
Work with Small Groups 3.18 – 3.83 2.73 – 3.57
Work with Communities/Organizations 2.93 – 3.57 2.72 – 3.57

*Mean scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to knowledge base and skill items (n=31) reported that our graduates started employment prepared for generalist practice (benchmark = 75% of employers).

2007-2008 Graduate Survey data indicate that 94% of students felt that practice courses adequately prepared them for beginning generalist social work practice (benchmark for success = 80%).

Licensure pass/fail rates are used as an indicator of preparation for beginning generalist social work practice. Over a period of five years, the pass rate for WVSU program graduates has averaged 72.7% (benchmark = 70%). Additionally, the individual average test scores of our graduates in 2006 and 2007 were 74 and 73, respectively. These scores place our graduates in the average “Usual and Customary” range of national licensure test placement for first time testing.

Objective 7: Apply knowledge of the bio-psycho-social variables that affect the
individual development and behavior, and use theoretical frameworks to
understand the interactions among individuals and between individuals
and social systems (i.e., families, groups, organizations and communities).

Field supervisors’ and students’ assessment of students’ ability to use theoretical frameworks to understand interactions among individuals and between individuals and social systems of various sizes is summarized as follows:
________________________________________________________________________
*Range of Mean Scores
Item Supervisors Students_____
Service to Individuals & Families 2.68 – 3.89 2.78 – 3.57
Work with Small Groups 3.18 – 3.83 2.73 – 3.57
Work with Communities/Organizations 2.93 – 3.57 2.72 – 3.57

*Mean scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 100% of employers who responded to human cycle development, person/perspective, and systems theory items (n=31) reported that our graduates meet this program objective (benchmark = 75% of employers).

Objective 8: Analyze the impact of social policies on client systems, workers, and
agencies.

Annual mean scores from field supervisors range from 3.00 to 3.86; mean scores from students range from 3.89 to 3.83 on an item labeled “policies and procedures” designed to measure this program objective. Scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 100% of employers who responded to social policy and political/government process items (n=31) reported that our graduates demonstrate competencies related to analyzing the impact of social policies on client systems, workers, and agencies (benchmark = 75% of employers).

Objective 9: Evaluate research studies and apply findings to practice, and under
supervision, to evaluate their own practice interventions and those of
relevant systems.

Annual mean scores from field supervisors range from 3.00 to 3.57; mean scores from students range from 2.65 to 3.57 on an item labeled “evaluating practice and programs” designed to measure this program objective. Mean scores from another item labeled “knowledge” range from 2.77 to 3.67 from supervisors, and from 2.74 to 3.46 from students. Scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to a research item (n=31) reported that our graduates can read and apply research findings (benchmark = 75% of employers).

Objective 10: Use communication skills differentially with a variety of client
populations, colleagues, and members of the community.

Over a five-year period, field supervisors’ and students’ assessment of students’ ability to communicate effectively is summarized as follows:
________________________________________________________________________
*Range of Mean Scores
Item Supervisors Students_____
Interviewing Skills 2.80 – 3.67 2.62 – 3.43
Written Communication 2.81 – 3.67 2.80 – 3.43
Relationships with Coworkers 3.14 – 3.89 3.16 – 3.86
Rural Context 2.84 – 3.87 2.72 – 3.71

*Mean scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to items related to interviewing and writing (n=31) reported that our graduates were adequately prepared in these areas (benchmark = 75% of employers).

Objective 11: Use supervision and consultation appropriate to generalist practice.

Annual mean scores from field supervisors range from 3.29 to 3.89; mean scores from students range from 3.28 to 3.86 on an item labeled “use of supervision” designed to measure this program objective. Scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to a research item (n=31) reported that our graduates use consultation appropriately and 100% reported that students utilize supervision effectively (benchmark = 75% of employers).

Objective 12: Function within the structure of organizations and service delivery
systems, and under supervision, seek necessary organizational change.

Annual mean scores from field supervisors range from 2.67 to 3.78; mean scores from students range from 2.76 to 3.57 on an item labeled “workload management” designed to measure this program objective. Additionally, mean scores from field supervisors range from 2.87 to 3.57, and mean scores form students range from 2.83 to 3.44 on an item labeled “linkages with other community resources” designed to measure the student’s ability to relate effectively to organizations in the service delivery system. Scores for all years exceed the benchmark score of 2.00.

Results of the 2008 Employer Survey (N=35) indicate that 97% of employers who responded to a research item (n=31) reported that our graduates use consultation appropriately and 100% reported that students utilize supervision effectively (benchmark = 75% of employers).

Assessment Summary and Conclusions

We found a positive trend in the way students and field supervisors rated student performance in the two field instruction courses on the Field Evaluation over the five-year period. Mean scores in the year 2008 were generally higher than mean scores in 2004. Students and field supervisors were closely aligned in their assessment of items. In all areas, benchmarks for success were exceeded. A closer look at the most recent annual data (2008) indicates that students scored highest in the following areas: professional values and ethics, human diversity, use of supervision, knowledge of generalist practice, and relationships with coworkers. Students scored lowest in the following areas: interviewing skills, written communication, workload management, evaluating practice and programs, and linkages with community resources. Although benchmarks for success were exceeded, identification of areas with lowest scores inform continuous improvement efforts going forward.

The results of the Employer Survey indicate the presence of a high degree of employer satisfaction with the graduates of the Social Work Program. Results also reflect a strong employer and program partnership resulting in the skills, competencies, and abilities of our graduates to meet the beginning level of generalist social work practice.

When combined with other measures, results from the Graduate Exit Survey and the Licensure Pass/Fail Report add to the evidence that program graduates are prepared for generalist social work practice at the baccalaureate level.

©2014 West Virginia State University  |  P.O. Box 1000 Institute, WV 25112-1000  |  (800) 987-2112 | Mobile Site | Webmaster