Comprehensive Standard #3.5.1 (Assessment of General Education)

The institution identifies college-level competencies within the general education core and provides evidence that graduates have attained those competencies.

Compliance Certification

The College of William and Mary is in compliance with this comprehensive standard.

Explanation

Requirements for the baccalaureate degree at William and Mary include the following:

• Freshman Seminar Requirement
• Digital Information Literacy (DIL) Proficiency
• Major Computing Requirement
• 7 General Education Requirements (GERs)
• Writing proficiency (lower-division and in the major)
• Foreign Language Proficiency

In the freshman year, students take a writing- and discussion-intensive freshman seminar. Most freshman seminars satisfy the lower-level writing requirement . (This writing requirement also may be fulfilled by completing the Writing 101 course or by meeting standards established for AP, IB, or other transfer credit.) In addition, during the first year, freshmen and transfer students with less than 39 credit hours must pass a Digital Information Literacy exam to satisfy the lower-level computing requirement. These first year requirements help students develop skills for concurrent and advanced study. Other requirements may be fulfilled throughout a student's tenure at William and Mary.

The writing and computing requirements in the major were addressed in the most recent round of program reviews. Internal and external review teams read samples of student papers and reviewed the computing requirements and use of technology in the department/program curricula. In their reports, review teams commented on the proficiencies and recommended ways to improve writing and computing in the major (see Comprehensive Standard 3.3.1 ). Departments and programs have identified specific learning expectations in the new PIE framework (see PIE database). Computing and writing expectations will be integrated into the PIEs as appropriate. The Freshman Seminar Program, Digital Information Literacy Proficiency, the General Education Requirements (GERs), and the Foreign Language Proficiency are described below.

Freshman Seminar Requirement

The purpose of the freshman seminar program is to help students develop their capacity for critical thinking, independent learning, and effective communication. The College Catalog describes freshman seminars as "the first step toward preparing students for the creative and rigorous thinking, writing, and research that they will be doing at the upper-division level." The requirement reads as follows in the 2004-2005 Course Catalog:

Each entering undergraduate student is required to pass one freshman seminar in their first year at the College. The only students who are exempt from this requirement are transfer students who enter the College with at least 24 credit hours earned after graduation from high school, which have been accepted for credit at The College of William and Mary. College credits earned through Advanced Placement or other placement tests will not be counted towards the 24 credit hours required for exemption.

Expectations for student learning in freshman seminars were drafted in 2004-2005 as part of the College's newly adopted process of institutional effectiveness (PIE). The learning expectations specify that students will:

  1. engage in critical thinking on topics pertinent to the subject matter of the course;
  2. work independently to understand texts and form judgments on topics pertinent to the subject matter of the course; and
  3. communicate effectively on topics pertinent to the subject matter of the course.

The freshman seminar program was included in the Spring 2005 pilot study for assessing General Education Requirements (see Comprehensive Standard 3.3.1 ). The pilot study indicated that freshman seminar portfolios could be used to demonstrate the extent to which seminar experiences resulted in the types of learning the faculty defined as appropriate for freshman seminars.

For the pilot study, instructors for three Freshman Seminars (CHEM 150W: Chemistry of Emotion, ENGL 150W: J.R.R. Tolkein and the Chivalric Code, JAPN 150W: Japan in the World: Cross-cultural Encounters and National Identity) were asked to construct course portfolios. Portfolios include the course syllabus, an instructor narrative linking course experiences to learning expectations, and a sample of student work that demonstrates student learning relevant to the expectations. In addition, the Office of Planning and Assessment provides a summary of survey responses from students enrolled in the course.

As described in the discussion of the PIE pilot study in Comprehensive Standard 3.3.1, Working Groups of two faculty members evaluated course portfolios. Course portfolios are available in the Office of Planning and Assessment. Contents are described below for each of the three portfolios. There is a complete portfolio for JAPN 150W. (Student work, survey responses, and Working Group ratings are confidential. Please refer to the instructions for accessing password protected material) The other portfolios provide somewhat different types of information. For example, the ENGL 150W portfolio included formal peer reviews , faculty comments and a grading check list, and CHEM 150W included grading check lists but did not include the peer reviews required in the course. JAPN 150W was selected because of its direct link to the expectations (see the Narrative below). In the 2005-2006 review, instructors will have access to sample materials as they prepare their portfolios. Freshman Seminars are scheduled as part of the five-year review cycle of the general education curriculum.

2005 Pilot Study: Components of Freshman Seminar Portfolios

The current assessment process emphasizes student learning and uses course portfolios with reviews of student work to document results of the seminar experiences. This emphasis and methodology reflect a shift from the initial assessment of the Freshman Seminar Program. The initial focus was on the extent to which seminars emphasized active small-class experiences. Curricular experiences were documented in student and faculty surveys that asked about opportunities for discussion, writing, and other modes of expression appropriate to the subject matter of the course. Students also were asked the extent to which their freshman seminars helped them develop skills. Faculty members received responses from students enrolled in their seminars. The Office of Planning and Assessment reported Aggregated Results of Quantitative Analyses to the Assessment Steering Committee and the Educational Policy Committee. Survey results suggested that freshman seminars emphasized the goals of the program and that seminar experiences improved students' abilities to think critically, write, and engage in small group and class discussions. The assessment scheduled for 2005-2006 will track these conclusions in reviews of student work and experiences in individual freshman seminars.

Digital Information Literacy Exam

The Digital Information Literacy or DIL competency is defined in terms of three integrally related processes: knowledge generation, knowledge access, and knowledge evaluation. Specifically, the lower level computing requirement is described as follows in the 2004-2005 Course Catalog (page 51):

Digital Information Literacy Proficiency

The purpose of the Digital Information Literacy (DIL) proficiency is to ensure that all students, upon matriculation at the College, have a basic understanding of digital information, how it is processed, and how to use it judiciously. To assess that basic understanding, all incoming freshmen, as well as newly admitted transfer students with fewer than 39 credits, must take and pass with a grade of C- or better the DIL exam. The exam consists of questions dealing with how computers process digital information; communicating using computers; security and privacy issues; analyzing research needs; finding information electronically; evaluating the information found; and information ethics. Those students failing to take and/or pass the exam by the end of the sixth week of classes after matriculation must enroll in and pass with a C- or better INTR 160, Digital Information Literacy.

In the fall of 2004, all first year students went through the DIL tutorial and took the DIL exam. All but 19 completed the work and passed. In the Spring of 2005, the 19 students were enrolled in one of the two sections of the INTR 160, a one-credit course taught by librarians. All 19 students passed the 14 week course. In the summer of 2005, librarians were revising some of the questions and will continue to serve on the EPC to work on this important initiative.

The history and development of the DIL are described in Swem Library's timeline and update of the DIL initiative.

General Education Requirements (GERs)

The College requires undergraduate students to fulfill seven General Education Requirements (GERs). Collectively, the definitions and elaborations of the GERs specify areas of knowledge and skills important to undergraduate liberal education at William and Mary. As noted in Comprehensive Standard 3.3.1 , the College has adopted a process of institutional effectiveness that encompasses all academic and administrative programs. For the GERs, the model specifies:

• the purpose of each GER in relation to the College's mission
• collectively defined faculty expectations for student learning
• an evaluation process centered on course portfolios
• a feedback process in which department and program faculties monitor GER courses with oversight by the Educational Policy Committee, the Dean's Office, and the Office of Planning and Assessment and Assessment Steering Committee

In 2004-2005, faculty members developed expectations for student learning for each GER. The expectations specify and elaborate the meanings of the requirements. In Spring 2005, we conducted a pilot study to determine if course portfolios would provide evidence the faculty needed to monitor the general education curriculum and enhance student learning. In the pilot study, at least two faculty members were asked to construct course portfolios for each GER (and the Freshman Seminars). To construct the portfolios, individual faculty members were asked to:

• provide a narrative that would help Working Group members evaluate the course relative to the learning expectations for the GER
• identify learning experiences that address the collectively defined learning expectations,
• provide examples of excellent and marginal student work to show how activities meet learning expectations
• review and interpret survey responses that show how students in their courses rate the alignment of course experiences and GER learning expectations (survey responses are provided by the Office of Planning and Assessment and are included in each portfolio)

The portfolios were rated by Working Groups of faculty members using a common scoring guide for each GER. For the pilot, much of the material (especially student work) is paper copy and some of the material (student work, survey results, and Working Group ratings) is confidential. Course portfolios from the pilot study are available in the Office of Planning and Assessment. However, contents of each portfolio are described below with examples of portfolio material for each GER. (Please refer to the instructions for accessing password protected material)

2005 Pilot Study: Components of GER Portfolios

The second cycle of GER assessments will begin in 2005-2006. As discussed in Comprehensive Standard 3.3.1 and as noted above, the next round of assessments will be done electronically to the extent possible. In the first round, the Assessment Steering Committee and the Assessment Office (now the Office of Planning and Assessment) reported results in aggregate form. At the outset, the faculty had decided not to conduct assessment at the individual level. We did this to facilitate the development of an assessment culture and to establish the proper formative focus for assessment efforts. The next round will include course-level information that can be used by instructors, departments and programs, and committees to enhance student experiences and student learning.

At the end of the pilot study, the Working Groups concluded that the portfolio methodology is an effective way to monitor the curriculum. They are convinced that student learning at William and Mary is best measured by individual faculty members within the context of a specific learning experience. They also recognized that the assessment approach adopted as part of the PIE model requires transparency and intentionality in the delivery of the curriculum. The PIE will allow individual faculty members to adjust course experiences that do not result in the intended learning expectations. Working Groups noted the importance of involving the departments and programs in the review of GER courses. There is concern about the labor intensive processes of constructing and rating portfolios, but faculty members involved in the pilot study are confident that the process is appropriate and manageable.

History of GER assessments at William and Mary

Assessment of the general education curriculum continues to evolve and improve (see General Education Assessment Timeline ). In the earliest stages, Peter Ewell, Vice President of the National Center for Higher Education Management Systems (NCHEMS) and the College's assessment consultant, helped the Assessment Steering Committee, the Office of Assessment, the Educational Policy Committee, and a Faculty Steering Committee to develop a strategy for the first round of GER assessments in 1994-1995. We launched a pilot study using two methodologies in 1996-1997, the first year of the new curriculum. Based on results of the pilot study, the faculty approved a course portfolio methodology supplemented with a student survey for assessing the GERs. Course portfolios included course materials and examples of student work reflecting performance relevant to the GER criteria. GER questionnaires asked students and faculty to rate the extent to which GER criteria were met in each of their GER courses.

We completed the first round of GER assessments in 2003, and the Educational Policy Committee made recommendations to the faculty based on assessment results in 2003-2004. As noted in Comprehensive Standard 3.3.1 , Working Groups and the Educational Policy Committee concluded that GER courses collectively were meeting the general goals of the requirements. Recall that Working Groups recommended some clarifications in the GER criteria and suggested ways to improve the assessment process. The main concern of the Working Groups was the need for a streamlined assessment process that would reduce the workload of Working Groups and faculty teaching GER courses and, at the same time, provide greater oversight of the curriculum at the course and course section levels. The new assessment process addresses concerns of the Working Groups and aligns the GER assessments with the College's newly adopted model of institutional effectiveness.

The new PIE model (Process of Institutional Effectiveness ) is similar to the initial GER assessment process but it is more intentional in defining expectations for student learning, the evaluation process, and how results will be used to monitor and improve the curriculum. Development of the GER PIEs began in a special seminar held in August 2004 with the Director of the SACS review, faculty representatives, the Dean of Undergraduate Studies, and the Director of Assessment. The GER PIEs were refined by the Assessment Steering Committee and the August seminar group in fall 2004, and a pilot study was conducted in Spring 2005.

To streamline the process, portfolios will be electronic to the extent possible and Working Groups will meet in May to review portfolios, complete evaluations, and prepare recommendations for the Educational Policy Committee and the Assessment Steering Committee. In addition, academic departments and programs will get information about the course sections taught in their department or program. A new round of GER assessments is scheduled to begin in fall 2005.

Foreign Language Proficiency:

As noted in the 2004-2005 Undergraduate Course Catalog:

Students are required to demonstrate proficiency in a foreign language commensurate with the 202 level at William and Mary. Completion of the foreign language requirement is accomplished in several ways;

The Department of Modern Languages and Literatures posts policies for placement in language courses and has adopted a scoring guide for all courses of the foreign language proficiency requirement (101, 102, 201 and 202).

MLL Dept Scoring Guide (all languages):

The faculty for each language monitors courses that fulfill the foreign language requirement. For example, the French faculty developed a common syllabus for required courses. For French 201-202, course objectives are consistent across both semesters. French 202 is a continuation of French 201 and builds on the skills and knowledge areas developed in French 201.

All eight language areas in the Department of Modern Languages and Literatures organize their 101 through 202 courses sequentially.  Each level has its goals and, in all of the language areas, the faculty are guided by a periodic general discussion of goals, methods and course contents.  Such deliberations normally include all of the instructors in a given level, and these individuals agree upon a common syllabus and a common final examination, although periodic testing and quizzing is sometimes left up to the individual instructors. As a general rule, the Department adheres to the recommendations published by the American Council on the Teaching of Foreign Languages (ACTFL) for assessing students' skills in reading, writing, speaking and cultural knowledge. Although the level of accomplishment which the students attain in a particular skill may vary from language to language (certain non-Western languages, for instance, require the mastery of character sets and reading competence is more slowly acquired), the ACTFL guidelines take such issues into account. By the end of the 202 level, students should have attained the "intermediate mid" level in most of the skills. Naturally, Beginning and Intermediate language courses almost universally use language textbooks designed for the university-level student and such materials always take into consideration established national norms.

General Education: Additional Assessment Studies. In addition to assessments of specific general education requirements, two broader studies of general education were conducted recently. The findings reinforce results of the more specific initiatives. Comparative results of the 1992 and 2002 Senior Surveys showed that students graduating under the new curriculum rated their level of knowledge and skills in general education higher than students graduating under the old cafeteria style system of general education.

Comparison of Means: Self Reports of Skills & Knowledge
Skills and Knowledge
(1=low to 5=high) 		1992
(N=290) 		2002
(N=503) 		1992 to 2002
Difference
Effective Writing 3.99 4.24 + .25
Computer 2.66 3.79 +1.13
Natural Sciences 2.83 3.31 + .48
Scientific Method 3.09 3.50 + .41
Mathematical 2.96 3.23 + .27
Critical Thinking 4.09 4.38 + .29
Effective Speaking 3.47 3.88 + .41

In 2003, a transcript analysis indicated that many seniors had completed more than the minimal requirements for each General Education Requirement (GER). Results suggest that students have multiple experiences which reinforce the learning expectations associated with GERs.

  Seniors: took minimal # of GER courses Seniors: took more than minimal # of GER courses
GER 1:
Mathematics & Quant. Reasoning 		657 (63%) 382 (37%)
GER 2a (physical):
The Natural Sciences 		859 (78%) 244 (22%)
GER 2b (biological):
The Natural Sciences 		607 (57%) 463 (43%)
GER 3:
The Social Sciences 		382 (32%) 808 (68%)
GER 4:
World Cultures & History 		342 (28%) 863 (72%)
GER 5:
Literature & History of the Arts 		367 (36%) 746 (64%)
GER 6:
Creative & Performing Arts 		826 (71%) 337 (29%)
GER 7:
Philos., Religious, & Social Thought 		916 (79%) 238 (21%)

References