Literature Review Collaborative Document

history print discuss edit sections

Edit Section

Preface

The purpose of this document is to present a review of learning assessment research and "best practice" literature upon which recommendations for state policy on Missouri postsecondary learning assessment may be based. This is a highly contextual domain of discourse and activity. Few, if any, universals apply. Therefore our first obligation is to establish the context in which this review was undertaken. This review is the work of a voluntary group of assessment professionals, administrators, and faculty from all postsecondary sectors (public and independent, two- and four-year), as well as administrators and educators from the K-12 sector, as well as staff of the Missouri Department of Higher Education (MDHE). The group, Learning Assessment in Missouri Postsecondary Education (LAMP) was created to consider statewide issues surrounding learning assessment in Missouri and to make policy recommendations to the Commissioner of Higher Education. Two primary "drivers" of this effort may be identified. The first is a piece of legislation, hereafter referred to as SB389, passed by the Missouri legislature in May of 2007. This bill modified several provisions regarding the state’s higher education system, some of which led to creation of an MDHE initiative called the Curriculum Alignment Initiative. The second is a new statewide strategic plan for Missouri higher education entitled "Imperatives for Change: Building a Higher Education System for the 21st Century," adopted in July 2008. The Coordinating Board for Higher Education (CBHE) and the Missouri Department of Higher Education (MDHE) collaborated with presidents, chancellors, and other stakeholders on this plan, the development of which is mandated by state law. More detailed information about SB389, the Curriculum Alignment Initiative, and Imperatives for Change, can be found on the MDHE website at http://www.dhe.mo.gov/.

The particular focal points for this review that are meant to inform imminent policy-setting decisions made necessary by the Curriculum Alignment Initiative and the new strategic plan, Imperatives for Change, are (1) college access and placement, (2) articulation and transfer of general education courses, and (3) assessment of general education. The first two stem most directly from the Curriculum Alignment Initiative, and the third from Imperatives for Change. The Curriculum Alignment Initiative, attempting to address requirements of SB389, has produced college "entry" competencies and general education course "exit" competencies.

The college entry competencies are particular levels of knowledge, abilities, and dispositions higher education faculty representing core disciplines believe are necessary for entering students to possess to be successful in college. How to assess these college entry competencies and how to use the assessment results to inform college access and placement in regular, remedial, or developmental courses, are the crux questions for the college access and placement focus of this review.

The general education course exit competencies are particular levels of knowledge, abilities, and dispositions that higher education faculty representing core disciplines believe students should be able to demonstrate upon completion of specific introductory courses commonly taken by students to satisfy general education requirements. How to assess these course-level learning outcomes and how to use the assessment results to inform articulation and transfer decisions among Missouri's postsecondary institutions are the crux questions for the articulation and transfer focus of this review.

The new strategic plan, Imperatives for Change, establishes goals and objectives for which development of clear operational measures, baselines, benchmarks, and targets are required. One major objective, Objective 1C, of the plan is: "Missouri’s higher education system will demonstrate continual improvement or sustained excellence in student learning outcomes." Four indicators of progress in achieving this objective were specified. The first of those indicators is most relevant here: "Results of assessments of student learning in general education (Institutions will be provided the option of using national normed tests and/or participation in an MDHE administered project involving samples of student work evaluated by a statewide committee of faculty). Data generated should serve dual purposes of accountability, i.e., demonstration of learning gains, and improvement, i.e., use by faculty to make changes in curriculum content and delivery." How to assess student learning outcomes in general education and how to use the assessment results to serve the dual purposes of accountability and curriculum improvement are the crux questions for the college level general education focus of this review.

Having established the three main focal points of this review and the particular context from which they arise and the particular questions they must address, it is equally important to acknowledge and affirm that these particulars raise and rest upon much broader and more general issues and questions. Any coherence and consistency that may be achieved in this review will, to use a familiar expression, come not just from examining the trees and forgetting the forest. Examining the forest in this case means examining and choosing principles of learning and assessment that will both guide and undergird our review and recommendations. Thus our first focus is on principles of assessment, and establishing a basic understanding of higher learning assessment and agreement on terms of the analysis and dialogue.

LAMP Charge

The full text of the LAMP Charge is available on the MDHE website. The "deliverables" specified in the Charge include the following:
The following duties are necessary to carry out this charge:

1. Perform a review of Missouri postsecondary assessments currently in use
2. Perform a review of literature and professional knowledge regarding effective use of assessment of student learning for continuous improvement and for accountability
3. Deliver a report to the Commissioner of Higher Education by June 1, 2009, including:
   1. Summary and analysis of current Missouri practices
   2. Review of relevant assessment research
   3. Policy recommendations
   4. Impact on existing CBHE policies
   5. Possible pilot projects as proof of concept
4. Develop and implement a communication plan to publicize, allow feedback, and build support at the secondary and postsecondary levels concerning the development of a statewide assessment agenda.

This document responds to #2 and will become part #3.2 of the report delivered to the Commissioner. It is expected that the policy recommendations (#3.3) of the report will result from, or at least be partially justified on the basis of, this review of relevant assessment research.

Methodology

To perform this review of research and best practices, searches were conducted in three distinct areas of research publication and in the professional literature associated with each of the focus areas. The three distinct areas of research publication were (1) learning assessment policy research, (2) learning assessment research, and (3) learning research. The focus areas are, again, (a) access and placement, (b) transfer and articulation, and (c) general education.

Shavelson, writing about alternative designs for examing student outcomes from telecourses, observed that evaluators have a wide range of alternative designs from which to choose: "Which choice is best for a given situation depends on many factors, not the least of which are the types of decisions (and decisionmakers) on which the evaluation focuses and the feasibility of implementing the design." (Shavelson, R. et al, 1986, p. v) In a later section, an important distinction will be drawn between evaluation and assessment in the context of this review, but the observation applies just as much to the design of learning outcomes assessment as it does to program evaluation. [continue with Shavelson's examples and provide corresonding examples for this review]

Learning Assessment Policy Research

Research on learning assessment policy addresses the effectiveness of assessment policy in achieving its goals. An extensive literature review of learning assessment policy research was published by the National Center for Postsecondary Improvement (NCPI) in 1997. NCPI was a collaborative research partnership of Stanford University, the University of Pennsylvania, and the University of Michigan. While NCPI ceased operations in 2004, its research findings, publications, and toolkits continue to be available at http://www.stanford.edu/group/ncpi/, maintained by the Stanford Institute for Higher Education Research. In matters of learning assessment policy research, the related publications of NCPI have served as a base. Literature searches original to this review will be limited to the time period, 1997 to the present. Sources of learning assessment policy research mentioned in the NCPI "Benchmarking" report will be searched for new publications since 1997. These sources include federal agencies, state governments, regional accrediting associations, voluntary associations of colleges and universities, the National Governor's Association (NGA), State Higher Education Executive Officers (SHEEO), Education Commission of the States (ECS), and the National Center for Higher Education Management Systems (NCHEMS).

Learning Assessment Research

Research on learning assessment addresses the effectiveness of assessment strategies, techniques, and instruments in improving student learning, informing academic program improvement, and meeting accountability requirements. For example, when feedback is given, timely and actionable feedback improves learning much more than simple knowledge of results (Nyquist, 2003).

Learning Research

Research on learning includes both basic and applied research on how people learn. In this review, the emphasis is on learning research that may inform learning assessment practices and policies. For example, researchers have found that testing enhances learning more than additional study of the material, even in the absence of feedback (Roediger & Karpicke, 2006).

Best Practices

In addition to published research on learning assessment policy, learning assessment, and learning, this review covers professional literature on best practices within the focus areas. Beginning in 1989, the bimonthly publication, Assessment Update, has covered developments in higher education assessment. Other periodicals that regularly report on best practices in learning assessment in postsecondary education include Change, The National Teaching & Learning Forum, and AAC&U's Liberal Education and peer Review.

Principles of Assessment

This section on principles of assessment must begin by acknowledging and seeking to build upon the document, Guiding Principles of Assessment (GPA), developed by the Missouri Assessment Consortium (MAC) in 1992. The MAC statement of assessment philosophy currently posted on the MDHE website opens with the following assertion: "Assessment should be guided by clearly stated, externally validated student learning processes and outcomes that flow from and support the institutional mission." In other words, assessment should be guided by what we know about how people learn and focused on learning objectives that flow from and support the mission of the institution in which assessment takes place. While this opening statement asserts that principles of assessment should be guided by principles of learning, the clear emphasis of the opening paragraphs of the MAC GPA is on preserving the autonomy of degree-granting postsecondary institutions. This emphasis is best understood in light of the historical context of the document's creation, a time in which pressure from federal and state government was building on public institutions of higher education to provide more evidence that students were learning what institutions said they should be learning and were learning.

The MAC GPA identifies three purposes of assessment: "A) improvement of student learning and instruction, B) accomplishment of institutional mission, and C) accountability for achievement of educational goals." Irrespective of purpose, the following are identified as important features of assessment:

• Assessment should be based on multiple measures appropriate to the program and institution
• The data collected should be longitudinal and should include both quantitative and qualitative elements
• Assessment programs should be based on reliable research and proven practices
• assessment instruments and methods should be continually evaluated to determine their utility in the assessment process

Several organizations have created lists of principles of assessment. Perhaps the most frequently cited in higher education are those published originally in 1996 by the American Association of Higher Education (AAHE). AAHE was dissolved in 2005 but AAHE's 9 principles of assessment can still be found on many assessment websites. The following abbreviated list is adapted from a more complete version retrieved from http://www.facet.iupui.edu/resources/AAHE%20Principles.pdf

1. The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement.
2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes,and habits of mind that affect both academic success and performance beyond the classroom.
3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations — those derived from the institution's mission, from faculty intentions in program and course design, and from knowledge of students' own goals.
4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students "end up" matters greatly. But to improve outcomes, we need to know about student experience along the way — about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning.
5. Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Though isolated, "one-shot" assessment can be better than none, improvement is best fostered when assessment entails a linked series of activities undertaken over time.
6. Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility.
7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
9. Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation — to ourselves, our students, and society — is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement.

Authors of the AAHE Principles included Alexander W. Astin, Trudy W. Banta, K. Patricia Cross, Elaine El-Khawas, Peter T. Ewell, Pat Hutchings, Theodore J. Marchese, Kay M. McClenney, Marcia Mentkowski, Margaret A. Miller, E. Thomas Moran, and Barbara D. Wright.

A third set of principles often cited are those published as the National Association of State University and Land Grant Colleges' (NASULGC) "Statement of Principles on Student Outcomes Assessment" Interestingly, these principles are not posted on the NASULGC website. The NASULGC principles state that programs for student outcomes assessment should:

1. focus primarily on the effectiveness of academic programs and the improvement of student learning and performance;
2. be developed in collaboration with the faculty;
3. be appropriate to the particular mission and goals of the institution;
4. use multiple methods of assessment;
5. be fiscally conservative and not impose costly programs on institutions;
6. be linked to strategic planning and program review processes within the institution.

The published lists of assessment principles above focus primarily on program-level and institution-level assessment, and assume a high level of knowledge of, and experience with, the terms of discourse and literature on assessment in higher education. The following "Assessment 101" section may help those who have not participated in that discourse or read extensively in that literature.

Assessment 101

The term "assessment" has many meanings in ordinary language and in various technical languages. Assessment in this context means student learning outcomes assessment in postsecondary education.

Many recent books and articles on assessment in higher education date the beginning of current concerns with assessment in higher education in the United States from the 1980s. Frequently cited as prompts are publication of A Nation at Risk (NCEE, 1983), Involvement in Learning (NIE, 1984), Time for Results (NGA, 1986), and Boyer's College (1987). A good case can be made that these and other publications during this time stimulated a new national concern with assessment for accountability purposes, but assessment as a means of measuring and improving learning in higher education has a much longer history. By some accounts, assessment as a means of measuring learning was practiced as early as the 4th century, B.C.E., during the Han Dynasty in China (Biggs, J., 2001). However, the purpose of assessment then, and in contemporary times through the 1940s, was primarily to screen and select those most capable, or incapable, of learning. Informal assessment to improve learning is of course as at least as old as recorded accounts of teaching, made famous in Plato's accounts of Socrates. But the contemporary use of formal assessment to improve learning in higher education might be dated from the beginnings of the competency-based reform movement in higher education during the late 1960s and early 1970s (Grant, G. et al, 1979).

Purposes of Assessment

The approptiateness of any method of assessment or assessment instrument depends on the purpose of assessment. The following purposes of assessment are considered in this review:

1. Improve Student Learning
2. Improve Program of Instruction
3. Improve Educational Effectiveness of the Institution
4. Document Student Learning, Program Improvement, and Educational Effectiveness to Outside Stakeholders (Accountability)

Over the past twenty years, assessment for the purpose of accountability has become a dominating concern in higher education. The problem, many observers now agree, is that methods of assessment and assessment instruments developed or chosen solely or primarily for purposes of accountability do not necessarily serve to improve student learning, improve programs of instruction, or improve educational effectiveness at the institutional level. The challenge is to develop or choose methods of assessment and assessment instruments primarily for the purpose of improving student learning that can also serve purposes of program improvement, educational effectiveness of the institution, and accountability to external stakeholders.

Differentiating Assessment from Evaluation

In many contexts, no distinction is made between the meanings of assessment and evaluation. In this context, it is important to distinguish assessment from evaluation. Assessment is a process of measuring a performance or product of learning and giving feedback which documents growth and provides directives to improve the performance or product. Evaluation is a judgment or determination of the quality of a performance or product in relation to a goal or standard. Some efforts to distinguish assessment from evaluation attempt to define them in ways that make them seem mutually exclusive (e.g., Parker, P. et al, 2001). Some efforts force the meanings of assessment and evaluation apart by equating the former with formative evaluation and the latter with summative evaluation as first distinguished by Michael Scriven (Scriven, M., 1967). In this context, it would be most accurate to say that assessment includes multiple acts of evaluation, but is more than evaluation. Documentation of growth and actionable feedback to improve learning are as essential to assessment as is evaluation.

Assessment for/as Learning versus Assessment of Learning

Distinguishing assessment "for" learning or assessment "as" learning from assessment "of" learning is perhaps not necessary if the previous differentiation of assessment from evaluation is already recognized and accepted. Unfortunately, in practice, assessment is not routinely differentiated from evaluation and assessment "of" learning is taken to mean the same thing as summative evaluation, a judgment of a performance or product at the conclusion of a learning experience. This has led to the development of the distinction in assessment literature between assessment "for/as" learning and assessment "of" learning, with assessment "for/as" learning intended to mean the formative process that here we equate with assessment. But the phrasing of assessment "for" learning and assessment "as" learning can still contribute extra meaning even when it is recognized and accepted that all assessment is formative by definition. The valuable extra meaning supplied by using the prepositions "for" or "as" is the intention that the assessed demonstration of learning is itself a learning experience, or that the complete process of assessment-performance, evaluation, documentation, feedback-be as brief and tightly connected as possible. Assessment of a "real-world" performance or performance in a high fidelity simulation of a "real-world" setting would be an example of assessment for learning. Learning to lengthen or deepen a meditative state using biofeedback equipment, would be an example of a very brief and tight performance-evaluaton-documentation-feedback loop.

Levels of Assessment and Levels of Performance

It is also important to identify and distinguish levels of assessment and levels of performance because the term "level" is used in both cases but means something very different. By levels of assessment, we are referring to course, program, and institutional, levels of student learning assessment data collection or data analysis.

Levels of Assessment

1. Course-Level Student Learning Assessment

All courses have, or should have, specific intended student learning outcomes. For example, students in an Introduction to Macro-economics course need to be able to calculate real GDP. The assessment of course-level learning outcomes can take place throughout the course and can be measured through a very wide variety of typically faculty-based tools such as quizzes, tests, papers, portfolios, journals and class assignments or other artifacts. Formative course-level assessment requires multiple in-course assessments to improve student learning. End-of-course assessments, such as a final exam, or final paper or project evaluation, are summative with respect to the individual student's learning in that specific course, but can be formative if part of a sequence of courses in which the student's learning in later courses can be improved based on the feedback received in a previous end-of-course assessment.

2. Program-Level Student Learning Assessment

All degree programs in postsecondary education have, or should have intended program-level student learning outcomes. For example, students in a Bachelor of Science in Nursing program should, by the time they graduate from the program, be able to explain and implement triage to a patient. The assessment of program-level learning outcomes can take place throughout a student's program in more than one course. End-of-program exams are sometimes also referred to as learning assessments, but the value of such exams is obviously limited to program improvement. In other words the end-of-program exam, just like end-of-course exams, are summative with respect to the particular student tested; they are potentially formative only with respect to improvement of the academic program. When program competencies are tracked throughout a student's coursework, the college typically has a paper or electronic tracking system to insure sufficient success on program competencies. Program-level learning outcomes can also be assessed at the end of the program. And end-of-program exam may be locally developed by program faculty or it may be a standardized exam given to students graduating from similar programs across the state or nation.

3. Institution-Level Student Learning Assessment

The most common examples of institution-level student learning assessment are assessments of the general education program required of undergraduate students across many programs, and proficiency assessments of general competencies expected to be attained by some or all students across many programs by the end of their program. For an example of the first type, at or near the time that a student completes all or most of her or his general education requirements for a two-year or four-year degree program, the student might be required to take one or more general education assessments, such as a writing assessment and an assessment of critical thinking or broad content knowledge, that target intended learning outcomes of the general education program. Such assessments could be formative with respect to the student, if the student gets feedback that the required level of performance has not been achieved and there are opportunities for the student to improve. Even if summative for the student, such a general education assessment can be formative for the institution if the results are used to continuously improve the general education program. Examples of the second type, assessments that are taken by students across many programs at the very end of their academic programs may be similar to general education assessments but calibrated to higher levels of performance, or they may be substantively different than assessments given to assess outcomes in the general education program, such as integration of multiple competencies in a field of specialization.
Some colleges refer to their college-wide, end-of-program learning objectives as "Common Student Learning Outcomes" or as "Common Student Abilities", etc.

Some regard institutional performance indicators such as retention rates and graduation rates, as part of institution-level assessment, but such indicators are not assessments of learning. Our review is limited to assessment of student learning.

Levels of Performance

By levels of performance we are referring the the level of attainment of the ability identified as a learning outcome. For example, it is very common to see competencies in communication and critical thinking identified as key learning outcomes at different levels of education from high school to graduate school. But for any general competency, there is a continuum of ability or achievement and we do not expect the same level of performance in high school that we expect in earning an associates degree, a baccalaureate degree, a master's degree, or a doctoral degree. There is of course overlap in the levels of performance a student may demonstrate. A high school student may perform at a level that is exceptional in terms of our expectations for high school and that would be adequate if not exceptional at a collegiate level.

The two important points here are: (1) to understand the different meanings of levels of assessment and levels of performance, and (2) to understand that levels of performance must be included in the articulation of learning outcomes at all levels of assessment and levels of education before appropriate assessments can be developed or chosen.

Access and Placement

By access we mean access to postsecondary education. By placement we mean placement in regular college (community college, four-year college, or university) courses or placement in remedial or developemental courses. In this section, we are searching for research and best practices literature that helps answer questions like the following:

• Are their essential entry competencies important to access and college readiness that have yet to be addressed by the Curriculum Alignment Initiative?
• How can we best assess CAI Entry Level Competencies prior to postsecondary entry to alleviate the need for remediatal/developmental/pre-collegiate coursework at or after entry?
• What still needs to be done to align CAI Entry Level Competencies with DESE educational assessment standards like Course Level Expectations (CLE)?
• In cases where CLEs are adequately aligned with entry-level competencies, are the End-of-Course examinations (EOC) of the CLEs sufficient to assess for access to postsecondary coursework?
• What kinds of supplemental assessment are required if EOC's are not sufficient and/or for exceptions like late transfer students, out-of-state students, advancement from remediation/developmental coursework to college level etc.?
• How do we ensure that Dual Credit students meet the same expectations as other students?
• How can we best assess the entry-level competencies at postsecondary entry to most effectively address needs for remedial/developmental/pre-collegiate coursework at or after entry?

Assessment Policy Research

While it might seem that, as a society, we are well on our way to achieving the goal of universal access to postsecondary education, large numbers of high school graduates enter postsecondary education institutions unprepared for college-level study (Greene & Foster, 2003). Even ten years ago, according to Breneman & Haarlow (1997), the costs of remediation were estimated at $1 billion or more at public institutions alone. It would seem a "no brainer" that states would look for ways to reduce the need for remediation, but while many states have assessment policies governing assessment of college readiness at entry (at least in English and math) and placement, few have policies in place to address the problem of preparation.

It is well documented that high school students who complete a so-called college preparatory curriculum are generally better prepared for college than those who do not. But far too many students who do complete a college preparatory curriculum are still found to need remediation courses once they enter college. A study conducted by the Ohio Board of Regents in 2002 found that 25 percent of Ohio high school graduates with a known core curriculum required remediation in math or English (Long & Riley, 2007). Even higher percentages of presumably well-prepared California high school graduates have been found to require remediation in math and/or English upon entry at California State University and University of California campuses (Long & Riley, 2007). The problem obviously runs deeper than clearly poor preparation in high school. The deeper problem has been identified as a mis-alignment of course material, tests, and standards between high school and college (McCabe, 2001; Venezia, Kirst & Antonio, 2003).

The use of college placement exams as diagnostic tools in high school is one promising policy that has been pursued in several states, but in . (Tierney & Garcia, 2008).

Assessment Research

Definition of Access and Placement
The Missouri Assessment Consortium (MAC), in creating the MAC Handbook, sought to fulfill the need for a reference resource of assessment practices and experiences at public four-year institutions in Missouri. The Handbook also provides definitions of key terms used in assessment that this paper will utilize in providing a basis for research.
Placement and Diagnosis. In the context of assessment for placement into collegiate-level coursework, diagnosis has been defined, according to MAC, as “the meaningful association of a test score with a local education experience. In other words, after careful study the institution has determined that students earning a score below a given point will not be successful in a particular course or pattern of courses without institutional intervention and individual scheduling decisions.” Placement into remedial/development/pre-collegiate level coursework occurs when the institution establishes “’cut scores’ for placement in enrichment or remedial/developmental sections.”
Access
National Statistics
• Students who take a core curriculum are not prepared for college (ACT, 2007 National Data Release)
o ACT College Readiness Benchmark attainment of students taking core curriculum, by number of benchmarks attained (2007 high school graduates).
o The ACT College Readiness Benchmarks are minimum scores on the ACT English, Mathematics, Reading, and Science Tests that reflect at least a 50% chance of achieving a B or higher grade or a 75% chance of a C or higher grade in entry-level, credit-bearing college courses.
 54% of high school graduates met 1 to 3 benchmarks
 27% met all 4 benchmarks
 19% met no benchmarks
• Nearly 3 in 10 students need remediation—percentage of college freshmen enrolled in remedial coursework in fall 2000, by type of remedial course (source: US Dept of Education, National Center for Education Statistics, 2004, The Condition of Education 2004 as cited in ACT making the Dream a Reality)

Missouri Statistics (I need to flesh this out more with prose)
Number and Percentage of First-Time Students Enrolled in Remedial / Developmental Coursework
High School Graduating Class of 2007 Enrolled in College in Fall 2007

Two-Year Institutions	Total Students Enrolled	Math	Math %	English / Writing	English / Writing %	Reading	Reading %	Any	Any %
Crowder College	496	82	16.50%	0	0.00%	0	0.00%	82	16.50%
East Central College	524	265	50.60%	122	23.30%	108	20.60%	312	59.50%
Jefferson College	833	278	33.40%	223	26.80%	37	4.40%	385	46.20%
KC Metro - Blue River	345	155	44.90%	76	22.00%	31	9.00%	173	50.10%
KC Metro - Business and Tech	65	18	27.70%	7	10.80%	7	10.80%	22	33.80%
KC Metro - Longview	933	472	50.60%	199	21.30%	154	16.50%	547	58.60%
KC Metro - Maple Woods	699	285	40.80%	135	19.30%	85	12.20%	343	49.10%
KC Metro - Penn Valley	220	121	55.00%	86	39.10%	69	31.40%	149	67.70%
Linn State Technical College	282	161	57.10%	0	0.00%	0	0.00%	161	57.10%
Mineral Area College	499	151	30.30%	86	17.20%	14	2.80%	188	37.70%
Missouri State Univ - West Plains	308	162	52.60%	71	23.10%	64	20.80%	187	60.70%
Moberly Area Community College	635	274	43.10%	0	0.00%	45	7.10%	289	45.50%
North Central Missouri College	242	51	21.10%	44	18.20%	28	11.60%	80	33.10%
Ozarks Technical Community Coll	1,613	694	43.00%	389	24.10%	20	1.20%	813	50.40%
SLCC - Florissant Valley	666	450	67.60%	361	54.20%	387	58.10%	531	79.70%
SLCC - Forest Park	471	328	69.60%	250	53.10%	282	59.90%	383	81.30%
SLCC - Meramec	1,273	709	55.70%	382	30.00%	369	29.00%	805	63.20%
SLCC - Wildwood	220	115	52.30%	60	27.30%	63	28.60%	130	59.10%
St. Charles Community College	1,139	720	63.20%	624	54.80%	224	19.70%	883	77.50%
State Fair Community College	550	260	47.30%	172	31.30%	132	24.00%	318	57.80%
Three Rivers Community College	531	178	33.50%	226	42.60%	65	12.20%	287	54.00%
Subtotal	12,544	5,929	47.30%	3,513	28.00%	2,184	17.40%	7,068	56.30%
Four-Year Institutions	Total Students Enrolled	Math	Math %	English / Writing	English / Writing %	Reading	Reading %	Any	Any %
Harris Stowe State University	233	97	41.60%	0	0.00%	82	35.20%	158	67.80%
Lincoln University	273	124	45.40%	94	34.40%	0	0.00%	148	54.20%
Missouri S&T	613	0	0.00%	0	0.00%	0	0.00%	0	0.00%
Missouri Southern State University	484	160	33.10%	9	1.90%	0	0.00%	161	33.30%
Missouri State University	1,861	0	0.00%	80	4.30%	0	0.00%	80	4.30%
Missouri Western State University	786	362	46.10%	331	42.10%	211	26.80%	488	62.10%
Northwest Missouri State University	936	150	16.00%	98	10.50%	0	0.00%	216	23.10%
Southeast Missouri State University	1,152	416	36.10%	53	4.60%	0	0.00%	437	37.90%
Truman State University	792	0	0.00%	0	0.00%	0	0.00%	0	0.00%
University of Central Missouri	1,105	88	8.00%	94	8.50%	4	0.40%	162	14.70%
University of Missouri-Columbia	2,675	0	0.00%	0	0.00%	0	0.00%	0	0.00%
University of Missouri-Kansas City	567	0	0.00%	0	0.00%	0	0.00%	0	0.00%
University of Missouri-St. Louis	333	0	0.00%	0	0.00%	0	0.00%	0	0.00%
Subtotal	11,810	1,397	11.80%	759	6.40%	297	2.50%	1,850	15.70%
Grand Total	24,354	7,326	30.10%	4,272	17.50%	2,481	10.20%	8,918	36.60%

Recommendations
Focus on essential standards—the ones most important for future success in college
Common expectation for all high school students to take a rigorous core curriculum, whether graduates are going to college or directly into the workforce
Clear performance expectations to that students, parents, and teachers receive consistent messages about “how good is good enough”
Rigorous high school courses making it more likely that incoming freshman actually are prepared for credit-bearing classes
Early monitoring and intervention with middle school students to keep them on target
Ten states are administering college and career readiness tests to all high school students as a result of statewide assessment systems (Achieve, 2009, Closing the Expectations Gap)
Seamless longitudinal system of information that can help monitor student progress every step of the way, from preschool through college
(ACT, 2008. Making the Dream A Reality: Action Steps for States to prepare all students for college and career)

Comparability of standardized measures across institutions. Advantages to this approach include: the ability to make comparisons across institutions and/or peer groups; assessment of a common set of skills (Dwyer, Miller, & Payne, 2006; A Culture of Evidence: Postsecondary Assessment and Learning Outcomes).

Learning Research

Best Practices

Mary Abraham's investigation of ACT COMPASS might go here?

Oklahoma Initiatives (OSU, 2008)
In 1994, the Oklahoma State System for Higher Education adopted several initiatives in their efforts to reduce remediation including: enhancing teacher preparation; increasing standards for college preparation; establishing better communication and feedback to Oklahoma high schools; initiating programs to enhance cooperation between state institutions; and improving Oklahoma college and university graduation rates. In 1994, the Oklahoma State Board of Regents adopted the Student Assessment Policy requiring each institution to develop and implement a comprehensive assessment program with mandatory student placement. Institutions are required to administer a standard comprehensive assessment tool, in this case, the ACT, and to use an ACT score of 19 as their "first-cut" in the areas of English, Math, Science Reasoning, and Reading. Scores below 19 require students to enroll in remedial courses or undergo secondary assessments. Although all institutions use the ACT as the first entry-level assessment, secondary evaluation testing instruments vary according to the institution. Most institutions use ASSET, AccuPlacer, COMPASS, and/or the Nelson-Denney Reading Test, and each institution is responsible for establishing their own cut-scores. These pre-collegiate level courses do not count toward degree requirements and a supplementary per credit hour fee is assessed the student for these courses. Colleges offer orientation courses, computer-assisted instruction, tutoring, and learning centers, in an effort to increase the rate at which students who take pre-college level courses succeed. Institutions are required to report to the Oklahoma State Regents the methods, instruments, and cut-scores used for entry-level course placement, as well as the student success in both remedial and college-level courses. High school students wishing to concurrently enroll in courses with established ACT cut-scores will not be allowed to enroll in those courses if they score below the minimum standard. A student who scores below the established ACT score in reading is not permitted enrollment in any other collegiate course. Secondary institutional assessments and remediation are not allowed for concurrent high school students.

Results show that since the inception of Oklahoma’s assessment policy, the percent of first-time freshmen enrolled in remedial courses has decreased in the state system. From 1996-97 to 2006-2007, the percentage of first-time freshmen enrolled in remedial courses decreased from 40.3% to 36.5%. At research institutions, the percentage dropped from 21.3% to 6.7%, and at regional institutions, the percentage dropped from 34.0% to 33.0%. At community colleges, the percentages remained the same at 49.9%.

Articulation and Transfer

This section is focused most specifically on articulation and transfer of credit for introductory courses in fields of study that are often taken to satisfy general education requirements of two- and four-year postsecondary institutions. Here we are searching for research and best practices literature that helps answer the following questions:

* What are advantages/disadvantages of statewide exam in beginning general education courses?
* What grading policies and procedures would have to be in place for grades to demonstrate achievement of exit competencies?
* How can we respect institutional autonomy while ensuring the transfer of knowledge and skills, not just the transfer of credit?
* Are there ways to "tune" learning goals or curriculum across the state so that grades might be sufficient demonstration of exit competencies mastery?

The Curriculum Alignment Initiative identified introductory courses in the following fields as those most frequently taken to satisfy general education requirements:

Arts and Humanities

* Introduction to Philosophy
* Introduction to Theatre

English and Communication

* Freshman Compositions Sequence
* Introduction to Communication

Foreign Languages

* First Semester Foreign Language

Mathematics

* College Algebra

Science

* General Geology
* Introductory Astronomy
* Introduction to Biology
* Introduction to Chemistry

Social Sciences

* American Government
* Introduction to Psychology
* U.S. History to/from 1877

Perhaps, in this section, we can take a page from research and best practices on course- and program-level assessment. These courses are in this list because they are widely used at the institutional level to satisfy general education program requirements. That is accomplished by aligning program-level learning objectives with course-level learning objectives and using course-embedded assessment of course-level objectives to satisfy program-level objectives.

Assessment Policy Research

Assessment Research

Learning Reseearch

Best Practices

College Level General Education

College level general education has two meanings. The first meaning is a general education component of curriculum completed during the first two years of college, whether completed at a two year college or a four year college. When general education is discussed in this way, it is understood to be a curriculum component distinct from an undergraduate major or field of specialization. As such it may be viewed as serving two main purposes. The first purpose is to provide breadth of study and ensure a "well-rounded" education over against the depth of study provided by the major. The second purpose is to serve as a foundation for the major. This foundation has both general cognitive skill elements and basic content knowledge elements.

The second meaning is more akin to liberal education. In this context, the term "liberal" has no political connotation, such as in liberal versus conservative political ideologies. The intended connotation is closer to liberation; a liberal education enables one to become and remain a self-regulated and lifelong learner. Rather than being regarded as being the first of two components of a college education, the phrases general education and liberal education are used in this context to mean the total, or at least the core, outcome of a four-year baccalaureate degree. In this meaning, both breadth and depth of study, serve the purposes of general education.

Assessment Policy Research

It should be noted that, relative to the first meaning, that general education courses have been distributed across the entirety of the college curriculum, perhaps an attempt to bring together the foundational and the liberal.

Assessment Policy

Keeping in mind the principles of assessment cited earlier in this document, it should be noted that assessment focuses on student outcomes and accountability related to these outcomes. "Our Students' Best Work A Framework for Accountability Worthy of Our Mission: Statement from the Board of Directors of Association of American Colleges and Universities," (2nd edition, 2008) identifies what it calls "Essential Learning Outcomes" and then makes ten recommendations for a new accountability.

Essential learning outcomes include the following:

Knowledge of Human Cultures and the Physical and Natural World through study in the sciences and mathematics, humanities, histories, languages, and the arts, focused by engagement with bigh questions, both contemporary and enduring.

Intellectual and Practical Skills, including inquiry and analysis, criticial and creative thinking, written and oral communication, quanitative literacy, information literacy, teamwork and problem solving practiced extensively, across the curriculum, in the context of progressively more challenging problems, projects, and standards for performance.

Personal and Social Responsibility, including civic knowledge and engagement—local and global, intercultural knowledge and competence, ethical reasoning and action, foundations and skills for lifelong learning anchored through active involvement with diverse communities and real-world challenges

Integrative and Applied Learning, including synthesis and advanced accomplishment across general and specialized studies demonstrated through the application of knowledge, skills, and responsibilities in new settings and complex problems

The ten recommendation for new accountability includes the following:

1. Make liberal education the new standard of excellence for all students.
2. Articulate locally owned goals for student learning outcomes.
3. Set standards in each goal area for basic, proficient, and advanced performance.
4. Develop clear and complementary responsibilities between general education and departmental and other programs for liberal education outcomes.
5. Charge departments and programs with responsibility for the level and quality of students' most advanced work.
6. Create milestone assessment across the curriculum.
7. Set clear expectations for culminating work performed at a high level of accomplishment.
8. Provide periodic external review and validation of assessment practices and standards.
9. Make assessment findings part of a campuswide commitment to inquiry and educational improvement.
10. Increase public visibility and transparency of learning goals and student achievement tests.

Jeanie Crain

Assessment Research

http://www.aacu.org/resources/generaleducation/index.cfm

Revising General Education - And Avoiding the Potholes: A Guide for Curricular Change
Toward Intentionality and Transparency: Analysis and Reflection on the Process of General
Education Reform (Peer Review, Fall 2008)
What Will I Learn in College? What You Need to Know Now to Get Ready for College Success
High Impact Educational Practices
Student Preparation, Motivation, and Achievement (Peer Review, Winter 2007)
Academic Freedom (Liberal Education, Spring 2006)
The Creativity Imperative (Peer Review, Spring 2006)
Why Do I Have to Take This Course? A Student Guide to Making Smart Educational Choices
Academic Freedom and Educational Responsibility
Advancing Liberal Education: Assessment Practices on Campus
American Pluralism and the College Curriculum
The Art and Science of Assessing General Education Outcomes: A Practical Guide
Creating Shared Responsibility for General Education and Assessment (Peer Review, Fall 2004)
Cultural Studies and General Education (Liberal Education, Summer 2004)
General Education and the Assessment Reform Agenda
General Education and Student Transfer; Fostering Intentionality and Coherence in State Systems
General Education: A Self-Study Guide for Review and Assessment
General Education: The Changing Agenda
Greater Expectations: A New Vision for Learning as a Nation Goes to College
Integrative Learning: Mapping the Terrain
Quantitative Literacy (Peer Review, Summer 2004)
The Status of General Education in the Year 2000: Summary of a National Survey
Strong Foundations: General Education Programs
Taking Responsibility for the Quality of the Baccalaureate Degree

Jeanie Crain

Learning Reseearch

Best Practices

Summary of Best Practices in General Education1

AAC&U’s statement on General Education:

A quality general education is an essential part of every undergraduate student’s experience and should reflect an institution’s core academic commitments. In the ideal, general education outcomes are achieved through a coherent sequence of dynamic learning experiences, in general courses and in students’ majors, and through curriculum-embedded assessments tied to important educational goals. In the ideal, general education is everyone’s shared concern.

Intentional (alignment among goals, outcomes, actions, results)2
Based on institution’s mission and broad goals
Outcome based (curriculum, pedagogy, assessment derived from expected outcomes)
Receives institutional support in terms of student learning, resource allocation, and faculty reward structure
Faculty generated and with faculty endorsement and commitment
Integrated into the major or student’s program of study (across the baccalaureate degree)
Focuses on development of life-long learning and developing knowledge, skills and dispositions for participation as a citizen (in a democracy and globally)
Emphasis on integration or making connections across courses/disciplines
Is viewed as one piece of a liberal education, not the totality of a liberal education
Best faculty teach in the program
Based on active learning
Assessment is integral to the program (embedded assessment (within course assessment), programmatic assessment of student learning))
Accountability for student learning

Trends in General Education reform (based on information from institutions attending AAC&U’s Institute of General Education)

In content:

Increased focus on development of skills and competencies (e.g., in writing)
New approaches for understanding culture and multiculturalism
Stress on moral and ethical questions/reasoning
Emphasis on goal of educating students for citizenship

In curriculum structure:

Developing all-campus requirements—common experiences, away from distributional curriculum
Smaller programs, fewer but more purposeful courses
Connecting student learning across courses
Increase in interdisciplinary courses (across the baccalaureate degree)
Requiring satisfactory demonstration of learning (achievement of learning outcomes)
New elements in structure, such as FYE, capstone, etc.
Raising standards, increasing requirements or criteria to meet standards
Hierarchical, four year programs

In pedagogy:

Faculty development focused on understanding how students learn
Movement from stress on content to stress on learning methods of inquiry
Emphasis on applications of what students learn
Increase in collaborative learning opportunities

In process of reform:

Attending to process of developing and implementing reforms
Faculty act as trustees of entire curriculum
Supporting faculty development of new courses, their own development of newer pedagogies
Multiple strategies for change
Integrating assessment into program proposals from beginning
Connecting general education content and structure to the guiding philosophy of the institution
Strong Foundations: Twelve Principles for Effective General Education Programs (AAC&U) Strong General Education Programs:
Explicitly Answer the Question, "What Is the Point of General Education?"
Embody Institutional Mission
Continuously Strive for Educational Coherence
Are Self-Consciously Value-based and Teach Social Responsibility
Attend Carefully to Student Experience
Are Consciously Designed So That They Will Continue To Evolve

PART II: Forming an Evolving Community Base upon a Vision of General Education: Strong General Education Programs:

Require and Foster Academic Community
Have Strong Faculty and Administrative Leadership
Cultivate Substantial and Enduring Support from Multiple Constituencies
Ensure Continuing Support for Faculty, Especially as They Engage in Dialogues Across Academic Specialties
Reach Beyond the Classroom to the Broad Range of Student Co-curricular Experiences
Assess and Monitor Progress Toward an Evolving Vision Through Ongoing Self-Reflection
1 Gleaned from several publications, but most published by AAC&U

2 According to AAC&U, intentional practice leads to (1) clear programmatic purposes for general education, (2) resonance with the institution’s distinctive mission, and (3) transparent, powerful goals and outcomes of learning. General education is more than a curriculum; “it is possibly the most important manifestation of an institution’s educational mission.” An intentional approach assures that this “important core of learning, shared by all students, no matter their area of concentration, reflects what is distinctive about the institution; its educational philosophy, culture, values, history, and student body.”

J. Crain