Tracking state accountability measures
States move to assess college performance

How well are universities performing in Texas? The governor wants to know, and has ordered an accountability system to find out.

After months of coordination with legislators and educators, including University of North Texas system chancellor Lee Jackson, the Texas Higher Education Coordinating Board delivered a system of accountability in late October. It will be considered further, for detail and clarification, by the state Legislature during its early 2005 session.

Among the determining factors for success, as outlined in Texas, are enrollment numbers, broken down by gender, ethnicity and first-generation students; graduation rates, including those among African-American and Hispanic students and among subject areas (computer science, engineering, math, physical science, nursing, health and teaching); student-faculty ratio; percentage of basic courses taught by tenure-level faculty; employment and professional licensure after graduation; and research funding procured.

Will these factors, measured and accounted for, lead to greater student success in Texas?  That remains to be seen. Discussions of systems of accountability are a slippery matter, as many states and the federal government are coming to realize. Comparing institutions that vary widely can be difficult—community colleges and research institutions, for example, have vastly different purposes. Graduation data often fail to adjust for transfer and part-time students. Initiatives to increase minority enrollment and first-generation college students may result in longer time to graduation, while a highly selective admissions process could skew graduation rates higher. The question of how each of these areas will be judged—and by whom—becomes especially important when funding is tied to success, although in Texas that element has yet to be determined. “Whether or not there will be sanctions [for falling short of requirements] remains to be seen,” says Ray Grasshoff, public information officer at the coordinating board.

Concern about how accountability measures are determined and applied were at the heart of a report the American Federation of Teachers issued last year. Called “Student Persistence: More Than Counting Caps and Gowns,” the report documents how student performance is based on a host of factors, some of which are beyond institutional control. For example, research shows that students in difficult financial circumstances take longer to complete college and are more likely to drop out, as are first-generation college students, students who did not take a rigorous high school curriculum and students with families to support. (The report can be found at www.aft.org/pubs-reports/higher_ed/
student_persistence.pdf)

The recently formed National Commission on Accountability in Higher Education, launched by the State Higher Education Executive Officers (SHEEO), is publishing a comprehensive report on assessment this month (go to www.sheeo.org/account/acct-home.htm). The commission, which includes former governors, legislative leaders, state higher ed executives, institution leaders and business representatives, considered input from an array of consultants, including a dozen state accountability agencies and nine national associations.

One group that is not well-represented on the commission is faculty, who also tend to be left out of state accountability initiatives as well. AFT leaders, who have endorsed the union’s commitment to promote opportunity, quality and accountability on our campuses as a first principle of the profession, find it alarming that states could impose accountability requirements without an adequate understanding of the academic procedures in place to ensure quality.

For example, Florida has been pursuing accountability for more than a decade. Since a 1991 statute established evaluation of state universities and community colleges, the state has experimented with incentive funding and measured such characteristics as earnings after graduation, alumni satisfaction, licensure exam scores, attendance at graduate school and, in research institutions, the number of patents, copyrights and articles. In 2001, the state senate passed a bill requiring that “at least 10 percent of the state funds appropriated for the K-20 education system are conditional upon meeting or exceeding established performance standards.” The 10 percent in question totals about $2 billion and excludes such programs as health insurance, major gifts, financial aid and some specific funds. Now the department of education is collecting data to formulize performance-based funding.

The seven accountability measures approved by the board of governors of the state university system are:

• increased access to and production of degrees;
  
  
• rate of passing licensure and registration exams;
  
  
• academic learning compacts designed to measure comprehension in specific subjects through exams, portfolios, oral defenses and similar tools;
  
  
• percentage of underserved populations that enroll and complete programs;
  
  
• graduation rates;
  
  
• meeting statewide needs for teachers, health professionals, economic development, emerging technologies and high wage/high-demand jobs; and
  
  
• building academic research capacity.

Each measure is prioritized, and tied to a dollar figure, according to an institution’s budget. If goals go unmet for two years in a row, the board of governors may reallocate money within the state university system. If goals are exceeded, universities may request new dollars.

In Illinois, people like Leo Welch, director of legislation and governing board affairs for the Illinois Federation of Teachers Community College Council, are most concerned with the implications of assessment tied to funding. Faculty groups there have complained about the flawed design of a two-year-old pilot assessment project sponsored by the Pew Charitable Trusts. The project was supposed to generate results that could be included in Measuring Up 2004: The National Report Card on Higher Education, the biennial review put out by the National Center for Public Policy and Higher Education. Not only did it miss that benchmark, but at a Dec. 7, 2004, meeting of the Illinois Board of Higher Education, a program director acknowledged that “many logistical and technical challenges” had led to inconclusive findings. Unless faculty experts participate in the assessment, says Welch, “our primary concern is of an ineptly designed instrument and analysis and the political/financial consequences that could be attached to it.”

Other states with various accountability systems include California, with the California Postsecondary Education Accountability Act of 2004, currently under consideration; Kentucky, with 19 key indicators of progress requiring reports on specific objectives; North Carolina, where the university system claims that improvements have resulted from accountability; North Dakota, where educators are focused on using education to improve the state’s economy and quality of life; Oklahoma, where the state regents’ Higher Education Report Card measures progress against established goals; South Carolina, where a system of funding tied to performance is under reconsideration; South Dakota, which uses a proficiency exam for college juniors; Tennessee and its long-established funding-for-performance system; and Virginia, with its reports of institutional effectiveness, a statewide assessment.

In addition to these many examples at the state level, there are numerous national assessment studies available now, including the annual Measuring Up: The National Report Card on Higher Education from the National Center for Public Policy and Higher Education; multiple national accreditation agencies; and the newer Public Accountability for Student Learning in Higher Education from the Business-Higher Education Forum. There’s also the National Survey of Student Engagement, which, for its fifth year, is providing an alternative to the popular U.S. News and World Report survey of colleges and universities. NSSE addresses such issues as frequency of contact with faculty, number of papers written and helpfulness of administrators. This year, the Value Added Assessment Initiative is testing another assessment possibility with its three-hour Collegiate Learning Assessment, designed to test the growth of critical thinking, analytical reasoning and written communication.

On behalf of faculty unions, the AFT is keeping a close eye on the imposition of outside accountability measures that require institutions to provide explicit quantitative data, including standardized assessments, to “prove” that education is successful.  These approaches can be worrisome, says William Scheuerman, AFT vice president and president of the United University Professions/AFT at the State University of New York. “They have the effect of reducing faculty and staff autonomy as well as the power of institutions, such as higher education unions, that defend that autonomy and keep it in place.”