Counting caps and gowns
Those who expect graduation rate data to provide insights on quality had better think again

Accountability seems to be the current prescription for much that ails education. A central element of the No Child Left Behind legislation that reauthorized the Elementary and Secondary Education Act, accountability bromides are now being freely dispensed in the context of reauthorizing the Higher Education Act.

We hear of “creative” proposals, such as linking funding to institutional performance as measured by graduation rates. This is touted as a means of both achieving educational reform and forcing cost efficiencies on an industry seemingly in need of restraint.

Yet the American Federation of Teachers believes this thinking misses the mark. Measuring college graduation rates would not have a positive effect on higher education at all. This is not a question of opposing accountability.

“Some of the ‘solutions’ to persistence ‘problems’ being talked about these days misapprehend the nature of the issue, miss its real causes, and therefore advocate policies that would hit the wrong targets and do more harm than good,” says William Scheuerman, AFT vice president and chairman of the Higher Education Program and Policy Council.

The AFT believes that institutions of higher education, particularly public institutions, must be accountable for providing students with a quality education and for properly managing public funds. In fact, this kind of accountability has been happening. Traditionally, it has rested in three places: (1) with the states (2) with the nation’s unique system of private accreditation and (3) with the tremendous diversity and competition built into the system.

Attending higher education is completely optional and costs money. College students are adults who select the education they want with their feet and their pocketbooks. They decide what quality means; they decide whether an institution is meeting their needs. No fair reading of the facts can conclude that higher education is unaccountable.

At the same time, we recognize that degree completion is not as high as it could be, and that there is a significant gap in persistence between affluent students and low-income and minority students. AFT believes that there is much the federal government could do—in partnership with the states and with the colleges—to make things better. However, we also believe that taking a snapshot of an institution’s graduation rate tells us next to nothing about either institutional success or student success.

Because of the direction reauthorization discussions are taking, the AFT asked John B. Lee and Lawrence E. Gladieux, independent researchers with long experience in access and persistence, to summarize what the data have to tell us about persistence and point us toward solutions that make sense. The AFT report arising out of this research is being released to the public and shared with Congress this month. What follows is a summary of its main points and AFT’s recommendations.

WHAT DO GRADUATION RATES TELL US?

Unfortunately, current sources of information on graduation rates provide a sketchy picture of college persistence. No definitive, system-wide data are produced on a regular basis that tell us how the nation is doing on degree completion—in the aggregate, by region, by institution, by institutional type or by any student subgroup.

For policy makers and anyone else who wants to help boost student persistence and completion rates, it is important to understand the limitations of existing data and analysis.

The federal government now has two sources of data on student persistence:

Student Right to Know (SRK) compliance: In 1998, Congress passed legislation requiring institutions participating in federal student aid programs to disseminate information to prospective and enrolled students about the costs of attendance and aid available from the institution, degree offerings, and a variety of other information—including completion rates of certificate- or degree-seeking full-time, undergraduate students entering such institutions.

The Student Right to Know Act also requires institutions to submit a once-a-year snapshot containing all this information to the National Center for Education Statistics (NCES) at the U.S. Department of Education. The original legislation did not envision the publication of graduation rates in student manuals as a means of holding institutions accountable for their performance.

However, more recently and as Congress prepares to reauthorize the Higher Education Act in 2004, proposals have been floated to reward or punish institutions based on their estimated SRK institutional graduation rates.

Longitudinal survey research: A much better source of information about student persistence can be found in longitudinal surveys conducted by NCES. These surveys trace students through the postsecondary system—and provide a variety of useful information on persistence (defined in the surveys as earning a degree or staying enrolled). Carefully conducted longitudinal research can help in understanding factors that contribute to persistence and what policies would make the most difference in improving rates of student success in postsecondary education.

WHAT GRADUATION RATE DATA DON’T TELL US

The SRK standard measures the percentage of a matriculating cohort of students that originally enrolled as full-time students and that graduated within 150 percent of the expected time. Many students get missed.

The institutional graduation rate doesn’t include all students. The SRK measure includes a series of arbitrary rules as to which students are and are not included in the calculation. The resulting data do not represent all students or every outcome at the college.

First, the measurement does not take into account those who begin their education part-time, which is a substantial population at schools serving non-traditional students. Part-time students take longer to complete degrees, and the percentage of undergraduates who attend college part-time has risen steadily in the 1980s and ’90s. More than 42 percent of college students attend part-time.

The SRK measure also fails to take into account the fact that students increasingly tend not to stay in the same place, doing the same thing, throughout their education. For example:

• If a student transfers from one four-year college to another prior to earning a degree, the student is recorded as a dropout at the first college, even if he or she receives a degree at the second college.

• One-third of the students who matriculate at a community college attend another community college before they finish. These graduates are never counted as a success. They do not count as a success at the community college from which they graduate because they did not begin as full-time freshmen there. Even worse, they actually count as a dropout at the initial community college because they did not graduate from that school.

• Sixteen percent of community college students start their education with no specific graduation goals, yet they are included in the cohort.

• Students can be excluded from the cohort if they are deceased, disabled, entered the armed forces, pursued foreign aid service such as the Peace Corps or undertook a religious mission before graduation. However, institutions report that many, if not most, of the students who leave for these reasons never communicate this to the college. If they did not, they are counted as dropouts.

Community colleges face the biggest problem calculating graduation rates because they have so many missions. They provide terminal vocational degrees, academic transfer degrees, and also offer many students the opportunity to take a number of classes to gain a specific skill. This leads to some unwarranted results.

The graduation rate data do not include all institutions. As of 2001, fewer than half of postsecondary institutions had reported useable graduation rate data after many years of preparation. On average, as of August 2003, 77 percent of public institutions had reported 2001 graduation rates to NCES. However, the percentage of private institutions reporting was 43.4 percent and for-profits reported at a rate of 32.4 percent. The fact that so many institutions do not or are not able to provide graduation rate data suggests that any attempt to use them in an official function is premature.

The graduation rate fails to distinguish between extended time-to-degree and dropping out. Still another shortcoming of using snapshot institutional data is that it serves to obscure two separate policy issues—extended time to degree and dropping out. Students who are still enrolled after 150 percent of expected graduation time represent a growing trend in higher education.

On average, bachelor degree recipients in 1999-2000 who had not stopped out of college for six months or more took about 55 months to graduate. Attending multiple institutions increased the time to complete a B.A. For example, those who attended one institution averaged 51 months to complete a bachelor’s degree, compared with 59 months for those who attended two institutions and 67 months for those who attended three or more institutions.

Students who started at community colleges took about a year and a half longer to complete a bachelor’s degree than students who began at public four-year institutions (71 versus 55 months), and almost two years longer than those who began at private colleges and universities (50 months).

The type of institution from which graduates received a degree was also related to time to degree: graduates of public institutions averaged about six months longer to complete a degree than graduates of private not-for-profit institutions (57 vs. 51 months).

THE REAL ROADBLOCKS TO PERSISTENCE AND GRADUATION

Because a student’s reasons for attending college may be so broad or so particular, it doesn’t make sense to use a graduation rate or some other institutional measure as a gauge of student success.

Students don’t always follow a straight line from enrollment to graduation. Some take a critical concentration of courses that prepare them for work, especially in high-demand fields like culinary arts and technology. Other students have personal and family obligations that draw them away from the classroom. (See sidebar, “Why students drop out,” below.)

In fact, student characteristics are much more important in predicting college graduation than any characteristics of the college program itself. The “classic” college students who take rigorous high school courses and have high admission test scores are most likely to graduate. They enroll full-time in four-year colleges right after high school.

Colleges and universities that enroll this type of high-achieving student will have higher graduation rates than those whose mission is to provide access to a broader range of students. Not surprisingly, you can predict a college’s graduation rate by knowing how selective are its admissions standards.

So, attaching great value to graduation rates will reduce a college’s willingness to enroll students at high risk of dropping out. And measuring schools against a single outcome will standardize their operations as they find similar methods to improve their rating.

HOW TO IMPROVE STUDENT SUCCESS

The AFT recognizes that student persistence is a complicated issue, but it remains important that every student who wants to complete academic study and is willing to work hard gets a fair chance to do so. Instead of imposing a punitive accountability measure, AFT recommends a new line of cooperation among colleges and universities, the federal government and states.

1. Address students’ financial barriers. A fundamental reason students drop out of college is lack of money. The AFT recommends full funding of the Pell Grant program and a doubling of the maximum Pell Grant.

2. Ensure a supportive academic environment. The AFT recommends increased support for the TRIO programs, which help students prepare for college, and other programs designed to improve achievement.

3. Ease the transfer process. The federal government should help and encourage states to create more uniform and effective transfer arrangements between two-year and four-year institutions.

4. Lay the groundwork in high school. The AFT recommends that Congress develop a new program to support high school/college collaboration in high school curriculum development.

5. Study the root causes of persistence problems. The AFT calls on Congress to fund student-centered research into the causes of persistence problems.

Perhaps the most important conclusion of this study is that student characteristics define the chances of succeeding in college much more than do college policies. Setting a simple outcome measure that doesn’t give ample weight to this reality and that overlooks the complex nature and widely varying purposes of postsecondary education will only make things worse. That is not to deny a federal role in helping students succeed, but to call for an accurate portrayal of what really helps and hinders students so that the government’s efforts—and taxpayers’ dollars—are not wasted.