Last Good Friday, while many folks here in south Louisiana were enjoying crawfish boils with their families and otherwise had a nice day off, I was nursing a nasty seasonal sinus headache and trying to make sense of John White’s latest presentation of pixie dust and spring flowers. I don’t know if it helped me feel any better but at least it gave me an excuse to stay in bed and write.
I felt like it was a crappy post because I somehow felt the need to open the hood so you could see how I approach and analyze data, but I also knew that I still couldn’t reconcile three very different narratives that come out of the data:
John White: Common Core Works! Reforms Work! Our students are all career and college bound! More students are graduating and more are going to college! Aren’t I just AMAZING?!?!?!?!?
Those Who Want To Shut Regionals and HBCUs Down: You SUCK. There’s too many of you anyway. No one wants to go to your CRAPPY SCHOOLS. That’s why enrollment is down. They want to go to ULL and LSU and LaTech but not to your sucky little school. There isn’t enough money to fund you all so screw you.
Me: The game is rigged. We tell students that they are college bound but then they can’t go. We are graduating more students and students with core and they aren’t going to our universities because they can’t get in. Regional universities are turning away a large number of students who used to be able to go AND succeed because they no longer meet our admission criteria. Some are going to community colleges instead but more are going either to privates, out of state, or worse, NO WHERE.
There are major problems with each of these narratives. Including, I just found out, mine.
Mine is still much closest to the truth, but thanks to Crazy Crawfish aka Jason France, I realized that I have to question at least one and maybe two of my assumptions…
What if we really aren’t graduating more students, and more students with core?
And what if more students aren’t really going to college after all, but we are just doing a better job of finding out where they are going if they aren’t staying in-state or going to public colleges and universities?
This is because there are some datasets that we can trust. But a big source of data that we can’t. Anything that comes out of LDOE. Which means that anything that relies either directly or indirectly upon LDOE data is suspect.
Let’s look at a semi-complete list of data sets that are available to assess whether our students are graduating from high school, are ready for college, and are actually going to college.
First, the state level data:
SIS or LDOE Student Information System. Crazy Crawfish explains this here. This is the master file of students who are in registered Louisiana schools, both public and private. However, many of the reports that are on LDOE’s website include public students only – traditional public, charter, direct-run state, and public alternative schools.
STS or Student Transcript System. This is the database of high school students who are in the state’s electronic transcript system, that we in higher education use in order to verify that students graduate with Core 4 and the minimum GPA required for college admission. I have access to student level data only for students who have applied to my institution. However, once a year Regents releases STS Core 4 graduation reports so that we can see how many students graduated overall and with core by high school. We can also see how many students are enrolled in each high school grade. I was EXPECTING that STS and other student level data at LDOE would match. BUT, according to Crazy Crawfish, THEY DON’T. This is why “my” numbers (what I get from STS/Regents) and LDOE’s numbers don’t match. SIS reports more graduates than STS.
SIS is overinflated.
Here is the difference between what was reported from STS (Regents) and SIS as submitted through National Student Clearinghouse (LDOE):
|Year||STS – Public||SIS/NSC||Inflation|
Which means I have no idea which to believe. Do I believe STS/Regents? or do I believe SIS/LDOE? I think in this case the lower number is the most reliable one. But really, I’m not sure if either are reliable. They could BOTH be inflated.
ACT Reports from LDOE – Beginning in spring 2013, all Louisiana public students were required to take the ACT in the 11th grade. In order to do this, the ACT and LDOE established a state testing date in March of each year. There are additional national testing dates in September, October, December, February, April and June. Many Louisiana students take the ACT several times in order to try to get TOPS. But the state pays for all public students to take it in March of their junior year. Some students sit for the ACT again in March of their senior year. LDOE’s ACT reports only include the March public test date score report information. They don’t include the national test date scores. John White’s contention that LDOE’s data are better than ACT’s because they use “best score” calculations is a crock because the state pays for only ONE score per student and his report does not include all the national test date scores. It is woefully incomplete data. There are at most two score reports per student… and for many students there is only one score report reflected in his report, even among students who may have taken the ACT multiple other times on national test dates. What is the best score of one?
Louisiana Board of Regents SSPS (Statewide Student Profile System) reports: These sets of reports include all public college university enrollments, both for first time freshmen and overall. Federal graduation rates are calculated just off first time, full time freshmen who graduate in either 3 years (two year schools) or 6 years (four year schools). But anyone who is interested in seeing who we actually enroll can see that some institutions are enrolling–and graduating!–lots of students who don’t count in our graduation rates, including transfer students and readmitted students.
There are other reports on the Board of Regents website, but SSPS is the master source for publicly available enrollment information for our public institutions.
National data sets:
National Student Clearinghouse – The National Student Clearinghouse is a national non-profit organization that, in exchange for campus data, provides free reporting services for partner institutions. This is a highly reliable source of information (especially compared to LDOE!) and participating is a huge win-win for higher education. When I first arrived in Louisiana, many of us in higher education were not yet in Clearinghouse but with the advent of next-gen SIS and ERP systems, plus with increased federal and state accountability and reporting requirements, Clearinghouse turned from a nice-to-have-if-we-can-just-get-our-data-together-to-participate to a must-have. Effective this year, all Louisiana publics will be in Clearinghouse (right now it is virtually all except maybe 1 or 2 smaller institutions). According to Crazy Crawfish (and this is verified on LDOE’s website), LDOE switched from verifying college enrollment with the Board of Regents – which only has enrollment information for public institutions – to verifying with National Student Clearinghouse. The first year they did so I’m fairly certain that they underreported state public enrollment as a result–because not all of us were in Clearinghouse yet, including the large institution where I used to work–but now they’re picking up virtually everyone. In addition, LDOE knows how many students are going to privates vs. publics (although they don’t report that!) two vs. four year, and in state vs. out of state (which they DO report).
ACT – The Real One in Iowa, Not the Fake One in Baton Rouge
There are two major sources of data that are available from ACT for enrollment planning and public policy research; the publicly available national and state Profile Reports, and the subscription only ACT EIS (Enrollment Information Service). Much of what I have presented here have been from one of those two datasets. I’ve avoided John White’s fake data altogether. Both these sources include ALL ACT scores for ALL students, public, private, and home school/non-registered school. Including the scores from John White’s report. The difference is that the Real ACT from Iowa provides a complete picture of what is going on ACT-wise in our state, and indeed across the nation. John White has a sliver of glass forged from the depths of LDOE’s swamp gas.
Integrated Postsecondary Education Data System or IPEDS – US Department of Education – this is the master source for postsecondary enrollments in the US because it is MANDATORY. The National Student Clearinghouse actually reports data to IPEDS on behalf of institutions participating in Clearinghouse. But IPEDS also includes the roughly 2% of institutions who still aren’t Clearinghouse members. The downside of IPEDS is that it takes 9 months from the time enrollment data are reported to when they are available to the public in preliminary reports. IPEDS is the official source for graduation rate data, which as I explained above is only calculated off of first time, full time freshmen. A hypothetical freshman from the class of 2005 who dropped out of college after Katrina because his/her campus was underwater, returned five years later, and finally graduated with a bachelor’s degree DOESN’T COUNT as a success but as a failure because said student didn’t stay enrolled in school and didn’t graduate in six years. I also don’t use IPEDS for average ACT scores because it is based off of quartiles, not actual averages. I rely on ACT for that. But I do use IPEDS to see where students are going besides public institutions.
So… with all these disparate sources of data, it is really important to know that not all of them are going to give you a complete picture of what is going on, especially if you do not have access to student level identifiers (which I don’t other than for students who came to my institution). The trick is to look at enough RELIABLE sources and check them against each other.
Which John White doesn’t do. He ONLY looks at his own data, and wants you to Believe.
We do know this. Enrollments at our public colleges and universities are not skyrocketing like John White’s graduation rates. And according to Crazy Crawfish, the increased college going rates might not be real. It’s just that now that we are going through a recognized, reliable source of enrollment information, National Student Clearinghouse, we are FINDING more students enrolled in college after high school graduation. But it doesn’t mean that more students are actually GOING.
Which means that we still have a huge population of students who are going NO WHERE after graduation. Which means we should be building up higher education, not tearing it down. All of it. From community college through flagship.
Next, I’m going to present some national consulting data which show that Bobby Jindal’s plan to make students pay for the entire cost of college is going to blow up big time. Perhaps by design. This is backed up by recent research which shows that Louisiana students are on the verge of being priced out of the market to bolt out of state. Where schools like Jackson State, Ole Miss and Southern Miss are waiting with college acceptance letters and generous TOPS-equivalent scholarships to students who our state says aren’t qualified to attend Nicholls and Southeastern and SUNO.
Let me give you a teaser… lower achieving students subsidize the education of higher achieving students.
Whoa, really? That blew my mind too.
And who are we in the process of excluding from public higher education in Louisiana?