Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Categories
- 33.4K All Categories
- 28.1K LSAT
- 17.1K General
- 5.2K Logical Reasoning
- 1.4K Reading Comprehension
- 1.7K Logic Games
- 70 Podcasts
- 193 Webinars
- 12 Scholarships
- 194 Test Center Reviews
- 2.2K Study Groups
- 111 Study Guides/Cheat Sheets
- 2.5K Specific LSAT Dates
- 31 November 2024 LSAT
- 18 October 2024 LSAT
- 9 September 2024 LSAT
- 38 August 2024 LSAT
- 28 June 2024 LSAT
- 4 April 2024 LSAT
- 11 February 2024 LSAT
- 22 January 2024 LSAT
- 38 November 2023 LSAT
- 43 October 2023 LSAT
- 14 September 2023 LSAT
- 38 August 2023 LSAT
- 27 June 2023 LSAT
- 30 Sage Advice
- 5K Not LSAT
- 4K Law School Admissions
- 13 Law School Explained
- 10 Forum Rules
- 636 Technical Problems
- 286 Off-topic
LSAC Reports on Accommodated Test-taking - Has 7sage discussed yet?
LSAC released a report on accommodated test-taking trends:
https://www.lsac.org/docs/default-source/research-(lsac-resources)/tr-17-03.pdf
"This report examined trends and performance of accommodated test takers for the June 2012 through February 2017 LSAT administrations. Trends with regard to the request for and approval of testing accommodations, types of accommodations approved, and the demographic makeup of the accommodated test takers, were examined. The overall performance of the accommodated test takers was examined and compared to the performance of the Nonaccommodated group, and the performance of Accommodated/Extra Time test takers who repeated the test a second time was also studied. In general, it was observed that the number of accommodation requests submitted by test takers with documented disabilities and both the number and percentage of accommodations approved greatly increased over the current report years. These increases reflect policy changes dictated by the Consent Decree beginning with the June 2014 LSAT administration. However, the proportion of those who received approval for an accommodation and who then went on to take an accommodated LSAT remained fairly steady at 65–77%.
With regard to the distribution of accommodated test takers across various demographic subgroups, these subgroups were in some ways similar to, and in some ways different from, those in the Nonaccommodated subgroup. Male accommodated test takers were more prevalent among the Accommodated group compared to the Nonaccommodated group. While the representations of Native American and Hispanic/Latino test takers in the Accommodated group were similar to those found in the Nonaccommodated group, the African American and Asian test-taking subgroups were underrepresented and the Caucasian/White test-taking subgroup was overrepresented in the Accommodated group compared to the Nonaccommodated group. Trends with regard to LSAT performance for accommodated test takers have changed during the current report years, with those in the Accommodated/Extra Time subgroup scoring higher than those in the Nonaccommodated group in 18 of the 20 administrations and those testing in the Accommodated/Standard Time subgroup scoring higher than those in the Nonaccommodated group in 12 of the 20 LSAT administrations. Score gains for Accommodated/Extra Time repeat test takers were almost the same as those observed for the Nonaccommodated group but higher for those who first tested under nonaccommodated conditions and then switched to accommodated/extra-time testing conditions. The trends presented in this report are purely descriptive in nature. While trends with regard to the Accommodated group have been described and compared to trends in the Nonaccommodated group, explanation of the underlying causes of any differences observed is beyond the scope of this report. More specifically, those included in the sample of accommodated test takers being analyzed are, in several respects, selfselected. These test takers chose to take the LSAT and to request accommodated testing conditions, and then self-reported their subgroup membership with regard to such factors as gender, race/ethnicity, and age"
It also released a report on the predictive validity of accommodated test-taking with respect to first year law school grades:
https://www.lsac.org/docs/default-source/research-(lsac-resources)/tr-17-04.pdf
"Predictive validity of LSAT score, undergraduate grade point average (UGPA), and Index score (which includes both LSAT score and UGPA combined) was assessed using first-year average (FYA) as the criterion. Results from this study suggest that LSAT scores, UGPAs, and Index scores for Accommodated/Extra Time test takers tend to overpredict FYAs. Additionally, results indicate that LSAT scores, UGPAs, and Index scores predict FYAs relatively well when accommodations unrelated to timing were given."
Conclusion:
"Results from this study suggest that LSAT scores obtained under accommodated/extra-time testing conditions are not comparable to LSAT scores obtained under nonaccommodated testing conditions. In particular, LSAT scores among test takers in the Accommodated/Extra Time subgroup tend to overpredict law school performance as measured by FYAs. This finding of overprediction is consistent with prior findings for LSAT scores and scores on other large-scale standardized tests (e.g., Braun et al., 1986a, 1986b). In addition, relative to others in their entering class, the ranked standing of these test takers with regard to their first-year performance tended to be substantially lower than their ranked standing with regard to their LSAT score. A similar result was found for their Index score, but this phenomenon was not as evident for their entering-class UGPA ranked standing. In contrast, no substantial evidence was found to suggest that LSAT scores obtained by test takers in the Accommodated/Standard Time subgroup are not comparable to those for the Nonaccommodated group. "
Comments
Just a reminder to everyone. It is okay to have an opinion on this, but we need to remain civil about our discussions. This topic can generate heated debate and we need to respect each other. Please refrain from personal attacks and keep it civil. A similar discussion was shut down earlier because it was getting out of hand!
Be kind to each other, even if you disagree!!
Whether you're for or against accommodations, I think you can agree the current system needs major improvement.
The purpose of accommodations is to level the playing field. It isn't a mere coincidence that accommodated test takers are scoring much higher than non-accommodated test takers.
As @LSATcantwin says, yes, there has been a debate about this....
I didn't think it included discussion about the data though - very interesting and thought-provoking results.
And before we get too carried away look at how small the "repeat" group is that had the most significant gains less than 600 people from 2012-2017.
And overall in the 5 year period less than 6000 people took the LSAT with some form of accommodations.
2016/2017 2318 people took accommodated testing and less than 88% of those had extra time compared to the over 109,000 that took the test last year. So less than 1.8% of test takers had extra time.
And the interesting this is the approval process has yeilded about the same percentage of approvals. So the "it so much easier to get accommodations" argument doesn't really fly. More people applied so a greater number approved, but it's proportional to "pre settlement " but a lot of the barriers that prevented people seeking accommodations were removed. So an increase should have been expected but the standards remained high.
Actually, the proportion getting approval increased dramatically. "The percentage of testing accommodations approved per year also increased across the 5 testing years from 46% in 2012–2013 to 79% in 2016–2017, averaging 1,542 per year. "
For anyone who is interested in this issue (and about myriad other problems with the LSAT), please read this excellent article. https://lawreview.law.pitt.edu/ojs/index.php/lawreview/article/view/102/102
Excuse my ignorance on the subject, I'm not that well versed in the minutiae but if accommodations on test taking are not that strong of a predictor of FYA, would this put law schools in the position of choosing a non-accommodated test taker over an accommodated test taker with an equal score? So if a competitive law school has two applicants both with a 168 LSAT score, would the non-accommodated one receive preference given this data? Do law schools know who has gotten accommodations?
@thrillhouse @uhinberg I'm asking from a genuine place of curiosity. I've been so focused on the exam itself, that I rarely venture into these things.
David
No they don't know who was given accommodation. LSAC use to flag accommodated test takers, but had to stop that policy after a lawsuit.
I wonder what percentage of the people testing under extended time are also taking advantage of (in the most neutral sense of the expression) similar educational accommodations in their first year of law school as compared to in undergrad.
Also, I feel like there should now be waivers offered for 504 plan/IEP/Educational Testing introduced somehow now. Otherwise, this ends up strongly favoring well-resourced testers.
LSAC holds very strongly that accommodated test takers should be flagged, but they were sued and lost in court, so there is no flag now. I do believe, however, that LSAC does not use the results of accommodated test takers for equating purposes.
"those in the Accommodated/Extra Time subgroup scoring higher than those in the Nonaccommodated group in 18 of the 20 administrations."
Feels like a lot of people are using the accommodation leniency to skirt the system
If the policy was really just to make things fairer for those with accommodations and legitimate problems, we should be seeing equal performance...not this incredibly skewed, better performance among those with accommodations.
Some people would say that people who get accommodations may be more into studying than the average non-accommodated test-taker. This was a point made in the other thread. But obviously this is not a proven point or even necessarily likely to be true - it's just another potential factor to consider. In any case, there are probably a mix of factors at issue that account for why accommodated average is higher than non-accommodated average, possibly including your point in this post.
"However, the proportion of those who received approval for an accommodation and who then went on to take an accommodated LSAT remained fairly steady at 65–77%."
ALSO... higher according to an older report means a slightly better 2 point difference the average is a 151, so this implies accommodated test takers are 153 scorers, on average.
Ok but I'm out on this convo, because I've been in HR too long to not support and understand that being accommodated allows people to meet their potential. If they are scoring better maybe it just means they were going to score better anyway, just like some people are 171 test takers and others are 167s.
That's not referring to the proportion approved. That's referring to the proportion who went on to take the exam out of those who were approved. For example, consider this situation: 100 people applied, 50 people were approved. Of those 50 people, 25 went on to take the exam. In that situation 50% were approved and 50% who received approval went on to take the exam. The next year, 100 people applied, 90 were approved. Of those 90 people, 45 took the exam. This means there was a 90% approval rate, but 50% who received approval went on to take the exam. The stat you are referring to above is about the % who received approval that went on to take the exam. But the approval rate went up to about 80% when it used to be about 45%.
What do you mean by potential? Do you mean the highest score possible?
I believe she means it in the sense of, allowing people to score how they would otherwise without their documented disability.
Also someone mentioned the premise from the other thread that accommodated testers were "more into studying". That's not exactly what the argument was. It's more along the lines of, it is likely that if someone takes all the steps to receive accommodation, which is a time-consuming process, they are likely also studying for the test. And as we all know, in every LSAT administration there are a significant number of testers who did not study at all and are taking it cold, and many who don't study nearly as much as they should. It's likely that accommodated testers have much fewer people in their ranks who either haven't studied at all or didn't study much. If you removed all the folks who don't study from the non-accommodated testers, I'd bet that the average score would rise significantly as well. It's just a theory of course, and not provable. But it would make some sense.
It might be provable. We would just have to control for time spent studying. That could either be done through an actual controlled experiment or via simply asking test takers how much time they spent studying and using that to construct a statistical model. Once the time spent studying was controlled for we would expect the accomodated testers to score much closer to the same as non-accomodated testers if studying was the main source of the difference. If the accomodation provided an unfair advantage that explains the scoring gap, then we wouldn't expect the gap to shrink when studying was controlled for.
I guess to clarify, I mean provable with available data. We don't currently have any information about who studies for how long.
I don't really have too much of an opinion on the accommodations themselves, it certainly seems like a good thing to help everyone achieve their potential and I'm always all for that. However, after reading this report, it seems pretty clear the system as it is now isn't working the way it should be. I think it made more sense to have your score flagged like they used to before the LSAC got sued. This way people got their accommodations but there was some sort of "check" against getting it. You had to really weight the pros and cons of applying for and receiving accommodations. I don't think the old way was ideal, but it seems to make more sense than the current system.
When you can basically get extra time without much of any draw back, it's inherently going to lead to increases in applicants and some less-than-honest people applying for extra time. It seems like in an attempt to make it more fair for some applicants they've made it less fair for others...
7Sage has done such an incredible job building such an accessible and affordable way to study for the LSAT. The LSAC just needs to figure out a way going forward to ensure everyone has equality of opportunity to do well on this test, because I truly do believe everyone should have the same chance to shine on test day
On that note, I think LSAC is working with Khan Academy to offer free LSAT prep material.
I saw that! I wonder how it will turn out? No one will ever top JY's free youtube videos
Agreed. It went from 1% to 3 % of all test takers getting accommodations. Yesterday's "Thinking lsat" podcast has a very interesting take on this. Very fact not feeling based.
I literally just got done listening on my commute home from work. It's certainly an interesting subject.
@"Alex Divine", I'm about halfway through that episode. I'm really glad they took the time to go through the report. Does anyone know how often they publish this report?
There are two additional things we should bear in mind here:
(1) The "extra-study" hypothesis about accommodated test takers has to be a bit sophisticated given that (a) there is a difference between retake scores between the non-accommodated population and the accommodated population (and presumably, people prepare more/better for retakes) and (b) there is a difference in the accommodated population--between those receiving extra time accommodations and those receiving other accommodations.
(2) Distributed averages are misleading. Just as a 2 degree rise in average temp can have catastrophic climate effects on certain regions--in that certain regions can have 20 degree shifts in temp while others remain relatively stable-- an average of a few points difference can mean that a very small percentage of accommodated persons received a massive boost (massive = >10 points) in score relative to the non-accommodated population. This would presumably mean that for those that didn't fall in that small percentage, the provision of accommodations is working as it was intended.
In actuality, say 3000 ppl get accommodations in a given cycle. Say that massive boosts only go to 10% of that population (where massive again is relative to the average improvement of non-accommodated testers). That leaves 300 ppl.--You'll think that this is a significant population if you're looking at things from the perspective of T-14 seats and the weight that is placed on high LSAT scores at those schools.
Still, perhaps score boosts are more evenly distributed throughout the accommodated extra time population. And, more importantly, there is no proof that if one were to receive a massive score boost within this population that it would mean that the accommodation was somehow illegitimate.
My take is that, yes, there are obviously some free-riders here that game what is a system with a just cause and that's shitty, and those people are shitty. This is like anything else (e.g., welfare benefits, affirmative action). Still, there is no reason to throw the baby out with the bath water. Perhaps, LSAC will need to change its policy in the future if trends continue. Perhaps. But I don't think those of us who are non-accommodated should be worrying about it (understandable is that might be). And that's even if you're gunning for a "T-14", facing stiff competition. Far more helpful to spend that time worrying about your PS.--A.c.S