We updated the way we calculated difficulties. Nothing got objectively easier or harder, but you might notice that some questions shifted a star or two in difficulty from where they used to be.
***
We changed the formula underlying our difficulty calculation. We used to rate a question by how strong a student has to be to have at least a 50% chance of getting it right. Now we rate it by how strong you have to be to have at least a 75% chance of getting it right. The main effect is that questions where a high scorer isn't much likelier than a low scorer to get it right -- questions that are a bit of a coin flip for everyone -- now get rated as hard. Under the old system some of them could have been rated as easy.
We think the updated calculation results in better difficulty ratings.
14 comments
Im new to 7sage. Is starts the same as lawhub levels 1-4? How come the questions are not the same as lawhub advtange's difficulty?
@Amanda.K It's not clear how LawHub rates the difficulty of questions. Ours is based on data from hundreds of thousands of students, so I think ours is probably more reliable. They have 4 levels because they don't discriminate as much on the top end -- we have more stars to capture more distinctions in difficulty. (Some users even want more than 5 stars!)
Now it makes me curious... which questions have secretely been "evil 5 star questions" 👀
Awesome! 🔥 PT138.S1.P3.Q17 went from a one-star to a five-star (screenshot taken last week when I was showing it to my friend):
@PowerUp_Tutoring Omigod I got that one wrong initially. Makes so much more sense now.
Kevin. It's been day 9 without my Sage AI. I miss my parasocial relationship.
@antitrust_fan glad to know I'm not the only one
@antitrust_fan same. I depend on that little guy. He’s a necessary condition to me studying. 🫠
@antitrust_fan Ran out of usage? It resets August 1!
@Kevin_Lin Kevin there's so little for $70/month. Paying for two AIs seems a little absurd so it's hard to justify the extra cost for Live, especially when I don't really want anything in Live.
I've been personally victimized by 'easy' questions, and I feel seen.
@afk big facts 😭
@afk when i get one wrong and it has a 138 next to it i lose my mind
@pennstater44 Understatement, for sure.
I set up an LR drill that uses incorrect questions as a type of "vindication drill" that's been unexpectedly helpful. An added bonus is how I get to re-read the notes I had written when I initially answered the question wrong. Saying nicer things to my future self wouldn't be a bad thing LOL