David_Busis
Head of ProductDo you use/refer to Accuracy and Expected accuracy in Priorities by tag?
Hey all,
We want to make our analytics even better, and we need to hear from you. Do you look at the bar charts of "Accuracy" vs. "Expected accuracy" on the Priorties by tag page?
If so, how often? How important are they? What do you think they mean?
12
71 comments
Yes, I use it often to build my drills. BUT it does not include drill and section analytics so it is difficult to see improvement unless I take another PT. I would be great to see all analytics included so I can monitor my performance better. Also, some understanding of how the accuracy is calculated would be helpful.
@Bblocker09 We'll include drills and sections into priority calculations soon! We're working on it now.
Like some others have said, I would love it if I could let the algorithm know what my target score is so that my analytics would change accordingly
PLEASE INCLUDE DRILLS AND SECTIONS TO THIS NOT JUST PTs
@dritter5283@gmail.com Yes, agreed. I'd really like to have this represent a map of what's going on in my brain so I can know what I am good and bad at
@dritter5283@gmail.com Coming soon!
Another problem with my chart at least is the accuracy doesn't update when I do drills. Like if I click on "show 8 questions" there will be 13 there with a higher accuracy... that's weird - I assume these are just the numbers off the practice tests?
@JessicaFly We'll include drills and sections into priority calculations soon! We're working on it now.
I look at everything, and hope drills gets put into analytics, imo
@Naomiboyer04 We'll include drills and sections into priority calculations soon! We're working on it now.
I think it would be nice if the update can also include changes to accuracy from drills or timed sections. Then I can better determine which areas are still weak after I practice them.
@plz We'll include drills and sections into priority calculations soon! We're working on it now.
Yes, it is the main way I determine which types of questions to drill. My only question is how expected accuracy is determined (and whether it shifts over time based on drill/PT performance)
@AidanYilmaz Expected accuracy is the accuracy of people who scored similar to you.
@David_Busis Shouldn't expected accuracy be what you want to score???
I think they would be more helpful if you could set a goal accuracy. (i.e. 100%). That way your recommendations are based on an absolute and not a relative standard.
@MarcosBarrios NOW THIS...IS A GERAT IDEA. "This is my goal...how do I get there?" This would actually drive greater study efforts for people using the analytics to make informed studying decisions.
when it came to analytics from the old website, I really enjoyed the line graph we used to have. I think having analytics from your PTs and drills combined, with the option of using one pool, or the other or both to get a gauge of where you stand in that moment in time
@EdwardGz We'll include drills and sections into priority calculations soon!
I don't use it, but it would be cool if we can change expected accuracy. One thing I'd LOVE for analytics is the ability to only look at a few PTs. For example, in my first two PT (diagnostic and one after Core Curriculum), I was terrible at finding the MC so missed all of these. Now I get them right, but it still says highest priority. It would be helpful to know what is high priority as you progress, like highest priority from my last 3 PTs so I can really find my current weaknesses
@hey_maret Great ideas! We're working on this, but I don't know when we'll have something.
I find them helpful, especially in being able to see all the questions that are going into the calculus, and review whether I would still get them wrong today. But this ends up kinda having to be a manual assessment of where I'm at. I wish there was a way to reset these analytics, or have them broken into e.g. past week, past month, past year. I'd for certain use them more if there was a less tedious way of differentiating what my weaknesses were when I started studying versus what they are now. This would also be cool to show how far we've progressed, if we could see that we don't get any questions wrong in what used to be our greatest weakness.
I use them all the time!! I find it really helps target my drilling.
Can be helpful but are susceptible to overweighting. For example, I got 1 MBF wrong and no matter what it will remain as high priority forever. Also they should change programmatically as you do more drills (i believe they only take into account PTs. For example, if I am rapidly improving and taking more tests, the value of these stats goes down. Is it helpful to see the breakdown by question type? Yes. But it'd be more helpful if they were updated to account for changing skill levels/expertise.
@jimbob +1 to this. I read them and find them useful, however, I noticed some of them are either "I score 0% of expected 88%" (because I just started studying, or haven't encountered it on a PT).
I just want to know which question types I'm getting wrong and to sort by highest priority. It's SUPER USEFUL to know what to drill.
Perhaps have some toggle that would allow us to distinguish? Like a toggle "include drills"?
@jimbob @Metroidude These numbers will include data from drills soon!
@David_Busis Sweet -- also if you guys could cache selections when creating drills that'd be huge. It's a PITA to make a drill and then walk away for a few moments and it goes away. I think this could be a low lift high impact add. Also a quick bug report for you is that when you are looking at a results page (finishing a PT or section), and you click on just the flags, or if you resort the list by difficulty, etc, then the bar charts do not dynamically change, only the figures around them do
@jimbob We'll include drills and sections into priority calculations soon! We're working on it now.
Personally I don't think they do much. I am scoring close to the expected accuracy yet no where near where I want to be in terms of my goal score. I'd like to know what the expected accuracy in all the categories would add up to in a specific test.
Yes, I look at them after every practice test (once Blind Review is also finished). I find them to be important because it gives me a sense of direction regarding what I need to work on and how much I am actually improving.
I want it to include PTs, Drills, and Sections!!! PLEASE!! Maybe even toggle between the 3 individually as well??? ORRR even select specific PTs or groups of PTs or certain drills etc.
@EthanLucas It's coming soon!
Yes I use these, but I don't understand expected accuracy is calculated? is it just the Blind Review score?
@AlizaGGG Expected accuracy is the accuracy of people who scored similar to you.
Yes, I look at these charts. They are helpful. They don't look this good yet, but I use them. Create a goal-driven analytics system for this platform. This way we can say, "I want a minus 3 on RC and minus 2 on LR" then an analytics-driven study plan can be formulated to increase point uptick. Low-hanging fruit just waiting to be created for this program.
I think every analytic is helpful (even if available out of curiosity), but I don't think knowing expected accuracy is helpful given that we should strive to get 100% accuracy regardless of what is expected at our level of performance. Practice makes perfect, so just knowing my actual accuracy on each question type is helpful; expected accuracy is nice to know.
Yes, every few weeks. They are helpful in understanding how much I need to work on things. I think "expected accuracy" means what 7Sage's algorithm thinks my accuracy should be given my average score, and "accuracy" means how I actually score.
I definitely look at them.
I'm not sure what expected accuracy means. As I understand it, it is the accuracy of those that have the same average scores as me on the site.
Preferably, I'd like to be able to tweak the expected accuracy to a "goal score". For example, my practice tests hover at 150. My goal is to reach 160, so, I will enter 160 as part of the Expected Accuracy calculation. Then, I can prioritize using what has the most average questions per test along with what kind of accuracy I need in that question type to maximize how fast I improve.
@DianaSoltani Expected accuracy is the accuracy of people who scored similar to you.
@David_Busis Is there any way we can change that. Say, I want to compare to people who scored +5 points more than me - and i can track my biggest gap that way. Or, I want to compare with people who scored -5 points lower than me and see if I am lacking somewhere in relation to low scorers. I want to do analytics on people who score higher and see what problems they don't (or do) have trouble with so that I can work to being in a similar standing.
I think the 'expected accuracy' is a bit confusing. I am not sure what it's measuring. For instance, I could have a tag where I am getting an expected accuracy of 40%, but my actual accuracy is 45%. Neither look good to me, but it seems those sometimes are labeled low priority because accuracy > expected accuracy
Perhaps working as intended, but not sure what it means.
Of course, I presume that it's gauging what someone at your average LSAT PTest score should be scoring to show the areas you are lagging behind in, or do not need to study until you improve to the point you lag behind in new areas. But again, an assumption, and I think the fact it's not accessibly and readily explained right there has lead to me not using them.
I second maybe setting a goal score as someone else stated. Because it feels somewhat aimless training these scores with no visible finish line.
I also understand you cannot set an 'expected accuracy' to 100% because it would be impossible to achieve that without 1000s of questions for each type to get correct to get the numerator to match the denominator, or without the stats resetting. So I think conceptually its good, it could just allow for more control and more explanation.
@Jsonf Expected accuracy is the accuracy of people who scored similar to you.
I thought it meant like here is what im getting vs what people who do well on the lsat are getting
I mainly focus on my personal accuracy. I am not sure what expected accuracy means, but I think it refers to the % accuracy of people who are scoring similarly to me on PTs?
@sillygoose26 more or less! Expected accuracy is the accuracy of people who scored similar to you on the questions you took for a particular tag.
Yes, I look at my personal accuracy but I don't look at the expected accuracy. I have no idea where that's coming from and it is often very different from my actual accuracy (.... and, like, why would it be? If I've gotten 11% of these questions right why do you expect me to get 55% of them right? It just doesn't make any sense to me).
@SaraSmith Expected accuracy is the accuracy of people who scored similar to you on the questions you took for a particular tag.