Subscription pricing
David Busis
Head of ProductDo you use/refer to Accuracy and Expected accuracy in Priorities by tag?
Hey all,
We want to make our analytics even better, and we need to hear from you. Do you look at the bar charts of "Accuracy" vs. "Expected accuracy" on the Priorties by tag page?
If so, how often? How important are they? What do you think they mean?
12
71 comments
I thought it meant like here is what im getting vs what people who do well on the lsat are getting
why did all of my expected accuracy drop to like 20%?
I mainly focus on my personal accuracy. I am not sure what expected accuracy means, but I think it refers to the % accuracy of people who are scoring similarly to me on PTs?
Yes, I look at my personal accuracy but I don't look at the expected accuracy. I have no idea where that's coming from and it is often very different from my actual accuracy (.... and, like, why would it be? If I've gotten 11% of these questions right why do you expect me to get 55% of them right? It just doesn't make any sense to me).
Yes I do! Really helps when I want to practice random drills
Yes, but I wish it was more clear where the data was coming from. Additionally, I wish it was more dynamic comparing PT, drills, sections, etc... in a "togglable" format.
Yes, I look at them after every practice test (once Blind Review is also finished). I find them to be important because it gives me a sense of direction regarding what I need to work on and how much I am actually improving.
yes
yes
when it came to analytics from the old website, I really enjoyed the line graph we used to have. I think having analytics from your PTs and drills combined, with the option of using one pool, or the other or both to get a gauge of where you stand in that moment in time
I want it to include PTs, Drills, and Sections!!! PLEASE!! Maybe even toggle between the 3 individually as well??? ORRR even select specific PTs or groups of PTs or certain drills etc.
PLEASE INCLUDE DRILLS AND SECTIONS TO THIS NOT JUST PTs
Yes, I use it often to build my drills. BUT it does not include drill and section analytics so it is difficult to see improvement unless I take another PT. I would be great to see all analytics included so I can monitor my performance better. Also, some understanding of how the accuracy is calculated would be helpful.
Yes I use these, but I don't understand expected accuracy is calculated? is it just the Blind Review score?
I don't use it, but it would be cool if we can change expected accuracy. One thing I'd LOVE for analytics is the ability to only look at a few PTs. For example, in my first two PT (diagnostic and one after Core Curriculum), I was terrible at finding the MC so missed all of these. Now I get them right, but it still says highest priority. It would be helpful to know what is high priority as you progress, like highest priority from my last 3 PTs so I can really find my current weaknesses
I think it would be nice if the update can also include changes to accuracy from drills or timed sections. Then I can better determine which areas are still weak after I practice them.
Another problem with my chart at least is the accuracy doesn't update when I do drills. Like if I click on "show 8 questions" there will be 13 there with a higher accuracy... that's weird - I assume these are just the numbers off the practice tests?
Yes, I look at these charts. They are helpful. They don't look this good yet, but I use them. Create a goal-driven analytics system for this platform. This way we can say, "I want a minus 3 on RC and minus 2 on LR" then an analytics-driven study plan can be formulated to increase point uptick. Low-hanging fruit just waiting to be created for this program.
Yes, it is the main way I determine which types of questions to drill. My only question is how expected accuracy is determined (and whether it shifts over time based on drill/PT performance)
Also... if LR and RC tags are both based on just your PT scores, if you consistently do much better in one section, say, RC, then will the system show you as basically all good in every RC tag? I feel like that may make people miss out on easy practice to get that initial score bump.
Say you take a PT for the first time ever. You get -4 RC avg but -13 LR avg. Is it safe to assume you have less marginal growth in a section type you are already 'good' at? I feel like maybe your model is making that assumption because I have relatively few RC tags (although I know there is just less of them). Or not, no idea. But if it is!
I think every analytic is helpful (even if available out of curiosity), but I don't think knowing expected accuracy is helpful given that we should strive to get 100% accuracy regardless of what is expected at our level of performance. Practice makes perfect, so just knowing my actual accuracy on each question type is helpful; expected accuracy is nice to know.
I look at everything, and hope drills gets put into analytics, imo
Not necessarily, i use it to see my weakest points, but it gives me a warped view of what my scores could be in the future. My very first score was a 143, and the analytics said i could expect "x" amount of increase. However, I far exceeded the analytics expectations, and it doesn't now help, it just tells me I'm way more accurate in that field of subset of questions. Not super helpful, but the priority is very helpful.
I think they would be more helpful if you could set a goal accuracy. (i.e. 100%). That way your recommendations are based on an absolute and not a relative standard.
I find them helpful, especially in being able to see all the questions that are going into the calculus, and review whether I would still get them wrong today. But this ends up kinda having to be a manual assessment of where I'm at. I wish there was a way to reset these analytics, or have them broken into e.g. past week, past month, past year. I'd for certain use them more if there was a less tedious way of differentiating what my weaknesses were when I started studying versus what they are now. This would also be cool to show how far we've progressed, if we could see that we don't get any questions wrong in what used to be our greatest weakness.