Why Psychometry Tests Don't Work
Those of you who live in the UK may have heard of the CEM Centre and their baseline tests, which a lot of secondary schools force their pupils to take. Those of you who go to a secondary school in the UK may have taken one - sometimes multiple times.
What the tests are actually for is a mystery to some of us, especially those taking them. The answers are variously:
1) To measure actual ability against grades
2) To predict your grades
3) To measure how smart you are
In any case, whichever answer you pick, you're not going to get very good results out of the baseline tests.
"Gasp!" you exclaim. "Why not?"
Scroll down to find out the answer if you're that desperate, I say. I'll get there in my own good time.
There are a few different reasons why psychometry tests are flawed - about two main ones, I should say, and then some more. For simplicity's sake, I'm calling what are technically known as ability and aptitude tests psychometry.
The first reason is that psychometry tests are not adapted for the job they're doing. Most ability and aptitude tests you take from year 7 up till about year 11 are mathematical, verbal, and non-verbal reasoning tests, and are suspiciously similar to the tasks done in IQ tests. (If you study psychology and you're about to tell me that I'm wrong, feel free to do so - I'm just blowing off steam here, although I do consider the tests flawed.) IQ tests originate from the Binet-Simon tests, originally developed to see if developmentally disabled children needed to be put in special ed classes, and were seized upon and revised by the U.S., at first by eugenics supporters, to credibly diagnose mental retardation. From there we pretty much jumped to using it as a general intelligence test, even though IQ scores can differ on different tests for the same individual at the same age.
The point I'm trying to make here, I guess, is that the ability and aptitude test-makers took a test which even today gives different results for the same individual at the same time and was originally developed for a fairly small field and decided it would work for everyone. This is somewhat like me testing if my toaster works by shoving bread into it and turning it on, then deciding I can test every electrical appliance by shoving bread into it and turning it on.
Secondly - and this reason follows on from the first - psychometry tests do not measure your actual intelligence. They measure your ability to do psychometry tests. There's a reason kids need coaching to pass verbal and non-verbal reasoning tests - I did, and I'm not stupid by a long way. If it truly measured aptitude instead of the ability to do certain things, it would be far more intuitive and we wouldn't need to teach people to do it. This is why you can get people - naturally smart people, mind, and in some cases you might be able to confirm that with their IQ scores - who score lower than they should on psychometry tests and yet pass their exams with flying colours, and why you can get people who do ridiculously well on psychometry tests yet are working as refuse collectors because they don't display other qualities associated with intelligence - like a good memory, an ability to grasp things quickly, or the ability to solve problems and think creatively.
So what do I get out of this? I get an inherently flawed system, used for something it shouldn't be used for, being used to measure everyone's aptitude and how well they do in relation to that. I understand that psychometrics is still a relatively young subject and that it's very difficult to measure intelligence, but if a layman can see the problems with your method, doesn't that mean you should change it?...
A last paragraph. If you study psychology/are a psychometrician/are simply very knowledgeable and informed on this subject, don't hesitate to blow my arguments to bits. I don't know very much about this subject - I know just enough to suspect something might be a bit off, and that's about it. I also have some anecdotal evidence on my side which needs to be accounted for...
What the tests are actually for is a mystery to some of us, especially those taking them. The answers are variously:
1) To measure actual ability against grades
2) To predict your grades
3) To measure how smart you are
In any case, whichever answer you pick, you're not going to get very good results out of the baseline tests.
"Gasp!" you exclaim. "Why not?"
Scroll down to find out the answer if you're that desperate, I say. I'll get there in my own good time.
There are a few different reasons why psychometry tests are flawed - about two main ones, I should say, and then some more. For simplicity's sake, I'm calling what are technically known as ability and aptitude tests psychometry.
The first reason is that psychometry tests are not adapted for the job they're doing. Most ability and aptitude tests you take from year 7 up till about year 11 are mathematical, verbal, and non-verbal reasoning tests, and are suspiciously similar to the tasks done in IQ tests. (If you study psychology and you're about to tell me that I'm wrong, feel free to do so - I'm just blowing off steam here, although I do consider the tests flawed.) IQ tests originate from the Binet-Simon tests, originally developed to see if developmentally disabled children needed to be put in special ed classes, and were seized upon and revised by the U.S., at first by eugenics supporters, to credibly diagnose mental retardation. From there we pretty much jumped to using it as a general intelligence test, even though IQ scores can differ on different tests for the same individual at the same age.
The point I'm trying to make here, I guess, is that the ability and aptitude test-makers took a test which even today gives different results for the same individual at the same time and was originally developed for a fairly small field and decided it would work for everyone. This is somewhat like me testing if my toaster works by shoving bread into it and turning it on, then deciding I can test every electrical appliance by shoving bread into it and turning it on.
Secondly - and this reason follows on from the first - psychometry tests do not measure your actual intelligence. They measure your ability to do psychometry tests. There's a reason kids need coaching to pass verbal and non-verbal reasoning tests - I did, and I'm not stupid by a long way. If it truly measured aptitude instead of the ability to do certain things, it would be far more intuitive and we wouldn't need to teach people to do it. This is why you can get people - naturally smart people, mind, and in some cases you might be able to confirm that with their IQ scores - who score lower than they should on psychometry tests and yet pass their exams with flying colours, and why you can get people who do ridiculously well on psychometry tests yet are working as refuse collectors because they don't display other qualities associated with intelligence - like a good memory, an ability to grasp things quickly, or the ability to solve problems and think creatively.
So what do I get out of this? I get an inherently flawed system, used for something it shouldn't be used for, being used to measure everyone's aptitude and how well they do in relation to that. I understand that psychometrics is still a relatively young subject and that it's very difficult to measure intelligence, but if a layman can see the problems with your method, doesn't that mean you should change it?...
A last paragraph. If you study psychology/are a psychometrician/are simply very knowledgeable and informed on this subject, don't hesitate to blow my arguments to bits. I don't know very much about this subject - I know just enough to suspect something might be a bit off, and that's about it. I also have some anecdotal evidence on my side which needs to be accounted for...
Comments
Post a Comment