The blog that's gotten one comment ever! Thanks mom!
Monday, March 23, 2009
Sometimes I wonder...
...if I'm really bad at my job. I try, I really do. But the failures are always there, and the successes rare and unheralded. Success at my job is the absence of failure. My job is to find failure. It wears on the soul.
It reminds me of the paradoxical position I found myself in running the placement test at my last school. A big part of my job was doing constant validation studies on the test. But when you do that, you always expose the flaws. You need to do that to fix them, but fixing them can easily take an entire test cycle (i.e. a year), and you won't actually know if you did fix them until the next test cycle finishes and you run validation studies again. So, despite the fact that the reading and listening sections were well-oiled, student-placing monster machines, problems in the trickier parts of the test (double-rated live group speaking task, double-rated timed essays) were much harder to pin down and fix.
So, because you're doing such a good job of finding the problems, and the fixes are slow, people start to think the test is bad, when actually it's good and your team is good because they actually know how to check and the test is basically open-source, unlike the proprietary designs by people like ETS, where everything is closed and you just have to take their word that they work (word over beers with people who work on them is always quite different--tests always have problems).
So ultimately, me doing my job well ends up making me look like I'm doing my job poorly, and gets my test killed. Luckily, it gets killed (after an almost-20-year run!) as I'm leaving and I don't have to try to stitch a project together from the ashes, or find a new research job.
What am I saying?
I hear you.
It sucks when your job is to be the messenger for bad news, especially when you're the only face people associate with it.
1 comment:
I've worried about that job of yours.
It reminds me of the paradoxical position I found myself in running the placement test at my last school. A big part of my job was doing constant validation studies on the test. But when you do that, you always expose the flaws. You need to do that to fix them, but fixing them can easily take an entire test cycle (i.e. a year), and you won't actually know if you did fix them until the next test cycle finishes and you run validation studies again. So, despite the fact that the reading and listening sections were well-oiled, student-placing monster machines, problems in the trickier parts of the test (double-rated live group speaking task, double-rated timed essays) were much harder to pin down and fix.
So, because you're doing such a good job of finding the problems, and the fixes are slow, people start to think the test is bad, when actually it's good and your team is good because they actually know how to check and the test is basically open-source, unlike the proprietary designs by people like ETS, where everything is closed and you just have to take their word that they work (word over beers with people who work on them is always quite different--tests always have problems).
So ultimately, me doing my job well ends up making me look like I'm doing my job poorly, and gets my test killed. Luckily, it gets killed (after an almost-20-year run!) as I'm leaving and I don't have to try to stitch a project together from the ashes, or find a new research job.
What am I saying?
I hear you.
It sucks when your job is to be the messenger for bad news, especially when you're the only face people associate with it.
I don't really have any advice.
Post a Comment