Matura exam and subject for bias

In Poland every year in May 350.000 students take Matura exam (High School Diploma examinations). Below is a chart with points distribution from Matura exam in Polish on foundation level in 2010


Do you see anything interesting in the picture above?

Almost all values fit into normal distribution, however for some values results seem not correct - values between 17-22 points. Actually results doesn't seem strange when you know that to succeed in this exam student needed to have 30% of maximal number of points which in this case equaled to 21 points. According to this chart it looks that people evaluating this exams had tendency to omit giving exactly 20 points. There may be many reasons for such tendency. We don't know if this was done consciously and on purpose. What we do know is that people who are expected to make any evaluation can be easily directed toward some direction because decisions are subject to bias. I've learned during AST Bug Advocacy course that in decision making we can analyze several biasing variables. For example common bias variables are : "Motivation", "Expected Consequences" , "Perceived probability" ,"Perceived importance of the task". We don't know if there were any consequences for examiners for failing student when he got exactly 20 points, but for sure this number had some impact on making pass / fail decision.

The Matura exam in Polish consist of foundation part and second not mandatory part on advance level. This part has no impact on pass / fail decision. Let's see normal distribution of points on advanced level.

There is no strange values around specific numbers. So it looks that with pre-defined boundary point for pass / fail decision, examiners are less likely to fail students which have one or two points below this boundary value.

Same applies to people who evaluate software on daily basis - testers. Our decision regarding Bug / Feature is also subject to bias. Glen Myers in his book "The Art of software Testing" writes that "testers who want to find bugs are more likely to notice program misbehavior then people who want to verify that the program works correctly". What is most interesting testers do this unconsciously.

You can read more about above phenomenon by searching phrase Experimenter Effect and for more about making decision under uncertainty search Signal Detection Theory

Alek