I had the idea of a psychological experiment for detecting race or gender bias, or any other sort of psychological bias about people's appearance whether conscious or unconscious. The experiment compares a human distribution that should be random and meaningless vs. a computer distribution which is actually random.
1. Assemble a group of 100 images of faces which are categorised by a particular race or gender. Face #1 might be Afro-Caribbean Male for example, face #2 South-East Asian Female.
2. Create more groups of 100 faces like the above. This experiment must be repeated several times so it would be best if different faces were used each time.
3. Importantly, the order of race/gender sorting must remain the same for each group and always fixed for all participants and for every round of the experiment, so, in the above example, face #1 would always be an Afro-Caribbean Male. Faces should be chosen to eliminate race or gender ambiguity.
4. Choose a number of subjects and divide these into 3 groups. Those in group 1 are told that the experiment is to test racial bias. Those in group 2 are told that the experiment is to test gender bias. Those in group 3 are told nothing (or, for example, that the test is simply about reaction time and spatial awareness, or some other such thing).
5. For the test, the subject is tasked as an employer and is asked to distribute roles to employees. 10 employees must be assigned a management role. 20 employees are to be assigned a technical role. 70 employees are to be assigned a worker role.
6. The subject either uses a computer or picture cards and is shown each of the 100 faces in order. A count-up timer is present to add a feeling of time pressure; this either appears on the screen or the researcher should visibly use a stop-watch. The subject must assign each face to either management, technical, or worker. Subjects can swap around faces; they can change their mind and shift the roles as much as they want.
7. When all 100 faces have been shown and all roles have been assigned the test is complete and the time is stopped and noted.
8. The experiment is repeated by the same subject several times because an average result from many rounds is taken. There can be a time gap (hours, days) between rounds of this test, and this is perhaps better as it stops fatigue and any memory bias.
9. The fixed order of the faces is necessary to stop simple tricks; eg. if a subject put the first 10 faces as manager, the next 20 as technical, the rest as worker then this would be apparent in the data, whereas it would appear as random if the faces were shown in random order.
10. The experiment is also completed by a computer using a purely random distribution. It comparison of the human distribution to this random distribution which will reveal any bias in the average choices of the human subjects.
