Are machine learning systems too biased for Agile teams?

About a month ago, the virtual swissICT event “AI in Agile recruiting – a reality check” (in German) took place. It got me thinking whether machine learning systems are too biased to be helpful for Agile teams.

Author: Jiri Lundak

How biased are machine learning systems?

The event gave me the impression that even in our modern times, we are not immune to prejudice. It showed, that this world has its roots in the prejudices of each and every one of us. On the other side, we ourselves are also influenced and shaped by the society that surrounds us. Accordingly, we personally also adopt the prejudices of those around us (whether in the company or the wider society).

At the event, Dr. Prof. Ana Fernandes, in her presentation “Conscious and Unconscious Discrimination in Hiring: Fertility, Gender and Ethnicity Bias”, clearly demonstrated, using specific examples, that bias is already a problem in recruiting not assisted with machine learning.

Machine learning systems are designed by humans and fed data. The lecturing experts made it very clear to us that machine learning systems cannot be trusted unconditionally. This is because we, as humans, carry our prejudices, consciously or unconsciously, into the database of machine learning systems. Ursula Deriu showed in her talk “Von der Allmacht der Künstlichen Intelligenz” (in German), that machine learning software – as it presents itself today – is inherently vulnerable to be biased. Also Dr. Prof. Mascha Kurpicz-Briki, in her presentation “Social stereotypes in AI – how does it happen?” (in German), vividly demonstrated how prejudices can become embedded in machine learning systems.

Often enough, the pool of raw data is not broad enough to counteract biases. A conscious effort must be made to train the models underlying machine learning to provide a good foundation for bias-free recommendations and autonomous decision making by automated systems.

Using machine learning – for what?

The event participants considered the use of machine learning to be particularly sensitive in the following areas of application: during job interviews, in finding out reasons for dismissal, and in assessing social skills. Additionally they concluded that “hire for cultural fit” is hardly conceivable due to an automatic assessment of CVs. It was also emphasized that humans should have the final say in assessing candidates. Machine learning systems should thus only make recommendations.

Machine learning support was particularly desired by HR managers in the pre-selection of candidates and in predicting employees who might leave the company. At the same time repetitive work and reviewing qualitative responses from surveys and comments, were also mentioned as potential uses of machine learning.

Uncertainty prevailed regarding the quality of automated scanning of resumes.

On the other hand, participants emphasized that attributes such as age, gender, race, nationality and sexual orientation should be deliberately excluded from assessment criteria. This would be breeding ground for bias in the models.

Machine learning in Agile recruiting – are they compatible?

In the past, I had the opportunity to sift through hundreds of CVs and conduct many interviews. The company I was working for at the time was a pioneer regarding agile approaches. This experience makes me skeptical about the use of machine learning in recruiting.

From my point of view as an agile coach and as a team member of agile teams, what is written in a CV is only partially decisive. Sure, it’s good to be able to read from it whether a programmer has already worked with Java when looking for people with such experience in the company. Of course it is good to know if a person has a lot or little experience in certain areas when it comes to hiring a person with a certain seniority.

To find this out, it is certainly useful to have a way to get an overview of the available candidates and to be able to select a subset of them (e.g. with the criterion “experience Java > 4 years”). If a searchable database helps you do that, all the better (AI is not necessarily required for this). The beauty of this approach is that it can even eliminate the need for pre-selection by HR staff, as members of an agile team can query the database themselves.

What really matters to agile teams

Machine learning systems – in their current state – are not able to analyze a candidates behavior to counter potential bias, when just feeding off of CVs and job reference letters. Let’s see why this is the case.

The event left me thinking for a few days. What do agile teams want? In my experience, they usually insist on selecting their future colleagues independently, “handpicked” so to speak.

Why do they insist on this? Because they have found that the CV only hints at what is really in a person in terms of skills and potential. Since they want to be as sure as possible in their choice, they will subject each candidate to a practical test, which involves observing them at work, thinking and behaving.

For agile teams, it is thus important to achieve a “culture fit”. But not only that! A “skill fit” is definitely important as well, but unlike what you might expect, it’s not about whether a Java developer knows library X or is familiar with technology Y. It’s more about whether she knows how to work. It is rather about whether she…

• …is able to approach problems systematically and not hastily
• …has acquired sensible programming habits (e.g. refactoring strategies, red/green testing, small changes, etc.) and knows when to use which technique
• …is willing to take advice and learn from others
• …can communicate in a targeted and understandable way.

Why CVs say so little about candidates

You can’t read these skills out of a CV. Why not? Because the CV hides the most important things. I remember a case when a candidate seemed to have been involved in many projects and was familiar with many technologies. Nevertheless, he was not able to put a simple piece of program code into a better form.

Does the potential future colleague have a lot of programming experience, but is unable to admit that she doesn’t know something? Does the candidate’s way of working consist of delving into a problem and not asking for help when it comes up? Is a programmer quick to develop but gives little thought to design or quality assurance?

Even if you look closely at a person’s references, you always have doubts about whether the person who wrote the reference really worked closely with the candidate to make a nuanced assessment.

It becomes clear that machine learning systems can only evaluate what is written. Unfortunately what is not written might be even more important to a person assessing the value of a candidate for one’s team.

Job references are misleading

Some think that for a good assessment job reference letters are needed. Actually they where invented to give an accurate view of a candidate. Ideally they would be well suited as raw data for machine learning systems, so they can make a good choice without being biased. Unfortunately this is rarely the case.

One always imagines the writing of a job reference to be so direct, immediate and truthful. Finally, it is said about job references that they must be “true, benevolent, complete, consistent and clear” [1]. The figure below represents the ideal world. A small team. The line manager is close to the team and works closely with the employee he or she must evaluate.

Considering that certain large companies have outsourced the writing of job references to external service providers who work on the basis of some quickly clicked together text modules, doubts about the quality of the references are reasonable. See the next figure:

This drawing shows that nowadays it is not at all self-evident to get a job reference issued by a person who knows you intimately. This example shows, that because of increasing distance and several participants in the process, it can be doubted, that a reference is produced that gives a good, well-rounded picture of the employee.

Job references and machine learning systems

But there is another problem that amplifies the the woes machine learning systems have to make decision without becoming biased.

As someone who has been allowed to write references myself, I know how difficult it is to write a reference that meets the requirements of being “true” and “benevolent” at the same time. Actually formulaic formulations, which contain hidden insinuations about the actual performance of an employee, have developed over time. This because the issuer of the reference must not block the chance of a new job for the former employee. Fortunately, such formulations are increasingly being dispensed with in the meantime.

But this does not simplify the problem of job references for people who have not done a good job. The balancing act is very difficult. This leads to the question: What corresponds to the – sometimes harsh – truth, and what I am actually allowed to write in a job reference without it harming the employee’s career? In such cases, we as human beings have the tendency to write too positive a reference, thereby glossing over the statements to be made.

This is another reason to question job references as a central basis for decision-making in the application process.

Development teams take too little responsibility

Development teams (certainly agile ones) like to welcome highly productive new employees in their midst. At the same time they often like to delegate the task of selection to third parties (be it the internal HR department or an external headhunter). Although, Proper employee selection should be a core competency of any team. This process is time-consuming and requires in-depth engagement with candidates.

The dream of delegating or even automating this process is tempting. However, the proper selection of team members is so important that it cannot actually be delegated. Although some administrative support (e.g., in writing job ads) can be obtained, the good impression, from direct interaction with candidates, should help the team make an informed decision in the selection process.

In the end, the team has to live with the outcome of the selection.

Conclusion

With the use of machine learning in recruiting, we hope to see a more streamlined hiring process.

As we’ve found, machine learning systems today are still biased because of their developers and data providers. This is not going to change anytime soon. But initial efforts (in research) are underway to actively counteract biases.

Of course, biases can also come into play when an agile team itself selects team members. Nevertheless, putting the responsibility of selection on the team helps to make a better team decision.

Machine learning can help in peripheral areas of the recruitment process. However, the selection of suitable candidates should be based on hands-on collaboration with them and the team.

In this way, biases will not influence the selection. We certainly can’t expect that. However, we must tackle the problem with prejudices at its root, where it originates, namely in people’s minds.

The author: Jiri Lundak has been working independently as an Agile coach at the intersection of people and technology for many years. He has held many roles, including recruiting.

[1] “Creating job references” (in German), Weka, retrieved 21.09.2021.