Veröffentlicht am

ChatGPT for Agile Software Development – Boost or Risk?

ChatGPT sample code generation dialog

Agile software development is a widely adopted methodology that emphasizes flexibility, collaboration, and iterative progress. As software engineers strive to optimize their workflow, the use of ChatGPT can offer numerous benefits to enhance their agility. However, it is important to be mindful of potential risks that could hinder the agile process. Let’s explore both the advantages and potential pitfalls of incorporating ChatGPT into agile software development.

Ways ChatGPT Could Help

There are some ways ChatGPT could help software engineers be more Agile:

      1. Improved Communication: Effective communication is a cornerstone of agile development, and ChatGPT can facilitate this by providing real-time feedback and suggestions during team discussions. It can analyze code snippets, offer syntax recommendations, and even assist in identifying potential vulnerabilities. This could streamline the development process and fostering collaboration among team members.

      1. Rapid Prototyping: ChatGPT can aid in generating code snippets and prototypes, which can accelerate the development of minimum viable products (MVPs). This allows software engineers to quickly iterate and gather feedback from stakeholders. Doing so could lead to faster decision-making and validation of ideas, a core principle of agile development.

    1. Enhanced Documentation: Documentation is vital in Agile development to maintain transparency and knowledge sharing among team members. ChatGPT can assist in generating documentation templates, code comments, and other technical write-ups, saving valuable time and effort, and ensuring that project documentation remains up-to-date and accessible to all team members.

But… where there is light there is shadow.

Dangers to Agile Development with ChatGPT

A few points com to mind, when we want to highlight the risks:

      1. Over-reliance on Automation: Relying solely on ChatGPT for code generation and suggestions can potentially lead to a decrease in critical thinking and creativity among software engineers. Agile development values the human element, and excessive reliance on automation could result in a loss of innovation and diversity of ideas, which are essential for creating robust and unique solutions.

      1. Quality Control Challenges: While ChatGPT can provide suggestions for code snippets, it may not always guarantee the best practices or optimal solutions. If not thoroughly reviewed and validated, code generated by ChatGPT could potentially introduce bugs, security vulnerabilities, or violate coding standards. This may result in increased technical debt and rework, contradicting the principles of Agile development.

    1. Ethical Considerations: As a language model, ChatGPT can potentially generate biased or unethical code, especially when trained on biased data. This could result in unintentional propagation of bias or unethical practices in the software being developed, which could have ethical, legal, and reputational consequences, posing a risk to the integrity and fairness of Agile development.

Mitigating the Risks

There are though some things we can do to mitigate the risks:

      1. Strike a Balance: Software engineers should strike a balance between utilizing ChatGPT as a tool for assistance and leveraging their own expertise and creativity. It is essential to maintain critical thinking, innovation, and human decision-making in the agile development process.

      1. Validate Generated Code: Code generated by ChatGPT should be thoroughly reviewed, validated, and tested to ensure it adheres to coding standards, follows best practices, and meets quality requirements. This includes thorough testing, code reviews, and validation against project specifications to identify and fix any potential issues before deploying the code.

    1. Ethical Considerations: Be mindful of the ethical implications of using ChatGPT in software development. Training data should be diverse, inclusive, and free from bias. Review the code generated by ChatGPT for any potential biases or unethical practices, and take necessary steps to mitigate them. It’s important to have a strong ethical framework and guidelines in place when incorporating ChatGPT into the Agile development process.

An Experiment

My personal (admittedly still limited) experience using ChatGPT to generate and explain code confirms the risks associated using the tool (at least in its current state).

Actually, when using it intensely and double checking its output, one can still see its limits.

The code it generates looks mostly clean, but it does not follow a specific coding style. Actually the temperature setting of the ChatGPT dialogue engine is set to give “creative” answers. So you will get different variants on how you could implement an idea. If you have no personal idea on how a solution should look like you can go with any proposed code. But if you try to stick to a specific coding style, you oftentimes find yourself second-guessing what it proposes.

How far ChatGPT’s understanding of the code goes can be easily challenged doing these few steps:

      1. Let it generate some code (for a problem domain, you might be familiar with).
        → I let it generate code to calculate the distance between two planets at any point in time.

      1. It usually gives a verbal explanation on what the code does.
        → It explained to me, that it was using a Euclidean formula to calculate the distance.

      1. Then modify the generated code slightly (like a programmer would do, when moving it to an IDE and evolving it over time).
        → I just changed the calculation of the Euclidean formula slightly.

    1. Paste the modified version of the code back into ChatGPT and ask it to explain the code.
      → ChatGPT explained to me that the code used the Euclidean formula. But this could not be true, as I changed the formula to be invalid.

Conclusion

Just this little experiment shows the danger of relying too much on ChatGPT’s help, at least in its current state. There actually is no reasoning behind what it spits out. Unfortunately for many people that do not take its output with a grain of salt its convincing authoritative tone will produce lots of non valid output.

So yes, validating, testing and questioning its output is essential. We as developers need to step up our game, by knowing what the expected results should be, before asking ChatGPT. So Agile development skills will be in high demand more than ever before.

In the coming weeks I will have a look at various practices from Agile software development (like test automation, evolutionary software design, etc. ) and how they are impacted by the use of artificial intelligence tools.

If you are interested in improving your Agile implementation, then check out my upcoming book.

Note: My experiments were done using ChatGPT version 3.5. As soon as I can get my hands on ChatGPT 4 I will do a follow-up to this post.

 

  • CHECK OUT NOW

  • The Practitioner's Handbook Early Edition is available now

Veröffentlicht am Schreib einen Kommentar

Are machine learning systems too biased for Agile teams?

Are machine learning systems too biased for Agile teams?

About a month ago, the virtual swissICT event “AI in Agile recruiting – a reality check” (in German) took place. It got me thinking whether machine learning systems are too biased to be helpful for Agile teams.

Author: Jiri Lundak

How biased are machine learning systems?

The event gave me the impression that even in our modern times, we are not immune to prejudice. It showed, that this world has its roots in the prejudices of each and every one of us. On the other side, we ourselves are also influenced and shaped by the society that surrounds us. Accordingly, we personally also adopt the prejudices of those around us (whether in the company or the wider society).

At the event, Dr. Prof. Ana Fernandes, in her presentation “Conscious and Unconscious Discrimination in Hiring: Fertility, Gender and Ethnicity Bias”, clearly demonstrated, using specific examples, that bias is already a problem in recruiting not assisted with machine learning.

Machine learning systems are designed by humans and fed data. The lecturing experts made it very clear to us that machine learning systems cannot be trusted unconditionally. This is because we, as humans, carry our prejudices, consciously or unconsciously, into the database of machine learning systems. Ursula Deriu showed in her talk “Von der Allmacht der Künstlichen Intelligenz” (in German), that machine learning software – as it presents itself today – is inherently vulnerable to be biased. Also Dr. Prof. Mascha Kurpicz-Briki, in her presentation “Social stereotypes in AI – how does it happen?” (in German), vividly demonstrated how prejudices can become embedded in machine learning systems.

Often enough, the pool of raw data is not broad enough to counteract biases. A conscious effort must be made to train the models underlying machine learning to provide a good foundation for bias-free recommendations and autonomous decision making by automated systems.

Using machine learning – for what?

The event participants considered the use of machine learning to be particularly sensitive in the following areas of application: during job interviews, in finding out reasons for dismissal, and in assessing social skills. Additionally they concluded that “hire for cultural fit” is hardly conceivable due to an automatic assessment of CVs. It was also emphasized that humans should have the final say in assessing candidates. Machine learning systems should thus only make recommendations.

Machine learning support was particularly desired by HR managers in the pre-selection of candidates and in predicting employees who might leave the company. At the same time repetitive work and reviewing qualitative responses from surveys and comments, were also mentioned as potential uses of machine learning.

Uncertainty prevailed regarding the quality of automated scanning of resumes.

On the other hand, participants emphasized that attributes such as age, gender, race, nationality and sexual orientation should be deliberately excluded from assessment criteria. This would be breeding ground for bias in the models.

Machine learning in Agile recruiting – are they compatible?

In the past, I had the opportunity to sift through hundreds of CVs and conduct many interviews. The company I was working for at the time was a pioneer regarding agile approaches. This experience makes me skeptical about the use of machine learning in recruiting.

From my point of view as an agile coach and as a team member of agile teams, what is written in a CV is only partially decisive. Sure, it’s good to be able to read from it whether a programmer has already worked with Java when looking for people with such experience in the company. Of course it is good to know if a person has a lot or little experience in certain areas when it comes to hiring a person with a certain seniority.

To find this out, it is certainly useful to have a way to get an overview of the available candidates and to be able to select a subset of them (e.g. with the criterion “experience Java > 4 years”). If a searchable database helps you do that, all the better (AI is not necessarily required for this). The beauty of this approach is that it can even eliminate the need for pre-selection by HR staff, as members of an agile team can query the database themselves.

What really matters to agile teams

Machine learning systems – in their current state – are not able to analyze a candidates behavior to counter potential bias, when just feeding off of CVs and job reference letters. Let’s see why this is the case.

The event left me thinking for a few days. What do agile teams want? In my experience, they usually insist on selecting their future colleagues independently, “handpicked” so to speak.

Why do they insist on this? Because they have found that the CV only hints at what is really in a person in terms of skills and potential. Since they want to be as sure as possible in their choice, they will subject each candidate to a practical test, which involves observing them at work, thinking and behaving.

For agile teams, it is thus important to achieve a “culture fit”. But not only that! A “skill fit” is definitely important as well, but unlike what you might expect, it’s not about whether a Java developer knows library X or is familiar with technology Y. It’s more about whether she knows how to work. It is rather about whether she…

  • …is able to approach problems systematically and not hastily
  • …has acquired sensible programming habits (e.g. refactoring strategies, red/green testing, small changes, etc.) and knows when to use which technique
  • …is willing to take advice and learn from others
  • …can communicate in a targeted and understandable way.

Why CVs say so little about candidates

You can’t read these skills out of a CV. Why not? Because the CV hides the most important things. I remember a case when a candidate seemed to have been involved in many projects and was familiar with many technologies. Nevertheless, he was not able to put a simple piece of program code into a better form.

Does the potential future colleague have a lot of programming experience, but is unable to admit that she doesn’t know something? Does the candidate’s way of working consist of delving into a problem and not asking for help when it comes up? Is a programmer quick to develop but gives little thought to design or quality assurance?

Even if you look closely at a person’s references, you always have doubts about whether the person who wrote the reference really worked closely with the candidate to make a nuanced assessment.

It becomes clear that machine learning systems can only evaluate what is written. Unfortunately what is not written might be even more important to a person assessing the value of a candidate for one’s team.

Job references are misleading

Some think that for a good assessment job reference letters are needed. Actually they where invented to give an accurate view of a candidate. Ideally they would be well suited as raw data for machine learning systems, so they can make a good choice without being biased. Unfortunately this is rarely the case.

One always imagines the writing of a job reference to be so direct, immediate and truthful. Finally, it is said about job references that they must be “true, benevolent, complete, consistent and clear” [1]. The figure below represents the ideal world. A small team. The line manager is close to the team and works closely with the employee he or she must evaluate.

Team embedded in a small company
Team embedded in a small company

Considering that certain large companies have outsourced the writing of job references to external service providers who work on the basis of some quickly clicked together text modules, doubts about the quality of the references are reasonable. See the next figure:

Team embedded in a large company
Team embedded in a large company

This drawing shows that nowadays it is not at all self-evident to get a job reference issued by a person who knows you intimately. This example shows, that because of increasing distance and several participants in the process, it can be doubted, that a reference is produced that gives a good, well-rounded picture of the employee.

Job references and machine learning systems

But there is another problem that amplifies the the woes machine learning systems have to make decision without becoming biased.

As someone who has been allowed to write references myself, I know how difficult it is to write a reference that meets the requirements of being “true” and “benevolent” at the same time. Actually formulaic formulations, which contain hidden insinuations about the actual performance of an employee, have developed over time. This because the issuer of the reference must not block the chance of a new job for the former employee. Fortunately, such formulations are increasingly being dispensed with in the meantime.

But this does not simplify the problem of job references for people who have not done a good job. The balancing act is very difficult. This leads to the question: What corresponds to the – sometimes harsh – truth, and what I am actually allowed to write in a job reference without it harming the employee’s career? In such cases, we as human beings have the tendency to write too positive a reference, thereby glossing over the statements to be made.

This is another reason to question job references as a central basis for decision-making in the application process.

Development teams take too little responsibility

Development teams (certainly agile ones) like to welcome highly productive new employees in their midst. At the same time they often like to delegate the task of selection to third parties (be it the internal HR department or an external headhunter). Although, Proper employee selection should be a core competency of any team. This process is time-consuming and requires in-depth engagement with candidates.

The dream of delegating or even automating this process is tempting. However, the proper selection of team members is so important that it cannot actually be delegated. Although some administrative support (e.g., in writing job ads) can be obtained, the good impression, from direct interaction with candidates, should help the team make an informed decision in the selection process.

In the end, the team has to live with the outcome of the selection.

Conclusion

With the use of machine learning in recruiting, we hope to see a more streamlined hiring process.

As we’ve found, machine learning systems today are still biased because of their developers and data providers. This is not going to change anytime soon. But initial efforts (in research) are underway to actively counteract biases.

Of course, biases can also come into play when an agile team itself selects team members. Nevertheless, putting the responsibility of selection on the team helps to make a better team decision.

Machine learning can help in peripheral areas of the recruitment process. However, the selection of suitable candidates should be based on hands-on collaboration with them and the team.

In this way, biases will not influence the selection. We certainly can’t expect that. However, we must tackle the problem with prejudices at its root, where it originates, namely in people’s minds.

The author: Jiri Lundak has been working independently as an Agile coach at the intersection of people and technology for many years. He has held many roles, including recruiting.

[1] “Creating job references” (in German), Weka, retrieved 21.09.2021.

 

  • CHECK OUT NOW

  • The Practitioner's Handbook Early Edition is available now