GitHub advertises its intelligent Copilot as “the world’s most widely used AI tool for developers”. The IT service provider Avision, which specializes in software revival, has tested this extensively in practice – with mixed results.
Does GitHub’s Copilot finally provide developers with an intelligent pair programming partner that both reduces the complexity of the coding world and is an effective means of combating the shortage of skilled workers? To answer this question, there is no way around a practical text. Avision took a close look at the AI tool and put its potential to the test with a focus on TypeScript, Java, React, Python and Angular.
Up-to-dateness and training data
At the time of the test phase, the GitHub Copilot was accessing data from the end of 2021 – although the database is regularly updated, the model is still not up to date. As a result, the latest versions of languages and frameworks such as Angular or React are simply not available. This can quickly lead to further problems, as the tool may not be familiar with new language constructs or security standards, which poses potential risks and can reduce the quality of the code.
Quality of the results
The AI assistant can take over many simple but time-consuming work steps and inspires new approaches to solutions. However, the current version is not a simple copy-and-paste solution. In almost all cases, the human developers had to revise and adapt the generated results. In some cases, Copilot also simply delivers incorrect results that were not recognizable at first glance – this increases complexity and reduces the initial time saved through automation. Nevertheless, the tool is suitable for creating an initial code base on which developers can build. Even if there is still some catching up to do in terms of results, none of the testers would want to do without Copilot.
Potential and productivity
Despite the qualitative gaps and the lack of up-to-dateness, Copilot has enormous potential. We are only at the beginning of a continuous development that will undoubtedly lead to an increase in efficiency and productivity in the future. Until then, however, there is still a lot of work to be done to improve the reliability and accuracy of the results. However, this should not diminish the importance for the industry; AI assistants will be part of the permanent repertoire of developers in the foreseeable future.
Inexperienced developers and generative AI
The question of how developers without many years of coding expertise should deal with generative AI tools is particularly relevant. The risk factor here is that the uncontrolled use of these tools leads to a deterioration in code quality, especially if inexperienced developers tend to adopt the proposed solutions without reflection. On the one hand, this can lead to inconsistent and possibly faulty code that impairs the overall quality of the project. On the other hand, junior developers also lack the necessary learning effect if they only work with AI-generated code suggestions instead of developing it themselves. What leads to a stagnation of the individual learning curve in individual cases can pose a challenge for the promotion and personnel development of the entire industry in the big picture. It is therefore important that inexperienced developers continue to learn through practical experience and do not rely exclusively on generative AI tools.
Conclusion
The test results with GitHub Copilot are mixed. While there are undoubtedly helpful use cases and the tool can automate smaller tasks, developers need to critically scrutinize and review the suggestions generated by the AI at all times. The challenge is that errors or incompleteness are not always recognizable at first glance, which makes using the tool quite complex and time-consuming. Nevertheless, GitHub Copilot offers the promising potential to increase the productivity of developers – at least if the quality and reliability of the results increase. The bottom line is that it is and remains essential to handle the use of these tools responsibly and ensure that they complement and do not replace human expertise.
This press release is also available at www.pr-com.de/de/avision.
Press contact
Avision GmbH
Christina Karl
Marketing
Bajuwarenring 14
D-82041 Oberhaching
Phone +49-89-623037-967
christina.karl@avision-it.de
PR-COM GmbH
Melissa Gemmrich
Sendlinger-Tor-Platz 6
D-80336 Munich
Phone +49-89-59997-759
melissa.gemmrich@pr-com.de
