Friend or Foe: Can Computer Programmers Trust ChatGPT? ft

“It seemed magical to me,” says Pietro Schirano. “He kind of scared me because he was so good.”

You are talking about the latest version of ChatGPT, an artificial intelligence (AI) tool. You type something and it becomes an answer to you.

It’s amazing how humane, friendly and intelligent the responses seem. Below is an AI model called GPT-4.

Schirano, design lead at trade finance firm Brex, has been using ChatGPT to create computer programs.

“I want to build a Pong-like game,” he wrote, referencing Atari’s 1970s table tennis game. “What’s the best language to (use) so a lot of people can try?”

ChatGPT not only recommended a programming language that runs in the browser (JavaScript), but also generated the code for a simple version of the game.

What would have taken an experienced developer half an hour to write, ChatGPT took 40 seconds. It worked equally well for the classic arcade games Breakout and Asteroids.

There are many code samples for these games online, but ChatGPT is not searching the Internet for a ready answer.

Although built using a lot of web content, ChatGPT generates new text just for you by predicting the correct answer.

“ChatGPT was able to understand my question about how I make this game more popular,” says Mr. Schirano. “He has a lot of contextual understanding that someone who has never developed couldn’t get from Google.”

Using a search engine, you would have to figure out what coding language to use, how to use it, and how to finish any incomplete examples online. ChatGPT can provide you with all the game code, along with instructions tailored to your needs.

“Now even someone who has no idea what they’re doing can create a simple web application,” says Mr. Schirano.

Some say that recreating these old games was too easy because there are so many examples online.

So Mr. Schirano’s colleague Ammaar Reshi, design manager at Brex, pushed ChatGPT further. He asked her to recreate an old PC game for which there is no sample code online. It’s a 3D game called Skyroads where you drive a car on a road, jumping over barriers.

ChatGPT knew about the game through its Wikipedia page and Mr. Reshi explained how the game works. “There was a lot of back and forth with this one,” Mr. Reshi says. “Literally an hour of tweaking this, tweaking that.” He had to test the game, tell ChatGPT how to change the game’s behavior, and repeat.

Even so, Mr. Reshi was able to create a 3D game online, despite not knowing anything about the JavaScript language used to create it, and without ChatGPT having access to code samples for that game.

These are small games, and no one is suggesting that ChatGPT is going to make modern console games. But it could be used to create parts of a major game or to help create any other software.

But even with productivity improvements, some are hesitant to embrace AI for coding.

“The risks of ChatGPT for software development still outweigh the benefits,” says Tony Smith, CTO of Rightly, an organization that helps people remove their data from company databases. “The code these AIs generate is often buggy or inefficient.”

Mr. Smith asked ChatGPT to create a code to calculate how many days there are in a given month. “The code looks great, however there is a problem with the change to British summer time, so it thinks March has 30 days,” he says. “There is a subtle bug that will appear once a year.”

Others have discovered that AI-generated code can include security flaws. ChatGPT can model bad behavior seen online, bypass vulnerabilities, or use outdated techniques. Also, ChatGPT has limited knowledge of events after 2021.

Business risk is also increased if people are tempted to use code they don’t understand.

Kevin Bocek is vice president of security strategy and threat intelligence at Venafi, a company that makes security software to authenticate machines. He used ChatGPT to create Excel macros and PowerShell scripts, which are two different ways of giving repeatable instructions to a computer. Hackers often use them to launch an attack.

“Now I have code coming from ‘out there’ and I’m going to run it, and I have no idea what it does,” he says. “Now the opportunity for more code that could be harmful has increased.”

While the AI ​​can be used to create code, it will always be the responsibility of the developer to verify it in Venafi. “That’s something we take very seriously,” he says. “Humans review code multiple times. Ultimately, it’s professional developers who are responsible and accountable.”

Herd, a performance marketing and web agency, creates slideshow style elements on some of their websites. “Writing the JavaScript code individually for each section can be painful, so ChatGPT really speeds up that process,” says Matthew Hunt, a junior web developer at Herd.

However, you are concerned that ChatGPT may also remove some of the successful work. “Part of the joy of developing is learning exactly what a file does and how it all works,” he says. “If ChatGPT is doing this for us, we’re not really growing as developers.”

Dan Ciruli, vice president of technology at cloud management company D2iQ, takes a different view. He previously worked at Google.

Google Application Programming Interfaces (APIs) allow developers to connect their software to Google services. There were 200 APIs and seven supported programming languages, including Python. To provide instructions for each API in each language, Google would have to create 1,400 code samples.

“That’s something we struggle with,” he says. “We just couldn’t build samples in every (programming) language. But when I got access to ChatGPT, the first thing I did was ask it to show me how to call the Google Translate API in Python. Boom, there it was.”

The future of software development is likely to be a partnership between the developer and the AI ​​assistant, which is reflected in the name of a popular AI tool. GitHub Copilot is a commercial tool that automatically completes code as developers work. Code completion uses a faster but less accurate model than ChatGPT.

“Since you’re writing code, ideally every time you enter a character, you get a new response,” says GitHub CEO Thomas Dohmke.

Although Copilot has been updated with features to detect and block insecure code, humans remain essential protection. “Just like any other coding tool, you should always use GitHub Copilot in conjunction with human code review, testing practices, and security tools,” says Mr. Dohmke.

Will AI code generation lead to job losses? Schirano and Reshi can foresee that the junior roles could be replaced if AI takes on repetitive coding work.

Manuel Doc is a front-end developer at user experience agency Illustrate Digital. “I was so surprised by the code-level responses from ChatGPT that I feared my job would be in jeopardy soon,” he says.

“However, during my 1-2-1 with our lead developer, it helped me understand that I was not only hired to write code, but to analyze problems and provide solutions that ChatGPT cannot provide. It was very reassuring.”