AI is everywhere. Every time you download an app, login to a site, or buy a product, they’re all talking about how AI is being integrated. We’re definitely in the AI bubble.

One of the big advancements has been the ability for AI to write code, and replace all those pesky developers out there.

😕 Wait, that’s me.

In May, Microsoft announced GitHub Copilot Agent mode. Agent mode allows anyone to ask AI to write some code, and it’ll do it. Not just some code, but all the code. You can ask GitHub Copilot Agent to write an entire website, or an entire mobile application. The best part is, you don’t even need to know how to code. To be honest, though, that’s also the worst part.

I was curious. How good is GitHub Copilot Agent? Is it going to replace me?

So I took it for a spin, and I’ve been using it for the past few weeks creating all sorts of things.

TLDR: GitHub Copilot Agent is pretty good at what it does, but it has so many rough edges. Don’t go replacing humans yet.

The first thing I did was ask GitHub Copilot Agent to build a one-page React site that just showed me the weather in both Fahrenheit and Celsius. This was something I wanted to do, but never got around to. I already had an API to use, so I asked GitHub Copilot Agent to build it. And it did. In less than a minute.

Amazing!

I then spent the next few hours asking GitHub Copilot Agent to enhance it. I asked it to create a modern look and feel, and it wrote all the CSS and HTML for me. I asked it to add in input, so users could input a location, and an option to geolocate them. GitHub Copilot Agent had no issues doing all that. I asked for more weather details like feels like temp, highs and lows, and even the forecast for the next three days. No problem.

🤔 This was amazing, and concerning. Maybe AI can really replace me!

Then, we got ourselves into trouble. I decided that I wanted that app to be in Next.js. GitHub Copilot Agent said it’d create a new project in Next.js and copy everything into the new framework. Only it didn’t. It got about half way done, and it got confused, and quit. I then had to finish the job myself.

Remember, this is only a one-page site and Next.js is based on React so it shouldn’t have been too hard to do.

Now that my weather app was in Next.js, I continued to make updates.

Sometimes it was great, sometimes not so much. I’d ask for functionality to be added, and it’d do that, but also change some of the design for unknown reasons. I’d ask it to make the site mobile friendly, and it destroyed the desktop view and replaced it with the mobile design. I asked it to save the user’s location in local storage so that it could re-load their location on their next visit, and it had a hard time understanding how that flow would work when there was a location input, and geolocation, and yet the user could change their location. I eventually had to intervene and write the code myself because no matter how many times AI tried, it couldn’t successfully figure it out.

That’s when I really started to understand that prompting AI to do something only half the battle. You have to ask for what you want, but also tell it what you don’t want. You have to ask it to use specific frameworks, and packages, and standards, otherwise it’ll just pick some it likes.

AI is smart, but they’re more like a know-it-all new programmer. They can do a lot, but they don’t understand the full scope of what they’re being asked, and they don’t ask clarifying questions. Sometimes it forgets the obvious too. Like when I asked to add a shared footer to the site. It had no issues creating the file and adding it to one page, but it only added it to one page. 🤦

In a different app, I asked AI to create an AI image generator written in Python. Again, AI did a good job, but I wasn’t specific enough. It created the site using OpenAI’s APIs, which cost a lot and I don’t pay for. So I asked AI if there was a free API we could use. It said yes and updated the code to use Midjourney. Midjourney’s API allows for about 8 free photos and then costs too. Eventually we did find a free API, and I’ve been able to build out a pretty cool image generation app using Python. Again, sometimes it did great, other times I found myself being frustrated that it couldn’t do what I thought were simple things.

Fun fact, I don’t know Python very well. I have a fully functional app that does what I want though.

In all the different little projects I built out, it became clear time and time again that AI doesn’t often think outside the scope of what you ask.

• Is the site accessible?
• Is the code secure?
• Is the code optimized?
• Is there documentation?
• Is this based on [programming language] standards?
• Is the site mobile friendly?

These are all the things that AI can do, but doesn’t usually do them unless you ask. Even then, sometimes you have to ask it multiple times because it’ll make things accessible, but after adding new features you’ll realize that those new features aren’t accessible because it didn’t think of that.

🫣 On the flip side, AI can also over do things.

I asked AI to create a WordPress plugin. I gave it clear instructions of what I needed to be done, and it did an amazing job of outputting a basic plugin.

I then asked it if the code was optimized, which it honestly answered that the code wasn’t great, but it could optimize it for me. The code that it just wrote, it admitted, wasn’t good. WTH.

I asked it if the code was secure, and it said no, but it could update that. Great, but why aren’t we putting security first?

I asked it if it was based on WordPress standards, and it said no, but it’d update everything to follow WordPress standards. In doing so, it created a package.json, install and uninstall functions and files, and a composer file. However, my plugin didn’t install anything. It only swapped out a shortcode for some JavaScript. I pointed that out to AI, and it admitted that it over did it. It added files and functions it didn’t need, but it could clean them up for me.

🤖 It’s examples like that that show how amazing AI can be, but also how AI isn’t ready to replace people.

GitHub Copilot Agent is great. I love it for spinning up new projects, and writing code, and helping me fix something when I don’t really know how the code works, but human oversight is necessary.

This is what scares me about AI. If people use it without looking at what it’s outputting, then you have no idea what’s really going on. Sure, you can build a web or mobile app in no time, but is the user’s information safe? Is it accurate?

At the end of the day, companies can blame AI, but they’re at fault for what they created with AI.

And, in case you’re wondering, this post was not written in AI. That cool robot image at the top was created with AI, though.