Okay, so maybe I’m just impatient. But when a friend sent me a link to an interview on YouTube featuring Omar Yaghi, a Nobel laureate who is now using AI extensively in his chemistry research, I was interested but not inclined to watch it.
My hearing isn’t the best, but also interviews can be a bit rambling. On the other hand, I really wanted to know what he said.
So I simply gave the link to the new ChatGPT Codex app on my Mac and asked it to summarize the interview. Then the fun began.
First Codex asked for my permission to access the internet. It then found the interview on YouTube, but determined that captions and a transcript weren’t available. So Codex realized it would have to make a transcript itself.
It began by finding a low-resolution, quick-download version of the interview online that it saved to my computer.
Then it extracted the audio—and divided it into 36 segments. Why? Codex knew that Apple’s speech recognition has a one-minute limit, so it split the interview into 50-second segments. Also, this allowed Codex to transcribe the 36 individual segments simultaneously, saving time.
Codex noticed, though, that because these segment boundaries were arbitrary, some sentences crossed the boundaries, thereby interfering with accurate transcription.
It was smart enough to realize that this could mess up the flow of the conversation, so it made a second, overlapping set of 32 segments and transcribed those as well.
It then reconciled the two transcripts it had created, checked them against an official interview description, identified the major themes, and summarized Yaghi’s ideas.
The resulting summary was excellent, and I really appreciated Yaghi’s insights into AI. Thank you, Codex.
This whole process was autonomous, and I only learned these details after asking Codex how it accomplished this task (which took it over 20 minutes to complete).
Codex is an example of an AI agent, which can work independently for hours or even days at a time to accomplish a task.
I first had to give it access to my Mac’s desktop so it could download whatever was needed and do the processing there. During those 20 minutes, while I was working on something else, it briefly asked my permission for things such as accessing the internet.
Agents are powerful and can go rogue, as was widely reported this past summer. I could have adjusted the settings to give it more freedom to work without occasionally asking my permission, but Codex itself advised against it. I love the utility of AI but am not ready for a rogue agent on my computer.
I was curious, so I asked Codex how it knew to accomplish the task I gave it. It replied: “The [ChatGPT] model supplied broad technical know-how; Codex supplied a computer workbench; and the agent process connected observations, decisions, tools, and corrections into a workflow. That process is capable but not infallible. The numerous retries and approval dialogues are evidence of that: I knew the general path, but had to discover which exact path would work on your Mac.”
So smart.
Of course, I’ve been having fun asking it to do other things. It was even able to identify a sweet birdsong that Audubon’s bird app bizarrely said was a crow. (Codex said that it was 90 percent confident that the best match was an olive-sided flycatcher.)
I also had it make an app that I can use to update my website with each new monthly column. In the past this process could take as long as 45 minutes, but now it just takes a couple minutes.
Codex is available for free in the ChatGPT app for Mac and Windows. I found that if I ask ChatGPT to do a complex task, it automatically invokes Codex, as when I asked for the YouTube summary. Also available in ChatGPT is Work, which is, again, an agent that can function independently.
When you give an agent a task, it can plan the steps, gather information, use tools, create files, and then offer up a result.
Note that whatever AI model you use, it may initially default to a faster and less intelligent version. Look for a dropdown menu at the top or in the dialogue box if you want more in-depth thinking.
While Codex is free, it will limit you to a certain amount of usage each day. On the other hand, in August OpenAI said there’s no longer any daily limit on the amount of simple chatting you can do with ChatGPT. Limits do still apply to things such as file uploads, images, and other tools.
Such great toys—and tools. Amazing how far things have come in the nearly four years since ChatGPT was first released. Your Computer Frontiers columnist is having fun.