Debunking Devin: "First AI Software Engineer" Upwork lie exposed!

Channel:Internet of Bugs · Length:25:16 · Published:2024-04-09 · Views:557815

Watch on YouTube

Summary

Internet of Bugs analyzes Cognition's Devin demo frame by frame, arguing the video did not show Devin completing paid Upwork jobs as claimed, and showing what the task required versus what Devin produced.

Why HN recommends it

Cited as the debunking of Devin's marketing claims in skeptical discussions of AI software engineers.

Software practice, open source & careers · LLMs & generative AI · AI & data (other)

Subtitles (excerpt)

This is the Internet of Bugs, my name is Carl, and <i>that</i> is a lie. So this video is in three parts. First, we're going to talk about that claim. We're going to talk about what should have been done. What Devin actually did and how it did it and how well it did it. I have been a software professional for 35 years. I am not anti-AI, but I really am anti-hype and that's why I'm doing this. Devin was intro'd not quite a month ago now. And it was touted as the world's "first AI software engineer." And I don't believe that it's the first software engineer and I already made a video about that. I'll put the links in the description. But today is about the specific claim that's the first line of the video description, which says "watch Devin make money taking on messy Upwork tasks. That statement is a lie. You cannot watch that in the video. It does not happen in the video. It does not happen. What's worse though is that the hype and the fear, uncertainty and doubt from people repeating and embellishing on that claim because they're trying to get clicks or they're trying to go viral or they just want to be part of the zeitgeist. The hype around Devin in general is just crazy. And that statement seems to be what a lot of it is is pinned on. For the

On Hacker News

No public testing, no benchmarks, no clear information on context window size or restrictions for extensive use, no comparison with the newest Claude Sonnet 3.5 or O1, nothing. What we do get is a price of $ 500,- per month from a company that has been caught lying about this very product [0] and has never allowed independent testing. Cognition, I am sorry to tell you, but there is no reason to t… — Topfi, 2024-12
The last thing I saw on it was a video saying that calling Devin an AI software engineer that does Upwork tasks is a lie based in what was shared in an earlier video. [VIDEO] "I broke down the Devin Upwork video frame by frame, and here I show what Devin was supposed to do, what it actually managed to do instead, and how bad a job of that it did. On the whole that's not surprising given the curre… — jaredsohn, 2024-09
This debunking video( [VIDEO] ) of Devin really questioned the usefulness for me. It created a file in the repo and spent a lot of time debugging its own unnecessary code rather than reading the Read Me to understand that the code it needed to use already existed and just needed to be run with different inputs. It's not clear it we're even near a point where it can independently and meaningfully … — siliconc0w, 2024-05

Related videos