Nicholas Carlini - How I Use "AI"
2025-03-09
One of the most common retorts I get after showing these examples is some form of the statement “but those tasks are easy! Any computer science undergrad could have learned to do that!” And you know what? That's right. An undergrad could, with a few hours of searching around, have told me how to properly diagnose that CUDA error and which packages I could reinstall. An undergrad could, with a few hours of work, have rewritten that program in C. An undergrad could, with a hours hours of work, have studied the relevant textbooks and taught me whatever I wanted to know about that subject. Unfortunately, I don't have that magical undergrad who will drop everything and answer any question I have. But I do have the language model. And so sure; language models are not yet good enough that they can solve the interesting parts of my job as a programmer. And current models can only solve the easy tasks.
But five years ago, the best an LLM could do was write a plausibly-English sounding paragraph. And we were amazed when they could form coherent ideas from one sentence to the next. Their practical utility was exactly zero. Today, though, they've improved my productivity at the programming aspects of my job by at least 50% on the average project, and have removed enough of the drudgery that I built several things I would never have attempted otherwise.
These use cases are far away from what the AI-hype promises. What LLMs can do isn't revolutionary but it can still be useful.
After seeing these examples I am a bit more interested in using LLMs as a tool, aware of what they can and cannot do and keeping in mind what Molly White wrote in AI isn't useless. But is it worth it?