On Automation

Oct 7, 2026

As someone who’s been working as a software engineer for the past four years, I’ve experienced first hand the disruption wrought by AI: in a couple of years I went from coding by hand to coding with some AI assistance to now being largely relegated to the sidelines, sipping coffee while watching Claude/Codex go ham on the codebase. Every now and then, I might send some follow-up prompts to unblock. Most of the time, I’m trying to understand the code that the AI wrote so I know what the heck is going on.

Everyone in the industry knows that the days of typing code by hand are gone. The barbarians are at the gate. In this brave new world, it seems to me that a software engineer’s job broadly falls into 3 categories:

1. Exercising judgment on what is worth building, evaluating trade-offs between different approaches, providing context and specifying requirements of the system to build for the AI agents.
2. Building systems around the AI to help do its job more effectively. For example, inference engineers may still find it useful to write a new skill that teaches the agent what specific workloads to use for benchmarking or to even create a custom agentic optimization harness.
3. Reviewing the code generated by AI. AI can still make mistakes, especially in domains like GPU programming where the code might seem correct and have subtle correctness issues (e.g. race conditions). Another reason for reviewing the code written by AI is to understand it, so that you’re not completely left out of the loop: when AI can produce code faster than we humans can digest, understanding is the new bottleneck in development.

Over time, #2 and #3 will likely become less prevalent in a software engineer's day-to-day job. We are already seeing agent harnesses built around the model get absorbed into the model. Models will continue to get better and make less mistakes, and as for code reviews, well clearly some developers are checking in code without reading it, as preposterous as that might’ve sounded a few years ago. In the long term, the role of software engineers might mostly be about prioritization: given some compute budget, where should you direct it? What problems will you solve? Where will you be pointing the eyes of Stargate?

In the meantime, the demand for experienced software engineers still seems strong, at least in the Bay Area. For example, AI is very good at writing CUDA kernels nowadays, but good kernel engineers are still highly coveted, even as the work has evolved from writing code to directing and verifying AI. Instead of completely replacing software engineers, what we’re seeing is that AI has shifted the work to operating at higher levels of abstraction: designing harnesses, setting up autoresearch loops, monitoring and unblocking agents, etc. As a field, software engineering has gone through similar transformations in the past: developers used to code in lower level languages like Assembly or C until more abstract, higher level languages like Python and Ruby came along and found massive adoption. Now the field is going through another stage of evolution as developers increasingly spend time thinking about higher order system-level abstractions and trade-offs, and guide coding agents towards desired business outcomes.

Taking a step back for a bit, I’d like to talk about my own mixed feelings about the changing nature of software engineering. On one hand, I share the enthusiasm and optimism that some developers have voiced in the community: instead of worrying about small details like language syntax or obscure APIs, software engineers are now freed up to work on more creative and consequential higher-level decisions that actually involve problem-solving. I also think it’s great that AI has lowered the barriers to software creation and folks who didn't know how to code can now more or less spawn personalized software on demand, while experienced software engineers are now empowered to take on even more ambitious projects than was possible before. On the other hand, I understand the grieving reactions from software engineers who feel like AI has killed their passion for the craft, as someone who was also initially drawn to the field because I loved thinking deeply about technical problems. There was real intellectual satisfaction in seeing your program work after sitting in front of a computer screen for hours thinking about how to structure a program or mulling over which algorithm to apply. Nowadays, prompting LLMs and hoping for them to solve my problems in one shot feels more like playing a slot machine in a casino.

As I look back on the last few years as a professional software engineer, it’s crazy just how fast things have changed: the field is barely recognizable from what it used to be 4 years ago. I think this is partly why people in tech can often sound alarmist to those in other professions: software engineers know what it means to have their profession get uprooted by the technology and had to adapt to changes quickly. It’s as if we were the first to get hit by the wave at a wave pool. Personally, I’ve been doing a lot of reflecting during this time: if one job can be automated this fast, what comes next?

I think the big question looming over all of us right now is: what are humans uniquely suited to do in the age of AI? Will there by any areas where humans will ultimately still be needed to provide value? Can AI complement human capabilities instead of replacing us completely?¹ Let’s start by asking the inverse of this question: what won’t humans be doing in the next few years? As I established earlier, we’re already seeing this play out in some quantitative fields like software engineering and more recently, mathematics.

Recent breakthrough proofs driven by AI have forced mathematicians to seriously contend with the possibility of humans being entirely displaced from the field. As far as I can tell, reactions among mathematicians are mixed, but skewing towards the negative. Still, there are some optimistic takes, ranging from Christian Szegedy’s exhortation for mathematicians to take on bigger, more ambitious problems with AI, to hopeful-but-wary takes like Kevin Buzzard’s argument that AI might take over theorem proving but may still need human involvement for theory building or conjecture formulation, both of which he views as being central to mathematics.

As an outsider to the field, here’s my take: in the short term, mathematicians should be in the best position to direct AI to solve open and important problems in their respective domains, understand those results and formulate new conjectures based on the findings. Those without the requisite mathematical backgrounds, like me, don’t know what to ask AI to do even though publicly available models are clearly capable of solving open math problems. So it seems likely that mathematicians will still be needed in this capacity, at least in the short term. Within the mathematics community, AI might lead to a redistribution of opportunities as less experienced mathematicians (e.g. grad students) may suddenly find themselves in a position to solve, with the aid of AI, very difficult problems that might’ve taken them decades more to tackle.

In the long term, things are less certain and depends on whether the rate of AI progress will continue at this pace. Assuming it does, I don’t see why AI won’t be able to build mathematical theories or formulate new conjectures, and autonomously produce an ever increasing body of work in the field; this may be accelerated by some shared infrastructure for agent-to-agent communication, like an online scratchpad/forum for agents to discuss their findings, collaborate and track their findings. In this future, the role of human mathematicians may largely recede to the role of coming together to understand the AI-generated proofs and to establish and communicate their significance for the rest of society.

One common factor among domains where AI have had the most disruption is that they are easily verifiable. In mathematics, a proof is either right or wrong, and thanks to automatic theorem provers like Lean, verification in mathematics is cheap and fast. Similarly, in software engineering, it’s easy to verify that a program works as intended by writing tests and ensuring the code passes the tests. The more easily verifiable a task, the easier it is to perform Reinforcement Learning on Verified Rewards (RLVR), and thus AI models will become better at it and even recursively self-improve. Or as Peyman Milanfar puts it:

Speed of automation is in inverse proportion to the entropy of the subject domain.

Still, problems in some domains are more complex when it comes to verifying correctness: think something like CAD modelling. Industrial CAD models can be very complex with multiple moving parts, so how do you verify if the model has done a good job? This is what a new cohort of startups like Normal are trying to solve. The young startup recently released the FrontierCAD benchmark to more comprehensively measure AI's capabilities in CAD modelling: their parts benchmark measures things like whether the CAD model behaves as expected when applying rotations, while the assembly benchmark verifies whether the AI model is able to take an assembly drawing and STEP files of parts and assemble the components together according to the drawing.

As such benchmarks and verifiers get built in the coming years, more complex, long-horizon tasks like CAD modelling should eventually be rendered automatable via RLVR. We might then might expect that most economically valuable tasks can and will be automated in the limit through human data à la Mercor as long as the task is verifiable. Depending on whether this takes 2 years or 20 years, the implications for economy and society at large will look very different.

Conversely, tasks that are more “fuzzy” and thus harder to verify should in theory be harder to completely automate. Take for example, making video games. Nowadays, you can easily vibe-code a video game: the latest models like Opus 5.5 and GPT-6-Astra have impressive spatial reasoning, coding and tool use capabilities that enable anyone to make a game. But I'd argue that it's still hard to make these games fun. Anecdotally, I can’t think of a vibe-coded video game that has gone mainstream. Why is that?

For one, it's not cheap to verify whether a game is fun. One way I suppose you could attempt this to get many play-testers to try the game and score it based on how fun it is to play. This might be a fine proxy for how fun the game is if it is done at large enough scale but this would be prohibitively expensive and slow unlike verifying code or a math proof, so it wouldn't work in a RLVR setting. Another problem with the approach is that the rewards would be noisy: different people have different taste in video games. Lastly, the design space for making video games is huge! Even if we get somehow manage to obtain reward signals we want to optimize for, how would we know what to change during training? A video game has many components— mechanics, level design, music, story, and so on— that interact with each other in subtle ways to create a cohesive experience for the player who might perceive the game as fun and worth playing. Therefore it seems likely that fuzzy, messy, nuanced hard-to-verify jobs like being a good video game designer will be less susceptible to disruption by AI.

One caveat is that this assumes that we won’t see another step change in AI capabilities in the coming years. Things we describe as “fuzzy”, like taste, creativity or fun, may well fall to forces of AI automation through innovations in post-training. Anthropic, for one, seems to clearly have made some progress in instilling something like a sense of taste/aesthetics into their latest model releases; my X timeline was recently filled with examples of AI creations have ostensibly crossed some threshold of aesthetic appeal, like this viral music video created by Opus 5.5.

So far I’ve been talking about cognitive labour, where AI is causing the most disruption right now, but the discussion wouldn’t be complete without also mentioning physical labour, which constitutes a significant part of the economy. For now, I think it’s safe to say that humans are generally better than robots at physical tasks, like mixing cocktails. On the other hand, the case for robotics is stronger in more constrained environments like factories and laboratories, where the environment is less dynamic and robots only need to perform a predictably limited range of tasks. My guess is that job loss due to robots will meaningfully accelerate in such areas, driven by capitalistic pressures to cut costs and enabled by rapid advancements in robot foundation models. Broader diffusion of robots into our everyday lives, such as in our homes, will, I suspect, take longer still. The physical world has a long tail of complex, rare edge cases, many of them with serious safety consequences if the models fail to generalize, so I think it'll be a while before robots win our trust in such environments.

Instead, robot adoption will likely look like a gradual ramp, as robot capabilities progress and we become increasingly comfortable with delegating larger parts of our daily lives to a robot. Matic seems to be following this ramp-up strategy by first building a good vacuuming robot, establishing a customer base that derives value from it, and eventually expanding into other more form factors for household robots.

Personally, I am a huge fan of dishwashers and would love to see more advanced robots automating away more tedious chores that we do for maintenance, like cleaning or throwing trash out. It would also be great to have the robots perform dangerous jobs and save human lives. My worry, though, is a future where humans lose all economically productive cognitive labour to AI but nevertheless continue to outperform AI-enabled robots in physical labour. In such a scenario, humans might be reduced to meat machines that execute tasks in the physical world in response to instructions from super-intelligent machines. It would be good to avoid this version of the future.

An important question, then, seems to be: will humans still be doing cognitive labour in a post-AGI world? If so, what would that look like? Charlie O’Neil believes that the final form of (cognitive) work will look like humans directing AI to solve problems they care about, i.e. the work will shift to the meta-problem of deciding what to solve. This is the future I’d like to see, where AI liberates us all from minutiae of implementation details and empowers us to solve bigger, grander problems based on our personal and collective sense of what’s important.

Reflecting on what it means to be a human living in an age where intelligence is commoditized, David Bau posits that “as humans, our role is to care”. It’s something I’ve been thinking about recently, too. When I think of a future where androids roam about in the streets alongside humans, I often wonder about what motivations, goals and objectives these androids would have. Humans wake up every day and go about doing what they do because of many different reasons: some people work hard because they want to leave a lasting legacy, some work out and build their bodies to attract a mate, etc., and we all eat food because it’s pleasurable and we would go hungry and starve otherwise. Presumably, AI would neither feel the need to procreate nor experience hunger. What, then, will AI work for if not for values that humans instill in them? Human beings may be less efficient and less intelligent than AI, but we are deeply complex beings in ways that I think are under-appreciated relative to intelligence. In particular, I think we can agree that we all have an innate capacity to care about things, even if these are different from person to person.

So that’s where my mind is at. In the last four years, the world has changed and along with it, my work. As it turns out, intelligence isn’t as sacred or special as we thought; what’s durable might be the human capacity to care about something and the agency to apply yourself to it with whatever tools you have. It’s a humbling experience. It also presents many societal challenges that we’ll need to navigate as a species. But overall, I am optimistic about the future and feel grateful to be living through such a crazy period of rapid transformation. It’s both scary and exciting. It feels like the old world is dying and rules are being re-written. Personally, I’m choosing to take a step back to think about what I care about, and give myself the room to be excited about things again.

I suspect that’s the most important question for all of us now: what do you care about?