Rendered at 15:24:44 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
avinoth 2 days ago [-]
This is like one of Google’s infrastructure blog. Google scale things are only relevant and appropriate for them. We had lots of bitter lessons from smaller engineering teams following the process with the justification that “this is how Google does it”.
Similarly, Anthropic can afford to do this because they have virtually unlimited tokens and labs to train models for their specific usecase. The rest of us need code comprehension lest we become dependent on continuous supply of tokens (which is a state these labs badly want to achieve)
So, unless you work at a lab, or have unlimited tokens guaranteed through the life of your company, it would do both the company and you good to continue to read the code and not let these 30k PRs per engineer be the norm.
An excerpt from the article makes my point
> Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to. When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today.
thatsabadlook 2 days ago [-]
Even then do we really have evidence that what they are doing is reasonable or practical for them at all? They are edging on like 3 9s uptime, what leaked of Claude code looked like soup, tons of security issues, they can't build a sandbox to save their company, and last I heard they aren't even profitable without adjusting their numbers for arbitrary things.
Think I'm good on taking advice from companies that aren't making money.
avinoth 2 days ago [-]
Exactly. What they’re doing, repeatedly, is setting a narrative that this is how software engineering should be. Always be reliant on tokens. Don’t worry about uptime or errors - agents can address it, don’t worry about security - agents can address it, don’t worry about code comprehension - agents will always be there for you.
samuell 1 days ago [-]
Nailed it
oblio 1 days ago [-]
> agents will always be there for you.
In Romania an investment fund (FNI) around 2000 ran some super famous ads saying "FNI is watching" (FNI is keeping watch).
6 months later they went bankrupt.
ban_this_man 1 days ago [-]
[dead]
panarky 1 days ago [-]
If they're cash-flow positive with 75% of compute going to research and training, what do you think will happen to cash flow when they shift to 50-50 training-inference?
Because "pacing the frontier" is more about flipping on the money printer switch than it is about safety.
thr0w 1 days ago [-]
> they ask Claude to write an explainer for exactly that slice
If their Claude is the same as our Claude, then this is a joke. Claude’s writing style is insidiously mind numbing. It takes constant pushing to get it to explain things in digestible English.
bulbar 23 hours ago [-]
It also mixes important with unimportant information. It tends to think that a commit/bugfix is important and will happily list it together with important properties of the feature.
At least Opus 5.5 does that. Its writing is better than that if other models, though. I had other models explain stuff in an undigestable language, technically correct, but practically not what anything is about.
elondaits 20 hours ago [-]
I hate it when it realizes something that was contextually obvious to me, and then treats it like a precious nugget of wisdom, including it in the README, the comments, and the commit message. Even worse when it’s a hypothetical situation. “Even if the raw chicken is bagged and refrigerated, we want to avoid a leaky bag.”
bulbar 16 hours ago [-]
I believe this is a variant of it: I dislike how often it mentions stuff that's not there or relevant (anymore), just because it was referenced somewhere:
"We use a double instead of a float there so that the addition will not overflow". Yeah, thanks, that was a bug 2 years ago, nobody cares that the line now does what it's supposed to do.
therealdrag0 13 hours ago [-]
Today I had AI do a design doc cleanup and conciseness sweep on a vibe code project I have.
It removed ~50% totaling 30k words! So annoying how everything I discuss with AI gets documented in 10 the length, and then every subsequent decision is documented in 5 places.
froggy 1 days ago [-]
Try having it explain something in “ISO Plain Language standard”. It made an enormous difference for me yesterday. The language will change from dense large paragraphs to simply-worded 2-3 sentence paragraphs. I was using gpt though.
engine_y 1 days ago [-]
It's like it's trying to impress you with poetry in comment:
"What the function does, it does well, and only just once"
panarky 1 days ago [-]
v5.5 eliminated much of the impenetrable prose of earlier versions.
thr0w 24 hours ago [-]
> v5.5
You don't still feel that mental friction when 5.5 responds with a wall of text and you drop into that first tedious sentence? I even had it generate a syntactically invalid English sentence the other day. It's still pretty bad.
wowoc 1 days ago [-]
It’s better, but still worse than OAI’s models in my experience
mbel 1 days ago [-]
Or we’re just getting used to it.
meheleventyone 23 hours ago [-]
Even getting it to write less obtuse English isn't enough IMO. It seems to have real issues with structuring documentation in a way that makes it easy to follow. Also it's a real demon for leaking its context as implicit assumptions into documentation.
Starlevel004 1 days ago [-]
The people who work at Claude seem to talk in claudlish, so I doubt they care.
IshKebab 1 days ago [-]
They pretty much fixed that. Plus you can influence the writing style to some extent. I tell it to use full sentences and avoid assuming I already understand the code.
Another thing is that it's very good at creating diagrams (I think someone wrote a skill for it so maybe not by default). I usually find the diagrams are even more valuable than the text.
Honestly I was skeptical but you really can just point Claude at some code and ask it to make a markdown + SVG explainer and it will do a shockingly good job.
I'd still rather read human documentation, but even before AI almost nobody writes any - especially for internal code. And they certainly don't make great diagrams.
fibonachos 1 days ago [-]
I’ve been using diagrams a lot lately. My team currently owns a wide surface area, so there is a lot of mental context churn as we bounce between parts of the system. HTML diagrams, or artifacts in the Claude desktop client, are super for helpful for bootstrapping/refreshing my mental model when starting a new project.
If we as engineers didn't use similar terminology to describe any code not written by ourselves for the last few decades perhaps people would take it seriously.
Tanjreeve 1 days ago [-]
Claude was trained on engineering wittering.
oblio 1 days ago [-]
Fairly sure all LLMs were trained on ads and sales and marketing copy as there's probably 1000 times more of that on the internet.
ban_this_man 1 days ago [-]
[dead]
sixtyj 1 days ago [-]
But the words are all right /s
If it was an exception only… but it is so common that I had to put ASD-STE100 into instructions to have precise answers.
sadengineer22 1 days ago [-]
I have heard from someone who works at Anthropic that they will have postmortems that sometimes devolve into "my claude says X" vs "well my claude says Y". Maybe this is the future of software engineering, but it seems pretty fucking sad.
Wowfunhappy 1 days ago [-]
Seems like a good time to get a third Claude to hear the cases of the first two Claudes and break the tie.
bulbar 23 hours ago [-]
Make it five so that you can write a blog article about how your use of AI is extra-secure and top game.
strken 1 days ago [-]
This is basically just being a manager in the before times. "My engineer said X!" "Well, mine said Y!"
Which is fine, whatever, I guess this is how I pay the mortgage off instead of having any clue what the hell Claude is actually writing. Hopefully I can semi-retire soon and go study dunnarts or take up welding or something else that's more rewarding than this nonsense.
Rapzid 1 days ago [-]
I would love to ask them why this scale is even necessary for a conceptually simple CLI app like Claude Code, but they wouldn't be able to answer because they don't even understand the scope of it apparently.
thatsabadlook 1 days ago [-]
It's absolutely insane that they can't keep track of a CLI application that makes shell calls. This is so far from rocket science even mentioning the term is misleading.
sebzim4500 1 days ago [-]
To be fair, rocket science is also easy. You just combine a propellant and an oxidiser in a reaction chamber.
In fact, everything is easy when you summarise it in one sentence and don't think very hard.
thatsabadlook 1 days ago [-]
Yea not at all. There are dozens of oss Claude code clones written by 1 person with a few contributors that work as well if not better. So it's really not some massive challenge. A decent cs student with enough coffee could get 40% the way there in a week without any llms. The same can't be said for rocket building or even design...
bulbar 23 hours ago [-]
What people believe are the core features has never been difficult to implement. Building "something like" Twitter or facebook or reddit has always been possible within a few days.
It's an iceberg, most of the complexity lies underneath the obvious stuff.
avinoth 12 hours ago [-]
But in Twitter or Facebook, the complexity is not in the app but with the scale. How do you support the network, graph, querying, fetching, etc. That is not the case for a self-contained binary like Claude code CLI that talks to the same API irrespective of 100 people are using or 1M people are using around the world.
thatsabadlook 17 hours ago [-]
No like there are dozens of open source Claude code alternatives that are as good or better than Claude code. This isn't a theoretical thing. It's not new either.
bulbar 16 hours ago [-]
Same goes for the reddit and twitter clones.
Do you use one of those open source alternatives?
thatsabadlook 13 hours ago [-]
Not a clone but yes I use alternative social media from twitter and reddit along with a few million other people. Its nice.
22 hours ago [-]
ban_this_man 1 days ago [-]
[dead]
swaptr 11 hours ago [-]
They do have quite a few products now, the desktop apps, websites, cowork, cloud session sandboxes, API, enterprise, etc. have their own gotchas, but after a core development, the 30000 commits a day does not feel justified, or is that an online training dataset prep, that's hard to tell. That's why I feel with their endless tokens, it feels just petty marketing and that's sad because it does not bring much to our engineering table.
unddoch 1 days ago [-]
It needs to gather a bunch of inputs and feed into the remote GPUs correctly and as efficiently as possible. Lots of room to optimize under the constraints that users love cheap tokens, demand a fair billing structure, and want responsiveness and low latency.
jbjbjbjb 1 days ago [-]
I don’t think you can infer the scale for Claude Code alone. They said it doubled but not what the starting point was. The 30k figure is for the entire Anthropic monorepo.
Rapzid 16 hours ago [-]
Well I personally didn't say anything about 30k PRs. I still stand by my belief the "known" scope of Claude Code doesn't seem to warrant the scale of "vibe coding" and lack of comprehensive understanding they are communicating to us.
volkercraig 12 hours ago [-]
30k commits going into a monorepo would make me want to kms having to rebase the fucking PR's every 35 seconds.
1 days ago [-]
tayo42 1 days ago [-]
Is Claude code just the cli or does it include a bunch or backend services supporting?
scottyeager 1 days ago [-]
It's also a desktop app, integrated into the mobile app, and it has cloud hosted session capabilities too. Plugs into Slack and Github, maybe others. So definitely there's backend stuff as well, beyond the inference.
ban_this_man 1 days ago [-]
[dead]
cheema33 1 days ago [-]
> Nobody on the team keeps a complete mental model of Claude Code in their head.
For sufficiently large codebases in large organization that is how it has been with humans, who come and go. In many respects, What Anthropic is doing is a massive upgrade from the status quo.
jt2190 1 days ago [-]
Yes this is absolutely true.
Another challenge is getting developers to stop assuming they know what the code does without reading it! Having AI “fact check” various documents against the source will help immensely.
myegorov 24 hours ago [-]
Best developers can run the code in their head without referring to source. It's called a mental model.
ares623 1 days ago [-]
Really? Hundreds of billions of dollars in debt is an improvement? The product that this code is supporting can't even pay for its debt.
Jesus. Give me 1 billion dollars and I'll keep your entire codebase in my head + personal notes for you until I die. Hell, I'll even force my descendants to do the same. Now _that's_ an improvement.
cheema33 10 hours ago [-]
> Really? Hundreds of billions of dollars in debt is an improvement?
We are not debating Anthropic finances in this article.
ares623 9 hours ago [-]
Interesting. That's like an exact same method on how they're advertising their finances. "Our expenses are irrelevant for the IPO"
glouwbug 22 hours ago [-]
Pharaoh vibes
suttontom 23 hours ago [-]
It would benefit many people (especially Anthropic employees) if, after an LLM gives you a response like this, to go into a different session and ask another LLM if the explainer is correct. Not just so that you hopefully get the correct information, but also to see how easily fooled you can be by a plausible-looking, confident response about some code that you have no reference for. Use 5.5 on any thinking level, you will encounter this. The spaghetti mess that is CC likely makes this more of an issue.
Tanjreeve 1 days ago [-]
Google's infra genuinely pushed the industry forward though, things like kubernetes and SRE methodology are generally useful even if you aren't Google. Reading this I'm not even sure this sounds like the best way to run an organisation with infinite free LLM tokens let alone an actual engineering org where it just an impact on development cost. The people who are getting maximum leverage from LLM reasoning and routing queries intelligently who've made big strides in reducing inference while increasing throughout are the ones who impress me the most.
bossyTeacher 1 days ago [-]
[flagged]
mk12 2 days ago [-]
I saw someone say the other day (in response to another blog post being well written) that Anthropic employees have a rule against using AI for writing/external comms? That seems clearly not the case here, normal people don't write like this:
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
jazzyjackson 2 days ago [-]
100%, although if people are getting 90% of their interaction from a robot, one may start to pick up some vocal tics and colloquialisms from machine world. a kind of robo-pidgin.
talldeck 1 days ago [-]
English isn't my first language, and I'm treating every opportunity to use the language as a way to learn, including extensive discussions with AI at everyday work. Sadly, I'm not noticing any unnatural wording in its replies.
I'm starting to worry about being gradually affected by AI's style through heavy use. It will make me so sad if one day my written piece sent to others is considered to be AI-generated, while it's written with real effort trying to make it sound natural.
grosswait 22 hours ago [-]
Just accept that it will happen, and it will likely happen to all of us. It has already happened to a colleague of mine, and they are a highly educated native English speaker.
TeMPOraL 1 days ago [-]
Irony being, robots are averaging our best speech patterns. So we're really just syncing humanity's speech patterns, with LLMs acting as proxy.
bonoboTP 16 hours ago [-]
No it's not averaging. That's only pretraining. Post-training is RL for high approval ratings which isn't necessarily what people speak like but what was rewarded.
jazzyjackson 22 hours ago [-]
Yes this homogenization of not just way of speaking but way of thinking is very undesirable to me. But hey some people might accept the borg.
cjkhowes 1 days ago [-]
I've definitely picked up some weird claudeisms. I even know the em-dash keyboard shortcut now.
wffurr 1 days ago [-]
Macs make it really easy to type a certain few special characters -- em-dash among them. I'm typing on a Windows laptop and I know there's alt+numpad codes but this laptop doesn't have a numpad and I can never remember those. I used to know a few many years ago, but I've forgotten all of them.
Access to a wider array of special characters is also one of the really nice things about touch keyboards on mobile or tablet. Keyboards are great for fast, accurate typing but not for going beyond the basic character set.
nickspacek 1 days ago [-]
I used to use semi-colons and parenthesis in my semi-formal writing, but now I find myself reaching for em-dash instead. I'm not sure if my tone has changed as well but LLM-writing has probably influenced me more than I realize.
levkk 2 days ago [-]
Yuuuup. Which I guess makes it even harder to detect what's AI and what's not.
timacles 24 hours ago [-]
Detecting what is AI and what is good writing is easier than ever now that the AI common speech patterns are established. Reading a well written piece it’s like walking through the park on a beautiful day in comparison to dragging your legs through quicksand
Pangram 4 is quite a bit better (that post is about the previous version). IME it’s quite accurate for anything over 500 words or so.
wisdomvictory24 2 days ago [-]
[dead]
lavezzi 2 days ago [-]
> That seems clearly not the case here, normal people don't write like this:
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
cowboylowrez 1 days ago [-]
agree 100 percent, I write prompts, and at some point I'm going to adjust my style when I see my prompt is ambiguous or something and I bet subconciously I'm going to try to talk like my robot pal when making these adjustments so I could see it happening
jansport123 2 days ago [-]
this definitely seemed like something claude generated, i know because i frequently do all my research in chat and then ask it to generate a doc based on what we talked about and when i read what it generates the verbosity, the style it writes in makes me delete the whole thing. I would never subject readers to that.
atty 2 days ago [-]
It felt weird reading it because every once in awhile there were some well written sentences and coherent paragraphs, then it fell back into Claudisms.
myth2018 2 days ago [-]
As a non-native, I see how susceptible I am to absorbing such style. There's no way at all for me to perceive anything wrong with the examples you present
mk12 2 days ago [-]
It's a bit more subtle than things like "it's not X, it's Y". And even that isn't "wrong", it's just the way AI talks, and therefore an indicator that the author offloaded their thinking to an agent. I've read so much Claude output that it's easy for me to recognize its voice, the same way I could recognize a friend's voice in writing. The examples I quoted are a weird kind of circuitous sentence structure with a certain shape and rhythm that scream Claude to me. Here is how I might rewrite them in human speak:
> Most of the path is automated; a person's attention goes where the risk is
--> We automate as much as possible, focusing human attention on risky parts
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
--> We run reviewer evals against every incident
But that's just my interpretation and my voice. I can't know for sure what the authors would have said if they took the time to write it themselves.
myth2018 1 days ago [-]
I see. Yeah it's a bit annoying. In my mother tongue, Portuguese, I feel a visceral pain when I read what I classify as bad style. But I never developed such esthetic sense with the other languages in which I can communicate in any level
versteegen 1 days ago [-]
Personally, a year or two ago writing like that did not really come across as particular bad or AI generated to me, although it was uncommon for humans to write phrases like that. Writing in an unusual way is not inherently bad, it can make your text interesting... but it is less direct and clear than alternatives. But the style becomes unpleasant if you keep seeing the same patterns over and over such as "X'ed against all of it"
wannabe44 2 days ago [-]
It feels like it's trying to sell you something in every sentence.
drewbeck 21 hours ago [-]
Yep. It’s the consistent emphasis in every statement that is so draining to me. A human might use a similar tone once or twice in a piece to drive home a particularly important point, but Claude does it with every point. It’s exhausting.
jambalaya8 21 hours ago [-]
agreed.
timacles 22 hours ago [-]
It’s optimized for emotional reaction instead of communication
bonoboTP 16 hours ago [-]
Heavy emphasis on "every" like this is my new Claude smell, after honest and genuine seems to be reduced.
afro88 21 hours ago [-]
The one that twigged me was in one of the first diagrams:
> Gradual release
> The latest channel first; the stable channel trails it
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves.
sebzim4500 1 days ago [-]
I don't think that's a claude-ism because it makes sense to cover this point at the start of the article as a defence against overexcited journalists.
It's only a claudeism when it says "X is not a Y" when no reasonable person would have been expecting that X could be a Y.
brof 22 hours ago [-]
Fair, I just feel like I see negated statements significantly more in AI than in human writing. (And, to me, it's poor writing: if it's not an X, it doesn't tell me much, but that's a personal preference.) The newer models also seem to try to stay almost excessively neutral and avoid over-claiming, by writing explicit disclaimers, like this does. So while we employed these patterns in human writing before, I may be becoming overly sensitive to AI writing smells now (like seeing the word "unusually" in any text). Thanks :)
wffurr 1 days ago [-]
If you spend all day reading Claude-written "explainers" of your own codebase, I can imagine the "house style" would rub off on you.
userbinator 1 days ago [-]
What came first, "Claudish" or those who trained it? I wonder if Anthropic's employees originally wrote like that. It's definitely irritating to read, and IMHO more so than the ESL-grade English which was quite common before AI.
MisterMunchkin 1 days ago [-]
Claudeslop didn’t come from humans, it’s an artifact of running RL on slop outputs. If you keep feeding the slop output back into the model and then training on it, it amplifies its own unique weird vocabulary and behaviours.
damontal 1 days ago [-]
The word “gate” was used.
2 days ago [-]
ta93754829 1 days ago [-]
load bearing comment
fn-mote 1 days ago [-]
To be fair, that phrase does not appear in the article.
status_quo69 2 days ago [-]
No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
jonahx 2 days ago [-]
> have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
patrickdavey 2 days ago [-]
"Tbf, it's worth pointing that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
daveguy 2 days ago [-]
Not only that, when you have a human designed and implemented system you know it's at least coherent enough that someone understood it at some point well enough to take responsibility for it. No such assurance with the llms.
dmd 2 days ago [-]
I'm guessing you've only ever worked at startups?
Large company codebases are very much not like that. I've worked on systems (e.g., banking) that have accreted still-running code for as much as 60 years. Nobody understands how it works. Nobody ever understood how the whole worked, and many of the parts were never actually intended to work together.
hnedeotes 1 days ago [-]
This is mostly a self made and kept problem. I also worked in codebases where the current developer(s) didn't know why things were the way they were and would take days to dig into why some part wasn't working - but the issue was, instead of taking the time to write tests and document, even if only when doing these point in time fixes, they just did the ad-hoc thing and solved whatever issue there was at the time without any worry whatsoever.
At some point I was tasked with improving the queries/retrieval times and started converting parts of the code and not only was it possible to write tests that would convey the actual meaning of the code and crystallise it but I started finding bugs in the implementation that no-one realized were even bugs because they no longer knew what it should do when queries grew to certain complexity and the search space was in the hundreds of millions of rows that thinking through them just with napkin calcs wasn't cutting it (specially when the bugs were subtle and the results still allowed the program to work "mostly" correctly).
hxugufjfjf 1 days ago [-]
And this holds true for most of the Internet. There is no one person who truly understands how it all works: DNS, routing, and all the other things we rely on to get our systems to work.
leonidasrup 1 days ago [-]
Network engineers and TCP/IP stack implementers have pretty good understanding of how the Internet works.
But I have see quite terrible finance software, patches over patches hold together by a duct tape. Also finance software has to implement large complexity of tax law, which is very large and always changing. It also doesn't help if the company developing finance software sees software development as a cost center and wants always to push the costs down (especially short term costs).
mqus 1 days ago [-]
Sure that happens, but over _years_. LLMs can make this happen over _days_.
csallen 2 days ago [-]
But is this really a problem? If you have testing and you have version control via Git, you can roll back changes and investigate them.
I also don't even know if it's really true. Sure, maybe it's not coherent enough that a person understands it, but if an LLM wrote it, you can be ~99% certain (in my experience) that that LLM will understand it, and usually in less than the time it took to write it. And unlike humans, the LLMs don't die and take their knowledge with them, or change companies, or get ornery when you ask them to explain.
Also, in my experience, Claude doesn't really do what the parent is saying and dive head-first into complex problems and bork your code. It is fairly humble in the face of complexity. It constantly recommends breaking down larger problems into smaller ones that are more easy to grok.
But most importantly, anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful. Especially when they're writing code that's important. Unless you're an AI researcher pushing the bounds of what LLMs can do, which let's face it, the vast majority of programmers are not, it's almost always the case that you'll have your LLM writing intelligible code that you can understand and have it explain easily enough.
evolve-maz 1 days ago [-]
If anthropic is making as many PRs as stated in articles, then rolling back is unlikely an option. More likely it'd be a new commit to fix it, and with a feature flag to preserve the "wrong" behavior (even if wrong, since people will probably depend on that function since its original behaviour).
Those sorts of changes are really rough to manage. Maybe still manageable by LLMs but we'll find out.
myegorov 1 days ago [-]
Is the Hugging Face incident from July really a problem? Or Australian healthcare portal breach? Who let their guard down unable to detect or roll back the change for days on end?
csallen 1 days ago [-]
As I said at the end of my comment, unless you're an AI researcher pushing the bound of what LLMs can do. I don't think most of us are running ten million dollar AI swarms with unreleased models that are escaping their sandboxes. That's research, not software engineering, and this discussion is about whether or not software engineers can reasonably expect to understand their LLM generated code.
Also, for what it's worth, LLM research is hardly the first scientific research to have accidents with negative external consequences.
myegorov 23 hours ago [-]
Frontier labs are not the first to socialize externalities and privatize the gains. Not even the first to shift the responsibility on to some non-entity (agents, corporations). I think you have a very reductionist view of software. Software is used to design the buildings you inhabit, bridges you cross, vehicles you drive, medical devices you wear...
dieortin 2 days ago [-]
> can roll back changes and investigate them
Sure, you can do that, but irreparable harm might have been already caused. Sure, that’s unlikely when your product is an LLM wrapper. But in something like a database, it could be catastrophic.
csallen 2 days ago [-]
I mean it could be, sure. But I think people are generally speaking more careful with higher stakes code than they are with lower stakes code. I know a lot of people who are YOLO'ing it with LLMs, but who ask for more understanding, more reviews, more tests, etc., in trickier areas. Which is only natural. Just like people are more careful when carrying a baby than when carrying a pillow.
(Yes, there will be individual exceptions to this, just like with everything else in life.)
what 1 days ago [-]
> Just like people are more careful when carrying a baby than when carrying a pillow
lol, I assume you’re not a parent then. Or at best a new parent.
hxugufjfjf 1 days ago [-]
What do you mean, exactly?
TeMPOraL 1 days ago [-]
They're talking about operational benefits of having backup kids.
timacles 19 hours ago [-]
And rolling back changes
jdlshore 2 days ago [-]
> anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful.
Is this a definitional argument? As in, if you’re not thoughtful, you must not be a decently skilled engineer? Because I see no reason to believe that people are going to be thoughtful about LLM-generated code. Just the opposite. It’s incredibly tempting to LGTM it.
csallen 2 days ago [-]
> I see no reason to believe that people are going to be thoughtful about LLM-generated code.
I think that's true for low value code. But the more important and crucial and high-stakes code is, the more effort people put into it, generally. Even with LLMs.
pofferfish 2 days ago [-]
I would love to know what the Venn diagram of people who complain about people not understanding their code, to people who can't describe the boot process of their system is.
customguy 1 days ago [-]
"their code" == code they actually write, that originates with them, "their system" == anything, from gentoo to something off-the-shelf
So the real question would be, how many people designing / implementing the boot process of any given system don't understand what what's going on.
marcindulak 1 days ago [-]
Here is an explicit example of the additional lack of coherence due to LLMs in the Anthropic Claude Code public repo itself: the human maintainer trusts LLM's incorrect understanding of the current state of code in the repo.
> "Issues here are auto-closed by a bot when they've had no activity for a while and are missing information needed to reproduce (usually a version and steps, or a /feedback ID). Nothing is triaged solely by counting"
Therefore, I wonder whether the internal review process described in the Anthropic's blog for internal repos also results in such issues, and whether Anthropic has a internal way to measure and mitigate that. The blog post says:
> "When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today."
The Claude Code public repo problem shows that LLMs don't always "reflect the code as it is today", and can be a source of additional misunderstanding.
daveguy 1 days ago [-]
Exactly this. Using an LLM to understand the original LLM slop is just another chance to get it wrong 20% of the time. These things just aren't reliable.
SR2Z 2 days ago [-]
I am not sure that "someone understood it well enough to take responsibility" means what you think it means. I have definitely come across old systems which have had addons bolted on to them in so many places that the overall behavior of the system is unpredictable and only mostly works.
daveguy 1 days ago [-]
"I've seen bad code in the past" does not address the fact that human code by an experienced software developer is almost always less complex and more clear, with less repetition, dead code paths, and odd architectural choices than what an llm generates. If you don't recognize this, then you either haven't been reading the code or are not an experienced software developer. 80% success on a benchmark means they fail 20% of the time in the exact domain they were trained on.
SR2Z 12 hours ago [-]
I am an experienced software dev, and I am telling you this as clearly as I can: human generated code is oftentimes buggy, indecipherable slop.
You need to put 80% in context. There's a reason why FizzBuzz lasted as long as it did - lots of developers, even senior devs, are terrible at writing code, subjected to absurd time constraints, or required to color inside the lines of a leaky abstraction. Sometimes all three!
I've seen many situations at my current job where asking an LLM to do a refactoring made things much better - but if I had to do it by hand I'd just give up and move on to more essential work.
pofferfish 2 days ago [-]
Someone understanding something and the codebase being coherent are different things. I can only think that you have been lucky with your career placements if you are making that statement.
What you are saying is leaning on 2 premises that seem to not hold water
1. Code written by people is always more readable and maintainable than LLM code
2. Code written by an LLM is always less readable and maintainable
Can you really make those statements in all intellectual honesty?
daveguy 1 days ago [-]
False assumptions.
The human doesn't have to always be more readable, to be usually more readable. And vice versa. LLMs given the opportunity to generate a "feature" almost always generates more complexity than a software engineer would develop.
antihipocrat 2 days ago [-]
I'd like that to be true, but many software service startups were stitching together node packages and selling the result long before LLMs became useful.
cmrdporcupine 2 days ago [-]
> you know it's at least coherent enough that someone understood it
30 years working in this industry and I'd say that's the exception rather than the rule. When it comes to large systems.
monkpit 2 days ago [-]
Yes, [citation needed]
aturek 1 days ago [-]
> You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
I love this phrase, and I think it captures the difference between the "60 years of legacy code" mentioned elsewhere here, and the vibe-coded monstrosity that we all know we're going to be encountering in the next 5 years. The fact that there were human brains with intent making all the little (short-sighted, myopic, ignorant, etc) changes in the legacy code means that you DO have at least some shot of gaining enough context to make safer changes.
There is no real intent in LLMs and particularly in blind vibe coding, so you'll end up reading stuff that works, or appears to work, but there was no human brain behind the change for you to model and thus predict.
TeMPOraL 1 days ago [-]
This just means we can't take the (often bad) shortcut of guessing what the code does because we feel we can predict what the author was thinking. Instead, we have to treat existing code as a natural sciences exercise - use the same methods we apply to the world around us, which has its logic and structure, but no human intent behind it.
jonahx 1 days ago [-]
GP is saying that "the search space" of "the natural world" is much bigger than the search space of "stuff created on purpose by humans". In both cases, there will be guesswork, but in one case less.
I think there is something to this, but am even more concerned by the pure scale. Consider 100k, 1M, and 10M LOC human legacy projects (already a big problem, usually!), and now compare to the likely outcome of 100M, 1B, 10B, etc LOC llm projects. That's where you may start to see the outcomes I reference in my OP.
TeMPOraL 21 hours ago [-]
> GP is saying that "the search space" of "the natural world" is much bigger than the search space of "stuff created on purpose by humans". In both cases, there will be guesswork, but in one case less.
I agree. What I am saying is, we have effective tools and processes for each of these search spaces, so code no longer being "created on purpose by humans" is not the end of the world.
jonahx 14 hours ago [-]
> we have effective tools and processes for each of these search spaces
Do we? Science (the tool for the natural world) is limited and there's so much we don't understand.
The recent superhuman achievements by llms in math are promising, but what we've seen so far isn't enough imo to YOLO building arbitrarily large systems in code and just trust that the llms will be able to handle it. Especially when the stakes are high (human lives, etc).
TeMPOraL 7 hours ago [-]
> Do we? Science (the tool for the natural world) is limited and there's so much we don't understand.
Limited in what way? Its basic structure is sufficient to eventually let us discover anything there is to discover. It's a fully general problem solving framework.
jonahx 2 hours ago [-]
> Limited in what way? Its basic structure is sufficient to eventually let us discover anything there is to discover. It's a fully general problem solving framework.
That "eventually" is doing a lot of heavy lifting. It's limited in the most literal, empirical way. We don't have answers to most things. If we stood in relation to a gargantuan code base as we stand in relation to the natural world, with an equal proportion of known vs unknown, we'd be pretty screwed.
As an aside, even with the "eventually" qualifier, the claim isn't true. The generation of hypotheses, in particular, and even sometimes the experimental setups to test them, happens through an "out of band" process -- namely, human creativity and intuition. Which is effectively a computational process we don't understand, and limited in its own way (despite its incredible productions). The history of scientific breakthroughs is the history of astonishing insights out of nowhere, from the brains of individual people, sometimes literally from dreams. The experimental method provides a reliable way to test the insights more or less "mechanically", but it would be mostly impotent without the other part.
peab 2 days ago [-]
most large software companies are already too large for anyone to have a complete mental model of.
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
tripleee 2 days ago [-]
I don't think they're the same. AI code isn't just complex because it's huge.. it's complex because it makes no sense
If you have a large human-written system with a little thought put into it you should be able to find the specific pieces that handle the behavior you want and work on that, and if it's good then changing it should have a small blast radius and be covered with regression tests
In contrast, I've fully vibe coded a 15k line tool for myself with Opus and I have no idea how on earth it's architected after reading through it. It has "tests" but I don't trust them. It's just all over the place with no coherent design to it. It works, but it's making me more and more nervous to change it
skippyboxedhero 1 days ago [-]
You can tell with some of the more recent models that they have been fine-tuned in such a way to produce a lot of descriptions that sound impressive but have no real meaning. In terms of production of semantic text, the recent models are significantly worse than a year ago.
It is not completely useless as a tool but there is still a huge gap in most real-world applications (I work in an org with multiple thousands of devs and I am top 5 AI user consistently, I have used Claude Code since pre-release). If you are building something like an SQL database which has decades of logic tests built by humans then it can work. If you do not have this (i.e. 99.99% of the time) then you still need to be able to issue prompts that give context. And if you start without that context, it will produce shit code in extremely basic scenarios. In particular, humans are far better at producing code that is easy to change (in practice, you need to create unintuitive, non-human rules to govern this i.e. creating many artificial boundaries which would be completely unnecessary for humans).
I am not sure whether the models just capped out in terms of coding or whether they are just trained by people who have never worked on enterprise software but the gap isn't closing. It has just been filled by people who lack the knowledge to identify when they are being bullshitted (the prevalence of consultants suggests that this markets is large).
I have worked on legacy stuff before. In most cases, there is a kernel of reason and logic that can be recovered. I have seen pure AI projects and they are the worst grade of TCS-authored abandonware. They can often do one specific thing but are basically unmaintainable outside of that.
bpye 1 days ago [-]
> You can tell with some of the more recent models that they have been fine-tuned in such a way to produce a lot of descriptions that sound impressive but have no real meaning. In terms of production of semantic text, the recent models are significantly worse than a year ago.
I frequently find this in LLM generated comments, both when using an LLM myself or when reviewing code that someone else used an LLM for. I find it is often the most obvious indication that nobody really read through what was generated.
tored 1 days ago [-]
Yes, it has become quite common that Claude’s text feedback, e.g. Claude found an issue in the existing code that requires a decision and Claude tries to explain it, it is a paragraph of bullshit words stringed together.
Then you investigate it and it is often something relatively simple that could have been explained with short sentence.
skippyboxedhero 1 days ago [-]
Have you found a way to stop this?
I used that engineering language skill which is good when it uses it...but it doesn't like to use it.
...there are clearly some areas of alignment that they are not looking at.
rwmj 1 days ago [-]
We had one that hallucinated benchmarks, with graphs and everything! (The code didn't actually work.)
bhaak 1 days ago [-]
I vibecoded a project without looking at the code as an experiment, and I had the same experience as you. It really drives home the point that, left to their own devices, LLMs produce abysmal code.
If you give them no constraints, they’ll produce code that just works, and it’ll look exactly like that.
We already know the solution to this problem. Software engineering. Put the LLM through the same grinder as any software dev and out constraints into place. Tell the LLM that the code design is bad and that it should improve it.
In my case, I had to go through several rounds of refactoring before the code at least resembled a good codebase.
A fun fact is that the refactorings didn’t break any functionality. Only adding new features broke preexisting behavior.
peab 1 days ago [-]
I don't think this is the case anymore if you're using the latest expensive models, i.e fable.
ithkuil 1 days ago [-]
I'm trying to learn how to write software with agents where I keep control over the design while yielding control over the implementation.
This is not very different from what already happens to architects that delegate implementation to teams.
The challenge Is to find a way to encode the architecture in a way that both you and the agents can understand and agree on.
I'm still learning how to do this effectively, but my initial experience is quite positive. Not because agents are great but because humans are bad; the degree of coherence I've seen agents following is bigger than many human teams I've led in my career.
My benchmark is a quite big project for which the architecture is already well known, namely the reimplementation of the OpenVMS OS (kernel and user-land). There are many traps and many opportunities for the agents to just go ahead and diverge from the goals, so I found it a good playground toearn how reign them in but let them be effective at "drawing the rest of the owl"
majormajor 1 days ago [-]
One of the fun things that you currently need to babysit Claude, etc, on, is that while they've been tuned to hell now to do more testing and even logging and such, they don't currently do enough log-and-debug-oriented specific passes/revisions to distinguish meaningful logging for meaningful debugging from noise that won't necessarily tell you what you need to know.
And you can ask them to address this. And you can ask them to look for that in all your PRs. Etc etc. And that helps a lot.
But it's not something that can be brute-forced in the same way "this failed a test, let me fix it" is. So without a long-term-memory brain holding all the pain of past incidents and knowing and applying ahead-of-time judgement about what to build, it's a challenge to turn the general guidance into the specifics for any unique piece of software.
They're trying to get there of course. We shall see what's next.
("Human code turns into this too" is also a curious rejoinder from the "don't read the code anymore" crowd because there are entire disciplines and areas of study that have emerged and evolved from the problems of large ad-hoc multi-faceted systems being difficult and risky to deal with.)
jonahx 2 days ago [-]
Right, that's what I alluded to in my last sentence.
It is true that this gamble has existed in the form you mention for many years. Still, I sense something new here, both in scale and in the attitude of trust. Also, I find those massive, incomprehensible human-built systems frightening too.
dasil003 2 days ago [-]
Having worked continuously this decade on one massively complex consumer facing app that interacts heavily with real-world scenarios, I can say that the problems of cohesion existed well before the agentic revolution. Tons of features built by myopic teams simply because they couldn't understand enough of the system (both online and offline, meat-space interactions) to make good design decisions. AI actually helps here by being able to perform deeper reading and analysis than any human can do. But then the risk is that it can also produce way more random non-cohesive features.
So far I kind of feel like it nets out to the same bottleneck being good judgment from decision makers. And you might say, no, leadership will just put more and more pressure to add more complexity just because we can and it makes it appear that we are moving fast. But when it comes to real-world apps, there is a backstop which is that things that are too complex and illegible end up not being fit-for-purpose for human problems.
Claude Code and other general purpose AI tools are not like this, because they are literally trying to optimize it for every single kind of knowledge work with zero constraints on use case or target audience other than chasing the next juiciest one they can with essentially unlimited resource. 99% of software is not like this, and when the dust settles I suspect software won't be anywhere near as different as the doomers or the maxxers are predicting right now.
skippyboxedhero 1 days ago [-]
One problem is managing expectations from non-technical people. You can ship faster but you are also producing tech debt at that same faster rate, and possibly even faster as the gaps in your initial implementation/prompts are filled by Claude deciding to do random shit.
In most orgs, there is also a gap between leadership, even tech leadership and people actually using LLMs every day. And it is challenging for actual devs who are being told that they should be going much faster and the reality which is more mixed. Pushing back on people who are convinced that Claude is God is going to be tough if you are a junior.
I don't feel the job has really changed, I don't write code as much but that time goes into working out the structure...which is what I was doing anyway. But it has changed the general environment, how work is planned, expectations, etc.
curt15 1 days ago [-]
> There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug.
At this point, why even bother with source code? Source code is designed for humans to comprehend. If humans are no longer the intended audience, why not have machines output and maintain some more efficient intermediate representation of computer programs?
gieksosz 1 days ago [-]
I fixed many bugs over the last years with help of claude code in an unfamiliar codebase. it is never just Claude alone so we don’t rely on AI understanding 100%. It is whatever AI can understand plus what I do. If you use AI right your understanding will also grow over time.
2 days ago [-]
dimbletimbers 2 days ago [-]
> But.... That's not knowledge?
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
There are no shortcuts to true understanding.
Terr_ 2 days ago [-]
> wherein the machines are treating them like pets
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
giancarlostoro 2 days ago [-]
When you get to the point that you are doing 1000 PRs I have to seriously wonder what even are you building, I hear about this a lot from a number of companies, but their “too many commits / too many PRs” problem never seems to give anyone from the outside the feeling they are innovating at groundbreaking speeds.
1 days ago [-]
frereubu 2 days ago [-]
My favourite quote at the moment: "May not man himself become a sort of parasite upon the machines? An affectionate machine-tickling aphid?" Samuel Butler, Erewhon, 1872
hgh 2 days ago [-]
I think it's just a different focus on what matters.
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
xyzsparetimexyz 2 days ago [-]
>The software they develop around their model is, relatively, trivial
30,000 changes a day is not my idea of trivial.
californical 2 days ago [-]
I mean a harness around a model is pretty trivial, I’ve also built one (heavily relying on LLMs to build it).
With a few thousand lines of code over the course of a week or so of my limited evening energy, I have agentic loops, ide integration, and a limited selection of custom tools which are all very safe (no unintended changes), along with session management and awareness of git.
It works great, is customizable, and allows integration with any provider.
If they need 30k changes per day, it’s not due to the nature of the problem!
chicagobuss 1 days ago [-]
I think claude code is just way, way, way more than "just a harness" at this point.
You have to remember all of the surrounding features it works with now, like the infra behind the cloud agents, or how remote control works, the mobile apps, their connectors, team skills and mcp servers, on and on....
xyzsparetimexyz 1 days ago [-]
Do I need any of that? Pi works just fine
sebzim4500 1 days ago [-]
You probably don't, but that's true of all software. 99.99% of users are going to use a small minority of the functionality but it will be a different minority
xyzsparetimexyz 2 days ago [-]
Oh, I mean I can't disagree there. But they've dug themselves into a hole where they're implementing custom ray tracing in the game engine that claude code runs on or whatever.
californical 2 days ago [-]
> implementing custom ray tracing in the game engine that claude code runs on
Haha, I don’t doubt it! Pretty damning overall for long-term maintainability of vibe-coded projects, even as the models are great at a lot of individual tasks
status_quo69 2 days ago [-]
I mean, sure we can assume that (although the Anthropic article from the other day say that claude mostly builds Claude) but this is the Claude Code team which, I again am assuming so I could be wrong, focuses on the TUI/CLI interface to those models, not necessarily the models themselves. That coupled with the fact that this is how a large portion of their clientele interact with their moat, it would be odd to me if they didn't apply significant work towards this thing to keep the hype or productivity train moving
markbao 2 days ago [-]
> I really don't get the tendency here from Anthropic to describe what's most likely churn? 1000 "PRs" a day on a TUI to wrap a while loop?
From the article, the 30,000 changes a month are in the monorepo, and Claude Code is in a separate repo. So it’s not the TUI that has 1,000 PRs a day.
I could easily see 1,000 PRs a day on the backend. Big tech was doing more than that pre-AI.
zeroonetwothree 2 days ago [-]
When I worked at Facebook (pre LLM) we were around 50-100k per day. I imagine it's much higher now (quantity, probably not quality).
status_quo69 2 days ago [-]
Fair enough, I did invent the 1000 a day because the article was vague, I'll edit my submission accordingly to also be vague
fouc 1 days ago [-]
> No wonder most people at Anthropic believe our jobs are in jeopardy, they've already made a system wherein the machines are treating them like pets.
I guess it wouldn't be surprising for people prone to AI psychosis to be attracted to working at the frontier labs, resulting companies that are heavily in the throes of mass AI psychosis.
Being tongue-in-cheek here.
j45 2 days ago [-]
The Anthropic post the other day mentioned part of what helps Claude Code unlock some of this is the sheer ability to use more tokens than the rest to get something going and efficient.
tdhz77 2 days ago [-]
This is a good answer
ls-a 2 days ago [-]
[dead]
squidlib 1 days ago [-]
The ownership component is the toughest and least enjoyable part of coding with ai. AI saves the time of coding, but 100% of the responsibility remains on you. You then have to invest an imbalanced amount of time to understand and fix the issue vs the time it took to generate the code. Then on top of that is the emotional feeling of something you “owned” messed up, but it wasn’t really your doing - it was the AI that did the keyboard work and now I need to parent/manage the AI. This destroys trust in AI and the cycle restarts. But we keep coming back to the slot machine as we often get those returns where things just work, but like gambling, you’re not satisfied by those wins and you’re immediately on to looking for the next fleeting high.
remus 22 hours ago [-]
I think the question is then "how do I gain more confidence this code is correct?" This doesn't have to be reading through everything line by line, it can be writing more tests, or working with the LLM to dig into how certain edge cases are handled etc. There are a lot of things you can do to improve your confidence in a piece of code.
627467 23 hours ago [-]
> You then have to invest an imbalanced amount of time to understand and fix the issue vs the time it took to generate the code.
Obviously no amount of training will make AI human but: how is this different from you being the leader of a team whose output you own and are accountable for?
bonoboTP 22 hours ago [-]
The difference is that now people who signed up and were hired as engineers now have to do what managers used to do. It's also a different pay grade when managers are accountable for a team vs and engineer for Claude instances.
lubujackson 23 hours ago [-]
In a weird way, it reminds me of playing chess. If I play a human opponent I can be thoughtful and deliberate. If I switch to playing a bot, I respond to the speed of the responses no matter the level and get crushed.
It takes a while to settle in and play deliberately. I also need to switch gears because bots play much differenly than human opponents - they don't fall for bait, they don't have time pressure at all, they don't leave pieces uncovered by accident.
I think of coding with AI is similar, you need to have a framework where you build understanding along the way from plan to review. It is untenable to roll out a 6000 line feature then try to review it, even if the LLM is happy to do things that way, mere mortals can't keep up. But if you stay in your lane and code solidly, AI can be a straight speed boost. I feel like most people despairing over handing the reins to AI simply haven't found their personal sweet spot.
YmFzZTY0ZGFkZHk 2 days ago [-]
And so the wildly unprofitable AI company explains how it token maxes it's code and review process.
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
evolve-maz 1 days ago [-]
I look at it as a giant experiment. Any research tends to be very investment heavy, and people are given a lot more leeway to try things which are wildly unprofitable.
I'm glad they're doing this, because it lets me see an extreme of LLM development.
markbao 2 days ago [-]
We spend a few thousand dollars a month in AI code review costs.
Compare that to the cost of human code review which can take up a third of an engineer’s time. You’re looking at $50-100K a year per engineer fully loaded in code review costs.
If AI assists in code review (not even replaces – we still do human review ourselves), you could spend thousands a month per engineer and still come out ahead.
Ironic that this comment talks about cost without weighing the numbers.
therealdrag0 13 hours ago [-]
It’s gotten pretty good too. My employer has a team that wrote their own code review bot with the checks and values we care about and it has gotten quite good over the last six months.
mohsinakram 1 days ago [-]
[flagged]
enraged_camel 2 days ago [-]
[flagged]
tedd4u 2 days ago [-]
"Anthropic told investors that it will profit on a measure called adjusted operating income.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
I wonder how they account for free credits given in this spend. We got given US$5k free API credits from them a couple months ago and even accounting for our Claude Code subscription spend, I think we're still ahead overall - they haven't made any money from us! And they just gave us another few hundred bucks a month of API credit as part of our subscription.
2 days ago [-]
rhdunn 2 days ago [-]
How so? Can you link to the numbers?
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
Source? Only reports I’ve seen was they were profitable for one month in last quarter due to xAI discount of renting out colossus . Be interested to learn otherwise.
lbrito 2 days ago [-]
That's factually incorrect, and shouldn't be allowed here.
After investments. Investments they can cut if they need to. Their daily operation is very profitable, they just decided to use that to bring cash in and go on the crazy growing trajectory, instead of taking it slow and steady. But if AI progress suddenly stops, they can just scale investments back and became profitable immediately. Calling them unprofitable is highly misleading. It's not like they are an early venture company still trying to figure out their product-market fit, with totally inflated numbers based merely on a theoretical number of potential customers (while having close to zero paying ones).
YmFzZTY0ZGFkZHk 2 days ago [-]
I'm not arguing that their product is not valuable or that it doesn't have customers.
I'm stating that their current business model, prices, and behavior they are trying to instill is not profitable. For example, token maxing every code change is great for Anthropic and terrible advise for Anthropic customers.
The moment they stop training, they become irrelevant. The moment they become irrelevant their valuation crashes down to normal valuation ranges for a company with their revenue and future growth
surgical_fire 2 days ago [-]
We're talking about being actually profitable, not fake profitable.
tracerbulletx 2 days ago [-]
“I think that, every time you see the word EBITDA, you should substitute the words "bullshit earnings.”
― Charles T. Munger
And what they're reporting is adjusted even further than EBITDA
adammarples 2 days ago [-]
Yes when you subtract expenses and capex
throwaway613746 2 days ago [-]
[dead]
koolba 2 days ago [-]
> Low - Docs, copy, internal-only features, additive changes behind a flag
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
brational 2 days ago [-]
It’s writing the docs for other agents obviously
TNWin 2 days ago [-]
Anthropic has amazing models. But they should really stop commenting on software practices. Most of what they say seem out of touch with what's happening in software shops. A lot of it seems hype driven. And the entire message looks to be written by AI.
cheema33 1 days ago [-]
> But they should really stop commenting on software practices.
I disagree. Many of us have an emotional attachment to the way we do things and have difficulty letting go. Anthropic is showing a different way of doing things. And I am not going to judge them too harshly if the final output is good. I use Claude Code every day. And it is good. Ahead of the pack.
dimator 1 days ago [-]
You're failing to differentiate what Claude Code means here. The models, no one argues that they're not good. If you mean Claude code the local agent, there are now dozens of smaller, faster, lighter harnesses that do everything the Claude tui does and more.
Yes, this is a different way of doing things, and it's clearly working for them, but there is no direct line between that and a "ahead of the pack" product.
72deluxe 1 days ago [-]
I don't understand saying you have an emotional attachment to doing something. Normally you do things in a certain way because it is functional and works. What we're seeing from these LLM companies is them proposing new methods of working which highlights insane ways of "working" (they don't "work") since they're reliant on an LLM, which is just self-marketing.
qnleigh 1 days ago [-]
Given the internal models that they have access to, I'm more worried about software companies being out of touch with the next generation of models will enable. Anecdotally I've seen that it takes a while for an organization to adjust to new LLM capabilities, even if everyone had access.
pokstad 2 days ago [-]
It’s all marketing nonsense. Along with the “AI is dangerous” and “we’re all going to lose our jobs” messaging they do, it’s just to sell the services.
gaal_dornick 2 days ago [-]
From the article: "merge with at most one human in the loop, who can be the author". Curious to see how this handles enterprise compliance requirements and insider threats?
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
llama052 2 days ago [-]
I’m sure it’s not compliant. We have the same requirement for soc2. However the audits are pretty light and a company big enough can afford to have an auditor who looks past those things.
c0rruptbytes 2 days ago [-]
i think SOC2 is about audibility more than anything - my guess - and i’m not an internal auditor or anything - is soc2 needs auditing for human changes, but a machine is just a “system behaving on its own”
vb-8448 2 days ago [-]
No human can stay on par with 30k pull requests per day ... they simply don't have control on what's going on and rely on AI itself to do the job.
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
cush 2 days ago [-]
30k PRs is such a meaningless figure though, you can't make anything of it. It entirely depends what's living in that repo. Adding localized strings, configs, things that change frequently, can easily move numbers like this by orders of magnitude. How big are the PRs? Maybe there's an auto-bug-fixer agent that runs around the codebase making thousands of single-line fixes per day. Who knows. The number means nothing in isolation. We used to scoff at bad managers for measuring the performance of engineers using lines of code, now we're using the same figures to judge AI. We've become the bad managers.
vb-8448 1 days ago [-]
Actually LOC is a decent measurement, it becomes a bad measurement when you use it in the wrong way, like any other measurement.
Agree that 30k PR alone doesn't mean much, but there aren't other details on what's inside. Based on I saw on how LLM programming works, it's hard to believe that there are single-line fixes. If I have to bet I'd bet on 200 LOC on average, that is 6M/day. Even if only a small portion reach production, it's still insane.
cush 1 days ago [-]
It’s kind of ironic how your comment starts with “LOC is a decent measurement” with zero explanation, then quickly devolves into rough estimates, assumptions, and finishing with “6M/day… it’s still insane”
You’re proving precisely what I’m saying. You don’t have any real information and your guess is as good as mine. It’s just as plausible to assume Anthropic runs 50x experiments for every feature they ship and only 2% of the code hits production. It’s just as plausible to assume half or more of their repo is tests. It’s just as plausible to assume there’s as much comments, skills, docs, and context engineering as there is code. It’s just as plausible to assume that there’s 10x more configs, localized strings, and content as there is executable code.
If you add all my assumptions together that leaves us with about 15x PRs worth of shipped executable code. And even with your generous 200 LoC per PR, it’s only 3000 Lines per day. Now that’s insane!
vb-8448 22 hours ago [-]
You’re right, mine are assumptions are just as much assumptions as yours.
However, if you’ve worked with coding agents and seen repositories that were mostly vibecoded, you’ve probably also seen how complex they became in a very short time. Even the estimate of 200 LOC per PR was conservative, actual number is very likely much higher.
In any case, even if a large portion of those lines were documentation, tests, or comments, those count as LOC too, and it’s simply not humanly possible to keep up.
PS: They could simply release more data and stop the debate, but for some reason, they don’t.
cush 21 hours ago [-]
> it’s simply not humanly possible to keep up
Are you talking about manual review? Of course it isn’t possible to read that much, I never said it was. The scale of changes requires new approaches to engineering. Some may feel these processes are less rigorous, but in my opinion, when done well, they’re actually more.
I have worked in many codebases using coding agents. Sometimes their changes are large and sometimes they’re small. The better repos have many guardrails to keep the PRs small. I’m not sure why you’d assume why my or your experience working with them would be even remotely similar to working with them at Anthropic though. I think it’s just fine to say you don’t know. They have no obligation to appease to people debating in an HN thread.
I think in general you’re going to continue to struggle to translate your one-off experiences to these 1000x 10000x scales. Most people can’t fathom scaling anything beyond 10-100x.
RGS1811 2 days ago [-]
How is the ability of a human to understand the code via hands-on familiarity not a direct dependency of their ability to do manual code review? I don't get it. Having claude write an explainer (something I do a lot!) doesn't get you to the point where you can confidently assess a complex diff. And you've funneled your priors on what the diff says and does through the same system that already reviewed and approved it. This article isn't bad but it doesn't give enough weight to the human context-building part of the problem.
mahogany 2 days ago [-]
I think there is no magic trick here. The answer is simply that humans will no longer understand the code.
Although, following Anthropic’s ethos and taking this to its conclusion, I’m not even sure why they need human reviewers at all. What, they bring in a human to look at a diff of an “important” section of code that was written by agents and has changed a dozen times since last week through 12 PRs of churn? Obviously they don’t know what’s going on. So what’s a human to do? Do what the other approvers (if any) did: fire up a Claude to command it “verify this change has no regression; check carefully for edge cases”, then come back after a half hour of High Thinking to see “Change is clean (with some non-blocking findings below, in order of significance)” and click Approve. Not sure what the purpose of the human is in such a process.
abalgir 18 hours ago [-]
Fair assumption. I think humans, at least those who like to code and build solutions, will become better builders than coders. Understanding the code will become less important, but knowing how to put it all together will be essential.
jazzyjackson 2 days ago [-]
Yes it used to be said, the code is not the asset, the people who know what the code does and how to change it are the asset.
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
abalgir 19 hours ago [-]
I agree with you. A code reviewer from the same company, even with different models or efforts, shares the same blind spots. To review my code, I use a different company; if I code with Claude (usually Sonnet 5.5-medium), Codex reviews it, and vice versa. I get much better code this way. But more importantly, I always tell the reviewer to build its own test-scenario ledger and refute the coder's claim that it is done. And the verdict is shared with the coder, but the coder cannot reject it.
alungeanu 2 hours ago [-]
[flagged]
dieortin 2 days ago [-]
> One colleague said slowing down does lower your error rate, but mostly because you're doing less of everything
That’s not how rates work
dozerly 2 days ago [-]
They aren’t claiming that the doing less by itself changes the rate in this sentence. They are saying that the rate changes when you slow down, probably due to having more time to focus, which in my opinion is likely.
Rapzid 1 days ago [-]
Haha, yeah.. Also with most real world, vs LLM FOMO hype fantasy land, consumer software has two very real tiers of bugs; regressions, and bugs in new features.
Regressions are usually the absolute worst.
Up to now we've just been eating shit on the DX side as far as regressions in Codex and Claude Code because everything is all "go-go-go OMG I don't want to fall behind" and also.. Access to new models is tied to harness updates.
But a ton of industries have users that are not so willing to eat the regressions in the name of more features and changes than they can keep up with.
Edit: Also moving too fast.. You may have been less likely to "get it right" than ever before leading to design churn. Again, depending on industry and users, they may not be very tolerant to releasing the 5th complete overhaul of some feature in so many weeks..
grey-area 2 days ago [-]
At 30k commits a day, is there anything that can meaningfully be called review?
stillpointlab 2 days ago [-]
I'm thinking about high volume mass manufacturing as an example here. I like those "How it is made" type YouTube videos. You see a production line that has some kind of QA process where cameras + automated systems do spot checks at high speeds and these crazy actuators are punching out bad examples. These processes often also involve spot checks where some statistical sample is selected and viewed carefully.
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
fipar 2 days ago [-]
I really like those videos too, but a very big difference is, in manufacturing QA, the system outputs a very high number of copies of a very small number of possible types, so spot checks work fine from an operations research point of view, as the variance in output has a known upper bound.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
grey-area 2 days ago [-]
That’s a category error. Software isn’t stamping out the same part again and again; if it were review would be simple. Analogies often mislead.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
stillpointlab 2 days ago [-]
I understand the skepticism and I can't claim I am right. But some points come to mind.
First, I think we overestimate how different code is from other code. We even joke about CRUD apps and how cookie-cutter they are. A lot of changes in those apps are: add some new features to a data access layer (SQL, ORM, whatever), add new routes, add new UI surface, etc. There is an entire industry around "Design Patterns" and people live by the DRY axioms. Things like Ruby on Rails exists because rebuilding these systems over and over is wasteful. I feel we are just moving further down that same path.
Second, your question of "why would we want" is totally backwards. Progress doesn't follow your desires. Progress doesn't require justification. Sometimes we adapt the environment to our needs, other times we adapt to a changing environment. Which case we are currently in is still undecided and you are free to fight for the future you prefer. But demanding an industry justify itself before it is allowed to change fails in the face of evolution. If it works, we will justify it in hindsight.
g-b-r 2 days ago [-]
The second paragraph hardly makes any sense, was it the output of an LLM?
stillpointlab.com , "Get more from your LLM"
jbvlkt 2 days ago [-]
Exactly. 30k changes a day look like brute force. No one wants that except the one charging for generating those changes.
1 days ago [-]
0c3ca83 2 days ago [-]
Those manufacturing videos are of someone making the exact same thing over and over. Months and years of testing go into ensuring that there is no change in the system, and continuous monitoring is put in place so that if any changes slip in, it can be undone. The design is set in stone, as it should be, and variations are bugs. There are months to years of cheap prototypes, iteration, design changes, pipeline tuning, and verification to ensure that nothing will need to change for the next million items.
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
cadamsdotcom 2 days ago [-]
The repetitive piece - the piece that needs that process-control - is moving from shipping code to its creation.
That's new. All analogies to 2022 software engineering need to be thrown out.
Software construction is now more like an industrial process, with the artisanal work shifting towards both
1. aligning what's built with what's needed (talking to customers!)
as well as
2. implementing and tuning your software construction process (ie. hi - we're Anthropic. We're burying ourselves in a mountain of code but it's okay because the fix is a second mountain of code, try some tokens the first hit's free!)
Back to my original point...
Writing code is now on the production line; making quality output demands industrial grade/scale verification (some involving humans of course).. which is a fascinating engineering challenge, and it's brand new!
0c3ca83 2 days ago [-]
Sure, I'd you want the exact same code created every time, I can automate that with cp. It's perfectly reliable, and checksums can catch the few times it fails.
Verification means nothing for industrial processes when the desired outcome is a moving target, so the first part of an industrial process is to develop a fixed target and aggressively remove variation.
Moving the target around is a research problem, before you feed the result into an industrial process.
Software solved industrialization so well we even forgot that we go from research to industrial with `npm` deployments or whatever your package manager of choice is.
cadamsdotcom 2 days ago [-]
You should go learn about test driven development and test first development in general.
It absolutely can be industrialized: the agent writes tests that conform to your requirements; the harness makes sure those tests pass before work can stop.
Have a look at https://codeleash.dev/docs/tdd-guard for my (dated but still interesting) take on this. Claude Code hooks are really useful to do the enforcement.
It's open source so no need to read my crappy generated docs - you can point your agent at the repo and ask it questions.
0c3ca83 12 hours ago [-]
You should learn about industrial processes.
0c3ca83 1 days ago [-]
What is that repo? Satire, for sure. Maybe unintentionally satire, it's hard to say.
11 hours ago [-]
11 hours ago [-]
grey-area 1 days ago [-]
Stop trying to use analogies to reason. Analogies are for persuasion and colour, not for thinking.
The process of anthropic, turning gold into shit software, is hardly something to emulate. What is the evidence software should now be produced in an industrial way? Because arthropic (the ones selling you the shovels) says so?
jeremyjh 2 days ago [-]
Those machines are not doing design work. They automate production of designs that had long development and testing lifecycles. Code is design. The compiler is the automated production process.
cush 2 days ago [-]
Honestly systems like this resemble real engineering more than manual human review does
swaptr 2 days ago [-]
I dont believe it, its FOMO marketing to me.
IshKebab 1 days ago [-]
Why not? Code review is a crazy time sync. If I'm working on a project without code review (e.g. a personal tool) using Claude I can easily churn out 100 commits per day. And it isn't even slop these days.
It does make me wonder what on earth they are working on. Feels like most of their products could be basically finished already.
On the other hand Spotify somehow employs 3000 programmers, and that's just a music player.
mirekrusin 2 days ago [-]
It's conducting/steering/oversight/supervision, not reviewing.
mstaoru 2 days ago [-]
Before all came to this, I've been promoting reviews at $dayjob not as gatekeeping but as method to exchange information and build more team spirit. We are an internal service team working by "projects", which makes us a very disjointed team with barely any communication. (Part of it is also German work culture.)
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
tfarias 1 days ago [-]
Reviews were always meant to be about more than just finding bugs, thanks for mentioning that.
Nowadays I'll ask a question to, for example, try to understand why was a particular abstraction chosen, and I'll get a response from a robot, sometimes without even a disclaimer that the response came from a robot (it's still somewhat easy to spot due to the shape and style of the prose).
It does make me feel like there is nobody on the other side, especially working remotely, so I think I relate to your feeling of isolation.
Sometimes I care about how some change impacts the broader application moving forward, not really the specific technical tradeoffs for one construct or another, and that's, more often than not, very lacking.
movedx01 1 days ago [-]
> We don't have a formal process for this. Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to.
Nobody on the team keeps the mental model of Claude Code as it is today in their head, so nobody on the team is able to reason and decide about Claude Code in the future, is that right? So no long term architecture strategy can exist as no current architecture understanding exists. While it seemingly does work for a system which is a TUI, would the same be true for a complex distributed system today?
baq 1 days ago [-]
'Nobody has a complete mental model of <insert name of system>' has very much been the norm since the invention of the two week's notice. What's actually new is that an architecture strategy can now be cheaply reverse engineered out of any system, adjusted where needed and immediately applied to the development process in days or hours since 'the team' is now a single dude or dudette with a fleet of robots which can see the whole code base together with its durable arch docs at once.
stuaxo 1 days ago [-]
It doesn't do all that much - so nobody being able to fit it in their head is not a good sign.
afro88 21 hours ago [-]
Under low risk, no reviews required:
> additive changes behind a flag
That's low risk when the flag is off. Is there suddenly proper code review before the flag gets turned on? That's when the risk is actualised.
Also, when they say "risk" - risk of what? Bugs and security incidents? What about other risks: future rework, slowdown of development, obfuscation of bugs, abstractions that will end up being made leaky to work etc. Are these all not considered because they are confident Claude can manage all of that?
So many questions
koehr 1 days ago [-]
It's funny to me that they use Claude Code as advertisement for their coding models. This software is mediocre at best, full of useless bloat and really only good enough for the use case (unless, the use case is advertising their models to vibe coders, then it's pretty good and I'm not the target audience)
crtk 1 days ago [-]
Does anyone ever read what code these models produce? 50% or more is just crap: meaningless tests, convoluted code, a lot of indirection, unreadable with a lot of bugs. I am not surprised that they merge as much as they do, they just dont know what the code does but see it is somewhat working. Most people who are impressed by it, I am sure, are just shit coders themselves ngl
chrisjj 1 days ago [-]
> Most people who are impressed by it, I am sure, are just shit coders themselves
The less you understand code, the more you're impressed by coding chatbots - in part because you're less likely to read the bot's code, and if you do, less likely to realise how poor it is. Code gets judged good simply because it exists and costed little to create.
chris_money202 1 days ago [-]
I don't know if this is just the Claude Code team that works on the Agentic harness or the one that does the backend model hosting. But if it's the former, I think posts like this are exhausting because their stack is so simple that a setup like this is possible.
I would love to create agentic feedback loops like this but my project (and probably a lot in the software industry) are compiled. Compiling not only takes a lot of time, but we have to carry around nearly 200 GBs in toolchain dependencies, require access to license servers for 3rd party software used in the toolchain and to run the full integration test suite on the project can take hours even with parallel testing in the VM while using multiple VMs. And these VMs aren't small, 64 core/128GB. So outside of unit tests, we can't really merge code fast and test it with high confidence without spending an absolute absurd amount on VMs in order to enable the agents to move quickly.
misschresser 1 days ago [-]
This was an exhausting post. both the writing style and the thought that this is what bleeding edge currently looks like.
Some of the most capable engineers in the world high on their own supply… I have no doubt they think this is aspirational but even as I know LLMs will forever change how we code, I’m not entirely sure 30,000 PRs for a coding agent is a volume that makes any sense.
visarga 2 days ago [-]
All good ideas, I have been doing something similar for half a year. The most important thing is to reduce bugs and ensure the code does what you wanted. Writing code tests, logging, interacting with the running app are essential. For intent realignment I like to have the agent re-read all user messages posted in that project. It only takes a few minutes even on a 10,000 message project. Then use this fresh derivation of user intent to check code and tests. For long spans of work I prefer to generate a todo list which remains as a document in the project. Each closed gate tracks feedback from the agent on what happened. This task file is also reviewed twice - once before implementation and once after. So the formula is - review agents - tests - user message logs.
dominotw 2 days ago [-]
i am tired of listening about other ppls agent setups. Everyone thinks they've discovered something cool and unique but They are all the same.
Its like telling other ppl about your dreams. Not interesting.
kaashif 2 days ago [-]
I used to keep a dream journal because I thought it might be interesting to recall all of my dreams.
Eventually I did, but it was hours and hours of effectively random stuff happening that I had to write down or dictate with speech to text.
It was a waste of time and I'm now glad I forget almost all of my dreams. My point is even my own dreams are uninteresting.
I'm not that interested in my own agent setup either - usually the tools become better on their own after a few months and render it all obsolete.
visarga 1 days ago [-]
You don't need to read my message if you think you know what is the best formula for your AI usage. Whenever someone complains to be "tired" of reading some online message I take it as an act of verbal aggression. I rarely disclosed my formula online, and probably never will again after such reaction. Do you want to regulate HN? Was my message off?
dominotw 1 days ago [-]
how would i know to not read your message before reading it.
insin 1 days ago [-]
Sir, have you heard of our lord and saviour, Markdown?
zeroonetwothree 2 days ago [-]
Much like the output of agents themselves
acedTrex 2 days ago [-]
For real, my eyes completely glaze over, there is nothing interesting in it.
nzjrs 2 days ago [-]
So true.
vatsachak 2 days ago [-]
Mfw tui needs a billion prs per day.
Opus 5.5 is obviously stellar but come on what are you doing lol
pokstad 2 days ago [-]
They need to add tamagotchi pets to the TUI so you stay awake while approving commands.
Rapzid 1 days ago [-]
Staff software engineers(because all the seniors and below have already been sent to work at the soylent green factories) will have AI monitoring software installed on their computers.
If it detects they aren't paying attention, fall asleep, or haven't approved enough PRs per hour; off the work at the factory.
weknowbetter 9 hours ago [-]
Claude Code is essentially the pinnacle of over engineering. Agentic harnesses are simply just not nearly as complicated as Anthropic seems dedicated to making them.
I'm really happy with Pi agent harness and last time I checked I could do everything CC can with my set of Chinese flash model coded Pi extensions.
vavkamil 2 days ago [-]
> Security gets its own automated review too. ... You either fix it or close the thread with a short reason, which is how you clear a false positive.
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
maxgashkov 2 days ago [-]
I'd be more interested to read on what 10% of the code is still getting written by humans and how it doesn't become a bottleneck at the pace they described.
lifeisloving 2 days ago [-]
Important/hard things are usually bottlenecks, its a good thing and usually by design. People should have more self imposed bottlenecks.
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
maxgashkov 2 days ago [-]
My interest is mostly from the management perspective -- in 2025 I have seen a prime example when a heavily vibe-coding team chugged out half-baked products that other teams later had to deal with and this was not pretty and created all sorts of interpersonal issues; the same I fear may be happening here as well.
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
lifeisloving 2 days ago [-]
I didnt really mean it towards you, I meant it more generally but I can see how it came off that way.
While Im rooting for Anthropic to collapse under the weight of slop and their own contradictions. There's a leaked interview for Anthropic from earlier this year (I think) for their inference team. The interviewer seemed very competent and the quality of the questions were good.
This article is more likely a marketing gimmick in service of shelling shovels (for devs to dig their own grave) "hey look guys we automate everything with claude, you can too, just give us all your money!"
bonoboTP 1 days ago [-]
All this immense volume of changes in code but practically what does that translate to in the coding experience? CC isn't massively different from a few months ago, even vs the start of the year. I can list a few, like remote control which is nice but I wonder what all this amount of code churn actually achieves.
Tadpole9181 22 hours ago [-]
Claude Code is not in the monorepo. The monorepo is for their internal tooling and backend infrastructure.
bonoboTP 21 hours ago [-]
If so, it's phrased very unclearly: "merged pull requests to Claude Code's codebase more than doubled". They could be more specific that it's backend infra.
Tadpole9181 20 hours ago [-]
TFA states it clearly IMO?
> Claude Code lives in its own repository, separate from the monorepo, but it's seeing a lot of pressure too.
dylanhouli 2 days ago [-]
Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to.
stillpointlab 2 days ago [-]
My advice is to start a totally vibe coded side project. You will never gain confidence until you experiment, find the failure cases, learn to deal with them, etc.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
72deluxe 2 days ago [-]
I did this, but now I have a useful piece of software that I have zero clue regarding how it works. My only chance of "maintaining" it is giving it to another LLM to explain.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
stillpointlab 2 days ago [-]
One thing I do is ask the LLM to write me a walkthrough of the code. LLMs are pretty good at generating documentation like HTML slideshows or whatever else.
I mean, I've had useful software that I don't understand how it works where I am stuck hoping some small company continues to maintain it. And I've had some of those companies go out of business and leave me with useless software that I eventually have to replace once it no longer works on my machine.
Having the code that I can dump into an LLM to understand myself or have it alter it seems better than that.
arcanemachiner 2 days ago [-]
Mermaid is a great output format for this type of information. Flowcharts, sequence diagrams, etc.
72deluxe 1 days ago [-]
It does, but then this admission that I have to feed the code into an LLM for it to generate documentation on how it works is just proving my point that we are now dependent on an LLM to understand the code that the LLM has generated.
It's the equivalent of promoting a sausage factory to be the solution to all of mankind's food problems. It's garbage.
chrisjj 1 days ago [-]
> One thing I do is ask the LLM to write me a walkthrough of the code.
I tried that. ChatGPT's result was laden with loud, clear and wrong fabications which I might have believed had I not examined the code.
Unreliable program be unreliable.
zeroonetwothree 2 days ago [-]
I've been working on a large (~10MLOC) mostly vibe-coded project for months now. It's actually quite frustrating:
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
chrisjj 1 days ago [-]
> The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself.
I see the same in e.g. simple CSS. I ask ChatGPT to apply styling X to aĺl instances of say a UI control type. Then to another. Later I ask for a tweak of X. Bot updates some but not all instances. It didn't create a single class for all. No intelligence seen.
The most remarkable fails are simpler though. E.g. at version 303 I ask "Revert to version 299". I get 299 with 303's edits. Bot explains it interpreted "Revert" to mean something no intelligent programmer would.
And most importantly, these are just the ones I directly notice. To see the total, I have to read through and understand the most repulsive code I've seen in my life.
The only way to be satisfied with bot code is by having exceptionally low expectations. And never ever let yourself be kidded that your helper is more than a stochastic parrot gorged on stackoverflow etc.
xboxnolifes 2 days ago [-]
I've done this, and always get subpar results. I cant get the LLM to make the minor tweaks I want to see, and now I have no mental model of the code to make the fix myself. Sometimes they arent minor things.
mattm 2 days ago [-]
I have done this. It's exactly the reason I went back to doing code review. I don't look at the code with a fine toothed comb, but it does help me catch bad patterns early.
stillpointlab 2 days ago [-]
I guess the question is: did the project you worked on fail? As in, you couldn't achieve the outcome you were trying to achieve? Or at some point did you sneak a look at the code and recoil in revulsion? What was the failure state?
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
hn_acct-1029393 2 days ago [-]
Not parent, but project I was involved in was shipped very quickly, had many bugs, and burned a lot of customer trust.
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
stillpointlab 2 days ago [-]
I think people are moving the goal post here. My post suggested starting a vibe coded side project, one where failure is low stakes and has no consequences. You are talking about shipping a production system to customers.
If you cannot tolerate failure then don't experiment. Don't mix the two.
xyzsparetimexyz 2 days ago [-]
Ridiculous suggestion. If someone already takes issue with ai generated code (as well they should, it often is _still_ pretty bad), getting them to write more of it is useless.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
stillpointlab 2 days ago [-]
I'm not sure what is ridiculous about suggesting people take a stance of openness and curiousness when confronted with new situations. I genuinely don't understand what people have to lose by building some throw-away experiments without reading the code.
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
xyzsparetimexyz 2 days ago [-]
>I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
chrisjj 1 days ago [-]
> 'Now is the cheapest time it will ever be to experiment'?
First hit is free ;)
troupo 2 days ago [-]
[flagged]
dang 2 days ago [-]
Please don't do this here.
(You're welcome to make your substantive points thoughtfully, of course.)
Issue: I am still unable to stop myself from doing line-by-line code review
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
g-b-r 2 days ago [-]
The thing is that's not an issue in the slightest, it doesn't need therapy!!
It's those who don't do it who'd need a talk..!
stillpointlab 2 days ago [-]
I think that depends on where you work. If your company demands that all code is reviewed then anyone who doesn't do that should be spoken to.
However, I suspect there are teams that are making different choices. In the most obvious example, I can do whatever I want on my personal projects (I hope you agree). And if I started a business and hire in programmers, I have some expectation on defining the culture, making hiring decisions based on my preferences, etc.
Whether or not such teams succeed will be a matter of fact, which I do not believe has been decided at this time.
9dev 2 days ago [-]
I hate to say this, but you're now living in a world where the value of artisanal code is rapidly approaching zero. If you still cling to beautifully written code as your core qualification, you're having a rough future ahead of you.
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
That's not wrong right now, we'll see in the future.
But I have no interest in working as a vibecoder, there are many other jobs that I'd enjoy more
troupo 2 days ago [-]
No. The issue is: "I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) during review"
How does "low-stakes environment" fix that?
stillpointlab 2 days ago [-]
Because one may learn that those fears are overblown. Or they may learn that they cannot build any kind of software without reviewing every line of code.
The person is saying "I am still unable to stop" and that is most likely because they feel a sense of responsibility. If they choose a project where there is no responsibility, where it literally doesn't matter if the project fails, they may feel able to stop. And if they manage to make that project work, if they can succeed, then they will build up confidence.
It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code. In fact, keeping their job in the long term may (or may not) require having both skills.
daveguy 2 days ago [-]
yolo'ing a code complexity generator doesn't make it suddenly stop being a code complexity generator.
troupo 1 days ago [-]
> one may learn that those fears are overblown.
They are not overblown. LLMs do all the things the person described.
> Or they may learn that they cannot build any kind of software without reviewing every line of code.
We already had bad horrendous software before the onslaught of LLM-slop. Why contribute to that?
> And if they manage to make that project work, if they can succeed, then they will build up confidence.
Confidence in what? It doesn't fix the underlying issue. And the underlying issue isn't "I can't stop" but "redundant code, duplicate code, overly complex architecture, etc." that LLMs regularly produce.
> It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code.
"If I close my eyes maybe this goes away"
jazzyjackson 2 days ago [-]
I guess they’re just saying “you’re holding it wrong”
Last time I chose to vibe code, I did it in a language I didn’t know so that I wasn’t tempted to try and comprehend the choices the robot made. I ended up with the application I wanted.
Caveat: it was code I would never have to touch again, I ran a script to generate the 140,000 static html pages that the rails app provided and then just uploaded them all to R2 buckets (million reads per month on free tier)
troupo 1 days ago [-]
> I guess they’re just saying “you’re holding it wrong”
No, they are saying "who cares if all those issues with code are true, just don't review it"
> Caveat:
It's not really a caveat. I also have throw-away code. But we're not talking about throw--away code. We're talking about the systemic issues in LLM-produced code, which are completely ignored and the answer is always "who cares, close your eyes and don't review the code".
jazzyjackson 22 hours ago [-]
Right. You don’t review the machine code or the interaction with registers and branch prediction. If the behavior matches the desired outcome what does it matter?
The classic consideration is, this code will need to be maintained, built on top of… but it’s not like that was ever a solved problem, big projects remain fragile and corporate turnover means no one knows the entire stack. So I see the vibe code out there as just another imperfect way to build barely good enough software.
I’ve also written code I built slowly and intentionally as a piece of art, beautiful code I was proud of. Didn’t make me any money :)
troupo 21 hours ago [-]
Ah yeah, the good old "you don't check compiler output" argument.
LLM output isn't compiler output. It's a lot of shitty code that contains duplication, subpar algorithms, bad data access patterns etc. We were complaining about awful software before LLMs. Why is it now absolutely okay to write shitty software just because it's LLMs?
Also: literally no one is talking about "writing software as art". We 're talking about this: "many things (redundant code, duplicate code, overly complex architecture, etc.)"
jazzyjackson 10 hours ago [-]
My point wasn’t that compilers are just as bad as LLMs, but that whether humans or robots are writing code, it’s still going to be Just Good Enough to ship.
troupo 6 hours ago [-]
How do you know it's "Just Good Enough" to ship? I will repeat: We were complaining about awful software before LLMs. Why is it now absolutely okay to write significantly more shitty software just because it's LLMs?
For example, for all the claims of how LLMs are better than humans, or how you can ship gazillion LoC easily, we've yet to see any significant number of actual good software to come out of this slopathon. Even the frontier labs with unrestricted access to their own models who boasting how they merge 30000 PRs per day [1] ship unbelievable buggy shit. But it's okay because it's LLMs who cares?
I think the intended takeaway is to learn to solve those problems without line by line review. Start by being hands off on a low stakes project, and see how the model fails. Then, come up with, and try, solutions that aren't you doing archeology.
troupo 2 days ago [-]
This somehow assumes that the person above doesn't already do all the magic the people say work 100% of the time, and finds these issues. How does running into literally the same issues in a low-stakes project help?
BobbyJo 2 days ago [-]
I think it's a good assumption. Their comment implies they aren't harnessing the coding agent very strongly given the failure modes they mention.
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
troupo 1 days ago [-]
> Their comment implies they aren't harnessing the coding agent very strongly
Ah yes. "all the magic the people say work 100% of the time"
> Do you think it's better to experiment with high stakes projects?
This again assumes that people working on high stakes projects only work on one single project.
This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
This again assumes that people experiencing something different from the others are inevitably stupid.
etc. etc.
BobbyJo 1 days ago [-]
> Ah yes. "all the magic the people say work 100% of the time"
I don't see how this rebutts the implication.
> This again assumes that people working on high stakes projects only work on one single project.
No it doesn't. It's a very straightforward yes or no question.
> This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
He's just describing very common failure modes that come from a lack of structure/linting.
You're framing things like this is some grand conspiracy. OP is literally describing flat tires and your rebuttal is "these magical round tires i keep hearing about." It's very strange.
LLMs will dedupe code if you give them structures to find the duplication. It's not magic. I do it every day.
troupo 21 hours ago [-]
> You're framing things like this is some grand conspiracy. OP is literally describing flat tires and your rebuttal is "these magical round tires i keep hearing about." It's very strange.
It's not. OP is literally stating a fact that he sees in his work with LLMs. Without even saying what he's working on.
Your (and others') immediate assumptions:
- he only works on one project
- this only project is high stakes
- they are not magicking their project with right magicks. Sorry, not "harnessing agents"
- they lack structure/linting
Ah yes. It's very strange of me to call out these assumptions.
Now, let me step in the OP's shoes as a person who sees the same issues across multiple projects and doesn't believe in magic (and has tried all of them, except the very expensive ones).
Here's the entirety of OP's text, emphasis mine: "Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to."
So:
- no amount of "harnessing the coding agent very strongly" will fix those issues.
Also, "harnessing an agent strongly" is as meaningless as "there's some magic that will fix it". Agents will always continue with code duplication and sloppy architecture. It's a fact of reality.
You can run a "swarm of review agents" that will catch some of that, but this is just a) acknowledging the problem, and b) handing off the problem to the same LLMs that produced the problems in the first place and hoping that they will magically fix them
- no amount of linting will help with things like code duplication, redundant code, or overly complex architecture. No linter catches bad architecture. Very few languages have tools to do semantic code analysis and catch duplicate or redundant code.
- no idea what you mean by "structuring". To structure code correctly you need human input because <see issues described>. Any other "structure" is just running a bunch of the same LLMs on top of various markdown files telling them "make no mistakes" in various ways.
I am not kidding or lying that just today I was "structuring" Opus 5.5 to remove duplication from code it was creating as I was watching it. Despite all the "structuring" and "harnessing" etc. And catching previous duplication introduces by Fable and Astra despite all the "structuring" and things like https://github.com/elixir-vibe/ex_dna
benoau 2 days ago [-]
This is why I like GitHub Copilot's autocompletion, I'm right there guiding and policing the code as it appears.
g-b-r 2 days ago [-]
You're unable to stop being responsible? Tragic!
spiderice 2 days ago [-]
Is it responsible to move slowly when your competitors are moving quickly? Depends on how your competitors perform. If they perform well, then your "responsibility" suddenly becomes your extinction.
tripleee 2 days ago [-]
Apple's iPhone Duo seems like it'll be a huge hit.. how many years after the competitors?
They've built an entire strategy around moving slow and doing it better
g-b-r 2 days ago [-]
If they move quickly by shipping crap they're just running towards a wall, the responsible thing is to not follow them.
xyzsparetimexyz 2 days ago [-]
fomo nonsense. not everyone has to jump off the cliff just because you are.
fantasizr 2 days ago [-]
if you take yourself out of the loop it will hurt your skills and career. Engineering blogs used to have some best practices you emulate, but the big ai companies you cannot mimic without their infrastructure
2 days ago [-]
dyauspitr 2 days ago [-]
No one cares about good code. It’s a means to an end. Your code doesn’t need to look pretty or be well documented or any of that stuff anymore because people are not looking at it. The only thing people care about is performant and error free code.
tripleee 2 days ago [-]
People saying this are telling on themselves that they have no idea what quality code is. It's not about looking pretty or nice variable names.
Code quality matters to AI just as much as it did to people
bbctr1 2 days ago [-]
Do you have any data backing this statement? Or is it just your experience?
bpeng2000 2 days ago [-]
Personal experience:
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
tripleee 2 days ago [-]
-
piker 2 days ago [-]
Yep.
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
Rudybega 2 days ago [-]
I haven't had this issue since like last year with frontier models. What are you using?
troupo 2 days ago [-]
All the models do this. I'm literally right now watching Opus 5.5 duplicating code even after telling it to fix duplication.
tripleee 2 days ago [-]
-
SmirkingRevenge 2 days ago [-]
The models are pretty good at finding code duplication and eliminating it though, and it's usually part of my AI review process at this point
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
Funnily enough, the models learned from GGC and LLVM, yet, cannot produce something half-decent.
hn_acct-1029393 2 days ago [-]
I’ve had multiple occurrences (with the latest models) of AI reading a poorly named db column and assuming it was one thing when it was another that resulted in a problem. It does not know when to stop or when to continue with determining what is true.
1 days ago [-]
gwynforthewyn 2 days ago [-]
Literally 50 years of engineering history and culture cares about whether code is good. There are libraries worth of books about it, not as a means to an app but for its own beauty. Just read “The Elements of Programming Style”, for example.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
Quothling 1 days ago [-]
This is true but a lot of that history and culture has not been capable of producing software which has better quality than what came before. Around here banks are spending hundreds of millions trying to exit mainframe systems, and they've been failing for the better part of a decade. This despite having hundreds of people working on "modern" software paradigms in "modern" languages.
I do think the capabilities of AI depends a lot on how you work. If you're doing very low abstraction, fail fast defensive programming in something like Rust then it's fairly good at writing code with high quality. If you're working in some OOP language with tons of freedom, then my experience is that it's usually not better (or worse) than most human programmers, but certainly much worse than some.
I've got decades of experience in enterprise development and I've also been an external examiner for CS students. We have people with no coding back ground vibe coding internal tools in Python which are terrible, but not really worse than what we've previously bought from some of the cheaper coding houses around here. At the same time we have AI writing internal SDK's and libraries in Rust which will frankly write better code than I do on a thursday afternoon after too many meetings and too little sleep because the children kept me awake all week.
Good code can mean a lot of things though. If you're deep into clean SOLID DRY OOP programming I suspect that an AI will produce unreadable horrible code, much worse than most humans. On the flip-side the AI probably knows the difference between a generator and looping over a List. It probably also knows more about JVM optimization than most of those banking programmers trying to replace COBOL with JAVA.'
I doubt Antropic is a good sample though. Bun is written in Rust and it's a single purpose software, which seem to be an optimal environment for LLM's.
g-b-r 2 days ago [-]
> it doesn’t impact me
so long as his software isn't used for anything that can affect you
nerdo 2 days ago [-]
Not true, LLM’s will struggle with badly organized and designed large codebases just the same, though likely with less errors and profanities.
copperx 2 days ago [-]
The biggest concern is that AI can doxx you and find where you live if your code sucks. If that doesn't scare you, nothing will.
weakfish 2 days ago [-]
> No one cares about good code. It’s a means to an end.
I do, my coworkers do, my friends do, etc etc.
swat535 2 days ago [-]
I think a more generous interpretation of the parent comment would have been: "No one [in business] cares about good code" .
Your end users certainly don't either, heck they'll gladly sit there and scroll through their phone until the 50GB React app loads.
weakfish 2 days ago [-]
That’s a fair interpretation, thanks for the clarification!
xyzsparetimexyz 2 days ago [-]
I don't care about the opinions of end users with poor taste as much as I care about the opinions of end users with good taste.
g-b-r 2 days ago [-]
Good code is a means to the end of the code being error (and vulnerability) free, maintainable and performant.
And if people are not looking it, those people don't care about their customers.
andrekandre 2 days ago [-]
> those people don't care about their customers
if it was any other industry like medicine, food, or construction etc, could they get away with just shipping products where they have no idea whats in it?
where things like "it tastes good to me" or "cement didn't crumble when i hit it with a hammer" is good enough to ship we'd be inundated with lawsuits and customer claims... its a total joke where the software industry is headed
g-b-r 1 days ago [-]
Absolutely
drewg123 2 days ago [-]
The way you phrase it makes me realize its almost like a GPL contamination. Eg, once you pull in GPL code, your project is infected and can never use a BSD (or other GPL incompatible) license.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
gwynforthewyn 2 days ago [-]
There’s a little irony about the example you picked of the GPL, just because in the US the output of an LLM cannot be covered under copyright. The GPL works as an assignment of rights under copyright.
This is, of course, tangential to your point.
troupo 2 days ago [-]
Code that isn't duplicated in 15 places, that doesn't read the whole database to fetch just 1 record, that doesn't do 15 database reads where one would suffice, that uses an optimized algorithm instead of an exponential one recreated badly from scratch... all that is also good code, and is just as important in LLM-slop era.
xyzsparetimexyz 2 days ago [-]
No code is error free. All code will need a fix of some kind eventually. When it does, it's better that the internals are not a total mess. In the same way that you don't want the insides of your car to be total alien nonsense either.
hyperpape 2 days ago [-]
I'm somewhat surprised they would say Claude is writing 90% of their code at this point. Did they get tired of reporting new milestones, or has there been a plateau?
pokstad 2 days ago [-]
[dead]
lbrito 2 days ago [-]
Token peddler says best way to do something is using more tokens.
This is not HN-worthy
ativzzz 2 days ago [-]
It looks like they are specifically optimizing for incidents/reverts and shipping faster
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
enraged_camel 2 days ago [-]
From my perspective as a power user, the product has improved substantially over the past six months. It has gained new features that are actually useful, lots of UI/UX/perf improvements, and bug fixes as well.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
zeroonetwothree 2 days ago [-]
I'm curious what has improved so much. To me Opus 5.5 is very good but so was 4.6. I try to forget the versions in between...
lavezzi 2 days ago [-]
not even one tangible example?
UberMouse 1 days ago [-]
The Workflow engine, the Mod system. I've tried to explore other coding harnesses and nothing is as powerful as claude code (barring investing a lot of effort into custom plugins for Pi or something). Other harnesses have workflow engines, but none are as powerful as Claude Codes.
UberMouse 1 days ago [-]
And lets not forget the `claude agents` view and the backgrounding system. A++
enraged_camel 1 days ago [-]
I really like the the new ability for agents to start and manage other top-level agents. Makes it super easy to have one agent as the orchestrator for a feature and have it launch new sessions, answer questions from those agents, review their work and build merge trains. This was possible before, but required a fair amount of manual work. Now the experience feels first-class and it's really nice.
rileymichael 2 days ago [-]
what are people using for automated code review these days? the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd), but it seems like they've removed that from their docs with the launch yesterday. we've looked at various products in the space (coderabbit, greptile, etc.) but have gripes with each.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
NorthSouthNorth 2 days ago [-]
We've had great success with a custom gh action on a self hosted runner. It sets up codex and all the same stuff a dev has on their machine:
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
tills13 2 days ago [-]
this: gh actions using an opencode harness (so we can use cheap models through OpenCode Zen). With the right model, and the right, scoped access, it's just a prompt away.
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
1 days ago [-]
tepmoc 1 days ago [-]
Depends on model, promt and amount of agents that review. I found that default code-review in Claude code is pretty mid for finding all issues especially if you downgrade model and effort.
During my evals on my own 3 agents that review in parallel they are much more cost efficient and cover 75% issues. Even on model like haiku it’s better score than default.
Multiple passes or raising effort will cover 90+. And this was tested on large diff, about 3500 LoC changes
PJDK 2 days ago [-]
I've set up a claude code on a vm with a bunch of skills that can use git and the Azure Dev Ops MCP via a service user, the regular build agent pops a record in a db that serves as a queue/review and a bash script picks things up and fires off claude code in headless mode.
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
walthamstow 2 days ago [-]
Using a different lab's model to review the code to the one that wrote it. Claude model plans/codes, OpenAI model reviews, or vice versa, or DeepSeek or GLM or whatever.
a3nz3g 2 days ago [-]
I do it this way too. I think having the same lab's model review the code in a fresh session also helps, so I use both: I generally start a new Claude Code and a Codex and have them review the code with a focus on 3 things: 1. bugs, 2. cut corners, 3. code style.
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
How about your pattern?
warkdarrior 2 days ago [-]
> the claude code review product originally mentioned it cost "$15-25 per review" (which is absurd)
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
rileymichael 2 days ago [-]
in a vacuum, sure. the average pull request will have multiple commits once opened (i.e. to address feedback or perhaps it was opened too early), each triggering a review. it's very easy to rack up $100+ per pr (and an eng may open a dozen prs in a single day)
skillstackhq 1 days ago [-]
[flagged]
shukla1289 1 days ago [-]
[flagged]
piro0919 1 days ago [-]
I use Claude Code for code review, and honestly it catches more than human reviewers have in my experience. It's been a big help.
stymaar 2 days ago [-]
Poorly I'm afraid, given how many new bugs make their way to the claude code CLI every update.
jwpapi 2 days ago [-]
What I’m more worried about is feature identification and how to monitor them. How can you get the product to the next level?
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
gonzalohm 2 days ago [-]
That explains why the UI keeps changing.
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
enraged_camel 2 days ago [-]
The UI has improved substantially over the past six months. New capabilities are prominently featured and less-used ones are de-emphasized or removed. Every product goes through this type of evolution. The difference with Claude Code is that at the speed they iterate, it happens a lot faster and is therefore more noticeable.
gonzalohm 2 days ago [-]
You can call it however you want, but in the end it is just poor engineering. Imaging building a bridge that needs construction every week
enraged_camel 2 days ago [-]
Software is very different than other engineering disciplines. It is meant to be malleable. I think you need a better analogy.
andrekandre 2 days ago [-]
customers don't like big changes to workflows they get used to; adding a new feature or a new button or menu item isn't a big deal but rearranging things so often start to feel random and shows little respect for the customer (or signs of any cohesive product design)
gonzalohm 2 days ago [-]
Is it, though? Imagine NASA software having to change in-flight every day...
enraged_camel 1 days ago [-]
Claude Code is not in-flight software though.
gonzalohm 1 days ago [-]
That's what I mean, it's a piece of badly engineered software
enraged_camel 21 hours ago [-]
Sorry, I'm not following. It's badly engineered because it changes frequently, or because it does not control spacecraft?
btbuildem 2 days ago [-]
It makes sense for Anthropic to pitch "more code = more better" but as anyone who's been in this game long enough knows, "less is more" often enough. And good luck coercing an AI coding agent to solve a problem by removing things.
Personally, I find Claude's degenerate urge to just spew uncontrollable amounts of code quite counterproductive. Yes I get it, you get paid by the token. I'd like to see more reasoning, more understanding, and less generating.
Truth is, we are still very very far from actual AI software engineering. Some people are working on useful tooling, and (predictably, like anything good and useful) it is taking a long time. LLMs are great, and hopefully they will help us accelerate building the tools that bring this profession to the next level.
30k PRs in a month (or whatever) is not the brag they make it out to be.
compounding-err 2 days ago [-]
I think this is the future. Im more bullish than bearish. The only issue is that eventually human verification can no longer be possible and we will have to live under AI-verification assumptions....
KasianFranks 2 days ago [-]
We're already there with our pipelines and suspect a few more out there more concerned with measuring outcomes at this point.
drdexebtjl 2 days ago [-]
They are in for a surprise when they become publicly traded and have auditors requiring actual humans signing off on any code that touches financial data.
drooby 22 hours ago [-]
You're in for a surprise if you read the post.
The post explicitly says they use codeowners to require a human in the loop for billing related code.
drdexebtjl 19 hours ago [-]
This isn’t just billing related code. If they tell investors they have X active users, for example, the code that tracks active users is subject to the same standard.
vs4vijay 21 hours ago [-]
I built this PR Manager to help me review the PRs using Coding Agents. It sits on menu bar on macOS and has various lifecycle actions of PRs.
feedbacks are welcome.
I am happy people are testing these things out. I am also just going to wait a bit longer before really adopting things like this, I don't really trust it yet. A year ago this looked very different, and next year my guess is things will look different as well.
Yeah, the software factory idea has not crossed the chasm for me yet.
mcbishop 1 days ago [-]
> At Anthropic, our jobs have moved from reading every diff to the decisions machines are bad at:
> ...Architecture that sets a pattern others will copy...
I'm happy that even Anthropic agrees this is best left to humans.
aryehof 1 days ago [-]
> Code review does two jobs:
1. Find and fix issues before they ship
2. Build shared context and codebase understanding across the team
So code review has become newspeak for “testing” and “education”?
Well not really testing. We the users are the testers and it falls to lucky us to really discover and find the errors, edge cases and security issues. The consequence of the rush to continually release…
Is the AI advance in the field of software testing nothing more than “now an AI agent can do a code review?”.
crooked-v 2 days ago [-]
Well, that explains the Github repo with 5000 issues, maybe about a third of which are random regressions of previous behavior.
devin 2 days ago [-]
I snorted laughing at this. "We are experiencing tremendous growth" and it's just a codebase with 1,000 agents endlessly adding markdown documentation for every 1 line change.
AlexErrant 2 days ago [-]
I just learned that manually invoked skills MUST be the _first_ thing in the prompt, e.g. you can't have `disable-model-invocation: true` and say "Hi, /grill-me on X".
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
svachalek 2 days ago [-]
Oh /loop was the crowning achievement. Basically guarantees you'll run a prompt at max context without caching over and over.
steveklabnik 2 days ago [-]
I have done exactly this as recently as yesterday, so I’m not sure why it doesn’t work for you.
AlexErrant 2 days ago [-]
To be clear: you can certainly do "Hi, /grill-me on X" anytime you want. You can even write "hi, /grill-me and use /search-docs", even though the /search-docs skill absolutely doesn't work. But the Claude Code _harness_ won't autoload the skill into context, so claude-the-model has to go "hmm, where can I find that skill, when I call the Skill tool with that name it no worky, so now I have to search the filesystem for that skill...".
The <command-args> I know because I just audited my raw Claude transcripts via ccglass. You can verify by just asking Haiku "hey, /grill-me does this skill autoload for you?" and then "/grill-me hey, how many times does this text appear in your context?"
You can trivially doublecheck me. The above prompts won't even eat 1% of a 5 hr on 20$ plan.
steveklabnik 1 days ago [-]
Okay, yes, I agree that that’s how it works. I thought when you said MUST you meant that it doesn’t work; it does work it’s just that it burns a turn or two to do the lookup there. I certainly agree it would be nice to not do so.
(And I didn’t know the double args thing, that’s very interesting! I should have said I only meant the first bit.)
Rapzid 1 days ago [-]
Codex had over 15k issues last I checked including legit months old issues with the sandboxing/bubblewrap config.
Human contributions to issues, comments or submissions, were drowning in a sea of AI vomit. Tons of the issues had those AI issue submissions with the classic confidence of having identified the problem, but that smell I know a lot of us recognize of it not actually knowing what the problem is at all because it hadn't failed to fix it 5 times yet.
I believe their peak month was >13k issues opened, but worth bearing in mind: "This issue has been automatically locked since it was closed and has not had any activity for 7 days.".
layman51 2 days ago [-]
I was running into something similar where I had it working on a single Python file. Maybe part of the issue is that I had memory turned off, and the conversation might have spilled into two conversations. I wasn't using version control either, but since it was a single file with about 1000 lines of code, it didn't seem too hard to eventually get all of my features back.
_heimdall 1 days ago [-]
Its unclear to me how at least Anthropic hasn't been doing what amounts to recursive self improvement for a while now.
They heavily use Claude for both code generation and code review of Claude's own code and related tooling. Is that not functionally Claude running recursive loops to improve the next version of Claude?
cebert 1 days ago [-]
Anthropic is doing some fascinating work. However, I find their communication style frustrating. It’s clear to me their employees have near unlimited token budgets. When Anthropic team members share their internal processes, I feel they don’t appreciate the constraints that many of us face in our work with budgets.
72deluxe 1 days ago [-]
I don't understand how their work can be considered fascinating if an insight into their internal processes shows that they are completely detached from reality. This would mean that their work output is also detached from reality. "Fascinating" isn't the word for this.
drooby 23 hours ago [-]
I use Claude Code every day. And I find the product to work smoothly, and effectively. From the user's perspective, that's good software. I'm impressed with what they've built. Truly.
They are using a novel approach to write good software. That's fascinating.
TBH this community has lost touch with reality. AI is making people lose their minds.
mixtureoftakes 2 days ago [-]
30 thousand changes merged per day? no. like just no. what are we doing? I'm trying to understand
devin 2 days ago [-]
There was a post some months back called "Lines of Code Got a Better Publicist" or something along those lines. This is just more of that. I am not impressed by 30 thousand "changes" per day in the slightest without significant added context. When I logged in and prompted Claude today, do I feel like I got +600,000 change value from when I logged in a month ago?
grey-area 2 days ago [-]
I’m horrified by it. There is no possible justification for it but it does explain their buggy products.
Rapzid 1 days ago [-]
This might be one of those things were everyone on the team takes a 30 minute lunch instead of 60 and just nobody clocks out to get the 1 hour extra over time.. And everyone just kinda ";) ;) ;) Working from our desk while eating lunch ;) ;) ;)"
dfunckt 1 days ago [-]
Absolutely, at this rate most software would be complete in a few hours. I may lack imagination but wtf are you changing? Endlessly refactoring? That’s spinning your wheels renamed as productivity.
JamesSwift 2 days ago [-]
lol right? So youre telling me they shipping 150,000 features this week? Or, maybe, they whack-a-moled 149,995 perf or bug regressions and shipping 5 new things. :shrug:
JackSlateur 2 days ago [-]
I, too, can write and merge 30000 changes per day. Even 300000 if I've up to.
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
for i in {1..10000000}
do
echo "I am working !" >> README.md
git add README.md
git commit -s -m "feat: working with AI"
done
(automerged PR, JOB IS DONE)
aacid 2 days ago [-]
The premise is clear, is one agent can do what developer can, but quicker, why not speed up the whole workflow to match it?
ActionHank 2 days ago [-]
Have you used Claude Code?
Have you seen the benefit of the churn of 1000 PRs a day?
leoqa 2 days ago [-]
It’s only about 10 per engineer. I’m not at Anthropic and before 2026 I was regularly hitting 1400 commits / year on my GitHub. If agents made me 2x faster (they have, review is still slow; I currently have 40 PRs open) that’d be the same rate.
IneffablePigeon 2 days ago [-]
> 40 PRs open
Ah, so you've privatised the benefits and externalised the costs.
_verandaguy 2 days ago [-]
Right, but why do things this fast to begin with, especially for a product that's already publicly available?
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
sp4cec0wb0y 2 days ago [-]
More code output isn't a metric of success per se. Goodhart's law.
Andrex 2 days ago [-]
Humans don't operate at AI speeds. We're making the car go from 0 to 5000 mph in 1 second and the drivers are turning to goo in the seat.
xyzsparetimexyz 2 days ago [-]
haha I do like that analogy.
rddbs 2 days ago [-]
Genuine question I’ve been wanting to ask folks working on different types of things: how are your teams doing agent-assisted code reviews? Not automated reviews, but what techniques and tools are humans using to keep up with the larger volume of PRs that seem to trend larger and larger?
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
pokstad 2 days ago [-]
Using agents for a first pass is a good way to speed up the review cycle. This way contributors can get near immediate feedback before a human needs to weigh in. This doesn’t replace the human review, but augments it. Each time your agent misses something, update your code review guidelines.
senderista 2 days ago [-]
If their only metrics are revert/incident rate then they're missing the long-term, cumulative impact of compounding complexity due to poor architectural decisions.
sigotirandolas 1 days ago [-]
My thought as well, I think this approach only works for them because:
* They can afford frequent CC incidents since their value is having the best models, and 95%+ uptime is still OK-ish.
* The core functionality of CC (the harness "agentic loop") is simple, an if that works then for most purposes CC works.
* CC just stores a bunch of config and history files, there isn't any financial or irreversible safety loss if they mess it up once in a while.
Once you work on something where a minor slipup can result in a huge real world mess, then I don't see how their approach can work.
thisisauserid 2 days ago [-]
Remind me to invest more in the clouds they are burning those tokens in.
godber 1 days ago [-]
Tokens are also a scarce resource for some of us:
> THE SCARCE RESOURCE: HUMAN TIME AND ATTENTION
This workflow may not be accessible to token poor teams. Though I’m not all the way through the post, so I don’t know if token scarcity is even acknowledged.
bob1029 1 days ago [-]
> Claude Code lives in its own repository, separate from the monorepo
I have some bad news about that "monorepo" designation.
stkdump 1 days ago [-]
That's not news though. In fact almost no company beyond maybe 10 people that has a large central repository has no other smaller repos with other smaller production projects. They still always call it monorepo, because it approximately still behaves like one.
nathanqueme 2 days ago [-]
I built a library to have a way to check Claude Code's work against API docs, and asked it to write many tests on top, but even then I found Claude Code would paraphrase and drift from the docs. I also found that it passed its own smoke tests but failed when testing against the real APIs, so reviewing Claude Code's code is tricky.
alasano 2 days ago [-]
I simply don't want to review AI code in depth.
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
plaguuuuuu 21 hours ago [-]
The answer is that AI tools are not easy to use. It took me quite a long time to learn actually.
Eventual productization will probably make that effort meaningless but hey. whatever. I'm too old to sell my ass for cash too.
willtemperley 1 days ago [-]
I think this is exactly why the current "attack of the vibe-clones" will fail.
Vibe-coding complex software can work, but it's going to take a really disciplined team to manage this properly, like in this article.
beardbandit 2 days ago [-]
I'm astounded that I kinda ended up building the same agentic workflow they are talking about in this article and distributed it amongst the org (eng and product/designers).
On the other hand, I would have expected Anthropic to have a drastically more elegant SDLC.
aavisangle 22 hours ago [-]
I think they must be using the Fable or maybe more capable model for internal use purposes.
matltc 2 days ago [-]
Interested to know how they handle ordering changes. Sounds like with that many changes, you're bound to hit lots of conflicts.
typesanitizer 2 days ago [-]
FWIW, Pangram identifies several sections as 100% AI-written.
zeroonetwothree 2 days ago [-]
I would have been surprised were this not the case
julesrms 1 days ago [-]
30k commits per day on a console app?? Are these all one-line changes?
I've written a harness myself and fair enough, it's more complicated than you might expect.. but surely at this point the claude CLI is pretty stable, and this must mainly be a ridiculous amount of test churn.
Or maybe their team of clankers is just creating busy-work to make the humans think they're essential. That's not unheard of in the human behaviours that they're trained on.
72deluxe 1 days ago [-]
It's pretty bad, isn't it? I can't understand why that would need so many changes. It's like people/agents frantically working to look productive without any thought on the output or overall architecture. This is not something that should be copied or admired.
julesrms 1 days ago [-]
Absolutely.. If I had a project with 30k commits per day (or even per month!) I'd be keeping that secret, not blogging about it!
luisgvv 2 days ago [-]
And yet they can't fix the janky bugs when you scroll or show/hide chain of though, commands executed, configuring plug-ins from the options menu etc.
K0balt 1 days ago [-]
The furious pace of Ux “optimisation” that we see across the board (and especially at AI companies) is a hidden anti pattern. The time and frustration of learning a new ux is an acceptable burden when it frames real new capabilities or workflows. It’s not good to rework your ux user experience every week just because you can.
Not only that, but AI is categorically terrible at ux review. I don’t mean aesthetics, I mean attention flow, workflow in general and what the user anticipates as a logical path to feature discovery and exploration. Increasingly, features are hidden behind workflows that meet a single use case but fail to be universally logical or in many cases, even accessible at all outside of the test case.
So, people… use AI in code review. It’s actually pretty great at it if you do it right, and you can ship better software with fewer bugs than ever before. But please, don’t pressure your ux guy to “10x” with AI. In fact, now that you are “10x”ing your development cycle, you are probably going to need to hire -more- ux developers if you don’t want your product to feel like slop even if it’s gold under the hood.
petersumskas 1 days ago [-]
The writing in that blog has the distinct whiff of Claudish to me!
nakul_ 2 days ago [-]
I wonder how they maintain the quality of tests for such a massive codebase and frequent commits.
owebmaster 2 days ago [-]
That's the trick: they don't
corentin88 2 days ago [-]
Claude Code is great, but do you really push that much (new) code every day?
jgalt212 1 days ago [-]
I hope my competitors do code review this way.
jstummbillig 2 days ago [-]
That's a shockingly low number.
t-writescode 2 days ago [-]
1000/day in what sounds like a single repo is a low number?
jstummbillig 2 days ago [-]
I am not sure what it is you are saying. What I am saying: I would have supposed that Claude writes 99.x% of Claude's Code code, and that ~no code at Anthropic is written by humans anymore.
sandeepkd 2 days ago [-]
I agree finding out who is writing that 10% of the code in the AI company is important. Maybe time to identity them and put them on PIP since they are slowing down everyone else and giving a bad name to the AI company
shadow28 2 days ago [-]
The 30,000 number in the blog is the number of changes landing per day, not per month!
ActionHank 2 days ago [-]
to achieve what?
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
oofbey 2 days ago [-]
No. 90% sounds like a low number. That implies something else is writing 10% of the code? People? Other models? What on earth is writing that other 10%?
firemelt 22 hours ago [-]
this article seems written by opus 5
swingboy 2 days ago [-]
Sorry, but how the hell does a TUI harness have 30,000 changes _a day_?!
steveklabnik 2 days ago [-]
The 30,000 number is across their entire monorepo, which they say in the article Claude Code does not live in.
ekjhgkejhgk 18 hours ago [-]
I'm confused because I don't imagine the Claude Code team as particularly productive. They certainly don't standout compared to pre-AI times.
foolfoolz 2 days ago [-]
everything is shipped with auto reviews behind a flag. then you need a human approver to launch
good model
2 days ago [-]
kettlez 2 days ago [-]
If the entire body of this article isn't just an "LGTM" then I know they're lying.
bdcravens 2 days ago [-]
They use Codex?
smetannik 2 days ago [-]
spoiler: they don't
JackSlateur 2 days ago [-]
90% ?
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
ChrisArchitect 2 days ago [-]
Title is: How the Claude Code team does code review
2 days ago [-]
miroljub 2 days ago [-]
I don't care how they do it.
I vibe coded my harness that is superior to CC in functionality, and it still doesn't flicker.
I don't need to hear their bullshit.
surgical_fire 2 days ago [-]
Anthropic's solution to anything is to simply burn more tokens. Because of course.
Anyway, flagging as misinformation.
hugoj0s3 23 hours ago [-]
[flagged]
favori995749721 2 days ago [-]
[flagged]
rwieruch 1 days ago [-]
[dead]
pang_claw 1 days ago [-]
[flagged]
gary1381 2 days ago [-]
[flagged]
drewbitt 1 days ago [-]
[flagged]
deterministic 13 hours ago [-]
[dead]
r2ob 2 days ago [-]
[flagged]
pjmlp 1 days ago [-]
Do they still have humans left on the team? /s
0c3ca83 2 days ago [-]
tl;dr: We don't, claude does. Make of that what you will.
vektormemory 2 days ago [-]
We have to test many different api's and models as we have over 600+ in our harness now, Sonnet 5.5 via cloud code provides the highest quality results.
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
Similarly, Anthropic can afford to do this because they have virtually unlimited tokens and labs to train models for their specific usecase. The rest of us need code comprehension lest we become dependent on continuous supply of tokens (which is a state these labs badly want to achieve)
So, unless you work at a lab, or have unlimited tokens guaranteed through the life of your company, it would do both the company and you good to continue to read the code and not let these 30k PRs per engineer be the norm.
An excerpt from the article makes my point
> Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to. When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today.
Think I'm good on taking advice from companies that aren't making money.
In Romania an investment fund (FNI) around 2000 ran some super famous ads saying "FNI is watching" (FNI is keeping watch).
6 months later they went bankrupt.
Because "pacing the frontier" is more about flipping on the money printer switch than it is about safety.
If their Claude is the same as our Claude, then this is a joke. Claude’s writing style is insidiously mind numbing. It takes constant pushing to get it to explain things in digestible English.
At least Opus 5.5 does that. Its writing is better than that if other models, though. I had other models explain stuff in an undigestable language, technically correct, but practically not what anything is about.
It removed ~50% totaling 30k words! So annoying how everything I discuss with AI gets documented in 10 the length, and then every subsequent decision is documented in 5 places.
You don't still feel that mental friction when 5.5 responds with a wall of text and you drop into that first tedious sentence? I even had it generate a syntactically invalid English sentence the other day. It's still pretty bad.
Another thing is that it's very good at creating diagrams (I think someone wrote a skill for it so maybe not by default). I usually find the diagrams are even more valuable than the text.
Honestly I was skeptical but you really can just point Claude at some code and ask it to make a markdown + SVG explainer and it will do a shockingly good job.
I'd still rather read human documentation, but even before AI almost nobody writes any - especially for internal code. And they certainly don't make great diagrams.
If it was an exception only… but it is so common that I had to put ASD-STE100 into instructions to have precise answers.
Which is fine, whatever, I guess this is how I pay the mortgage off instead of having any clue what the hell Claude is actually writing. Hopefully I can semi-retire soon and go study dunnarts or take up welding or something else that's more rewarding than this nonsense.
In fact, everything is easy when you summarise it in one sentence and don't think very hard.
It's an iceberg, most of the complexity lies underneath the obvious stuff.
Do you use one of those open source alternatives?
For sufficiently large codebases in large organization that is how it has been with humans, who come and go. In many respects, What Anthropic is doing is a massive upgrade from the status quo.
Another challenge is getting developers to stop assuming they know what the code does without reading it! Having AI “fact check” various documents against the source will help immensely.
Jesus. Give me 1 billion dollars and I'll keep your entire codebase in my head + personal notes for you until I die. Hell, I'll even force my descendants to do the same. Now _that's_ an improvement.
We are not debating Anthropic finances in this article.
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
I'm starting to worry about being gradually affected by AI's style through heavy use. It will make me so sad if one day my written piece sent to others is considered to be AI-generated, while it's written with real effort trying to make it sound natural.
Access to a wider array of special characters is also one of the really nice things about touch keyboards on mobile or tablet. Keyboards are great for fast, accurate typing but not for going beyond the basic character set.
I think when you repeatedly expose and immerse yourself to such a degree in reading and writing language, it brushes off on oneself. So I could also absolutely believe someone from Anthropic wrote this.
> Most of the path is automated; a person's attention goes where the risk is
--> We automate as much as possible, focusing human attention on risky parts
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
--> We run reviewer evals against every incident
But that's just my interpretation and my voice. I can't know for sure what the authors would have said if they took the time to write it themselves.
> Gradual release > The latest channel first; the stable channel trails it
Such an awkward, Claude-ish way of saying it
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves.
It's only a claudeism when it says "X is not a Y" when no reasonable person would have been expecting that X could be a Y.
I really don't get the tendency here from Anthropic to describe what's most likely churn? A high pressure amount of changes/PRs a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
N.B. Changed "1000 PRs" to "A high pressure amount" to be vague since the article is vague here on what CC undergoes.
This is the crux. Take pause, because hidden here is a sea-change in our fundamental trust model of software...
There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone's head that can be appealed to, to slowly and methodically debug. By design, the system is too large for that, and everything hinges on Claude itself being able to find and understand the right slice of the system in any situation. As long as that very new invariant holds, everything will be all right. If it fails catastrophically, there is no recovery. You are the owner of a recovered alien spacecraft with tech you have no hope of understanding.
The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want. The risk might be worth the reward. But it's worth explicitly laying out the deal we're making.
Tbf, it's worth pointing out that many large legacy systems already work like that (sort of), with "Claude" replaced by "whoever happens to be maintaining it this year".
The real difference as I see it is that in the human maintained old style system, the human would take things slowly and ensure they understand any changes. Claude let loose will just say "yes, I know how to do this" and bork the system, because it doesn't have a concept of humbleness in the face of complexity!
Large company codebases are very much not like that. I've worked on systems (e.g., banking) that have accreted still-running code for as much as 60 years. Nobody understands how it works. Nobody ever understood how the whole worked, and many of the parts were never actually intended to work together.
At some point I was tasked with improving the queries/retrieval times and started converting parts of the code and not only was it possible to write tests that would convey the actual meaning of the code and crystallise it but I started finding bugs in the implementation that no-one realized were even bugs because they no longer knew what it should do when queries grew to certain complexity and the search space was in the hundreds of millions of rows that thinking through them just with napkin calcs wasn't cutting it (specially when the bugs were subtle and the results still allowed the program to work "mostly" correctly).
But I have see quite terrible finance software, patches over patches hold together by a duct tape. Also finance software has to implement large complexity of tax law, which is very large and always changing. It also doesn't help if the company developing finance software sees software development as a cost center and wants always to push the costs down (especially short term costs).
I also don't even know if it's really true. Sure, maybe it's not coherent enough that a person understands it, but if an LLM wrote it, you can be ~99% certain (in my experience) that that LLM will understand it, and usually in less than the time it took to write it. And unlike humans, the LLMs don't die and take their knowledge with them, or change companies, or get ornery when you ask them to explain.
Also, in my experience, Claude doesn't really do what the parent is saying and dive head-first into complex problems and bork your code. It is fairly humble in the face of complexity. It constantly recommends breaking down larger problems into smaller ones that are more easy to grok.
But most importantly, anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful. Especially when they're writing code that's important. Unless you're an AI researcher pushing the bounds of what LLMs can do, which let's face it, the vast majority of programmers are not, it's almost always the case that you'll have your LLM writing intelligible code that you can understand and have it explain easily enough.
Those sorts of changes are really rough to manage. Maybe still manageable by LLMs but we'll find out.
Also, for what it's worth, LLM research is hardly the first scientific research to have accidents with negative external consequences.
Sure, you can do that, but irreparable harm might have been already caused. Sure, that’s unlikely when your product is an LLM wrapper. But in something like a database, it could be catastrophic.
(Yes, there will be individual exceptions to this, just like with everything else in life.)
lol, I assume you’re not a parent then. Or at best a new parent.
Is this a definitional argument? As in, if you’re not thoughtful, you must not be a decently skilled engineer? Because I see no reason to believe that people are going to be thoughtful about LLM-generated code. Just the opposite. It’s incredibly tempting to LGTM it.
I think that's true for low value code. But the more important and crucial and high-stakes code is, the more effort people put into it, generally. Even with LLMs.
So the real question would be, how many people designing / implementing the boot process of any given system don't understand what what's going on.
The comment https://github.com/anthropics/claude-code/issues/30407#issue..., written by Claude Code, and posted by the human GitHub account says:
> "Issues here are auto-closed by a bot when they've had no activity for a while and are missing information needed to reproduce (usually a version and steps, or a /feedback ID). Nothing is triaged solely by counting"
In reality all issues in this repo are subject to time-based auto-closing, unless they collect a number of upvotes. See details at https://github.com/anthropics/claude-code/issues/87647.
Therefore, I wonder whether the internal review process described in the Anthropic's blog for internal repos also results in such issues, and whether Anthropic has a internal way to measure and mitigate that. The blog post says:
> "When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today."
The Claude Code public repo problem shows that LLMs don't always "reflect the code as it is today", and can be a source of additional misunderstanding.
You need to put 80% in context. There's a reason why FizzBuzz lasted as long as it did - lots of developers, even senior devs, are terrible at writing code, subjected to absurd time constraints, or required to color inside the lines of a leaky abstraction. Sometimes all three!
I've seen many situations at my current job where asking an LLM to do a refactoring made things much better - but if I had to do it by hand I'd just give up and move on to more essential work.
What you are saying is leaning on 2 premises that seem to not hold water 1. Code written by people is always more readable and maintainable than LLM code 2. Code written by an LLM is always less readable and maintainable
Can you really make those statements in all intellectual honesty?
30 years working in this industry and I'd say that's the exception rather than the rule. When it comes to large systems.
I love this phrase, and I think it captures the difference between the "60 years of legacy code" mentioned elsewhere here, and the vibe-coded monstrosity that we all know we're going to be encountering in the next 5 years. The fact that there were human brains with intent making all the little (short-sighted, myopic, ignorant, etc) changes in the legacy code means that you DO have at least some shot of gaining enough context to make safer changes.
There is no real intent in LLMs and particularly in blind vibe coding, so you'll end up reading stuff that works, or appears to work, but there was no human brain behind the change for you to model and thus predict.
I think there is something to this, but am even more concerned by the pure scale. Consider 100k, 1M, and 10M LOC human legacy projects (already a big problem, usually!), and now compare to the likely outcome of 100M, 1B, 10B, etc LOC llm projects. That's where you may start to see the outcomes I reference in my OP.
I agree. What I am saying is, we have effective tools and processes for each of these search spaces, so code no longer being "created on purpose by humans" is not the end of the world.
Do we? Science (the tool for the natural world) is limited and there's so much we don't understand.
The recent superhuman achievements by llms in math are promising, but what we've seen so far isn't enough imo to YOLO building arbitrarily large systems in code and just trust that the llms will be able to handle it. Especially when the stakes are high (human lives, etc).
Limited in what way? Its basic structure is sufficient to eventually let us discover anything there is to discover. It's a fully general problem solving framework.
That "eventually" is doing a lot of heavy lifting. It's limited in the most literal, empirical way. We don't have answers to most things. If we stood in relation to a gargantuan code base as we stand in relation to the natural world, with an equal proportion of known vs unknown, we'd be pretty screwed.
As an aside, even with the "eventually" qualifier, the claim isn't true. The generation of hypotheses, in particular, and even sometimes the experimental setups to test them, happens through an "out of band" process -- namely, human creativity and intuition. Which is effectively a computational process we don't understand, and limited in its own way (despite its incredible productions). The history of scientific breakthroughs is the history of astonishing insights out of nowhere, from the brains of individual people, sometimes literally from dreams. The experimental method provides a reliable way to test the insights more or less "mechanically", but it would be mostly impotent without the other part.
The idea that someone at say, Google, will be able to fit even half of the system architecture in their heads is not realistic. It also hinges on being able to find and understand the right slice of the system in any situation.
But we have all sorts of guardrails to make that easy - i.e logs, traces, etc
If you have a large human-written system with a little thought put into it you should be able to find the specific pieces that handle the behavior you want and work on that, and if it's good then changing it should have a small blast radius and be covered with regression tests
In contrast, I've fully vibe coded a 15k line tool for myself with Opus and I have no idea how on earth it's architected after reading through it. It has "tests" but I don't trust them. It's just all over the place with no coherent design to it. It works, but it's making me more and more nervous to change it
It is not completely useless as a tool but there is still a huge gap in most real-world applications (I work in an org with multiple thousands of devs and I am top 5 AI user consistently, I have used Claude Code since pre-release). If you are building something like an SQL database which has decades of logic tests built by humans then it can work. If you do not have this (i.e. 99.99% of the time) then you still need to be able to issue prompts that give context. And if you start without that context, it will produce shit code in extremely basic scenarios. In particular, humans are far better at producing code that is easy to change (in practice, you need to create unintuitive, non-human rules to govern this i.e. creating many artificial boundaries which would be completely unnecessary for humans).
I am not sure whether the models just capped out in terms of coding or whether they are just trained by people who have never worked on enterprise software but the gap isn't closing. It has just been filled by people who lack the knowledge to identify when they are being bullshitted (the prevalence of consultants suggests that this markets is large).
I have worked on legacy stuff before. In most cases, there is a kernel of reason and logic that can be recovered. I have seen pure AI projects and they are the worst grade of TCS-authored abandonware. They can often do one specific thing but are basically unmaintainable outside of that.
I frequently find this in LLM generated comments, both when using an LLM myself or when reviewing code that someone else used an LLM for. I find it is often the most obvious indication that nobody really read through what was generated.
Then you investigate it and it is often something relatively simple that could have been explained with short sentence.
I used that engineering language skill which is good when it uses it...but it doesn't like to use it.
...there are clearly some areas of alignment that they are not looking at.
If you give them no constraints, they’ll produce code that just works, and it’ll look exactly like that.
We already know the solution to this problem. Software engineering. Put the LLM through the same grinder as any software dev and out constraints into place. Tell the LLM that the code design is bad and that it should improve it.
In my case, I had to go through several rounds of refactoring before the code at least resembled a good codebase.
A fun fact is that the refactorings didn’t break any functionality. Only adding new features broke preexisting behavior.
This is not very different from what already happens to architects that delegate implementation to teams.
The challenge Is to find a way to encode the architecture in a way that both you and the agents can understand and agree on.
I'm still learning how to do this effectively, but my initial experience is quite positive. Not because agents are great but because humans are bad; the degree of coherence I've seen agents following is bigger than many human teams I've led in my career.
My benchmark is a quite big project for which the architecture is already well known, namely the reimplementation of the OpenVMS OS (kernel and user-land). There are many traps and many opportunities for the agents to just go ahead and diverge from the goals, so I found it a good playground toearn how reign them in but let them be effective at "drawing the rest of the owl"
And you can ask them to address this. And you can ask them to look for that in all your PRs. Etc etc. And that helps a lot.
But it's not something that can be brute-forced in the same way "this failed a test, let me fix it" is. So without a long-term-memory brain holding all the pain of past incidents and knowing and applying ahead-of-time judgement about what to build, it's a challenge to turn the general guidance into the specifics for any unique piece of software.
They're trying to get there of course. We shall see what's next.
("Human code turns into this too" is also a curious rejoinder from the "don't read the code anymore" crowd because there are entire disciplines and areas of study that have emerged and evolved from the problems of large ad-hoc multi-faceted systems being difficult and risky to deal with.)
It is true that this gamble has existed in the form you mention for many years. Still, I sense something new here, both in scale and in the attitude of trust. Also, I find those massive, incomprehensible human-built systems frightening too.
So far I kind of feel like it nets out to the same bottleneck being good judgment from decision makers. And you might say, no, leadership will just put more and more pressure to add more complexity just because we can and it makes it appear that we are moving fast. But when it comes to real-world apps, there is a backstop which is that things that are too complex and illegible end up not being fit-for-purpose for human problems.
Claude Code and other general purpose AI tools are not like this, because they are literally trying to optimize it for every single kind of knowledge work with zero constraints on use case or target audience other than chasing the next juiciest one they can with essentially unlimited resource. 99% of software is not like this, and when the dust settles I suspect software won't be anywhere near as different as the doomers or the maxxers are predicting right now.
In most orgs, there is also a gap between leadership, even tech leadership and people actually using LLMs every day. And it is challenging for actual devs who are being told that they should be going much faster and the reality which is more mixed. Pushing back on people who are convinced that Claude is God is going to be tough if you are a junior.
I don't feel the job has really changed, I don't write code as much but that time goes into working out the structure...which is what I was doing anyway. But it has changed the general environment, how work is planned, expectations, etc.
At this point, why even bother with source code? Source code is designed for humans to comprehend. If humans are no longer the intended audience, why not have machines output and maintain some more efficient intermediate representation of computer programs?
Thank you! This is a huge sticking point for me. There is no knowing the code without experiencing the code. If nothing else, you have to read the tests to know they verify what you think they do. Even then, you might know it does what you want, but can’t know it doesn’t do anything it shouldn’t.
There are no shortcuts to true understanding.
I know some people will groan hearing the phrase again, but: "Reverse centaur." A centaur is a synergistic combination of two things, but now those things are in the wrong position: The human is the computer's tool.
Naturally, how "wrong" it is tends to depend on whether you ask the people who do all the work or the people who own all the computer-capital.
There's clearly enormous (human) work that goes in to building and tuning their models.
The software they develop around their model is, relatively, trivial. It's not clear that enormous amounts of (human) energy need to go in to that code.
It seems largely a function of applying human energy towards the things that are marginally most important to generating you money and putting you in front of competitors.
30,000 changes a day is not my idea of trivial.
With a few thousand lines of code over the course of a week or so of my limited evening energy, I have agentic loops, ide integration, and a limited selection of custom tools which are all very safe (no unintended changes), along with session management and awareness of git.
It works great, is customizable, and allows integration with any provider.
If they need 30k changes per day, it’s not due to the nature of the problem!
You have to remember all of the surrounding features it works with now, like the infra behind the cloud agents, or how remote control works, the mobile apps, their connectors, team skills and mcp servers, on and on....
Haha, I don’t doubt it! Pretty damning overall for long-term maintainability of vibe-coded projects, even as the models are great at a lot of individual tasks
From the article, the 30,000 changes a month are in the monorepo, and Claude Code is in a separate repo. So it’s not the TUI that has 1,000 PRs a day.
I could easily see 1,000 PRs a day on the backend. Big tech was doing more than that pre-AI.
I guess it wouldn't be surprising for people prone to AI psychosis to be attracted to working at the frontier labs, resulting companies that are heavily in the throes of mass AI psychosis.
Being tongue-in-cheek here.
Obviously no amount of training will make AI human but: how is this different from you being the leader of a team whose output you own and are accountable for?
It takes a while to settle in and play deliberately. I also need to switch gears because bots play much differenly than human opponents - they don't fall for bait, they don't have time pressure at all, they don't leave pieces uncovered by accident.
I think of coding with AI is similar, you need to have a framework where you build understanding along the way from plan to review. It is untenable to roll out a 6000 line feature then try to review it, even if the LLM is happy to do things that way, mere mortals can't keep up. But if you stay in your lane and code solidly, AI can be a straight speed boost. I feel like most people despairing over handing the reins to AI simply haven't found their personal sweet spot.
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
I'm glad they're doing this, because it lets me see an extreme of LLM development.
Compare that to the cost of human code review which can take up a third of an engineer’s time. You’re looking at $50-100K a year per engineer fully loaded in code review costs.
If AI assists in code review (not even replaces – we still do human review ourselves), you could spend thousands a month per engineer and still come out ahead.
Ironic that this comment talks about cost without weighing the numbers.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
From https://www.morningstar.com/news/marketwatch/2026091414/the-...
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
[1] https://finance.yahoo.com/technology/ai/articles/anthropic-v...
[2] https://fortune.com/2026/09/29/anthropic-leaked-ipo-prospect...
[3] https://www.cnbc.com/2026/10/15/anthropics-third-quarter-rev...
https://youtu.be/T-oXyXwD6sE?si=tauzXVyGmQ6MhI_5
I'm stating that their current business model, prices, and behavior they are trying to instill is not profitable. For example, token maxing every code change is great for Anthropic and terrible advise for Anthropic customers.
The moment they stop training, they become irrelevant. The moment they become irrelevant their valuation crashes down to normal valuation ranges for a company with their revenue and future growth
And what they're reporting is adjusted even further than EBITDA
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
I disagree. Many of us have an emotional attachment to the way we do things and have difficulty letting go. Anthropic is showing a different way of doing things. And I am not going to judge them too harshly if the final output is good. I use Claude Code every day. And it is good. Ahead of the pack.
Yes, this is a different way of doing things, and it's clearly working for them, but there is no direct line between that and a "ahead of the pack" product.
- How was this approved for SOC2 compliance? Our SOC2 reviewer explicitly mentioned that all code needs to reviewed by at least one other human before it goes to production. Claude Code says they are SOC2 compliant (https://code.claude.com/docs/en/security).
- This assumes all engineers will not do anything malicious. What is preventing a Claude engineer from spying on their ex's or some other companies Claude searches by pushing malicious code? Recently an OpenAI engineer was caught planting a recording device at their own office.
PS: I really want to know what is the churn rate (and the cost per LOC) here. My gut is saying the cost per LOC is insane, and they are basically rewriting over and over the same code.
Agree that 30k PR alone doesn't mean much, but there aren't other details on what's inside. Based on I saw on how LLM programming works, it's hard to believe that there are single-line fixes. If I have to bet I'd bet on 200 LOC on average, that is 6M/day. Even if only a small portion reach production, it's still insane.
You’re proving precisely what I’m saying. You don’t have any real information and your guess is as good as mine. It’s just as plausible to assume Anthropic runs 50x experiments for every feature they ship and only 2% of the code hits production. It’s just as plausible to assume half or more of their repo is tests. It’s just as plausible to assume there’s as much comments, skills, docs, and context engineering as there is code. It’s just as plausible to assume that there’s 10x more configs, localized strings, and content as there is executable code.
If you add all my assumptions together that leaves us with about 15x PRs worth of shipped executable code. And even with your generous 200 LoC per PR, it’s only 3000 Lines per day. Now that’s insane!
However, if you’ve worked with coding agents and seen repositories that were mostly vibecoded, you’ve probably also seen how complex they became in a very short time. Even the estimate of 200 LOC per PR was conservative, actual number is very likely much higher.
In any case, even if a large portion of those lines were documentation, tests, or comments, those count as LOC too, and it’s simply not humanly possible to keep up.
PS: They could simply release more data and stop the debate, but for some reason, they don’t.
Are you talking about manual review? Of course it isn’t possible to read that much, I never said it was. The scale of changes requires new approaches to engineering. Some may feel these processes are less rigorous, but in my opinion, when done well, they’re actually more.
I have worked in many codebases using coding agents. Sometimes their changes are large and sometimes they’re small. The better repos have many guardrails to keep the PRs small. I’m not sure why you’d assume why my or your experience working with them would be even remotely similar to working with them at Anthropic though. I think it’s just fine to say you don’t know. They have no obligation to appease to people debating in an HN thread.
I think in general you’re going to continue to struggle to translate your one-off experiences to these 1000x 10000x scales. Most people can’t fathom scaling anything beyond 10-100x.
Although, following Anthropic’s ethos and taking this to its conclusion, I’m not even sure why they need human reviewers at all. What, they bring in a human to look at a diff of an “important” section of code that was written by agents and has changed a dozen times since last week through 12 PRs of churn? Obviously they don’t know what’s going on. So what’s a human to do? Do what the other approvers (if any) did: fire up a Claude to command it “verify this change has no regression; check carefully for edge cases”, then come back after a half hour of High Thinking to see “Change is clean (with some non-blocking findings below, in order of significance)” and click Approve. Not sure what the purpose of the human is in such a process.
Now we’re just going to outsource our capacity to design and review because it’s faster that way, as if the thing that was wrong with the software industry was that releases weren’t being pushed out often enough.
That’s not how rates work
Regressions are usually the absolute worst.
Up to now we've just been eating shit on the DX side as far as regressions in Codex and Claude Code because everything is all "go-go-go OMG I don't want to fall behind" and also.. Access to new models is tied to harness updates.
But a ton of industries have users that are not so willing to eat the regressions in the name of more features and changes than they can keep up with.
Edit: Also moving too fast.. You may have been less likely to "get it right" than ever before leading to design churn. Again, depending on industry and users, they may not be very tolerant to releasing the 5th complete overhaul of some feature in so many weeks..
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
First, I think we overestimate how different code is from other code. We even joke about CRUD apps and how cookie-cutter they are. A lot of changes in those apps are: add some new features to a data access layer (SQL, ORM, whatever), add new routes, add new UI surface, etc. There is an entire industry around "Design Patterns" and people live by the DRY axioms. Things like Ruby on Rails exists because rebuilding these systems over and over is wasteful. I feel we are just moving further down that same path.
Second, your question of "why would we want" is totally backwards. Progress doesn't follow your desires. Progress doesn't require justification. Sometimes we adapt the environment to our needs, other times we adapt to a changing environment. Which case we are currently in is still undecided and you are free to fight for the future you prefer. But demanding an industry justify itself before it is allowed to change fails in the face of evolution. If it works, we will justify it in hindsight.
stillpointlab.com , "Get more from your LLM"
If you want a software analogy to those devices getting churned off the assembly lines, it's the TCP streams coming out of your network card, with crazy checksums punching out bad packets.
We're orders of magnitude better at reliability with code, but we're regressing on that metric rather quickly. And the comment above deeply misunderstands both manufacturing and software.
That's new. All analogies to 2022 software engineering need to be thrown out.
Software construction is now more like an industrial process, with the artisanal work shifting towards both
1. aligning what's built with what's needed (talking to customers!)
as well as
2. implementing and tuning your software construction process (ie. hi - we're Anthropic. We're burying ourselves in a mountain of code but it's okay because the fix is a second mountain of code, try some tokens the first hit's free!)
Back to my original point...
Writing code is now on the production line; making quality output demands industrial grade/scale verification (some involving humans of course).. which is a fascinating engineering challenge, and it's brand new!
Verification means nothing for industrial processes when the desired outcome is a moving target, so the first part of an industrial process is to develop a fixed target and aggressively remove variation.
Moving the target around is a research problem, before you feed the result into an industrial process.
Software solved industrialization so well we even forgot that we go from research to industrial with `npm` deployments or whatever your package manager of choice is.
It absolutely can be industrialized: the agent writes tests that conform to your requirements; the harness makes sure those tests pass before work can stop.
Have a look at https://codeleash.dev/docs/tdd-guard for my (dated but still interesting) take on this. Claude Code hooks are really useful to do the enforcement.
It's open source so no need to read my crappy generated docs - you can point your agent at the repo and ask it questions.
The process of anthropic, turning gold into shit software, is hardly something to emulate. What is the evidence software should now be produced in an industrial way? Because arthropic (the ones selling you the shovels) says so?
It does make me wonder what on earth they are working on. Feels like most of their products could be basically finished already.
On the other hand Spotify somehow employs 3000 programmers, and that's just a music player.
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
Nowadays I'll ask a question to, for example, try to understand why was a particular abstraction chosen, and I'll get a response from a robot, sometimes without even a disclaimer that the response came from a robot (it's still somewhat easy to spot due to the shape and style of the prose).
It does make me feel like there is nobody on the other side, especially working remotely, so I think I relate to your feeling of isolation.
Sometimes I care about how some change impacts the broader application moving forward, not really the specific technical tradeoffs for one construct or another, and that's, more often than not, very lacking.
Nobody on the team keeps the mental model of Claude Code as it is today in their head, so nobody on the team is able to reason and decide about Claude Code in the future, is that right? So no long term architecture strategy can exist as no current architecture understanding exists. While it seemingly does work for a system which is a TUI, would the same be true for a complex distributed system today?
> additive changes behind a flag
That's low risk when the flag is off. Is there suddenly proper code review before the flag gets turned on? That's when the risk is actualised.
Also, when they say "risk" - risk of what? Bugs and security incidents? What about other risks: future rework, slowdown of development, obfuscation of bugs, abstractions that will end up being made leaky to work etc. Are these all not considered because they are confident Claude can manage all of that?
So many questions
The less you understand code, the more you're impressed by coding chatbots - in part because you're less likely to read the bot's code, and if you do, less likely to realise how poor it is. Code gets judged good simply because it exists and costed little to create.
I would love to create agentic feedback loops like this but my project (and probably a lot in the software industry) are compiled. Compiling not only takes a lot of time, but we have to carry around nearly 200 GBs in toolchain dependencies, require access to license servers for 3rd party software used in the toolchain and to run the full integration test suite on the project can take hours even with parallel testing in the VM while using multiple VMs. And these VMs aren't small, 64 core/128GB. So outside of unit tests, we can't really merge code fast and test it with high confidence without spending an absolute absurd amount on VMs in order to enable the agents to move quickly.
Some of the most capable engineers in the world high on their own supply… I have no doubt they think this is aspirational but even as I know LLMs will forever change how we code, I’m not entirely sure 30,000 PRs for a coding agent is a volume that makes any sense.
Its like telling other ppl about your dreams. Not interesting.
Eventually I did, but it was hours and hours of effectively random stuff happening that I had to write down or dictate with speech to text.
It was a waste of time and I'm now glad I forget almost all of my dreams. My point is even my own dreams are uninteresting.
I'm not that interested in my own agent setup either - usually the tools become better on their own after a few months and render it all obsolete.
Opus 5.5 is obviously stellar but come on what are you doing lol
If it detects they aren't paying attention, fall asleep, or haven't approved enough PRs per hour; off the work at the factory.
I'm really happy with Pi agent harness and last time I checked I could do everything CC can with my set of Chinese flash model coded Pi extensions.
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
What are llm freaks in such a hurry for? Its like everyone is manically racing towards a finish line without realizing the finish line is you out of a job. Its like everyone is excited to be completely obsolete and poor.
I'm looking at Claude Code and they chug releases daily and sometimes more often then this, if someone has to stop and code a critical path by hand or smth I'm wondering how this is handled.
--
On a separate topic, the "llm freaks" -- I know you intended this as an insult, but I'm actually proud to be called a freak again, and I thought this is the place where freaks are discussing things.
While Im rooting for Anthropic to collapse under the weight of slop and their own contradictions. There's a leaked interview for Anthropic from earlier this year (I think) for their inference team. The interviewer seemed very competent and the quality of the questions were good.
This article is more likely a marketing gimmick in service of shelling shovels (for devs to dig their own grave) "hey look guys we automate everything with claude, you can too, just give us all your money!"
> Claude Code lives in its own repository, separate from the monorepo, but it's seeing a lot of pressure too.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
I mean, I've had useful software that I don't understand how it works where I am stuck hoping some small company continues to maintain it. And I've had some of those companies go out of business and leave me with useless software that I eventually have to replace once it no longer works on my machine.
Having the code that I can dump into an LLM to understand myself or have it alter it seems better than that.
It's the equivalent of promoting a sausage factory to be the solution to all of mankind's food problems. It's garbage.
I tried that. ChatGPT's result was laden with loud, clear and wrong fabications which I might have believed had I not examined the code.
Unreliable program be unreliable.
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
I see the same in e.g. simple CSS. I ask ChatGPT to apply styling X to aĺl instances of say a UI control type. Then to another. Later I ask for a tweak of X. Bot updates some but not all instances. It didn't create a single class for all. No intelligence seen.
The most remarkable fails are simpler though. E.g. at version 303 I ask "Revert to version 299". I get 299 with 303's edits. Bot explains it interpreted "Revert" to mean something no intelligent programmer would.
And most importantly, these are just the ones I directly notice. To see the total, I have to read through and understand the most repulsive code I've seen in my life.
The only way to be satisfied with bot code is by having exceptionally low expectations. And never ever let yourself be kidded that your helper is more than a stochastic parrot gorged on stackoverflow etc.
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
If you cannot tolerate failure then don't experiment. Don't mix the two.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
First hit is free ;)
(You're welcome to make your substantive points thoughtfully, of course.)
https://news.ycombinator.com/newsguidelines.html
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
It's those who don't do it who'd need a talk..!
However, I suspect there are teams that are making different choices. In the most obvious example, I can do whatever I want on my personal projects (I hope you agree). And if I started a business and hire in programmers, I have some expectation on defining the culture, making hiring decisions based on my preferences, etc.
Whether or not such teams succeed will be a matter of fact, which I do not believe has been decided at this time.
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
But I have no interest in working as a vibecoder, there are many other jobs that I'd enjoy more
How does "low-stakes environment" fix that?
The person is saying "I am still unable to stop" and that is most likely because they feel a sense of responsibility. If they choose a project where there is no responsibility, where it literally doesn't matter if the project fails, they may feel able to stop. And if they manage to make that project work, if they can succeed, then they will build up confidence.
It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code. In fact, keeping their job in the long term may (or may not) require having both skills.
They are not overblown. LLMs do all the things the person described.
> Or they may learn that they cannot build any kind of software without reviewing every line of code.
We already had bad horrendous software before the onslaught of LLM-slop. Why contribute to that?
> And if they manage to make that project work, if they can succeed, then they will build up confidence.
Confidence in what? It doesn't fix the underlying issue. And the underlying issue isn't "I can't stop" but "redundant code, duplicate code, overly complex architecture, etc." that LLMs regularly produce.
> It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code.
"If I close my eyes maybe this goes away"
Last time I chose to vibe code, I did it in a language I didn’t know so that I wasn’t tempted to try and comprehend the choices the robot made. I ended up with the application I wanted.
Caveat: it was code I would never have to touch again, I ran a script to generate the 140,000 static html pages that the rails app provided and then just uploaded them all to R2 buckets (million reads per month on free tier)
No, they are saying "who cares if all those issues with code are true, just don't review it"
> Caveat:
It's not really a caveat. I also have throw-away code. But we're not talking about throw--away code. We're talking about the systemic issues in LLM-produced code, which are completely ignored and the answer is always "who cares, close your eyes and don't review the code".
The classic consideration is, this code will need to be maintained, built on top of… but it’s not like that was ever a solved problem, big projects remain fragile and corporate turnover means no one knows the entire stack. So I see the vibe code out there as just another imperfect way to build barely good enough software.
I’ve also written code I built slowly and intentionally as a piece of art, beautiful code I was proud of. Didn’t make me any money :)
LLM output isn't compiler output. It's a lot of shitty code that contains duplication, subpar algorithms, bad data access patterns etc. We were complaining about awful software before LLMs. Why is it now absolutely okay to write shitty software just because it's LLMs?
Also: literally no one is talking about "writing software as art". We 're talking about this: "many things (redundant code, duplicate code, overly complex architecture, etc.)"
For example, for all the claims of how LLMs are better than humans, or how you can ship gazillion LoC easily, we've yet to see any significant number of actual good software to come out of this slopathon. Even the frontier labs with unrestricted access to their own models who boasting how they merge 30000 PRs per day [1] ship unbelievable buggy shit. But it's okay because it's LLMs who cares?
[1] https://claude.dev/blog/solving-code-review-at-anthropic/
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
Ah yes. "all the magic the people say work 100% of the time"
> Do you think it's better to experiment with high stakes projects?
This again assumes that people working on high stakes projects only work on one single project.
This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
This again assumes that people experiencing something different from the others are inevitably stupid.
etc. etc.
I don't see how this rebutts the implication.
> This again assumes that people working on high stakes projects only work on one single project.
No it doesn't. It's a very straightforward yes or no question.
> This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
He's just describing very common failure modes that come from a lack of structure/linting.
You're framing things like this is some grand conspiracy. OP is literally describing flat tires and your rebuttal is "these magical round tires i keep hearing about." It's very strange.
LLMs will dedupe code if you give them structures to find the duplication. It's not magic. I do it every day.
It's not. OP is literally stating a fact that he sees in his work with LLMs. Without even saying what he's working on.
Your (and others') immediate assumptions:
- he only works on one project
- this only project is high stakes
- they are not magicking their project with right magicks. Sorry, not "harnessing agents"
- they lack structure/linting
Ah yes. It's very strange of me to call out these assumptions.
Now, let me step in the OP's shoes as a person who sees the same issues across multiple projects and doesn't believe in magic (and has tried all of them, except the very expensive ones).
Here's the entirety of OP's text, emphasis mine: "Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to."
So:
- no amount of "harnessing the coding agent very strongly" will fix those issues.
Also, "harnessing an agent strongly" is as meaningless as "there's some magic that will fix it". Agents will always continue with code duplication and sloppy architecture. It's a fact of reality.
You can run a "swarm of review agents" that will catch some of that, but this is just a) acknowledging the problem, and b) handing off the problem to the same LLMs that produced the problems in the first place and hoping that they will magically fix them
- no amount of linting will help with things like code duplication, redundant code, or overly complex architecture. No linter catches bad architecture. Very few languages have tools to do semantic code analysis and catch duplicate or redundant code.
- no idea what you mean by "structuring". To structure code correctly you need human input because <see issues described>. Any other "structure" is just running a bunch of the same LLMs on top of various markdown files telling them "make no mistakes" in various ways.
I am not kidding or lying that just today I was "structuring" Opus 5.5 to remove duplication from code it was creating as I was watching it. Despite all the "structuring" and "harnessing" etc. And catching previous duplication introduces by Fable and Astra despite all the "structuring" and things like https://github.com/elixir-vibe/ex_dna
They've built an entire strategy around moving slow and doing it better
Code quality matters to AI just as much as it did to people
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
https://news.ycombinator.com/item?id=49867306
Funnily enough, the models learned from GGC and LLVM, yet, cannot produce something half-decent.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
I do think the capabilities of AI depends a lot on how you work. If you're doing very low abstraction, fail fast defensive programming in something like Rust then it's fairly good at writing code with high quality. If you're working in some OOP language with tons of freedom, then my experience is that it's usually not better (or worse) than most human programmers, but certainly much worse than some.
I've got decades of experience in enterprise development and I've also been an external examiner for CS students. We have people with no coding back ground vibe coding internal tools in Python which are terrible, but not really worse than what we've previously bought from some of the cheaper coding houses around here. At the same time we have AI writing internal SDK's and libraries in Rust which will frankly write better code than I do on a thursday afternoon after too many meetings and too little sleep because the children kept me awake all week.
Good code can mean a lot of things though. If you're deep into clean SOLID DRY OOP programming I suspect that an AI will produce unreadable horrible code, much worse than most humans. On the flip-side the AI probably knows the difference between a generator and looping over a List. It probably also knows more about JVM optimization than most of those banking programmers trying to replace COBOL with JAVA.'
I doubt Antropic is a good sample though. Bun is written in Rust and it's a single purpose software, which seem to be an optimal environment for LLM's.
so long as his software isn't used for anything that can affect you
I do, my coworkers do, my friends do, etc etc.
Your end users certainly don't either, heck they'll gladly sit there and scroll through their phone until the 50GB React app loads.
And if people are not looking it, those people don't care about their customers.
where things like "it tastes good to me" or "cement didn't crumble when i hit it with a hammer" is good enough to ship we'd be inundated with lawsuits and customer claims... its a total joke where the software industry is headed
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
This is, of course, tangential to your point.
This is not HN-worthy
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
So dbs, seed files, etc, and then just a review prompt and some skills and a few scripts.
It triggers on a new published PR or if tagged @bot-name. It's way better than the other services we've tried (which was admittedly ages ago) because you can get it to work exactly how you want it and it's dead simple.
With Codex, for the time being at least, you can use a subscription, so it's basically free if you're already paying for one. But on API pricing it's still not too bad as it doesn't produce that much output.
I set this all up in about half a day. Codex does 99% of the heavy lifting.
bugbot was costing us $1k per month for a small team. EVEN IF we decide to go with a paid or say something like GPT Sol for our custom harness it'll cost us maybe $100. We've already implemented all of bugbot's interesting features like cross-PR memory.
the cool thing is we can leverage this for things that bugbot cannot do. Just today I added a security-reviewer action reusing almost all of the code that we wrote for the code reviewer.
During my evals on my own 3 agents that review in parallel they are much more cost efficient and cover 75% issues. Even on model like haiku it’s better score than default. Multiple passes or raising effort will cover 90+. And this was tested on large diff, about 3500 LoC changes
It's pretty effective and lets you do nice stuff like compare the code to the tickets, reply to comments and stuff like that.
This generally works for me. I think a solid spec is the key for the models to review against. I ask them to follow the spec, and if the spec doesn't say, follow first principles.
Also, if there is a user experience change, they need to notify me.
How about your pattern?
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
Personally, I find Claude's degenerate urge to just spew uncontrollable amounts of code quite counterproductive. Yes I get it, you get paid by the token. I'd like to see more reasoning, more understanding, and less generating.
Truth is, we are still very very far from actual AI software engineering. Some people are working on useful tooling, and (predictably, like anything good and useful) it is taking a long time. LLMs are great, and hopefully they will help us accelerate building the tools that bring this profession to the next level.
30k PRs in a month (or whatever) is not the brag they make it out to be.
The post explicitly says they use codeowners to require a human in the loop for billing related code.
Github Repo: https://github.com/vs4vijay/prx
https://en.wikipedia.org/wiki/Crossing_the_Chasm#/media/File...
> ...Architecture that sets a pattern others will copy...
I'm happy that even Anthropic agrees this is best left to humans.
So code review has become newspeak for “testing” and “education”?
Well not really testing. We the users are the testers and it falls to lucky us to really discover and find the errors, edge cases and security issues. The consequence of the rush to continually release…
Is the AI advance in the field of software testing nothing more than “now an AI agent can do a code review?”.
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
https://github.com/anthropics/claude-code/issues/92769
The <command-args> I know because I just audited my raw Claude transcripts via ccglass. You can verify by just asking Haiku "hey, /grill-me does this skill autoload for you?" and then "/grill-me hey, how many times does this text appear in your context?"
You can trivially doublecheck me. The above prompts won't even eat 1% of a 5 hr on 20$ plan.
(And I didn’t know the double args thing, that’s very interesting! I should have said I only meant the first bit.)
Human contributions to issues, comments or submissions, were drowning in a sea of AI vomit. Tons of the issues had those AI issue submissions with the classic confidence of having identified the problem, but that smell I know a lot of us recognize of it not actually knowing what the problem is at all because it hadn't failed to fix it 5 times yet.
their claude code changelog 'fixed' % over time.
https://i.imgur.com/KJcem6c.png
I believe their peak month was >13k issues opened, but worth bearing in mind: "This issue has been automatically locked since it was closed and has not had any activity for 7 days.".
They heavily use Claude for both code generation and code review of Claude's own code and related tooling. Is that not functionally Claude running recursive loops to improve the next version of Claude?
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
(automerged PR, JOB IS DONE)Have you seen the benefit of the churn of 1000 PRs a day?
Ah, so you've privatised the benefits and externalised the costs.
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
I think a lot of people like myself still want to feel in the loop but I don’t have the right tools to stay on top of the (generally good, productive, and still thoughtful) contributions from peers anymore.
* They can afford frequent CC incidents since their value is having the best models, and 95%+ uptime is still OK-ish.
* The core functionality of CC (the harness "agentic loop") is simple, an if that works then for most purposes CC works.
* CC just stores a bunch of config and history files, there isn't any financial or irreversible safety loss if they mess it up once in a while.
Once you work on something where a minor slipup can result in a huge real world mess, then I don't see how their approach can work.
> THE SCARCE RESOURCE: HUMAN TIME AND ATTENTION
This workflow may not be accessible to token poor teams. Though I’m not all the way through the post, so I don’t know if token scarcity is even acknowledged.
I have some bad news about that "monorepo" designation.
I used to do it but now I only review code organization and structure.
Mostly I define the product behavior I want and discuss the technical implementation beforehand.
Once that's in a spec I let it build in my durable implementation engine. Automated verification, review rounds from different agent types across different models, issues triaged into meaningful clusters and flagged for false positives before finally being fixed.
That loops until the code is up to spec.
There is no special secret, LLMs need repetition and multiple passes.
They'll one shot bigger and bigger things in their default harnesses but they'll be filled with issues no matter what.
Eventual productization will probably make that effort meaningless but hey. whatever. I'm too old to sell my ass for cash too.
Vibe-coding complex software can work, but it's going to take a really disciplined team to manage this properly, like in this article.
On the other hand, I would have expected Anthropic to have a drastically more elegant SDLC.
I've written a harness myself and fair enough, it's more complicated than you might expect.. but surely at this point the claude CLI is pretty stable, and this must mainly be a ridiculous amount of test churn.
Or maybe their team of clankers is just creating busy-work to make the humans think they're essential. That's not unheard of in the human behaviours that they're trained on.
Not only that, but AI is categorically terrible at ux review. I don’t mean aesthetics, I mean attention flow, workflow in general and what the user anticipates as a logical path to feature discovery and exploration. Increasingly, features are hidden behind workflows that meet a single use case but fail to be universally logical or in many cases, even accessible at all outside of the test case.
So, people… use AI in code review. It’s actually pretty great at it if you do it right, and you can ship better software with fewer bugs than ever before. But please, don’t pressure your ux guy to “10x” with AI. In fact, now that you are “10x”ing your development cycle, you are probably going to need to hire -more- ux developers if you don’t want your product to feel like slop even if it’s gold under the hood.
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
good model
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
I vibe coded my harness that is superior to CC in functionality, and it still doesn't flicker.
I don't need to hear their bullshit.
Anyway, flagging as misinformation.
Over the last few years we have added mcp tools, memory graph, md files and skills, automated everything to the point that on desktop the system runs itself, besides proceed/approved/explain.
We have 100's of PRs/jobs/projects and Claude builds down the list automated and only asks a few questions point form if needed.
Very impressive now, compared to 6 months ago.
You can view it if interested: www.vekmem.com