The XZ Utils backdoor is a very recent example of a contributor trying to abuse a backdoor he put there himself, after 2 years of posing as a helpful person.

A game that has been played for decades. A game that LLM review of merge requests should be making very much harder to play in the future.

The thing is: the review process now also has AI, and those AI assisted reviews are much more stringent than the traditional human only reviews. Neither are perfect, ideally both would give the submission all the attention it deserves - in real life that rarely happens, though at the moment the LLMs are fast and cheap enough that you really should let them review anything you are considering approving for merge - not necessarily take everything they output at face value, but I find that they find real problems that real humans often miss.

it’s all <approver>'s fault who is a fucking imbecile

And, as always, he’s not wrong.

when the day comes, and it will, when something is submitted and approved and breaks shit

That day has been coming, over 300 times a year for the past 30 years. People submit things that break shit all-the-time, the system is setup to deal with it, that’s the actual genius of FOSS development procedures, they’re built to handle it.

Now, the firehose isn’t just opened, the hydrant has been knocked off entirely, and there will be some growing pains to deal with the volume, but the basic procedures are well developed and I find that applying them to LLM outputs (before passing those outputs anywhere off my own desk) goes a very long way toward refining slop into genuinely robust solutions.

bogus bug reports “discovered by AI” that could not be replicated in any way, shape or form

And we NEED to be more serious about use of secure verifiable identities for people so we can properly filter inputs from internet contributions. Anonymous sources should be de-prioritized to just above the known consistently bad actors. That’s nothing about AI or LLMs, that’s just the world continuing to be far too trusting of basically anonymous contributions.

If you want to submit a bunch of AI slop under your secure ID, be prepared for that ID to be de-prioritized beneath anonymous contributions.

If you want submissions from your secure ID to be taken seriously, at least task your LLM with building a proof-of-concept replication of the bugs in the reports your other LLM session creates before submitting it for humans to spend their time evaluating.

We have a human test department overseas that gets a little eager to pump up their metrics once in a while, we haven’t resorted to penalizing them (yet) but when they send an “observation” with no way to reproduce it, no log files to back it up, we tend to ignore that observation until the review meeting and then in the review meeting ask “have you seen this again? do you have any way to reproduce it? do you have screen shots or log files?” and when all those things are missing, the observation is closed with a “cannot reproduce” tag, waiting for another tester to see it again and produce more evidence.

the article also seems overly confident

Anybody writing articles about AI, or evaluating articles about AI, needs to get very specific: what month are they talking about, because if your reference point for “overly confident” is based on where things were 8-12 months ago, that’s very different from where things are today, and many times experts in the field project progress to what they expect for the future - that is of course unreliable speculation, but so far most of the reasonable optimism from the last year or two for progress in LLM software authorship and maintenance proficiency has been borne out in reality. I’m not talking about the “software is solved” numbskulls, but the progress in terms of being a valuable tool set the way C compilers were valuable tools to former Assembly programmers - yeah… it’s gone there.

I also recall trying to use Borland Turbo C++ in 1991 and abandoning it because it was too buggy, wrote the project du jour in C instead, saved the C++ rewrite for 1996.

just because AI can find a working pattern doesn’t make it good

It’s a wee bit more sophisticated than that, and will tend to find 2 + 2 even when 16/4 - 8/4 gives the same result. And if you suspect it’s settled on an over-complicated solution you can add the prompt: “Search for simpler architectures which meet all requirements with less complexity.” It won’t always find the simplest possible, but generally speaking - I find LLM agents to be a bit better than my human colleagues in terms of finding good simple solutions and also finding problems within implementations.

None of us are perfect, and LLMs are trained on our writings. LLM agents within coding frameworks have progressed easily +10 years of human experience in their perfomance levels in the last 12 months. It’s still an blueberries to tangerines comparison, a lot of what makes the LLM agents “good” is their speed and thereby ability to experiment with alternatives in a reasonable time.

Now we don’t know who actually wrote the code

Did you ever? Really?

On the internet nobody knows you’re a dog.

I’m almost afraid to post about the various weird-cheap hardware I’m using now because the LLMs are setting up the cross compile environments and other PITA details that I didn’t have the time for in the past. I stuck with Raspberry Pi like a lot of people, and now you can’t buy a Pi Zero 2W for $15 except when the blue moon is on a Tuesday - don’t want that to happen to my newfound accessible toys.

I threw about 30 bucks in tokens

Dude, seriously check the subscription plans, unless you’re using your work tokens and they don’t mind.

My work pays about 100x as much per token as a $20/month subscription plan charges when you use all the subscription.

and work towards properly making corporations less foul

Well, you and your “human solutions” have been banging that drum for 50 years (that I know of) and all I see is negative progress. The ChatGPT suggestion was meant as a barb, of course, but in a “try something different” kind of way, because 50 more years of the same old same old bitching about the evil corporations might get you one more AT&T breakup level event, if you’re lucky and the powerful people of the world want it broken up too…

I think you are a little too into the random outputs of programs to be trusted on anything, really.

On the internet, nobody knows you are a dog.

As CEOs aren’t programmers, they aren’t really doing anything much beyond

playing God, master and commander of their own little universe - having the final say in every single aspect of the company. The good ones delegate. Not all CEOs are good.

LLMs aren’t sentient, they can’t take the role of a human nor accept responsibility when things go wrong.

No, but they can weigh alternatives and make psychopathically insensitive decisions to maximize profits - which is the implicit job description of CEO.

LLMs are just a faster way of being deskilled as by design,

In the 1990s compilers de-skilled my assembly programming. I’m O.K. with that, I’m sure I could pick it back up if it were necessary, better than I did before for my diverse experience outside assembly programming since then, but it doesn’t seem likely I’ll ever need to.

Google and the other search engines deskilled my library navigation abilities. I used to know how to whiz from card catalog to microfiche to the stacks to requesting an inter-library loan in a half dozen local libraries. If I ever need to go back to doing those things, first I’ll have to learn how they’ve changed in the last 25 years, but meanwhile - search engines are 1000x faster and more efficient. I can’t remember the last time I drove 30 miles, parked downtown and walked in the rain to use a search engine, but I do remember doing that and more to use libraries.

I think it’s time to crack that deskilled buggy whip maker joke now…

even better trust your fucking experts and ask them for their opinions

Self touted experts full of misinformation are nothing new. People have been selling stale and hallucinated “business intelligence” since forever.

a political narrative as well, designed to enable mass disenfranchisement of skilled people.

Like the cotton gin, the spinning Jenny, the power loom, etc. People could and still can pick seeds, spin thread, weave by hand - but why would they pursue that as a career anymore?

I will never give a corporation an inch, as they will steal a mile.

If you really want to change the state of things, there are more effective uses of your time to achieve that end than moaning about it on the internet. You might ask ChatGPT how you can achieve your goals more effectively. I wouldn’t take the responses as marching orders, but you might be surprised if you actually think about the effort vs efficacy ratio of your options.

on a scale that can absolutely terrify ever single politician in power

Do you remember Kent State? I’ll note that the draft has ended and not returned since.

big corporations are trying to replace their entire staff with just LLMs, which cannot write code, maintain code, and update it properly without a person being involved to audit it.

As Forrest says: stupid is as stupid does.

humans need to fully investigate and verify if the LLM detected an anomaly or not

Yeah, and once detected bugs and vulnerabilities (especially those with proof of concept exploits) are really easy to verify - and people who skip that step deserve all the ridicule and disrespect they can get, because they’re ridiculously disrespecting the people they’re reporting supposed bugs and vulnerabilities to if they haven’t at least verified they’re actual and not hallucinated.

publishing a fix based on human written code alone from there. The LLM cannot be involved further at that point, as deskilling is an ever-present danger with those things.

Here I diverge from your thinking. Deskilling has been an ever-present danger since promotion to management was invented. Some managers retain their skills, others follow the Peter Principle ever upward until they’re beyond their actual abilities. The need for skills remains, and I doubt computer coding knowledge will be vanishing just because it’s gotten easier to do it faster and better than ever before. If anything, I’m learning from the LLMs what’s possible when you have so much extra time on your hands to write unit tests, document trace matricies, keep all the design documents in-sync with the actual state of the code, etc. All the things that we have been learning how to do minimally to avoid wasting time on them, now can be done maximally, and there are benefits - particularly when working with dim-witted LLM agents as front-line coders (who work really really fast.)

CEOs have been brainwashed into believing

many things over the years. Speaking of a role better suited to an LLM…

Perhaps, if humans used them sanely, it wouldn’t be so bad.

You could as easily apply that statement to automobiles, earth movers, concrete plants…

Regarding languages: they never mattered a whole lot - by the time I was out of school I had BASIC, Assembly (several kinds), Fortran, Pascal, C, C++, LISP, probably several more - and it’s all more or less the same, each with their own quirks and special supports. Mostly you just use what works in the environment you’re trying to develop for. Same for libraries and APIs… More recently, I did a fairly sizeable project in Python with Claude, but much more important than anything written in Python were the user-facing requirements. When I was tired of Python taking 2-3 seconds to render a page, I consulted with Claude and we settled on Go as the next logical target. Because the requirements and their tests were fully developed / properly exercised and maintained, the re-code from Python to Go was 100% automated, all done by Claude, and it came out flawlessly - every feature identically implemented, running from the same database, only pages rendered in 0.1 seconds max instead of 2-3. Claude assures me that the Python development went 2-3x faster than original development would have gone in Go… I wonder how true that was, or if Claude is just parroting what it read somewhere; actually, at this stage I’m 95% sure it was parroting, though the models are getting better about quoting actual estimates based on LLM assisted development.

People who share textbooks, buy cheap ones…These students aren’t a profit driven entity

Actually, they are. They are a teeming horde of individual profit driven entities who might not be able to afford an education otherwise. One might even extend that analogy to Anthropic and friends, where it was more realistic for them to ask forgiveness / restitution rather than permission. Illegal, yes. But would staying legal have been the right move?

I wonder about the future of America…

No matter what happens in November, I still wonder about the future of the US, and the world.

My last interview was in 2014, for a transfer within a big company from one metro area office to another. I had been on-site in my present office many times (still, mostly working from home), so I took the “dress like you would dress during a normal workday” advice - brand new sharp pair of dark blue jeans (solid - no holes or wear), long sleeve dress shirt, loafers. I already “had the job” within the company, everybody on the actual team I was to be transferring to loved me, but… the final interview was with my new Boss’ Boss, and my to-be-new boss told me, “yeah, you’re going to have to come back to see him, he’ll flat out refuse to hire anybody who wears blue jeans to an interview.”

The internet really messed with things. We put a help-wanted ad in the local paper in 2000, specifically said in the ad “local candidates only” - I received 400+ applications in the first week, ~300 of them from out of town, 20+ of them from other countries. Sorting through the pile (they were still snail-mailed in those days) I came across a local candidate who worked with a company we regularly did business with, had our guy call his ex-manager and asked one question “would you re-hire him if you had the chance?” Answer was yes. We hired him, he was good.

My sincere apologies if your application was in the bottom of the stack and you were better qualified. Life is like that most of the time.

A lot of it is cyclical, hit it at the right time and they’ll hire any resume that comes across their desk. The other 90% of the time, yeah, get in line behind the guy that knows a guy…

Local division of a big name company was always advertising in the paper, I applied but they never replied. Once every 6 months or so they’d do an “open interviews night,” so I signed up. Sat down with the floor manager, he took one look at my resume and told me: “you don’t want to work here. What’s your pay, like X?” he was guessing based on my title and experience, he was within 2% "Yeah, I only get 80% of that and I’m the highest paid guy here. We work long hours and the work is both boring and demanding, long hours, unexpected overtime. Your kind of jobs are at our main facility (up north), down here we’re basically a sweat shop.

I interviewed for 3 weeks in Manhattan 1988, whoever would talk to me. Dressed well, suit, nice tie, matching black belt and shoes. Every single interviewer dumped on me: “no real experience, nobody cares about your degree or what grades you got, you’ve got nothing to offer.” Finally, I bought a $2 bright red necktie from a street vendor, had my shoes shined and went and saw an agent who interviewed me at a glass table. She took a long look at my shoes through that table, then at me, and declared she absolutely could get me a job with (investment banking firm). One glance at my resume: “you do computer stuff, right? They always need computer guys, and you’ll fit right in! They’re going to love you.”

Teach a man to fish, tomorrow he has already forgotten. If he was going to catch his own fish, he would have learned how to long before this.

I learned Autocad 14 on my own machine across a couple of months. Customized my input tablet and keyboard shortcuts to make my common tasks easier.

Then I went to a machine shop to have my widgets built, and tried to use their Autocad - and I was completely helpless, asking “how do you … ?” for the most common tasks. I never bothered to customize key-shortcuts again, I don’t work alone on a single machine enough to make it worthwhile.