Well, no matter how thoroughly you vet, it’s always good to have a tool to back you up.
For example, we once got a pull request, which was purely AI-generated but I couldn’t tell that right away. So, I skimmed it to make sure no malicious code is part of it, then I gave it to the CI runner. And that failed pretty much immediately during a compile check, which made it obvious that the pull request author had never tried to compile it.
In that moment, I could stop wasting my time with that pull request, rather than try to debug why it’s not working or having to vet it more thoroughly…
I thoroughly agree, you should always have CI tools to ensure it builds, passes tests, and meets whatever formatting and/or linting standards the team sets. I was specifically responding to “Rust makes it harder for a ‘contributor’ to sneak in LLM-generated crap”. If I get a contribution from an untrusted party, I will start with the assumption that it’s utter garbage, buggy, broken, and malicious and review it until I’m convinced it’s not. Not because I assume the dev is bad but because it’s safer to assume the code is garbage. If I get a contribution from a trusted party (e.g. a member of the dev team/employee/whatever) I will review the code carefully though not with as much paranoia. I don’t particularly care if my teammates are using LLMs, but if they’re submitting code they don’t understand that’s a great way to get ejected from the “trusted contributors” group, and if they’re an employee it’s a good way to get fired if they keep doing it after being warned not to.
Well, no matter how thoroughly you vet, it’s always good to have a tool to back you up.
For example, we once got a pull request, which was purely AI-generated but I couldn’t tell that right away. So, I skimmed it to make sure no malicious code is part of it, then I gave it to the CI runner. And that failed pretty much immediately during a compile check, which made it obvious that the pull request author had never tried to compile it.
In that moment, I could stop wasting my time with that pull request, rather than try to debug why it’s not working or having to vet it more thoroughly…
I thoroughly agree, you should always have CI tools to ensure it builds, passes tests, and meets whatever formatting and/or linting standards the team sets. I was specifically responding to “Rust makes it harder for a ‘contributor’ to sneak in LLM-generated crap”. If I get a contribution from an untrusted party, I will start with the assumption that it’s utter garbage, buggy, broken, and malicious and review it until I’m convinced it’s not. Not because I assume the dev is bad but because it’s safer to assume the code is garbage. If I get a contribution from a trusted party (e.g. a member of the dev team/employee/whatever) I will review the code carefully though not with as much paranoia. I don’t particularly care if my teammates are using LLMs, but if they’re submitting code they don’t understand that’s a great way to get ejected from the “trusted contributors” group, and if they’re an employee it’s a good way to get fired if they keep doing it after being warned not to.