Tips for co-coding with a LLM
OKay, so, it has become very popular to cowrite code with a LLM. And I cannot argue that it is a tremendous force multiplier, however, I do have some suggestions
#1) Painful as it might be, *read every diff*. Do not ever commit code you haven’t read.
a) While you’re reading, see if you can shorten the comments. LLMs overcomment – the reason this matters is if you are going to use
LLMs to maintain the code over long terms, you do *not* want to be burning tokens with excessive comments
b) Check the logic and approach. LLMs make mistakes.
#2) If the LLM says something improbable, *definitely* push back. *Expect* the LLM to draw incorrect conclusions from individual failures – one example, recently Claude concluded that the shopify metaobject filter didn’t work because you can only create a metaobject filter for a metaobject
which exists. It didn’t know that, got errors, and concluded the errors were fatal in scope to the operation. It was all set to create a implementation with 30 webhook registrations where only 3 were needed. I repeatedly pushed back, and led it down the correct road, until it understood the right way to do things.
#3) Point out when the LLM says something blatantly wrong. among other things, there isn’t any real reasoning going on – remember, LLMs have no aboutness, they’re in no way anchored in reality – they can’t tell a cat from a cow, they just are a sliding set of activations of weights moving through a transformer, also thought of as a feed-forward ANN with a attention head. By pointing out a blatently wrong statement, you may well nudge them into the correct alignment.
#4) *TEST* everything. One *good* thing about LLMs is they are wonderful at writing unit tests. Use this ability – but also end to end test the system under development unless you can tell by reading the code that it’s going to work – and maybe even then. Certainly don’t release anything to production that hasn’t been tested.
#5) Sometimes, write it yourself. You need to keep the ability to write code, and it will atrophy if you don’t use it. You want to occasionally exercise those pathways – neural skills are use it or lose it.