I've been using a paid subscription to ChatGPT Plus since May 2023 and I'm still happy with the usage. It's one of those tools you have to master to get most of the time spent with it. There are a lot of benefits to use it, of course there are also negative aspects and bugs. Some of them can be mitigated, and that's the purpose of the current post.
Disclaimer / Context of use 🧾
- I'm ChatGPT mostly for scripts and each conversation is focused on singular topics, i.e single-file codebases if possible.
- I don't use Open API Keys and I'm not interested to use OpenAI directly. I've been on a paid plan for OpenAI and was charged way too much for my needs. I advise anyone to use ChatGPT Plus which remain to this day worth it and good enough both for personal and professional.
- This article was not written nor reviewed by AI/ChatGPT.
Lessons learned - Do ✅
- Test test test after each change. ChatGPT will make you lose faith by repeating the same mistakes, by removing sensitive code, by providing incomplete solutions, always verify.
- Before running or committing anything generated by ChatGPT, always compare with the previous situation you had. Git diff the result of changes. If you have any doubt about a specific change, submit the git diff to ChatGPT for review and ask for explanations while also explaining why you have doubt (failing tests, errors at runtime, weird code removal, ...).
- Tell ChatGPT to always decouple code in functions.
- Instruct to use minimalist code, and avoid comments in code, respect your specific style (if you provide examples), instruct to write the code that takes the least amount of lines and space and to avoid long functions.
- Keep one conversation per context / type of problem / flow. Once done with the problem, validate the solution (test!), commit locally, and delete the conversation to keep your Chat history clean.
- Try to limit ChatGPT's attention to one single file or one single function at a time, to improve its speed and accuracy. I've experienced inconsistencies and code regressions when expecting ChatGPT to work with too many files and too many changes at once on too many files, as then ChatGPT.
- If you have lot of work you expect ChatGPT to perform for you, be patient and request one change at a time, then review the outcome, and only if it's validated, move to the next. ChatGPT sucks at multitasking. Keep the rest of your TODO list somewhere else to keep track of what remains to be requested from ChatGPT.
- You will be given better results if your tech stack relies on popular libraries and programming languages so keep that in mind and be prepared to deal with more mistakes if you pick niche programming languages.
- Ask to only output the code that require change, e.g add this at the end of your prompt :
just output the function(s) needing change - If you really want to multitask, open multiple ChatGPT sessions (browser tabs) in parallel, each focused on distinct changes, so you can multitask while ChatGPT stays focused on single tasks 🙂
- One change at a time. If ChatGPT keeps making error, step back, restart from to the latest stable version of your code, and ask to make minimal changes, step by step, while you validate each increment.
- When happy with your changes, ask ChatGPT to improve its performance or clean the code.
- When debugging your code with ChatGPT, provide stack traces, inputs, outputs and even complete your input with the copy of the code that seems buggy (based on the stack trace).
- Be suspicious if ChatGPT changes your code way too much or introduce weird changes to dependencies. Review, check, investigate.
Lessons learned - Avoid ❌
- Don't run nor commit anything generated by ChatGPT that you don't understand and always compare ChatGPT's solution with what you had before.
- Don't ask too many improvements or bug fixes at once or be prepared to deal with many new errors and regressions.
- Don't keep conversations for too long, as all the initial context will likely be forgotten about by ChatGPT and you will suffer, also it will cause a lot of scrolling and augment the size of the web page which will be slower to load and will likely crash your browser tab at some point.
- Don't switch context / files / problems in the same conversation. It's a waste of time and you will suffer later when trying to source specific content or make sense of anything.
- Don't share secrets/passwords with ChatGPT.
- Don't waste your time with ChatGPT if it seems unable to solve your problem. Either your problem is too big, either your input is crap, anyway you will likely move faster by starting from a fresh conversation with a smaller problem or by tackling a specific part of the problem on your own. Be confident in your own abilities.
- Don't expect ChatGPT to be as efficient on big codebases and complex problems as it is on small scripts and simple problems. So use it more often for the latter and keep the fun of solving the big problems yourself.
- Don't expect ChatGPT to understand a single thing you do nor why he generates his code. it's a dumb machine without creativity built in. It has to be treated as such and with caution.
- Don't expect ChatGPT to run well and fast on huge bloated scripts. That should encourage you to decouple your code into functions and modules and specialized files/modules/components/...
Relevant references
- I consider ChatGPT eases automation to the point those XKCD memes are less relevant than they used to be.
Extras
- If you lose focus on the ChatGPT session in your browser, it's likely the calculation process will interrupt.
- ChatGPT seems stuck at generating the output ? Refreshing the page might be enough, in other cases ChatGPT might automatically continue the generation or will show a button you can hit to force this action.
- I've commented on this topic and this post at https://lobste.rs/s/7a3qhh/how_s_your_experience_so_far_using_llms_for#c_igssaj
Thank you 🙂
Last updated: June 26, 2024 around 8:24 PM CEST in 🗺️ Brussels, Belgium.