The biggest weakness of the ChatGPT Webapp is the inability to tinker with the system prompt, which is what can really control how ChatGPT behaves and can correct a large number of undesired behaviors people keep complaining about.
This is most likely using system prompt engineering on the backend, so hopefully people will also realize that prompt engineering is not a meme by finally giving them a mechanism to customize output significantly. It also explains why it's limited to paying customers only, as it can be used to work around certain safety constraints, and I am skeptical that this new tool is hardened enough against it.
When I saw the headline, I was hoping this was the ability to tinker with the system prompt, or some sort of switches to enable "Lobotomized Response" or "Not lobotomized response" for people who want "safe" responses, and those who don't want filters. Going to stick with open source for now until there's un-lobotomized commercial offerings
The API lets you set whatever system prompt you want and could be considered a un-lobotomized version, it's not at all as likely to return "safe" responses when you use the API.
Is not just any message, it's a special message (by that vocabulary) that can set the tone for the next messages from the assistant, for example. Generally, pre GPT-4 (so GPT-3.5 for example), the system prompt wasn't as strong as it is with GPT-4. The model pays closer attention to whatever you put in the system prompt compared to other user messages.
For example, you can use the system message to force the assistant to always return JSON messages, instead of just plain text. Then whatever the user puts, it'll always output JSON message, even if they ask for YAML. You might have to tune the system prompt to be resilient to user requests, but GPT-4 is a lot better at this than GPT-3.
> The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. However note that the system message is optional and the model’s behavior without a system message is likely to be similar to using a generic message such as "You are a helpful assistant."
You have to do some intense prompt-engineering with the system prompt. The model considers the system to be trusted (roughly), some of the early 2023 exploits with ChatGPT still work if you do it on the system prompt.
The another element that's missing is control of output sampling. LLM models don't actually produce text. They produce a word probability map. Essentially a huge table of every single word(token) they know and the probability value of it being next. You run the model again and again to get each next word. You don't have to pick up the most probable word. Doing that is called greedy decoding. You can randomise a bit. Pick up one of less likely words if they have similar probabilities. This makes the output "more creative" sometimes. There are also more advanced ways of "steering the model" by maintaining a list of possible sentences and switching from one to another if it is considered better. You can run a smaller model on the output so far to judge if this answer is not becoming inappropriate etc.
Output decoding is a huge way to control the answers. Most users aren't even aware it exists. It's one of the reasons why comparing "naked" open source models to chatgpt is unfair. Chatgpt has all these extras on top.
I think the people who know what they're doing are probably going to come up with some quantitative ways to evaluate the performance of prompts. That's where the real engineering will come from. You could even have chatGPT generate prompts for you and evaluate the best performing ones.
Any moron can make up a prompt and say "yeah looks good enough".
I’ll take it. I spend about half my time developing/promptsmithing and the other half lawyering. “Wordsmith” sure beats some of the other lawyer epithets out there
I think prompt craft is a better definition. Engineering implies that there is a quantifiable aspect, and by their very nature LLMs are somewhat inscrutable black boxes, so craft feels more appropriate.
This is most likely using system prompt engineering on the backend, so hopefully people will also realize that prompt engineering is not a meme by finally giving them a mechanism to customize output significantly. It also explains why it's limited to paying customers only, as it can be used to work around certain safety constraints, and I am skeptical that this new tool is hardened enough against it.