Indeed. The issue is that of precision. I think it will never be possible to write nontrivial programs by describing them in anything like a conversational human language. I think this is true even if LLMs achieve some sort of godlike intelligence.
The problem is that human languages aren't intended to communicate that sort of precision. That's one of the largest reasons why very specialized technical professions (medicine, engineering, legal, etc.) heavily employ specialized jargon. It's necessary in order to make up for the deficiencies of natural human languages for these sorts of things.
In order to create a program, very high precision of descriptions are required. Programming languages can be viewed as a variety of such jargon. If you're doing it with (say) English, you'll always have to use it in a way that makes it no longer "conversational English".
I think LLMs are already influencing the development of somethin between natural language and a formal language.
Prompt engineering and getting structured data in and out of an LLM naturally lead you to something like precise English with JSON and pseudocode syntax sprinkled in
Tell me about it... I work on prompting a lot and you would be surprised how different the model interprets things like "suggestion" vs "recommendation".
The problem is that human languages aren't intended to communicate that sort of precision. That's one of the largest reasons why very specialized technical professions (medicine, engineering, legal, etc.) heavily employ specialized jargon. It's necessary in order to make up for the deficiencies of natural human languages for these sorts of things.
In order to create a program, very high precision of descriptions are required. Programming languages can be viewed as a variety of such jargon. If you're doing it with (say) English, you'll always have to use it in a way that makes it no longer "conversational English".