[Discuss] [arxiv] How Secure is Code Generated by ChatGPT

[Dan Ritter]

Kent Borg wrote: 
> ChatGPT is a fancy auto-complete, that has ingested a *lot* of material. It
> is excellent at being plausible, as a good BS artist will be.
> 
> And it can't think.
> 
> It can't reliably do arithmetic. Ask for a citation and it will frequently
> make up a broken URL. Because that is what fancy auto-complete can do, when
> it can't think.


This class of things is called "large language models". They are
models of language. They are not models of reality.

When you give them input strings -- "prompts" -- they extract a
string of words and punctuation that fits their model of what
will be statistically good.

If there are enough instances of "two plus two equals four" in
the training material, then prompting "what is two plus two?"
will get you a response of "two plus two equals four". There is
no arithmetic parser involved, and no equation solver. Wolfram's
Alpha has math parsers and equation solvers, but can't emit a
marketroid's babble about the comparative wonders of oranges and
machine screws.

-dsr-