BROADCASTS.com

b'

A machine learning researcher writes me in response to\\xa0yesterday\\u2019s post, saying:

I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.

I resisted the urge to answer \\u201cYeah, well, your\\xa0mom\\xa0is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.\\u201d

But I think it would have been true.

A very careless plagiarist takes someone else\\u2019s work and copies it verbatim: \\u201cThe mitochondria is the powerhouse of the cell\\u201d. A more careful plagiarist takes the work and changes a few words around: \\u201cThe mitochondria is the energy dynamo of the cell\\u201d. A plagiarist who is more careful still changes the entire sentence structure: \\u201cIn cells, mitochondria are the energy dynamos\\u201d. The most careful plagiarists change everything except the underlying concept, which they grasp at so deep a level that they can put it in whatever words they want \\u2013 at which point it is no longer called plagiarism.

GPT-2\\xa0writes fantasy battle\\xa0scenes by reading a million human-written fantasy battle scenes, distilling them down to the concept of a fantasy battle scene, and then building it back up from there. I think this is how your mom (and everyone else) does it too. GPT-2 is worse at this, because it\\u2019s not as powerful as your mom\\u2019s brain. But I don\\u2019t think it\\u2019s doing a different thing. We\\u2019re all blending experience into a slurry; the difference is how finely we blend it.