BROADCASTS.com

A machine learning researcher writes me in response to\xa0yesterday\u2019s post, saying:

I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.

I resisted the urge to answer \u201cYeah, well, your\xa0mom\xa0is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.\u201d

But I think it would have been true.

A very careless plagiarist takes someone else\u2019s work and copies it verbatim: \u201cThe mitochondria is the powerhouse of the cell\u201d. A more careful plagiarist takes the work and changes a few words around: \u201cThe mitochondria is the energy dynamo of the cell\u201d. A plagiarist who is more careful still changes the entire sentence structure: \u201cIn cells, mitochondria are the energy dynamos\u201d. The most careful plagiarists change everything except the underlying concept, which they grasp at so deep a level that they can put it in whatever words they want \u2013 at which point it is no longer called plagiarism.

GPT-2\xa0writes fantasy battle\xa0scenes by reading a million human-written fantasy battle scenes, distilling them down to the concept of a fantasy battle scene, and then building it back up from there. I think this is how your mom (and everyone else) does it too. GPT-2 is worse at this, because it\u2019s not as powerful as your mom\u2019s brain. But I don\u2019t think it\u2019s doing a different thing. We\u2019re all blending experience into a slurry; the difference is how finely we blend it.