BROADCASTS.com

b'

Gwern has answered my prayers and\\xa0taught GPT-2 poetry.

GPT-2\\xa0is the language processing system that OpenAI announced a few weeks ago. They are keeping the full version secret, but have released a smaller prototype version. Gwern retrained it on the\\xa0Gutenberg Poetry Corpus, a 117 MB collection of pre-1923 English poetry, to create a specialized poetry AI.

I previously tested the out-of-the-box version of GPT-2 and couldn\\u2019t make it understand rhyme and meter. I wrongly assumed this was a fundamental limitation: \\u201cobviously something that has never heard sound can\\u2019t derive these complex rhythms just from\\xa0meaningless\\xa0strings of letters.\\u201d I was wrong; it just didn\\u2019t have enough training data. Gwern\\u2019s retrained version gets both of these right, and more too. For example:

Thou know\\u2019st how Menoetiades the swift
Was dragged, of Hector and the fierce compeers
And Phrygian warriors. So, we will dispatch
Your bodies, then, yourselves to burn the ships
In sacrifice; with torches and with bells
To burn them, and with oxen to replace
Your gallant friends for ever. But I wish
That no man living has so long endured
The onset of his foes, as I have power
To burn or storm; for mighty Hector erst
Was slain, and now returns his safe return

This is all perfect iambic pentameter. I know AP English students who can\\u2019t write iambic pentameter as competently as this.

(by the way, both \\u201ccompeers\\u201d and \\u201cerst\\u201d are perfectly cromulent words from the period when people wrote poems like this; both show up in Shelley)

It has more trouble with rhymes \\u2013 my guess is a lot of the poetry it was trained on was blank verse. But when it decides it should be rhyming, it can keep it up for a little while. From its\\xa0Elegy Written in a Country Churchyardfanfic: