How much do language models memorize?

I enjoyed reading this paper. The experiments are well-designed and it's well-written.

Much work on generalization of ML models deals with asymptotic bounds. Here, there's a precise way of measuring these, even for relatively large models with millions of parameters.