X (formerly Twitter) ·
Deep learning excels at unlocking the creation of impressive early demos of new applications using very little development resources. The part where it struggles is reaching the level of consistent usefulness and reliability required by production usage.
Autonomous driving is the ultimate example. You could use deep learning to create an impressive self-driving car prototype in 2015 on a shoestring budget (Comma did exactly that, using Keras). Five years and billions of $ later, the best DL-centric driving systems are still L2+.
Every app demo based on GPT-3 follows this pattern. You can build the demo in a weekend, but if you invest $20M and 3 years fleshing out the app, it's unlikely it will still be using GPT-3 at all, and it may ever meet customer requirements
The reason why is that parametric models trained with gradient descent make it easy to automate something, but have little ability to deviate from the patterns they've learned. Meanwhile, the real world is full of surprises, and handling it requires the ability to adapt.
I did feel that. During my PhD I had some very good early results with cGANs. Months and months trying to improve those early results, and... it was frustrating how little improvement I’d get vs time spent, and more often than not, things would worsen actually.