One of the most important lessons we've learned from customer implementations is the diversity of templates that exist among various teams. Due to the fact that our Key-Value-Pair model must be trained on specific templates in order to enhance performance, obtaining sufficient training data (samples for each template type) is a significant difficulty, even more so when newer templates are utilized in projects.
While demonstrating these GPT-3-based solutions to customers, occasionally overselling occurs or they believe the models do everything. This is not true. GPT-3 frequently generates deprecated code and makes deprecated library calls. Occasionally, it just does not work properly and produces rubbish. Designing the appropriate prompt is a talent that must be gained via extensive use of GPT-3 and experimentation with how to configure the model parameters. As of July 2021, GPT-3 is not integrated with any cloud systems, posing security and integration difficulties.
Newer models like CuBERT (Code Understanding BERT) are constantly being developed and it is critical to use it for NLP/NLU jobs to increase quality.