Microsoft’s CEO, Satya Nadella, has issued a cautionary note, indicating that companies utilizing AI may encounter a dual financial burden: not only are they paying explicit token fees, but they may also be inadvertently forfeiting valuable proprietary data. Nadella highlighted that as businesses engage with AI, they continually feed proprietary knowledge into the system, including prompts, workflows, and model error corrections. This information is absorbed and retained by the model, potentially allowing the model provider to gain a deeper understanding of the business than the business itself. In a worst-case scenario, the provider could leverage this knowledge to establish a similar business, thereby directly competing with the original company.

Nadella advocates for a more equitable approach, suggesting that AI companies should permit other businesses to conduct “distillation” research on their models. This involves analyzing the model’s responses to grasp its underlying mechanisms and, subsequently, training new, cost-effective models. He argues that it is unfair for model providers to utilize external data for their model training while simultaneously preventing other businesses from scrutinizing their models in a comparable manner. Such a practice, he contends, amounts to a double standard.