The U.K.'s fireworks celebrations last week may be behind us, but a breach at AI infrastructure provider Fireworks AI has kept the focus this week on a different kind of explosive fallout in the form of security risks in enterprise AI supply chains.
The incident exposed customer datasets, incomplete model weights and sensitive account information through a third-party provider, once again raising questions about the security of the infrastructure supporting enterprise AI services.
The concern for CX leaders is how much customer information now passes through systems and providers beyond a company's direct control. Customers share personal details, account information and sensitive queries with businesses, often with little visibility into the external platforms processing that data.
For me, the Fireworks AI breach points to a wider challenge around customer trust and other developments this week show how broadly that challenge applies, with a number of developments suggesting implications for how enterprises protect customer information and maintain confidence in their services.
After all, the reputational cost of getting AI security and privacy wrong is high. As Ram Ramamoorthy, Director of AI Research at Zoho, told me in an interview this week:
“You want to be very privacy first... you are going to be in the news if you have a privacy breach, if you have a security breach... the reputational damage [from] a security breach or a privacy misconfiguration where data leaves the ecosystem is still too big when compared to the productivity gain that comes from using a lot of these tools.”
Protecting customer data remains a critical condition of adoption for enterprises weighing the efficiency gains of AI against the risks.
Fireworks AI Breach Exposes the Security Risks Behind Enterprise AI Supply Chains
The security incident involving Fireworks AI raises questions about how well enterprises understand the third-party systems processing their data, after attackers accessed customer account information, billing records, datasets and incomplete AI models.
Fireworks initially disclosed the security incident on October 9, after identifying the unauthorized use of internal credentials on October 6. At the time, it said attackers had accessed environment variables containing application programming interface (API) keys and tokens used in evaluation jobs across a limited number of customer accounts.
The initial investigation found no evidence that customer models, datasets, training data, personal information or inference traffic had been accessed. But an expanded investigation involving Palo Alto Networks' Unit 42 subsequently identified a wider impact.
In an update on November 9, Fireworks said the incident had exposed resource metadata and billing information, including customer addresses, invoices, payment history, account balances and credit usage. For a limited number of accounts, attackers also accessed secrets, datasets and incomplete model weights.
The incident has implications beyond Fireworks' direct customers. Notion confirmed that information accessed through its AI infrastructure provider included chat transcripts processed for its Insights feature, which helps customers understand their AI usage. On November 11, Notion confirmed that it had revoked exposed Notion API keys discovered in affected users' messages.
Fireworks said it had found no impact to its Inference, Training SDK and Nexus services, which operate with zero data retention (ZDR) by default. But the incident demonstrates how evaluation jobs, stored datasets, development environments and other supporting workflows can create separate exposure points.
The implications for CX leaders extend to customer conversations and the information employees share with AI assistants. Businesses need to understand which external providers process prompts, where associated datasets and credentials are stored, and whether their suppliers' security assurances cover the full AI supply chain.
The incident also raises questions about how quickly organizations can identify which downstream applications and customer interactions may have been affected by a supplier breach.
Maersk Investigates Unauthorised Access And Data Extraction From Legacy System
Shipping and logistics group Maersk is investigating unauthorized access to a local legacy system inherited through a previous acquisition, after identifying that data had been extracted from the environment.
In a November 11 security advisory, the company said its cybersecurity team had launched an investigation and taken steps to secure the affected system.
Maersk said initial findings strongly suggested that no sensitive data had been disclosed and there was no indication that other company systems or customer and partner environments connected through APIs and electronic data interchange (EDI) integrations had been affected.
The company said it does not currently anticipate disruption to customer operations, with its platforms for booking and managing shipments continuing to operate as usual.