The author is a Reuters Breakingviews columnist. The opinions expressed are her own.
By Aimee Donnellan
DUBLIN, July 14 (Reuters Breakingviews) - AI promises to make companies more productive. The same goes for hackers. While some blue-chip bosses struggle to spell out clear gains from large language models like ChatGPT and Claude, there's already a demonstrable improvement in the quality and sophistication of cyber attacks. It all suggests a troubling conclusion for CEOs: a chunk of the gains from AI will have to be siphoned off to pay for cyber insurance and ever-more sophisticated defense shields, sold by companies like $190 billion CrowdStrike CRWD.O and $270 billion Palo Alto Networks PANW.O.
The hacking attempts of old were often relatively easy to spot. They were littered with spelling mistakes, bad grammar or suspicious-looking links. Scammers told absurd stories about exiled African royals promising to share their riches. This poor quality was partly by design. Cybercriminals had a tough time breaking into company systems or individual bank accounts, and often resorted to wheezes that aimed to draw out only the most gullible possible victims.
As in many other areas, however, LLMs have allowed hackers to get faster and more focused in their hunt for weak spots. AI makes it trivially easy to generate and send thousands of highly personalised fake emails, written in the exact style of a victim's manager or colleague. Agents can rapidly scan millions of lines of code, identify vulnerabilities and launch thousands of simultaneous attacks. The same processes once took humans hours or days. Consulting firm McKinsey in late 2024 noted that the annual number of phishing sites detected ballooned by 138% shortly after ChatGPT's first release. There's no suggestion that attackers are using OpenAI's product specifically. Rather, the advent of LLMs more generally seems to have turbocharged bad actors' operations.
North Korea-linked attackers, always at the vanguard of the cyber wars, recently orchestrated a social engineering campaign that caught the industry's attention for its sophistication. It targeted Axios, a popular open-source tool that acts as a key piece of online plumbing. According to a post-mortem, the hackers masqueraded as a real company, including by cloning the founders' likeness. They invited the victim into a real-looking virtual workspace ostensibly on the popular messaging application Slack, complete with channels where workers were sharing LinkedIn posts. Next followed an invitation to a Microsoft Teams meeting with seemingly real attendees, during which something popped up saying that an item on the victim's system was out of date. This supposed Teams-related update was actually malware.
Such complex campaigns are getting more common. Google Cloud's Threat Intelligence service has written about other similar-sounding bespoke attacks, which seem to have involved AI "deepfake" video likenesses of real people of interest to the intended targets. LLMs could make it possible for a much broader pool of bad actors to industrialise these sorts of hacks, taking things well beyond ropey phishing attempts that most workers are accustomed to. If an experienced programmer can be hoodwinked, as in the Axios attack, what chance does anyone else have?
For cybersecurity firms, there's an obvious reply to this question: our products can keep your company safe. CrowdStrike, Darktrace and others are investing in new services that use AI to fight AI. Darktrace, for example, offers a behavioural monitoring system. It runs silently in the background, scanning emails and flagging unusual activity, including an unexpected tone shift or unusually urgent demand. Cyber firms are also tackling another nascent problem. As companies deploy multiple AI agents to automate work, hackers have found ways to corrupt or "poison" these semi-autonomous bots, weaponising them from within. It suggests greater future demand for new cyber products that detect when an internal agent has been compromised.
Investors are clearly excited about the opportunity for cyber behemoths, judging from the sky-high price-earnings multiples of CrowdStrike, Palo Alto Networks and Fortinet FTNT.O, which have surged this year even as the wider software sector has struggled with AI disruption fears. Combined revenue for this trio will exceed $46 billion by 2030, equity analysts reckon, according to Visible Alpha data. That implies a healthy compound annual growth rate of 16% from 2024's level. Their average operating margin will swell to a healthy 33%, the same broker forecasts show, compared with under 30% now.
Insurers like Chubb and Beazley are another key part of the picture. The cost of policy cover makes up about 10% on average of corporate cybersecurity budgets, compared with about 50% for third-party products and services, McKinsey reckons. Most of the rest goes to internal staff costs. The split, however, understates the underwriters' importance. Insurers increasingly require customers to strengthen their cyber defences before offering protection against catastrophic losses from hacks and the like. As attacks grow more frequent and severe, these policy requirements will only get tougher. In other words, the best salesperson for security software can sometimes be an insurance underwriter.
The flipside of rising sales for cybersecurity providers is the rising costs for customers. The question for CEOs, and investors, is to what extent higher IT security bills erode the wider AI productivity gains. From this perspective, it may be concerning that shielding against hacks is a neverending fight. More sophisticated LLMs beget more sophisticated attacks, justifying yet more spending on cyber protection to keep the bad actors at bay.
CEOs have been incessantly hunting for AI use cases over the past few years. It's ironic then that one of the clearest early applications for the technology lies in enabling more frequent and sophisticated attempts to hack big companies themselves. The cost of protecting against this danger threatens to take a bite out of AI's productivity promise.
PANW•
