Cyber threats dominate the risk landscape and individual data breaches have become depressingly commonplace.

However, the muah.ai data breach stands apart. It presents extreme risks for individuals affected by the breach. There are reports that the information obtained from the breach is being used for extortion, including forcing affected employees to compromise their employer’s systems.

The role of in-house cyber counsel has always been about more than the law. It requires an understanding of the technology, but also lateral thinking about the threat landscape. We consider what can be learnt from this dark data breach. 

The background and breach – AI “companions”

The muah.ai website allows users to generate and then interact with an AI companion, which might be “caring AI-powered girlfriends, supportive boyfriends, or virtual therapists”. The interactions include both text (voice chat) and image (photo exchange). The service is marketed as being “uncensored” and “NSFW”.  

The service detected a hack last week.^[1] The breach is said to affect 1.9 million users and includes both the chat prompts users gave to their AI companion and the users’ email addresses. The website uses an email verification process (i.e. to confirm the email addresses provided are genuine) and some of the email addresses appear not to be “burner” accounts. Instead, they easily identify the underlying individual. 

Extreme high risk to individuals

The breach presents an extremely high risk to affected individuals and others including their employers. The leaked chat prompts contain a large number of “NSFW” suggestions that, at best, would be very embarrassing to some individuals using the site. Those individuals might not have realised that their interactions with the chatbots were being stored alongside their email address.

This is not just a risk to the individuals’ privacy but raises a significant risk of blackmail. An obvious parallel is the Ashleigh Madison breach in 2015 which generated a huge volume of blackmail requests, for example asking individuals caught up in the breach to “pay $2,500 in bitcoin or have your infidelity exposed”.^[2]

However, for some individuals caught up in the muah.ai breach this goes well beyond embarrassment. The chat prompts include a significant number of requests to generate child sexual abuse materials. In the UK, the making or possession of this type of pseudo-image is a very serious criminal offence.^[3] These individuals face a real risk of prosecution, imprisonment and registration on the Sex Offenders Register.

Using the information to exploit insiders

There is, likely, limited sympathy for some of the individuals caught up in this breach. However, it is important to recognise how exposed they are to extortion attacks.

There are reports that threat actors have already contacted high value IT employees asking for access to their employers’ systems. In other words, rather than trying to get a few thousand dollars by blackmailing these individuals, the threat actors are looking for something much more valuable.

Employees with privileged access to information technology systems present a significant risk. The employee’s action could open the door for a ransomware attack on their company’s IT systems or, given the increasing activity from nation state actors in the cyber space, something worse. With some employees facing serious embarrassment or even prison, they will be under immense pressure.

What can be done?

The role of in-house cyber counsel involves more than just knowledge of the law. It requires an understanding of the technology, a healthy and open relationship with the technology team, and a lateral assessment of the threat landscape, including the development of practical solutions to mitigate those risks.

Having said that, the options to respond to this particular incident are limited. You could ask affected employees to come forward but it’s highly unlikely many would own up to committing, what is in some cases, a serious criminal offence.

This does provide an opportunity to consider wider insider threats. As part of your wider measures you might consider:

• Ensuring that employees are cyber-aware and alert to the risk of personal extortion and compromise. This includes giving employees the means to report attempted extortion attacks and offering support to employees who report attempted extortion attacks, including identity monitoring solutions.
• Appropriate staff vetting both when employees are hired and, in some cases, on an ongoing basis (to the extent permitted under local privacy laws).
• Applying a “zero trust” principle by assuming that even those inside your network are potentially malicious actors and so need to be continuously validated. This should be backed up by a process to properly define the access rights given to those staff.
• Using strict Privilege Access Management^[4] controls for employees with that level of access.

For more information see our Cyber Security Handbook, The Essential Handbook for In-house Counsel

^[1] See Hacked ‘AI Girlfriend’ Data Shows Prompts Describing Abuse, 8 October 2024, 404 Media.

^[2] See Spouses of Ashley Madison users targeted with blackmail letters, 3 March 2016, The Guardian.

^[3] In the UK, the CSAM regime applies to both photos and pseudo-photos. For example see section 1 of the Protection of Children Act 1978. Computer-generated images, cartoons, manga images and drawings are separately criminalised under section 62 of the Coroners and Justice Act 2009

^[4] See https://www.ncsc.gov.uk/collection/secure-system-administration/use-privileged-access-management