How AI Tools Make Data Breaches Easy
Every day, people continue to utilize AI in order to work faster and more efficiently. It is tempting to use AI to summarize information, analyze documents, and help with coding. More and more often, it is used to generate ideas or bounce ideas off of (it doesn’t seem to ever disagree either!). This comes with a downside, to say the least. People are using AI at work: is it safe?
Under the Radar Data Leaks/Access to Intellectual Property
AI chats are now the number one cause of data leaks in the workplace, beating out email,
according to a report by Cyera. Employees are often pasting sensitive company information
such as financials, strategy documents, inside reports, and personnel information directly into AI
chats for analysis. It is estimated that 77% of these chats involve real company data. 67% of the
time these chats occur within employee private accounts.
Real world example:
Healthcare employees are increasingly using AI tools to handle patient notes and assist with
documentation. Considering that many public AI tools are not HIPPA-compliant nor do not
provide Business Associate Agreements, significant privacy, compliance, and therefore medical
ethical concerns have surfaced.
While the staffers were not acting maliciously and simply trying to work faster, healthcare
organizations have had to step in with warnings or restrictions on AI usage after discovering that
patient data had been entered into public AI systems.
A Blind Spot for Security Teams
Private account use is a two-fold issue. First, IT teams are unable to monitor personal logins or
provide oversight. Primarily, this is what makes AI a blind spot for security teams.
Security software is also being bypassed because people are using screenshots or cutting and
pasting information into chat bots. These actions with internet windows open makes it appear as
simple internet usage rather than the downloading of documents which would instantly trigger
security software.
A Potential Component to Whaling Schemes
It is worth noting another tie in here that is not commonly discussed. Dark web sales of private
AI account logins are commonplace. ChatGPT logins have been stolen by the hundreds of thousands, thus giving access to entire chat histories. Attackers can use these as entry points.
Additionally, these chats can be a part of the complexity of a whaling scheme. A whaling
scheme is when a target (executive, celebrity, politician, etc) is accessed through a data breach.
Instead of an immediate attack, the person is studied in order to convincingly impersonate them for what will be the real attack, or even a series of illegal actions.
Let us illustrate: an executive at a company is a target. Having access to that person’s AI usage
(or someone close to them) is a perfect enhancement to the “discovery” portion of a whaling
scheme. AI searches and any analyses provide useful information to a scammer (s) studying
their target. Eventually when all information is gathered from all sources (emails, badge access,
phone calls, texts, etc.) the attacker will utilize all information and impersonate their target so
effectively that it will be easy for them to achieve their goal (obtaining funds or highly
sensitive proprietary information).
Protections: Perspectives and Policies
Most employees would never intentionally leak data. AI LLMs like Claude, ChatGPT, Gemini,
and Grok are tools and their use feels normal at this point. Here are some policies and
discussions to put into place:
- Inform everyone that chat interactions are basically file transfers.
- Have people ask themselves if this is information they would leave laying around a
conference or hotel lobby. If not, treat it with the confidentiality it requires. - Block access to generative AI from personal accounts.
- Require single sign-on (SSO) for all AI tools from company devices.
- Monitor keyboard strokes and clipboard activity for sensitive keywords or phrases.
- Talk to us about a deployment model for your company so AI can be utilized within the
boundary of your company’s IT.
If you have any questions or need a plan, we are your team for this. Contact us or call
908.895.0273 with any questions or to talk about next steps to take to protect your company.
