October 7, 2026 at 12:00 am
Comments posted to this topic are about the item Smart AI Agents and Data Security
October 7, 2026 at 7:11 am
My take is that the creators of these tools don't fully understand what they have built, so how the hell are the rest of us supposed to?
If these breaches take place unintentionally, what happens when AI is used to make deliberate attempts to carry out a breach?
AI is a useful tool. A rotary saw is a useful tool; weedkiller is a useful tool. With each of them, you have to handle them with care and respect. There has to be an investment in training and ongoing education to make sure such tools are used safely.
October 7, 2026 at 12:16 pm
Do you know how many credentials are shared?
all of the shared credentials create an auditing nightmare
October 8, 2026 at 12:25 pm
Well,
when we're talking about LLM 'AI', the security question has a simple answer in terms of how much the security can be trusted. The parable of the scorpion (scorpion and the frog) pretty much sums it up. You can't 'trust' it due to its very nature.
Now, if you're using and training an absolutely local LLM, ok, you can limit the 'blast zone', but at its heart an LLM does a simple thing. It stochastically (apparently randomly, but actually algorithmically) regurgitates anything that's put into it, in terms of outputs or actions. So, in terms of the impact per what it does, or what it releases is felt over its entire domain. So, it can leak, or as an agent impact upon, and in terms of confidentiality exfiltrate, anything it has access to. And in the case of an open, as opposed to local, LLM - that's - in terms of impact or data release, well, everyone.
The LLM doesn't 'care' that the data you pointed it to for a BI analysis is sensitive patient data, or people's banking details, nor that your Stochastic Parrot code generation tool has been pointed at your companies trade secrets (or indeed, as has been documented, that the passwords and other settings are things you have told it not to ingest) which is absolutely confidential. It's all grist to the mix - taken, just sitting there, waiting for the right prompt to spill it.
See, it's a scorpion. It can't help its nature.
I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.
Viewing 4 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply