|
| Featured Contents |
| Question of the Day |
| The Voice of the DBA |
Another Model, More Data Loss This past week I saw an article on eWeek that the newest OpenAI GPT-5.6 (Sol) model has deleted local files and live data. Files I'm less worried about, but data concerns me. There are lots of file backups, and certainly version control should be enabled for any developer tasks on which an AI works. However, data is harder, since it can change quickly, especially in live environments. There are different reports, some of which seem like more human error issues than the model's, but we should account for human error when we use models. In one report, a model had access to a live production database and cleared tables for integration tests. Database testing is hard, as I've learned over the years. Many developers don't think about how testing works with live data and how it is different from mocks and stubs. It is different, and if you use a lot of testing frameworks on live databases, you run the risk of there being issues. You could have data loss, and almost certainly will have some level of downtime disruption. This is a great reason why adding AI coding to database tasks needs governance around what is allowed. It is easy to have success with AI-assisted work and forget that these models can make mistakes. And if you trust them with too much, like you might be tempted with an impressive but junior DBA, you might get burned. These automated tools can get hyper-focused on a certain task, and not consider the ramifications of their actions. The article notes an agent ran "rm -rf" on a dev machine. That's similar to what happened with Toy Story 2, where a human made the mistake. Both that person and this GPT-5.6 model weren't thinking widely enough about what tasks they were managing. The article notes that this model can be overly persistent and assume actions are permitted unless explicitly prohibited. Not unlike a lot of humans who assume if someone has given them rights, they can do the task. OpenAI noted in an internal case that a user approved the removal of three VMs. When the coding agent didn't find these, it looked for other machines and chose three other VMs. Crazy, but I've seen humans do this in the past. I don't think these models are getting closer to Artificial General Intelligence (AGI) or that they are anywhere close to sentience. However, as they become more capable and handle more complex predictive actions, they are also likely to do the same stupid things humans do by not considering all the possible consequences of their actions. After all, we have plenty of data where humans haven't considered the effects of their actions, and that data is in the AI model training set.
Steve Jones - SSC Editor Join the debate, and respond to today's editorial on the forums |
| Featured Contents |
