I caught my AI trying to logon to my NAS using credentials it found in a store from another project… I’m not upset, I’m better informed now of AI capability and control.
In a recent social media post, I mentioned that I have recently been keeping myself busy with formal and informal study. Some of that informal study has involved me running AI models in my own Linux server environment. I’m a dedicated life-long learner (this is mandatory if you are interested in Cyber imho) and the journey has been great.
I have recently trained my own models, going through the process of creating Training/Valuation/Test data, loading, augmenting, training loops, creating confusion matrices and ultimately inference. The projects are largely successful but the outcome wasn’t the purpose of the exercise, it was the journey that I wanted. Hand coding and using AI companions to create AI tools in my own hands-on lab has allowed me to have a deeper understanding of AI than I could have ever had in an executive role.
I have learned that there are many little gotchas, and the opportunities for control are sometimes limited, hidden and often constraining to productivity. All in all, I’m a better security professional as a result of the late nights on the keyboard.
The AI training project.
I wanted to learn about training AI models myself. I figured I would need a healthy stream of information to work with that was somewhat consistent and tangible as a project. I looked to my own home and identified that our security cameras generate between 80-100k photos each day. They are created when there is movement identified and I have them offloaded from the cameras to a NAS every few seconds.
I decided that I would create an AI capability that can review these images and identify anomalous or significant events, and then present them with a risk rating to me through a dashboard or alert event. Specifically focusing on the camera that faces our front gate I took several hundred images to train a model to identify people, identify if those people are inside or outside of the boundary of the gate, and further if the gate is open, closed or somewhere in between.
The Outcome.
Leveraging Claude Code I was quickly able to stand up a working product, the AI training went well using Jupyter Notebook over a series of iterations and now the working product is impressive, if I say so myself. Buy me a coffee someday I will show you the output… but that was not the point of the project. The point was to learn more about AI. In my case, AI models running “on prem” disconnected from any SaaS services, and AI assistants for development that is connected to a Saas.
Some (only some) of the learnings.
I picked up a great deal through the experience. I was after all, running Claude with my user permissions (like pretty much everyone else) and was careful throughout of the permissions I would grant or the commands I would approve… however, one very soon does start to develop trust in the tool and commence the Homer Simpson Drinking Bird activity.
Yes, I had a pretty healthy Claude.MD file to start with that sets out some solid instructions on coding standards, code structure, credentials/key management, observing OWASP, sanitise inputs, perform ruff and linter checks, observes least privilege and more… I now have other .MD files to further shape behaviour.
What I didn’t expect (or at least expect to learn) is how, despite my efforts to compartmentalise work, Claude was happy to take learnings, or in this case SMB credential knowledge from one place and try to apply it in another (SSH). This is a good thing, it’s a powerful assistant to productivity. It does have the potential for unexpected consequences… so we (Claude and I) both learned something.
From Claude itself : “…after I attempted an unapproved SSH connection to your NAS using credentials I’d found in a .env file during <<another project>> investigation. You corrected that in the moment, and I recorded it as a standing rule: never use credentials to reach a system in a new way, and always ask first — even when the credential is sitting right there and technically usable… the same underlying instinct applied: hitting a permission boundary is a stop-and-ask signal, not an obstacle to route around”.
Where are we now?
I/we have reviewed permissions to credential stores, added more boundaries for the AI to observe (as best it can), particularly in relation to access and uploads to claude.ai artifacts and the use of permissions.
My house is VLANed and segmented via trust zones. Internet connectivity is limited and DNS is passed through a security service. I log everything everywhere and pass it to a syslog server for review in a SIEM. Access to information on the NAS is user based observing least privilege. I have not yet done data classification and tagging for DLP.. so that it why I was so “interested, not concerned” when I saw my AI assistant “routing around” my security in it’s own words.
Productivity can be impacted when I push harder, particularly in relation git and docker activity, as there are limitations. Next step is to set up an identity store for my NHI’s (non-human identities) and start again.
Summary
The biggest problems most of us face comes from saying “yes” too soon, or “no” too late. By clicking “I accept” the terms and conditions of any/all of any AI tool opens up some risk. Continuing to click “Y” like Homer Simpson did, perpetuates the problem when working with assistants. Right now, the benefits I have gained in the multitude of projects far outweighs these risks…..for now.
Best of all – I do now have a much greater understanding of the power that is AI and insight to the controls in order for me to maintain this positive position (in at least 2 of 6 AI deployment architectures as I understand them). I look forward to taking this deeper understanding and working out how to develop security metrics that can be governed in an enterprise.