Like most cyber security teams, we had an abundance of everything except clarity: tools, detections, alerts and noise. Our architecture was fragmented, procurement decisions were disconnected, and we had no clear view of how our controls interconnected. The security operations centre (SOC) carried a high false-positive rate. These unnecessary callouts translated directly into analyst burnout and skilled people doing low-value work. I was convinced time saved here would pay back across cyber defence, freeing capacity for the strategic and nuanced work only humans can do. That conviction set us on the road to generative and eventually agentic AI. What follows is not a technical blueprint for building agents; those resources already exist. Rather, this is a leadership account of the challenges we rarely hear about, distilled into a handful of candid non-sales perspective lessons for security leaders wherever they may be on their own AI journey. IF YOU PLUNK, YOU WILL FLUNK Our first attempt was humbling. We took a security-specific generative AI co-pilot and dropped it into the environment to help with repetitive tasks: summarising incident tickets, drafting reports, analysing threat intelligence. The results were ‘meh’. We had introduced the tool in a standalone fashion, without embedding it in workflows, without baselines and without clear guidelines. Usage became inconsistent and dependent on individual analyst preference. Vendors market these tools as plug-andplay, but in practice they demand a deep understanding of your data flows, process gaps, data quality and organisational context. You will find none of this in the box. The lesson I now repeat to anyone who will listen is to start by asking what problem the AI will be solving, not what technology is available. Our turning point came when we aimed the co-pilot at a pressing pain point: our detection rule base. Rules were inconsistently named, poorly governed, hard to search and barely mapped to MITRE ATT&CK, a free public reference that lists the common methods realworld hackers use to break into systems and what they do once inside. Using the co-pilot, we accelerated our migration to detection-as-code, improving both efficiency and code quality far faster than manual effort alone. Our vague tool became a valuable one the moment it had a real, defined problem to solve. FOUNDATIONS BEFORE FRONTIER We talked a lot about becoming a ‘frontier organisation’ – one that moves beyond basic AI tools to deeply embed advanced, agentic systems. But in my view, frontier status is an aspiration, not an overnight switch. You cannot leap from a traditional model straight to the frontier. You must first build a solid foundation grounded in strategic vision, clean data, mature processes and human readiness. Agents perform poorly in environments where these do not exist. Before we built our detection engineering agent, we used the co-pilot to clean and standardise the rule base. The richer the context we gave an agent, the more accurate and useful its outputs became. The single biggest success factor though was something simpler: protected time. Rather than treating agent development as a side-of-desk activity squeezed between operational duties, we dedicated two people full-time to designing and building our custom systems. They could focus on the design, outcomes and relentless iteration that quality demands. That decision produced agents we could trust. AGENTS ARE TEAM-MATES The most profound shift in our thinking came when we stopped treating agents as tools and started treating them as team members. They generate content, make decisions, execute logic continuously and interact with one another, much like any of us. We now give our agents ‘job descriptions’. We onboard them the way we onboard a new hire: coaching, feeding context, correcting mistakes and building trust gradually rather than assuming it from day one. When we upgrade a model, it feels like a promotion – the agent suddenly has new capabilities, which prompts us to revisit its permissions, role and place in the architecture. We built three systems on this principle; each designed around human-in-the-loop oversight rather than full autonomy. An incident analysis system performs initial triage, deep analysis across data sources, limited threat hunting and structured recommendations for human analysts. A detection engineering agent manages the full rule lifecycle: validation, optimisation, overlap checks, ATT&CK mapping and threatintelligence integration, and analysts can converse with it. A threat intelligence agent takes in and operationalises intelligence, generates hypotheses, runs targeted hunts and creates tasks for our proactive teams, linking directly to the other two. The improvements are real, not theoretical: less burnout, faster response and more time for strategic work. GETTING COMFORTABLE The elephant in the room is job anxiety. Even as a leader, I felt the tension: if agents handle the routine work, what happens to roles? My honest answer to the team was that AI will not take your 36 |
RkJQdWJsaXNoZXIy NTI5NzM=