Darktrace studentships

Darktrace, a global leader in cyber security AI, has partnered with Loughborough University to provide funding, resources and expertise for cross-disciplinary research at the intersection of artificial intelligence and cyber security.

Applications are invited for up to three PhD studentships for 2024 entry, as part of a partnership between Loughborough University and Darktrace.

The first out of five doctoral researchers under this partnership has already started and they will be able to team together to support each other on their research.  Each will also benefit from the support of other Doctoral Researchers at Loughborough and from Loughborough’s committed staff. We are a community based on mutual support and collaboration. Through our Doctoral College there are continual opportunities for building important research skills and networks among peers and research academics.

The successful candidates will be working in collaboration with Darktrace, a sponsoring company in the area of cybersecurity that will provide supervision and insights into real-world applications. In addition, the doctoral researchers will have access to a growing research group in cyber security and close collaborations with the The Alan Turing Institute.

The doctoral researchers will have access to state-of-the-art machine learning servers equipped with A100 GPUs and connected to the Loughborough University high-speed network facilities. Opportunities for exchanges and short visits to world-leading AI laboratories within our extensive collaboration network will be encouraged. The collaborative environment ensures exposure to diverse perspectives and facilitates knowledge exchange.

Details of the projects are given below:

Primary Supervisor: Prof. Tim Watson

Email: tim.watson@lboro.ac.uk

Cyber attacks tend to focus on the most adaptable parts of a system. Almost invariably this means people. Attackers vary from opportunist insiders persuading a colleague by email to bend the rules for personal gain through to nation-state groups that combine human agents, technical infiltration and significant resources to achieve their objectives of stealing secrets, damaging critical infrastructure or influencing elections. Cyber influence operation timescales vary widely too, from an email with a malicious link read and clicked in seconds to a long-term campaign that builds over several years.

AI, and in particular the spectacular recent advances in generative AI, create as yet poorly understood opportunities and threats. Mass phishing campaigns might now become targeted spear phishing at scale with Large Language Models (LLMs) conducting their own identification of key targets and open-source intelligence, and crafting exquisitely persuasive communications. Attacker tradecraft might continue to evolve, with LLMs conducting conversations using deepfake audio or video and convincing supporting evidence (documents, email chains, websites, chatlogs, network packet captures etc.). We need defences against this looming threat.

  • This PhD proposal covers a broad landscape of open research questions.
  • How can cyber influence techniques be tested for their effectiveness?
  • Can agent-based human simulations help?
  • How do LLMs alter the threat landscape for cyber influence operations?
  • Which cyber influence effectiveness tests will help us to develop countermeasures against these attacks in the most effective and economic way?
  • How do you perform system-level resilience tests rather than simply testing individuals?

Primary Supervisor: Dr Kostas Kyriakopoulos

Email: K.Kyriakopoulos@lboro.ac.uk

Resilient cybersecurity relies on people, processes, and technology and this is particularly true in cyber-physical environments, such as in Industry 4.0. In such environments cyber-threat susceptibility and risk remediation require a diverse, collaborative cohort of experts with different background. Some of these stakeholders are experts focusing on the cyber domain and others on the physical domain. Due to the multiple stakeholders, deriving mitigation procedures against cyber attacks within such environments is complex, but of upmost importance for the industrial sector.

To address the above challenge, this PhD project is positioned at the cross-section of the most exciting, growing, and influential technological concepts. Specifically, the project aims to explore the use of Large Language Models (LLM) in tandem with Causal Inference, both integrated in a Reinforcement Learning (RL) framework. The ultimate goal is to enable training autonomous decision-making agents that learn to defend cyber-physical environments under their respective constraints and unique challenges.

The successful PhD candidate will investigate new machine learning approaches leveraging models that capture the cause-and-effect dynamics between different domain knowledge of human experts. Furthermore, LLMs will be used to address the semantic dissonance and priority conflict among experts and optimise agents when deciding on action deployment towards risk remediation in critical environments.

The academic consortium has a strong track record in cyber-security projects, funded by Dstl/Ministry of Defence, and prior success in licensing fundamental research to commercial companies operating in the defence and other sectors.

A successful candidate is expected to demonstrate strong understanding in Machine Learning fundamentals, and be confident in programming skills and Machine Learning packages.

Primary Supervisor: Dr Andrea Soltoggio

Email: A.Soltoggio@lboro.ac.uk

This research project focuses on advancing the understanding and capabilities of large AI models, particularly transformers, through the innovative use of sub-network regions known as "masks." The goal is to break down and isolate knowledge components, enabling precise tracking and analysis of model decisions.

Large AI models often lack transparency, making it challenging to interpret and rectify inaccuracies. The proposed research aims to pioneer a novel transformer architecture where knowledge is segmented into distinct concepts represented by masks. By combining subsets of masks, specific outputs can be achieved. This approach facilitates causal inference, allowing us to unravel the contributions of each mask to final decisions. The ultimate goal is to design models that shift from correlation-based decisions to causation-based hypothesis testing and decisions.

The project builds on emerging parameter isolation algorithms in lifelong learning and knowledge composition, areas actively explored in Dr. Soltoggio's research group in collaboration with Prof. Georgina Cosma. Recent advancements in causality studies and the increased focus on interventions suggest a convergence of reinforcement learning with foundation models, providing a unique opportunity for groundbreaking research.

Candidates should have a Strong background in machine learning and mathematics as well as Proficiency in programming languages commonly used in AI research (e.g., Python, TensorFlow, PyTorch).

Primary Supervisor: Prof. Georgina Cosma

Email: g.cosma@lboro.ac.uk

As large language models (LLMs) are deployed for cybersecurity applications such as analysing threats in text data, being able to update and unlearn specific parts of their knowledge is critical.

However, the massive size and complexity of LLMs makes traditional retraining on new data practically infeasible. This PhD research will advance machine unlearning techniques to enable selective, efficient forgetting in LLMs for cybersecurity.

The aim is to design and develop algorithms that can erase specific learned representations from LLMs without degrading overall performance. Key challenges include isolating parametric dependencies related to targeted data and minimising catastrophic forgetting during the unlearning process. This research will also investigate innovative and efficient approaches, such as sparsity in overcoming catastrophic forgetting in LLMs.

Successful techniques could allow LLMs to adaptively forget selected (e.g. outdated or biased) knowledge, while retaining broader capabilities. This has the potential to transform how LLMs are updated and expanded for real-world cybersecurity applications. The research will focus both on developing new selective unlearning theory and demonstrating its feasibility in state-of-the-art LLMs. Outcomes of this research project align with the UK’s national AI strategy on developing trustworthy and responsible AI systems.

The PhD student will join the established research group [Neural Information Processing, Retrieval & Modelling] led by Professor Cosma. This group has a vibrant community of 7 PhD students and 5 research associates focused on advancing LLMs, with ongoing projects related to AI, reasoning, continual lifelong learning, neural information retrieval and machine unlearning. Dr. Soltoggio heads a complementary AI research group studying continual lifelong learning algorithms and memory mechanisms. Overall, this environment provides a scientific community and expert supervisors to support the proposed PhD research on selective machine unlearning for cybersecurity LLMs.