New research on AI and security, selected for relevance to language models, AI agents and cybersecurity. Summaries are the authors’ or publisher’s own words, linked to the original.
Peer-reviewed
The public trusts scientists with AI — more than it does anyone else
A unified view of system identification in the learning era
New preprints (not yet peer-reviewed)
On Private Knowledge Retrieval and Private Inference for secure multi-agent systems
Multi-agent systems provide a convenient framework for the massive deployment of agents based on Large Language Models (LLMs). It provides a sound framework where agents can interact with existing software to perform actions in an autonomous way.
BypassGym: A Benchmark for Evaluating AI Agents' Endpoint Defense Evasion Capabilities
Artificial intelligence agents increasingly automate cybersecurity tasks, including offensive operations. Frequent attacks across the web make endpoint detection and response (EDR) a critical defense, but whether AI agents can evade it remains unclear.
From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings
LLM agents are increasingly asked to make plans for people, such as scheduling drivers or assigning nurses. However, people's requests are often incomplete, and the supporting information for decision-making may be outdated, requiring agents to ask for clarification.
TERM: Target-Free Jailbreak Defense via Entropy-Guided Residual Modulation
Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate harmful responses.
Specification-based Security Evaluation of Real-World Agents Tool-Calling under Long-Horizon Context Tasks
As large language model (LLM) agents are increasingly deployed in long-horizon unattended scenarios, security evaluation of them before deployment becomes increasingly essential and challenging. Existing evaluation methods mainly follow the paradigms of LLM-as-a-Judge or Agent-as-a-Judge.
RoCoMark: Robust Code Watermarking Against LLM-Based Functionality-Preserving Rewriting Attacks
Large Language Models (LLMs) present significant challenges to source tracing of generated code, as LLM rewriting can substantially alter the syntactic and structural characteristics of watermarked code. However, existing watermarking methods often suffer severe performance degradation under LLM-based functionality-preserving rewriting attacks.
POEM: Output-Relative Block-Sparse Attention for Long-Context Prefilling
Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate harmful responses.
An Empirical Study on CVE-based Vulnerabilities Propagation from one System to another System (IntraProject vs InterProject)
Vulnerabilities in applications, including browsers, media players, online services and document readers, etc. are often exploited and cause a serious damage. The vulnerabilities could exist in your system maybe because of some coding errors, design oversights, outdated software, AI generated codes or maybe transferred from another system during development (e.g., code clones).