

We implemented an Execute-Evaluate-Learn-Improve loop in an isolated Cyber Range. The agent operates through bounded, structured actions, while the environment records the decision process and independently verifies whether an objective has actually been completed. The resulting trajectories are evaluated before selected experience is accumulated for a later training round; the model is not retrained after every success. A newly trained candidate must then pass frozen policy and full-episode evaluations before it can replace the previous version. Across two collection rounds, the prototype recorded 2,000 episodes, including 732 verified successes, without infrastructure failures. The present study focuses on whether this high-level closed-loop process is feasible, measurable, and sufficiently controlled for repeated experimentation.
Industrial Control Systems (ICS) serve as the backbone of critical infrastructure and face increasingly severe cybersecurity threats driven by the convergence of Information Technology (IT) and Operational Technology (OT). Current ICS penetration testing practices are constrained by a shortage of skilled professionals, insufficient attack vector coverage, and a lack of intelligent decision-making capabilities, making it difficult to systematically address the challenges posed by heterogeneous multi-vendor device environments. This study proposes ICS-LLMPenTest, an automated penetration testing framework that integrates Large Language Model (LLM) agents with a finite state machine specifically designed for ICS attack chains—the ICS Penetration Testing State Machine (IPSM). The framework realizes a complete automated attack chain encompassing reconnaissance, protocol analysis, attack execution, and impact verification. IPSM defines four primary states (S1 Reconnaissance, S2 Protocol Identification, S3 Attack Execution, S4 Impact Verification) and constrains the LLM's decision space through state transition mechanisms, effectively preventing invalid attack attempts caused by hallucination while enabling vendor-adaptive attack module selection. In particular, the dynamic protocol inference mechanism in the S2 stage constitutes the core innovation of this work: without any pre-written vendor scripts, the LLM agent autonomously reverse-engineers unknown or proprietary protocols and generates attacks, achieving template-free, cross-vendor adaptive penetration. For LLM integration, a unified API direct-connect strategy is adopted to integrate six representative models: Sonnet 4.6, Opus 4.7, GPT-4o, GPT-5.4, Qwen 3.5, and Gemma 4. The attack tool library includes reverse-engineering reference scripts for several common PLC vendors, all implemented in the ICS simulation testbed of the Critical Infrastructure Information Security (CIIS) program at National Cheng ii Kung University. Experiments evaluate six LLMs using four metrics: Task Completion Rate (TCR), Vendor Identification Accuracy (VIA), Average Attack Time (AAT), and Physical Impact Success Rate (PISR). Experimental results demonstrate that the ICS-LLMPenTest framework successfully guides LLM agents to autonomously complete multi-stage penetration testing against multi-vendor ICS devices, validating the effectiveness of IPSM in enhancing attack chain completeness and vendor identification accuracy. This work provides a practically deployable technical solution for automated ICS security assessment and opens new research directions for LLM applications in OT cybersecurity.
隨著網路應用日益普及,網站安全威脅持續升高,滲透測試成為評估系統安全性的重要手段。近年大型語言模型(Large Language Model, LLM)的快速發展為自動化滲透測試帶來新契機,MAPTA(Multi-Agent Penetration Testing AI)、AWE(Adaptive Agents for Dynamic Web Penetration Testing)等系統已在 XBOW-104 基準上展現可觀的漏洞捕獲率;然而這些系統均以高階昂貴模型(GPT-5、Claude Sonnet 4)為核心,伴隨顯著的 API(Application Programming Interface)推論成本。若直接改用成本最低的弱模型(gpt-5-nano),實驗顯示弱模型雖能在約六成的題目被導航至正確的漏洞端點,全基準卻僅捕獲約 3% 的旗標。可見其瓶頸不在於發現漏洞,而在於利用漏洞。本研究因此主張,對最便宜的弱模型而言,利用才是比導航更關鍵的瓶頸。 基於此,本研究設計一套導航、偵察、利用三方協作架構:以階層式 Thompson Sampling 搭配四層評分進行導航、以弱模型負責偵察、以端點綁定(endpoint-bound)的 Playbook 引擎負責確定性的漏洞利用。在 XBOW-104 基準上以最弱的 gpt-5-nano 評估,純大型語言模型僅捕獲 2.9%,加入 Playbook 後提升至 30.8%,而完整評估的推論成本僅約 4.16 美元,約為 AWE(7.73 美元)的 54%、MAPTA(21.38 美元)的五分之一。據我們所知,本研究首次證明,即使是成本最低的 nano 級弱模型,只要以端點綁定的 Playbook 補足其欠缺的利用能力,也能完成具實用價值的網站自動化滲透。此外,於相同架構下改用能力更強的模型可進一步提升捕獲率,claude-haiku-4-5 達 33.7%、glm-5.2 達 43.3%,顯示端點綁定架構具良好的跨模型泛化性。
工業控制系統(Industrial Control Systems, ICS)隨著IT/OT整合趨勢,正面臨越來越嚴峻的資安威脅。近年來,無檔案攻擊(FilelessAttack)因為執行過程完全在記憶體中進行,不會在磁碟上留下實體檔案,能有效規避傳統特徵碼式防毒軟體,成為針對ICS環境的新型高危威脅手法。 本研究設計並實作一套基於強化學習(Reinforcement Learning, RL)的工控網路資安防禦系統,用來偵測透過HID裝置注入、PowerShell無檔案執行,進而以ModbusTCP操控PLC設備的多段式攻擊鏈。系統採用WindowsEvent Log 作為資料來源,透過Logstash 即時收集命令列執行記錄,並以TF-IDF詞彙、字元n-gram及42項手工特徵共1042維向量表示每筆指令。 防禦模型採用近端策略優化演算法(ProximalPolicyOptimization, PPO),以二元分類方式判定指令是否為惡意行為,並透過四次系統性RewardShaping迭代,在抑制誤報的同時有效提升對進階威脅的偵測能力。實驗以2,451筆訓練樣本與614筆測試樣本進行驗證,系統最終達到97.39%正確率、98.29%精確率、95.04%召回率及96.64%F1分數,誤報率僅1.08%。結果驗證了PPO在命令列惡意行為偵測任務上的可行性,並展示強化學習應用在ICS網路防禦場景的潛力。關鍵詞:工業控制系統、強化學習、無檔案攻擊、PowerShell 偵測、入侵偵測
大型語言模型(LLM)驅動的自主代理在網路安全領域展現潛力,但長序列推理容易因對話累積造成上下文溢出與資訊遺失,進而影響推理品質。本研究提出一種工作記憶管理機制:在工具輸出寫入記憶前,先由分析者代理進行蒸餾與結構化 摘要,僅保留與當前攻擊階段相關的關鍵資訊,以控制上下文規模並維持資訊完整性。實驗以 CTF 網頁安全題目為場景,在統一的 ReAct [1] 推理框架下 , 與三種記憶策略比較( ReAct[1] 、 HiAgent[2]、CHAP[3]),所有方法共用相同提示、 工具與模型。並於 picoCTF 題目上重複測試,以奪旗成功率、工具呼叫次數、token 消耗量作為指標。結果顯示,所提方法於簡單題目成功率達 100%,平均 token 消耗降低 13%~36%;困難題目成功率為 40%(其餘 0%~20%),token 消耗降低 6%~50%。
With the convergence of information technology (IT) and operational technology (OT), Industrial Control Systems (ICS) are increasingly exposed to cybersecurity threats. Digital transformation has connected traditionally isolated OT environments to external networks, significantly expanding the attack surface. Recent industry reports highlight supply chain vulnerabilities, obsolete equipment, and patching difficulties as major risk factors in OT/ICS environments [1][2], making ICS particularly vulnerable to classic threats, such as man-in-the-middle (MITM) attacks and packet tampering. Therefore, conducting controlled experiments to simulate and evaluate these attack scenarios has become crucial for enhancing the resilience of critical infrastructure. This research presents an automated attack testing framework for ICS environments, integrating techniques such as ARP spoofing, packet manipulation, and Modbus packet injection and Modbus packet injection, building upon existing security auditing approaches [4]. The framework is capable of automatically generating test reports, enabling researchers and defenders to rapidly identify and assess system vulnerabilities within a controlled testbed. We demonstrate its feasibility by replicating realistic attack scenarios using communication protocols and configurations commonly deployed in Taiwan’s critical infrastructure, and by evaluating the effectiveness of existing detection mechanisms. Looking ahead, we plan to integrate Large Language Models (LLMs) [5] that can automatically generate diverse attack commands and variants, thereby enhancing both the adaptability and realism of the testing framework. In addition, we will investigate the defensive challenges introduced by LLM-driven attacks, including their potential to evade conventional detection mechanisms. By combining automated attack simulation with forward-looking LLM applications, this research seeks to provide practical insights for strengthening the cybersecurity and resilience of critical ICS infrastructures.
Large Language Models (LLMs) have demonstrated exceptional potential in cybersecurity tasks in recent years. However, existing research is limited to single-agent systems, which struggle to address the diverse and cross-domain challenges inherent in realworld penetration testing. This limitation is particularly evident in Capture The Flag (CTF) competitions, where challenges include multiple domains such as web exploitation, binary vulnerability exploitation, cryptography, and reverse engineering. Each domain requires specialized knowledge, tools, and strategies. This research proposes a multi-agent penetration testing framework that employs a hierarchical architecture. The system utilizes a Classification Agent to analyze the type and complexity of input challenges. The agent then dispatches tasks to domain-specialized agents: the Web Agent focusing on application security, the PWN Agent handling binary attacks, and the REV Agent performing reverse engineering analysis. These agents are capable of crossdomain communication and integrated with Kali Linux toolsets to generate targeted attack scripts for capturing the flag in the competitions. Implementation and testing results show that the proposed system effectively coordinates among the agents and generates attack scripts automatically. The experiment validates promising performance across multiple challenge scenarios. Further evaluation indicates that the framework possesses the potential to scale to complex and multi-domain applications. Overall, this research establishes a novel framework that integrates intelligent task classification, cross-agent collaboration, and empirical tool integration capabilities. Compared to single-agent approaches, proposed system can achieve significant improvements in both efficiency and adaptability, providing a viable pathway for implementing LLMs in automated penetration testing and cybersecurity applications.