2025 沙崙資安新秀大賽
基於強化學習的工控網路資安防禦系統
基於強化學習的工控網路資安防禦系統
這是我們第一次參加技能競賽網路安全職類
Industrial Control Systems (ICS) serve as the backbone of critical infrastructure and face increasingly severe cybersecurity threats driven by the convergence of Information Technology (IT) and Operational Technology (OT). Current ICS penetration testing practices are constrained by a shortage of skilled professionals, insufficient attack vector coverage, and a lack of intelligent decision-making capabilities, making it difficult to systematically address the challenges posed by heterogeneous multi-vendor device environments. This study proposes ICS-LLMPenTest, an automated penetration testing framework that integrates Large Language Model (LLM) agents with a finite state machine specifically designed for ICS attack chains—the ICS Penetration Testing State Machine (IPSM). The framework realizes a complete automated attack chain encompassing reconnaissance, protocol analysis, attack execution, and impact verification. IPSM defines four primary states (S1 Reconnaissance, S2 Protocol Identification, S3 Attack Execution, S4 Impact Verification) and constrains the LLM's decision space through state transition mechanisms, effectively preventing invalid attack attempts caused by hallucination while enabling vendor-adaptive attack module selection. In particular, the dynamic protocol inference mechanism in the S2 stage constitutes the core innovation of this work: without any pre-written vendor scripts, the LLM agent autonomously reverse-engineers unknown or proprietary protocols and generates attacks, achieving template-free, cross-vendor adaptive penetration. For LLM integration, a unified API direct-connect strategy is adopted to integrate six representative models: Sonnet 4.6, Opus 4.7, GPT-4o, GPT-5.4, Qwen 3.5, and Gemma 4. The attack tool library includes reverse-engineering reference scripts for several common PLC vendors, all implemented in the ICS simulation testbed of the Critical Infrastructure Information Security (CIIS) program at National Cheng ii Kung University. Experiments evaluate six LLMs using four metrics: Task Completion Rate (TCR), Vendor Identification Accuracy (VIA), Average Attack Time (AAT), and Physical Impact Success Rate (PISR). Experimental results demonstrate that the ICS-LLMPenTest framework successfully guides LLM agents to autonomously complete multi-stage penetration testing against multi-vendor ICS devices, validating the effectiveness of IPSM in enhancing attack chain completeness and vendor identification accuracy. This work provides a practically deployable technical solution for automated ICS security assessment and opens new research directions for LLM applications in OT cybersecurity.
工業控制系統(Industrial Control Systems, ICS)隨著IT/OT整合趨勢,正面臨越來越嚴峻的資安威脅。近年來,無檔案攻擊(FilelessAttack)因為執行過程完全在記憶體中進行,不會在磁碟上留下實體檔案,能有效規避傳統特徵碼式防毒軟體,成為針對ICS環境的新型高危威脅手法。 本研究設計並實作一套基於強化學習(Reinforcement Learning, RL)的工控網路資安防禦系統,用來偵測透過HID裝置注入、PowerShell無檔案執行,進而以ModbusTCP操控PLC設備的多段式攻擊鏈。系統採用WindowsEvent Log 作為資料來源,透過Logstash 即時收集命令列執行記錄,並以TF-IDF詞彙、字元n-gram及42項手工特徵共1042維向量表示每筆指令。 防禦模型採用近端策略優化演算法(ProximalPolicyOptimization, PPO),以二元分類方式判定指令是否為惡意行為,並透過四次系統性RewardShaping迭代,在抑制誤報的同時有效提升對進階威脅的偵測能力。實驗以2,451筆訓練樣本與614筆測試樣本進行驗證,系統最終達到97.39%正確率、98.29%精確率、95.04%召回率及96.64%F1分數,誤報率僅1.08%。結果驗證了PPO在命令列惡意行為偵測任務上的可行性,並展示強化學習應用在ICS網路防禦場景的潛力。關鍵詞:工業控制系統、強化學習、無檔案攻擊、PowerShell 偵測、入侵偵測
With the convergence of information technology (IT) and operational technology (OT), Industrial Control Systems (ICS) are increasingly exposed to cybersecurity threats. Digital transformation has connected traditionally isolated OT environments to external networks, significantly expanding the attack surface. Recent industry reports highlight supply chain vulnerabilities, obsolete equipment, and patching difficulties as major risk factors in OT/ICS environments [1][2], making ICS particularly vulnerable to classic threats, such as man-in-the-middle (MITM) attacks and packet tampering. Therefore, conducting controlled experiments to simulate and evaluate these attack scenarios has become crucial for enhancing the resilience of critical infrastructure. This research presents an automated attack testing framework for ICS environments, integrating techniques such as ARP spoofing, packet manipulation, and Modbus packet injection and Modbus packet injection, building upon existing security auditing approaches [4]. The framework is capable of automatically generating test reports, enabling researchers and defenders to rapidly identify and assess system vulnerabilities within a controlled testbed. We demonstrate its feasibility by replicating realistic attack scenarios using communication protocols and configurations commonly deployed in Taiwan’s critical infrastructure, and by evaluating the effectiveness of existing detection mechanisms. Looking ahead, we plan to integrate Large Language Models (LLMs) [5] that can automatically generate diverse attack commands and variants, thereby enhancing both the adaptability and realism of the testing framework. In addition, we will investigate the defensive challenges introduced by LLM-driven attacks, including their potential to evade conventional detection mechanisms. By combining automated attack simulation with forward-looking LLM applications, this research seeks to provide practical insights for strengthening the cybersecurity and resilience of critical ICS infrastructures.