Empowering LLMs with Toolkits: An Open-Source Intelligence Acquisition Method

Xinyang Yuan (), Jiarong Wang (), Haozhi Zhao, Tian Yan and Fazhi Qi
Additional contact information
Xinyang Yuan: Computing Center, Institute of High Energy Physics, Chinese Academy of Sciences, 19B Yuquan Road, Beijing 100049, China
Jiarong Wang: School of Nuclear Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China
Haozhi Zhao: Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Tian Yan: Computing Center, Institute of High Energy Physics, Chinese Academy of Sciences, 19B Yuquan Road, Beijing 100049, China
Fazhi Qi: Computing Center, Institute of High Energy Physics, Chinese Academy of Sciences, 19B Yuquan Road, Beijing 100049, China

Future Internet, 2024, vol. 16, issue 12, 1-20

Abstract: The acquisition of cybersecurity threat intelligence is a critical task in the implementation of effective security defense strategies. Recently, advancements in large language model (LLM) technology have led to remarkable capabilities in natural language processing and understanding. In this paper, we introduce an LLM-based approach for open-source intelligence (OSINT) acquisition. This approach autonomously obtains OSINT based on user requirements, eliminating the need for manual scanning or querying, thus saving significant time and effort. To further address the knowledge limitations and timeliness challenges inherent in LLMs when handling threat intelligence, we propose a framework that integrates chain-of-thought techniques to assist LLMs in utilizing tools to acquire OSINT. Based on this framework, we have developed a threat intelligence acquisition agent capable of decomposing logical reasoning problems into multiple steps and gradually solving them using appropriate tools, along with a toolkit for the agent to dynamically access during the problem-solving process. To validate the effectiveness of our approach, we have designed four evaluation metrics to assess the agent’s performance and constructed a test set. Experimental results indicate that the agent achieves high accuracy rates in OSINT acquisition tasks, with a substantial improvement noted over its baseline large language model counterpart in specific intelligence acquisition scenarios.

Keywords: large language models; agent; open-source intelligence; chain-of-thought (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/16/12/461/pdf (application/pdf)
https://www.mdpi.com/1999-5903/16/12/461/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:16:y:2024:i:12:p:461-:d:1538762

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().