The effect of the context length of large language models on the quality of extracted information in Polish in the process of project management

Abstract

PURPOSE: This paper presents GraphRAG, a novel tool that integrates large language models (LLMs) with knowledge graphs to enhance the precision and consistency of responses generated from unstructured text data. The primary objective is to improve the quality of information retrieval and synthesis for complex user queries requiring comprehensive understanding.

DESIGN/METHODOLOGY/APPROACH: The GraphRAG framework processes source documents by dividing them into smaller fragments (chunks) to facilitate knowledge extraction. Using community detection algorithms, such as the Leiden algorithm, GraphRAG identifies semantic clusters within the knowledge graph, enabling both local and global information retrieval. The tool employs a multi-stage analysis approach, leveraging prompts to detect entities and relationships in the text, which are then organized into structured graph nodes and edges.

FINDINGS: The experimental results reveal that smaller chunk sizes (e.g., 300 tokens) significantly improve the granularity of detected entities and relationships, leading to a more detailed knowledge graph structure. This approach enhances response accuracy for knowledge-intensive queries by enabling the LLM to focus on specific text segments, improving the precision of extracted information.

PRACTICAL IMPLICATIONS: GraphRAG has practical applications in any domain where accurate and context-rich responses are essential, such as customer support, decisionmaking, and research analysis. By balancing chunk size and processing efficiency, the tool enables scalable analysis while maintaining high data quality, making it a valuable asset for knowledge-intensive tasks.

ORIGINALITY/VALUE: This research contributes to the field by demonstrating an effective integration of LLMs with knowledge graphs to process large text corpora. GraphRAG’s method of combining local and global retrieval through knowledge graphs represents an advancement over traditional retrieval-augmented generation methods, especially in scenarios requiring detailed information synthesis.