To address the low efficiency of course knowledge graph construction in the education domain, an automated and accurate knowledge extraction framework is proposed to enable intelligent analysis and structured organisation of teaching resources. First, large language models (LLMs) are employed to perform data augmentation on a small set of manually annotated samples, thereby expanding a high-quality training dataset. The augmented dataset is then used to fine-tune open-source LLMs. By integrating chain of thought (CoT) prompting and positive-negative example prompting strategies, model performance in entity and relationship extraction tasks is enhanced. In knowledge extraction experiments conducted on the course Mechanics of Materials, the proposed FT-CoT-ICL method achieves F1 score improvements of 38%–43% and 23%–28% over traditional Zero-shot and in-context learning (ICL) methods, respectively, in entity extraction. For relationship extraction, F1 score increase by 44%–46% and 31%–35%, respectively. LLMs employing this method significantly outperform larger commercial LLMs in course knowledge extraction tasks while reducing inference costs, and superior performance is also observed compared with current state-of-the-art information extraction models. Experimental results demonstrate that this framework effectively activates LLM performance in entity and relationship extraction, thereby enabling automated construction of course knowledge graphs and showcasing its potential applications in knowledge engineering.