Accurate load forecasting is critical for efficient and reliable grid operations

particularly given increasing volatility caused by atypical events such as extreme weather

policy changes

and social disruptions. Traditional forecasting methods based solely on historical numerical data often fail to account for these anomalies in cyber-physical-social systems (CPSS)

leading to significant prediction errors. To address this limitation

a novel hybrid forecasting framework is proposed

integrating large language models (LLMs)

retrieval-augmented generation (RAG)

and chain-of-thought (CoT) prompting. The framework leverages LLMs to systematically extract structured event-driven features from unstructured information in CPSS. These enriched contextual features are integrated into a specially designed forecasting architecture

termed Loadformer to optimize numerical prediction accuracy. Experimental validation using real-world electricity load data from multiple Australian regions demonstrates the superior performance of the proposed approach. The results highlight the substantial benefits of integrating qualitative contextual understanding from textual data into quantitative forecasting models

improving adaptability and robustness in scenarios characterized by volatility and unpredictability.