Abstract: In order to solve the problem that the infrared thermal image of photovoltaic panels contains a large amount of noise and it is difficult to identify the hot spots caused by the uneven distribution of infrared images in different states,based on the Vision Transformer（ViT）model,the convolution neural network is used to improve the model feature extraction,and the compact multi head self-attention mechanism is used to improve the model structure. A photovoltaic infrared image hot spot recognition model,a compact vision transformer（ConCViT）,is proposed,by which pretrains the attention weight using CIFAR-10 data set. Taking small sample photovoltaic infrared images with low signal-to-noise ratio as the data set,a high accuracy hot spot detection model is trained. The experimental results show that the recognition accuracy of ConCViT model is 12.02% higher than that of traditional convolutional neural network,4.14% higher than that of deep convolutional self-coding network,and has faster convergence speed.