Automated recognition of insect category, which currently is performed mainly by agriculture experts, is a challenging problem that has received increasing attention in recent years. The goal of the present research is to develop an intelligent mobile-terminal recognition system based on deep neural networks to recognize garden insects in a device that can be conveniently deployed in mobile terminals. State-of-the-art lightweight convolutional neural networks (such as SqueezeNet and ShuffleNet) have the same accuracy as classical convolutional neural networks such as AlexNet but fewer parameters, thereby not only requiring communication across servers during distributed training but also being more feasible to deploy on mobile terminals and other hardware with limited memory. In this research, we connect with the rich details of the low-level network features and the rich semantic information of the high-level network features to construct more rich semantic information feature maps which can effectively improve SqueezeNet model with a small computational cost. In addition, we developed an off-line insect recognition software that can be deployed on the mobile terminal to solve no network and the time-delay problems in the field. Experiments demonstrate that the proposed method is promising for recognition while remaining within a limited computational budget and delivers a much higher recognition accuracy of 91.64% with less training time relative to other classical convolutional neural networks. We have also verified the results that the improved SqueezeNet model has a 2.3% higher than of the original model in the open insect data IP102.