Security analysis of P-SPN schemes against invariant subspace attack with inactive S-boxes

The security requirements of new applications such as cloud computing, big data, and the Internet of Things have promoted the development and application of security protocols such as secure multi-party computation, fully homomorphic encryption, and zero-knowledge proof. In order to meet these demands, there is a need for new symmetric ciphers that minimize multiplications in \( {\mathbb {F}}_{2^{n}} \) or \( {\mathbb {F}}_{p} \), where p is prime. One construction that addresses this demand is Partial SPN (P-SPN) construction, where the S-box layer is only applied to a portion of the state in each round. And there have been several research on the construction over the past years. The key to the design of P-SPN construction lies in the linear layers, but systematic exploration in this direction has been lacking in the existing work. In this work, we first establish a lower bound on the dimension of the maximal invariant subspace without active S-boxes for a generic P-SPN scheme. Subsequently, we concentrate on the linear layers of P-SPN construction. Through a meticulous examination of intriguing and beneficial characteristics for various matrices, we showcase that the security of a P-SPN scheme against invariant subspace attack depends on the degree of the minimal polynomial of the matrix. Inadequate choices of the matrices allow for large invariant subspaces that navigate any number of rounds without activating any S-boxes. A comprehensive proof for the Conjecture 1 proposed by Keller and Rosemarin is presented, which not only further improves the lower bound on the dimension of the maximal invariant subspace for the P-SPN rounds of STARKAD permutation, but also implies a lower bound on the dimension of the maximal invariant subspace for block matrices with special blocks. For a block circulant matrix with special blocks, a better annihilating polynomial exists and a lower bound on the dimension of the maximal invariant subspace can be identified. For circulant matrices and block circulant matrices with circulant blocks, we introduce methods to ascertain the range or exact value of the minimal polynomial degree. This determination advances the exploration of the invariant subspaces in these matrices. Especially if the number of S-Boxes in a P-SPN scheme is 1, we can attain the exact value of the dimension for the maximal invariant subspace. All the cases discussed here are invariant subspaces with inactive S-boxes. Our work intends to provide concise cryptanalytic methods for new proposals following P-SPN or HADES design principles. In addition, we derive a way to make sure that a circulant matrix C is resistant to invariant subspace attack with inactive S-boxes, thus providing design criteria for the construction of such matrices in the design of P-SPN schemes.