Enhancing Data Security in SPARK Cluster: A Novel Symbol-based Authentication Approach

User authentication is the process of confirming an individual's identity prior to granting them access to a connected device, an online service, or any other valuable resource. Its importance lies in its capability to protect data, applications, and networks for organizations by restricting access to authorized individuals or approved processes. In this study, the widely used Apache Spark technology was employed for storing and analyzing vast amounts of data, and a unique authentication framework was introduced. A dynamic symbol selection authentication offers a promising alternative to traditional alphanumeric passwords, as well as biometric and facial authentications. This authentication method has been thoroughly tested in the highly distributed Apache Spark cluster. The implementation utilizes SHA512 cryptography in various ways and compares the results with existing authentication and machine learning algorithms. The authentication scheme, combined with the powerful Apache Spark distributed system consisting of 10 nodes, yielded exceptional outcomes.