A RESTful messaging system for asynchronous distributed processing

Abstract

Traditionally, distributed computing problems have been solved by partitioning data into chunks small enough to be handled by commodity hardware. However, such partitioning is not possible in cases where there are a high number of dependencies or high dimensionality, such as in reasoning and expert systems, rendering such problems less tractable for distributed systems. By instead partitioning the problem, rather than the data, we can achieve a more general application of distributed computing. Partitioning the problem rather than the data may require tighter communication between members of the network, even though many networks can only be assumed to be weakly-connected. We believe that a decentralized implementation of propagator networks may resolve the problem. By placing several constraints on the merging of data transmitted over the network, we can easily synchronize information and achieve eventual convergence without implementing mechanisms needed for serialization. To this end, we present the design of a RESTful messaging mechanism, currently in the process of being implemented, that allows distributed propagator networks to be created, using mechanisms that result in eventual convergence of knowledge across a weakly-connected network. By utilizing a RESTful design of the mechanism, we can also achieve a reduction of bandwidth usage during synchronization through the use of caching.