We investigate a problem of composing distributed information in big data systems. We construct analytical cost models and formulate a generic Distributed Information Composition problem in Big Data Systems, DIC-BDS, to aggregate multiple distributed datasets using a composition operator of specific complexity to produce one final output. We rigorously prove this problem to be NP-Complete and design two heuristic algorithms as solutions.

An illustration of how information is composed in a big data system to form a tree, where the root of the tree represents the final output.

We propose blockchain-based decentralized IoT management systems for censorship resistance, which include a “diffusion” function to deliver all messages from sensors to all full nodes and an augmented consensus protocol to check data losses, replicate processing outcome, and facilitate opportunistic outcome delivery.

We also leverage public key aggregation to reduce communication complexity and signature verification. The experimental results from proof-of-concept implementation and deployment in a real distributed environment show the feasibility and effectiveness in achieving censorship resistance.