Interdependencies and information flow in time series
More to come...
The dependencies that can exist between only two variables have been extensively studied, generating techniques that range from Statistical Inference to Information Theory. Most of these approaches differentiate the role of the variables, e.g. identifying them as target or predictor. Unfortunately, the extension of these approaches to study multiple interdependencies is not straightforward, as bipartite splitting is in general not enough to characterize the potential richness of the interactions between three or more variables. The existing approaches for analysing interdependencies between many variables can be classified in two groups. The first study the information that is revealed by different marginal distributions (simple, pairwise, etc.). Although these techniques have been successful in monitoring the application of Ising models for describing biological and social systems, their impact has been limited mainly to the Statistical Physics community as the results have little intrinsic meaning beside the mentioned models. The second group of approaches study the interdependencies as different modes of information sharing, decomposing the total information in parts that are shared by different sets of variables3. However, these models have not gained popularity because most of them assume the existence of negative information, without providing a satisfactory interpretation of it.
Recently, researchers have developed a novel non-negative decomposition of the information shared by many variables, which characterizes public information that is shared by all the variables and private information that is contained only in specific elements. The main contribution of our work in this field has been to distinguish between redundant and synergistic interdependencies: the former corresponds to variables that share the same information, while the latter refers to information sharing modes where the statistical dependency exists within the whole but not between the parts. An example of synergistic sharing is given by two independent fair coins feeding an Exclusive-Or (XoR) logic gate. In this case, the three variables (the two inputs and the output) are pairwise independent, while the group has a non-trivial global structure. It has been shown that the whole system contains two bits of information, which are shared synergistically across the three variables. The synergistic nature of XoR’s gates helps to explain their crucial role in Cryptography and Network Coding.
Synergistic dependencies play a crucial role in various contexts. For example, symmetric key cryptography work based on synergetic dependencies that can be created between the secret key, public signal and private message, which are not seen when the key is not available. Also, the performance gains that are achievable by coordinated transmissions in the Multiple Access Channel –a popular model in Network Information Theory– are generated by synergistic dependencies between the transmitters and the receiver. Besides its relevance in information technologies, we believe that synergistic interdependencies might play an important role in biology and neuroscience, where systems are characterized by non-trivial collective behaviour.
The implications of synergistic interdependencies have only begun to be explored. My goal is to analyse the relevance of synergistic modes of information sharing in different contexts, developing a deeper understanding of their nature and consequences.
It has been claimed that, similar to the way in which evolution takes place among living species, human society evolves in time from simple to more complex forms of organization and behavior. One of the distinctive and more challenging characteristics of complex systems, which has been widely acknowledged in social scenarios, is that the aggregation of the activities of simple components or agents can generate complex and unpredictable outcomes. Therefore, just as thermodynamics and statistical mechanics went beyond classical mechanics in order to provide an adequate framework for the description of gasses and liquids, a new theory might be necessary in order to enable a deeper understanding of important phenomena that characterize modern society.
In fact, new information technologies are defying our traditional tools of analysis, which were forged in times when the world was simpler and easier to predict. In the early days of Internet, the promises of abundance of information and the anti-authoritarian structure were thought to be seeds that would bring great benefits to society. Unfortunately, it is now clear that the excess of available information and the limited processing capabilities of individuals trigger confirmation biases, which stimulate the exclusive use of information sources that support one’s existing beliefs or points of view. Furthermore, online recommendation algorithms constantly and invisibly filter user’s queries, presenting contents that might better satisfy the user’s profile and preferences. All these elements are creating so-called digital echo chambers, where disjoint groups of society are progressively reinforced in their beliefs —whatever they might be.
One big challenge is to clarify the effects and consequences of the large amount of information that is constantly generated and exchange between individuals in a digital society. In one hand, the massive deployment and use of Internet mobile terminals and devices is enabling massive information networks. On the other hand, social habits are evolving concurrently with the pervasive use of Internet, making social networks an essential tool for information exchange. For example, most people nowadays use the Internet to check other people’s recommendations prior to making decisions for traveling, buying a product or choosing a restaurant. In these cases, subsequent decisions are influence by earlier agents, which allows possible miss-information and cascades across the network. Such complex interactions may defy intuition and are difficult to predict, and therefore an in-depth understanding of the inner mechanisms is very much desirable.
In order to provide a small contribution to address all these issues, our approach is to explore social learning as a distributed signal processing method, building a bridge between the research done separately by economists and sociologist, and electrical engineers and computer scientists. We apply tools from communication theory and signal processing in order to study these problems from a fresh angle and deepen our understanding about this fascinating phenomena.