Mathematical theory of democracy (Q2393418)

The book has its genesis in the author's doctoral thesis and subsequent research work on social choice theory. It is divided in three parts, the first of which is a selective review of political systems and their analyses beginning with ancient Greece and culminating in modern times. The second part is a detailed presentation of the author's mathematical works placed in the context of social choice theory, voting paradoxes, and modern day political systems of representative government. The third, and final, part consists of applications of the methods developed in the second part to areas outside of political systems and social choice such as finance, polling, statistical quality control etc. The first part, which constitutes a little less than a third of the book, starts with a fairly detailed look at the kind of ideas of political systems and of the kind of governments which evolved in ancient Greece, in particular, democracy or mixed systems of democracy together with aristocratic and monarchical elements as practiced in Athens and Sparta, and their critical analyses carried out by scholars in ancient Greece and later during the Roman era and beyond, continues on with the systems prevalent in the Roman republic and subsequent city states such as Venice and Florence, and with the evolution of political systems in Britain, the United States, France and the Netherlands, and finally ends with contemporary representative democracy as practiced in most Western nations. This section is a very welcome prelude to the second part of the book and makes the author's work in the second part much more intelligible to the reader, if only because of the motivation it provides for the subsequent abstract theoretical parts, by underpinning the questions studied, and their methods of analyses, with concrete historical situations which gave rise to them. Aside from providing the reader an idea of the progression and development of democratic systems and political thinking, it makes interesting observations on notions of what constitutes democracy and how it has historically evolved over time, as well as discussions of voting paradoxes and their role in the evolution of social choice theory. The reader whose primary interest is social choice theory would find illuminating discussions here. For example, there is a section discussing Pliny the Younger's analysis of voting mechanisms in ancient Rome and the problems of unsatisfactory outcomes of voting methods as well as problems of manipulability they are open to. Another section discusses the ranking method of Borda and Laplace's justification of Borda's method within a probabilistic framework, showing that integer valued substitutes of real valued estimates of the merits of candidates in an election is a good approximation, with approximation errors becoming negligible as the number of voters becomes large, as well as delves into the problem of strategic voting and manipulability that the Borda scheme is open to and the consequent advantage it gives to mediocre candidates, something which was not only recognized at the time but actually borne out by voting results in the French Royal Academy of Sciences. The investigations by Condorcet of pairwise voting, stimulated by the problems encountered by Borda's method, leading to the paradox of cyclic majorities and his suggested voting method is discussed in detail in a subsequent section. Moreover, in a probabilistic framework, i.e., where the votes and merit estimates by the voters of the candidates are random variables, he proves that the modern day conjecture, i.e., that the methods of Borda and Condorcet almost certainly lead to the same ranking when the number of voters is large, is correct, something which is an original contribution to the literature. In the last chapter of the first part, Chapter 5, after a historical narrative and discussion of the evolution of democracy during the period of the American and French Revolutions until modern times, the author discusses several interesting contributions to the theory of voting and ends with a detailed presentation of Arrow's Impossibility Theorem in social choice theory, including some subsequent works on the possibility of choice rules satisfying Arrow's axioms when the number of individuals in the group is infinite, and especially stresses the hierarchical nature of decisive groups and less so the concept of a dictator and tries to draw some parallels with power structures in modern day democracies. There are several points on which the approach of this book is based. First, is the observation that while all of the Arrovian axioms have been called into question in the earlier literature, the least examined had been the axiom of non-dictatorship, possibly due to the emotional baggage the term carries; however, in the author's eye, the notion of an Arrovian dictator is not to be rejected summarily with an uncritical eye as might be tempting to do at first sight. It is the contention of the author that modern day democracies involve a hierarchy of powers which narrows down to a single individual with the ultimate decision making authority who could be labeled a `dictator' in the Arrovian sense but really governs and makes decisions in most situations with the help of a minority of individuals whose nested subgroups, always including the `dictator', forms the narrowing hierarchy of the power structure. Second, that the notion of democracy in ancient times was always intimately linked to choosing magistrates or executives or administrators by lot; however, this has changed in modern times to a notion of democracy which stresses universal suffrage together with the election of representative forms of government by some form of voting method. And third, that voting is a good way of accepting or rejecting proposals but that does not mean it is a sensible way of electing representatives. In ancient times it was recognized that voting was sensible when the electors formed a fairly homogenous group with shared values and which is politically well informed, such as the aristocracy, but not so when the group was a diverse heterogenous group with little in common and little opportunity or luxury to pay attention to public issues, such as the common people -- the demos. That there are problems with using voting methods in the election out of a field of candidates, especially when the degree of heterogeneity of opinion within the voting group is substantial was possibly realized, intuitively, in ancient times. Election by voting was used in selecting individuals for service from groups which were fairly homogeneous in their composition but not used as a general practice in decision making. Moreover, election of administrators by lot, perhaps after restricting suitably the field of potential candidates using filters based on the qualifications of the candidates and record of past service, is more egalitarian and democratic in that it gives everyone, or rather all of those adjudged to be qualified, an equal chance at the privilege of governance as well as in bearing the burden of governance. Any `paradoxes' of voting, when faced with elections from a group of candidates, regarded in this light are not really paradoxes to be somehow `solved' or bypassed. In short, the approach of the book basically turns attention away from concerns in social choice theory with dictatorial or oligarchical powers and other shortcomings of various procedures for group decision making and instead concentrates squarely on how to measure representativeness of the appropriate bodies elected or nominated and how to improve on the representativeness by suggesting modifications to election methods. As the author clearly recognizes, ``democracy is not regarded from the viewpoint of the quality of decisions'' but rather from the viewpoint of ``the quality of the \textit{representation} of public opinion by single representatives or representative bodies as \textit{agents} of the society''. Moreover, a major concern of modern day democracies is the declining participation by electors in the democratic process. To the extent that this is because of a perception of a `democratic deficit' in that election procedures are viewed as flawed and as a result end up with non-representative governments, an exercise such as the one undertaken in this book is of considerable practical importance. The viewpoint above leads essentially to eschewing concerns such as rationality of choice procedures, aggregation of individual preferences, possibility of a meaningful concept of collective or group welfare etc., and instead concentrates on a measurement problem of how to estimate \textit{representativeness} of a sample chosen, somehow, from a population and how the procedure of such sampling could be improved upon to achieve better representativeness. It is an approach of a statistician, pure and simple, and the major part of the author's work, which is expounded in the second part of the book, centers around proposing a probability approach to estimate representativeness of elected bodies, large or small, and using these in some concrete examples to give an idea of their effectiveness. Three indices are proposed as measures of representativeness for a decisive body \( D\), which could be an individual or a group. Imagine that there are a number of issues on which dichotomous questions, i.e., with yes/no answers, are asked. Since a polynomous question can, in principle, be broken down into several dichotomous ones, this is not a severe restriction, at least theoretically. Let a finite set of \(m\) elements \(Q\) denote the set of such questions \(q\). The importance of the questions is conceptualized as a probability measure \(\mu \) over \(Q\) so that an \(m\)-vector \(\mu =\{\mu_{q}\}\) denotes the weights or importance of the questions with \(\mu _{q}\) denoting the weight of question \(q\). Let \(I\) be the set of \(n\) individuals (electors or citizenry)\ and \(v=\left\{ v_{i}\right\} \) similarly stand for an \(n\)-vector of weights \(v_{i}\) denoting the importance or weight of individual \(i\). The representativeness of the decisive body on a question \(q\) could be represented by \(r_{qD}\equiv \sum v_{i},\) where the summation is over those individuals \(i\) whose answer to \(q\) coincides with that of the decisive body. A popularity index measures the average size of the groups represented by \(D\), i.e., \(P_{D}\equiv \sum_{q}\mu _{q}r_{qD}\). The universality index of \(D\) is the frequency or fraction of questions on which \(D\) represents a majority, i.e., \(U_{D}\equiv\sum_{q}\mu _{q},\) where the summation is over questions \(q\) for which \(r_{qD}\geq 0.5\). Finally, the goodness index of \(D\) is the average of the ratio of the group represented to the size of the majority over the range of the questions, i.e., \(G_{D}\equiv \sum_{q}\mu _{q}\frac{r_{qD}}{\text{ weight of majority on question } q}\). Note that the last one is of use because if the universality index of a body is not too large it may still provide a good representation because the size of the majority is small and as a ratio to that it could be fairly large. A geometric interpretation of the indices is provided, formulae for expected values and variances are computed, curves relating expected goodness of a body to the weight of the minority in the society are analyzed as the size of the decisive body increases, and it is noted that when situations are very controversial and the weight of the majority is small, a representative chosen by lot may do just as well as a large body and hence achieves the same result using less resources and hence may be more efficient. In Chapters 8 to 10 the indices and their extensions are used to study five German parties and their coalitions using results from the 2009 Bundestag elections, party positions and the results of a public opinion poll on 30 policy issues. The election winner does not turn out to be the best representative of public opinion, furthermore, the best representatives may get too few votes to even participate in the ruling coalition and, moreover, coalition formation can actually lower representativeness. A statistical likelihood test of whether the representative capacity of a body is significant or not is developed and applied to the Bundestag parties and coalitions, which reveals that none of them pass the significance test. The author believes that the poor representativeness is in large part due to the electors being influenced by modern-day media coverage of candidates, which focuses much more on the media personalities of the candidates rather than on their and their party's policy platforms and ``competence as representatives'', and he goes on to suggest an alternative election procedure to enhance the representativeness of elected bodies. The alternative procedure suggested is to include, in individual ballots, dichotomous questions about individual voter positions on a few key issues which appear in the manifestos of candidates or parties and then to evaluate the candidates by the degree to which their profiles match with that of the \textit{electorate as a whole}. A distinction is drawn between an individualistic political philosophy emphasizing representing electors as opposed to one which is public in nature, presuming that there is a ``public interest differing from the sum of individual wills'' and that society is like a single body and possessed of ``a political profile with regard to these issues'' resembling a general will, a concept such as that espoused by Rousseau, and emphasis is placed upon representing this electoral profile or general will. Accordingly, the results of two different types of election procedures, ``architectures'' in the terminology of the author, are contrasted in idealized examples. One of them chooses one candidate for each elector, on the basis of which of the candidates most closely represents the elector's profile in the sense of largest number of issues on which the candidate and the elector sees eye to eye. Then these choices or votes are used to decide on the winning candidate by some standard method, say first past the post kind of system. In the second method, for each of the issues, the representative(s) are found who most closely represent the majority opinion and then the candidate who represents the majority most frequently over the range of issues is selected. In other words, the indices of popularity etc., developed earlier, may be used in this method. If the election method involves allocating seats by proportional representation then such a method could be used to determine the proportions for each party using indices of party representativeness. Of course, a suitable combination of the first and second method or even a combination of vote on a candidate together with indices of representativeness weighted in some agreed upon way could be used in practice. Needless to say, the two methods applied to reallocation of seats based on the Bundestag election voting pattern and poll data results in a much more representative composition of the parliament and possible coalition governments. To the extent that it is perceived that there is a problem of representation in a particular election procedure in a particular society, certainly it does seem to make sense, in principle, of adopting some such variation of the election procedure. We say in principle because a more elaborate method such as this can be a source of frustration for voters and contribute to diminished participation, as well as turn out to be a much more expensive method which yields but little practical benefit in reality. As mentioned earlier, the remainder of the book is devoted to examining various other non-political issues such as polling, traffic control, stock exchange predictions etc. using the indices of representativeness. The preceding account basically summarizes the content of the book in its essentials. As already mentioned, the author provides a novel viewpoint, engaging analyses in a stimulating historical context, and interesting new methods and suggestions which certainly have the strong merit of being oriented towards practical concerns. Having said this in commendation, a few constructive criticisms will be offered in the remainder of this review. The title of the book appears somewhat misleading. It is true that the theory of social choice and/or voting methods and problems in these areas are an important part of the work examined here in that they are extensively discussed, however, the original part of the contribution of this work does not seem to contribute much, if at all, to such theoretical problems. If the reader is willing to presume that the end result of elections should be to have representatives who `best' represent the opinions of the electorate as a whole then the reader can proceed directly to the chapters dealing with the indices of representativeness developed and the subsequent follow up discussions and extensions, without losing much of substance or context as a result. This seems to be a straightforward statistical measurement problem to which some commonsensical estimates are suggested but without any kind of underlying guiding principle or maxims as such. The author himself takes pains to stress at one point that the axiomatic method initiated by social choice theory following Arrow's seminal work is not suitable, in his estimation, for the practical task of democratic governance and that the axiomatic method, while it works well for certain parts of mathematics and logic by deductively reasoning starting from obvious or self-evident premises, has gradually ceded ground to the inductive reasoning involved in the physical sciences. However, the bulk of the original work displayed here seems to involve neither deductive nor inductive reasoning. Mathematical methods may have been used to formulate statistical estimators and explore their properties but doesn't seem to have anything to do with exploring the concept of democracy or social choice and/or good decision making mathematically. It is quite unclear to what extent, if at all, this may be regarded as a theory of democracy. The author mentions at one point that ``elicitation of empirical knowledge requires the analysis of observations and development of quantitative methods. From this viewpoint, the invention of calculus in the 17th century was a great breakthrough. It is noteworthy that calculus, as opposed to geometry, has no axioms but only definitions and computations.'' It is noteworthy, in the context of this remark, that calculus, as well as astronomical observations and the inductive laws of planetary motion of Kepler, aided and indeed was essential in the formulation of Newton's gravitational theory but calculus, in and of itself, is not regarded as any theory. Neither is a formula such as the Gini coefficient a theory of inequality of income distribution, at least that's not the conventional understanding. A reader who is anticipating a mathematical theory in the conventional sense may be in for a disappointment. At various points of the text the author seems to subscribe heavily to a concept of general will which, he is at pains to stress, was important in the contributions of the likes of Rousseau, Condorcet and Borda -- a concept of society as a single body having an opinion over issues, in the manner of an individual, which is the profile of the electorate as a body -- and it is the purpose of his methods to discover that general will from data about individual opinions on such issues and find its closest match amongst the candidates. This viewpoint seems to finesse the whole problem of social choice. One of the major stimulation behind Arrow's problem was presumably the question whether it makes sense to presume that society has a general will of its own which is based squarely upon individual will in a meaningful and sensible way. After all it had long been a custom to talk of the `general good' or `well being of society as a whole' but at the end of the day, as Arrow demonstrated, there seems to be a fundamental difficulty with such a concept even in situations which, at first sight, seems well circumscribed and sensible. If such a problem is simply assumed away it is quite unclear why the book devotes so many pages to the discussion of social choice theory. Having said this it must be noted that the above criticism is more pertinent to the case of a small number of electors given the demonstrated convergence, in a probabilistic sense, of the results of the methods of Condorcet and Borda for large numbers of electors. In connection with the point made in the preceding paragraph, at various points the author notes that his researches originated in developments that took place in social choice theory subsequent to Arrow's work; in particular that the idea of having a `dictator' may not be so reprehensible after all because developments in social choice theory involving an infinite number of individuals made it clear that behind any (rational) choice rule there is an underlying hierarchy of nested groups whose will (when they are in accord) guides social choice. This seems to be interpreted as: if there is one person as the fountainhead of this hierarchy, namely the `dictator', then this is a person who consults and shares in the decision making powers with the groups who appear in the hierarchy and these groups are equally responsible for the decisions made. Moreover, it is stressed that even in modern day democracies this is the prevailing power structure in that there is a narrowing of decision making groups with ever larger spheres of influence ending in one person such as a prime minister or president, at the top of the hierarchy, who has the final say. From this standpoint there may be good and bad dictators with the good ones being good representatives of the society they lead. This interpretation of what a `dictator' means in the setting of social choice theory is very hard to understand and subscribe to. The stress in the entire discussion seems to be on whether the person (or a small decisive group) at the top of the hierarchy represents well, and frequently, the opinion of a large fraction of the populace on different issues \textit{at a particular point of time}. However, the attractiveness of the non-dictatorship axiom of social choice theory lies precisely in that one person's preference pattern decides social choice under \textit{all possible scenarios} of individual preference profiles (including that of the dictator). Imagine that the current situation is one where all individuals hold identical preference ranking over the alternatives and the social ranking is identical to this. There is supposedly a hierarchy of decision making with a chief at the top `who has the final say' but it is unexceptionable because he represents completely the community's `general will'. But if we now imagine that the chief's preference pattern overnight switches to oppose the rest of the community's (identical) preference pattern on every issue and the chief's view still carries the day that is the kind of decision making power that an Arrovian dictator is allowed to possess and which is why the non-dictatorship axiom seems so unexceptionable. It is hard to think of any modern day democracy where the constitution can actually allow for such a scenario of absolute power. In fact not even in so-called dictatorships is this possible since a dictator holds on to his power because he can find a group of other individuals who is willing to act in concert with him and possesses enough control over the administrative and security machinery to suppress anyone in opposition. It is necessary to allow wide variations of the preference profiles of the populace to make this axiom meaningful and relevant. Simply considering the desirableness or otherwise of a power structure with a particular preference profile in mind says very little. The grounds on which the author's acceptance of the notion of a dictator seems convoluted and based upon an interpretation of the notion which does not seem to be in accord with the meaning of the non-dictatorship axiom. But, then again, the fundamental premise of the study seems to be that what matters are not axiomatic principles but rather the practical aspect of measuring representativeness and so this is perhaps a moot point except in so far as this indicates that the entire discussion of dictators, good or bad, seems somewhat beside the point of the study. The author seems to interpret the conjecture (for which he provides an original proof) that the methods of Borda and Condorcet almost certainly lead to the same ranking when the number of voters is large as confirmation that the principal disadvantage of Borda's method that it is open to manipulation by voters disappears when the number of voters is large because the pairwise ranking method of Condorcet is independent of other alternatives. However, this needs to be viewed with some caution. After all if manipulation is indeed an objective of electors it seems likely that their preference patterns expressed are not going to be independent of one another, which is a basic assumption of the probability approach, but rather there are incentives for coalition formation with the common objective of manipulating the results of a ballot by agreeing on a common pattern of voting. Such coalition formation, or at least attempts to do so, are not unknown in elections within democracies, even fairly large ones, which take the form of some kind of vote trading. The points above have to do with the theoretical side of matters. On the more practical side, the suggestion of the author to modify election procedures may have some disadvantages of their own. Part of the reason is the underlying presumption that the main thing which is at stake here is whether election results in a representative government. Many may well object to that saying that what matters is a strong decisive government which is able to provide good strong leadership. From this angle, personalities matter a great deal. After all people do change their views over time and when in power they find their perspective has changed and platform promises and views of core supporters are not necessarily the driving factors anymore. A strong leader may be more reliable to chart a good course even if the voter doesn't see eye to eye on major issues because the voter trusts the integrity and judgment of the leader to appreciate better the voter's position when faced with the complicated realities of power. However, this is not a serious objection in itself since the author's suggestion can be used to modify, according to the electorate's taste, but not entirely replace election procedures currently in place. A bit more of a problem may be that consequences may flow from the necessary changes in electoral rules. It is suggested that the modifications to an election method by including voter opinion on a range of issues and by matching up each candidate's manifesto or publicly stated opinions, giving a weight to this along with a voter's actual vote, to decide which candidate or party won will be less subject to manipulation and put less weight on personalities and media coverage. However, as mentioned earlier, there may be good reasons why people do vote for candidates based on personalities rather than just stated political platforms. If rules are changed in the recommended way, there is no reason to expect that people will simply state their opinions on issues candidly as before and instead skew them in a way to match the profile of the candidate they would have voted for. It would also be to the advantage of candidates to shift positions following opinion polls to try to get as much weight in the final seat count out of the proposed mechanism, something which already happens when parties tend to converge to the centre as election looms closer and look indistinguishable to the voters which is often a major complaint. The combination of the two forces may lead to the very undermining of the data on which the proposed new method is supposed to rely on. In the opinion of this reviewer, overall, this is a well-written engaging book offering new ideas which are certainly worth considering when attempting practical reforms to modern-day democratic institutions and is well worth reading.