Breaking the Algorithmic Contract

Too often, we see the deluge of content streaming through our tech platforms as a wave washing over us. But the reality is that this is a “wave” we are choosing to ride.

It’s useful to think of our engagement with algorithms as a social contract. Political theorists have long used the social contract as a device to explain why individuals submit to the authority of a nation-state, how security, meaning, and protection of natural rights offer legitimacy.

In the same way, we need to consider why we see the algorithm as legitimate, why we surrender what I call our “curational autonomy” to algorithms. One reason is the relief we get from the “anxiety of choice” along with the illusion of safety from an increasingly complex world. We might allow tech companies to cluster and classify us to make our lives more predictable.

Doing so is good for business: A more predictable consumer means a higher return on advertising expenditures, and economic incentives encourage the optimization of prediction models. If a model cannot predict consumer behavior, it fails. But is that what people are for? As Walt Whitman wrote, “I am large, I contain multitudes.”

If we humans are not naturally predictable, and if models cannot predict capricious humans, what if humans are being made to act more predictably? If algorithms cannot develop models that fully predict us, will we be trained to increasingly become like algorithms and classify ourselves according to the model’s desires? In You Must Become an Algorithmic Problem: Renegotiating the Socio-Technical Contract, I argue that it is time to break the unspoken agreement we have with tech companies.—Jose Marichal

* * *

In our increasingly digital world, algorithmic models take every typed word or gesture (likes), eye movement, or swipe and breaks them down into either a node (thing) or an edge (attribute), which is placed into a database. This age of incessant collecting and analysing of our digital life is what I call an algorithmic age. In the algorithmic age, the priority is to create products that can predict our future behaviour. Prior to the 2010s, prediction was important, but understanding ourselves and other humans was the priority. We have moved into a regime where data is collected not simply to understand humans for marketing or surveillance purposes, but to create artificial replicas of human thought.

Research and lived experiences increasingly show that most people are not ‘swimming against the data tide’. They are content to accept a world of algorithmic classification despite its obvious harms because it gives them a (false) promise of comfort and security. We continue to be glued to our devices, increasingly using social media platforms as the foundation of our information diets. A 2024 Pew study found that 86 percent of users got their news from digital devices, and a growing number (54 percent) sometimes or often get their news from social media platforms. On social media sites popular with young people, 40 percent of Instagram users and 52 percent of TikTok users regularly get their news from each platform (St Aubin and Liedke 2024a). If we are under the throes of algorithmic overlords, we are not acting like it.

For this reason, we should think of our relationship to algorithms through the lens of contract theory. The concept of a social contract is a core element of political theory. Political theorists have used it to justify why individuals should form allegiances to a particular political system. It is a thought experiment designed to illustrate a relationship, one that cannot possibly be universal in practice since individuals have different reasons for their allegiance to a state. Nonetheless, it is one that provides legitimacy for state power. The three most prominent applications of contract theory come from Locke, Rousseau, and Hobbes, who posit that rational actors will willingly give up their theoretical position in a ‘state of nature’ either for protection of the self (Hobbes 1967 [1651]) or for an increased preservation of rights (Locke 1996). In Rousseau’s (1920 [1762]) case, leaving the state of nature is a fact and the only way to restore a sense of meaning and an escape from the judgement and status consciousness of modernity is to submit to your political community – the general will. In each case, the social contract justifies adherence to a political system. The system will either protect your physical person (Hobbes 1967 [1651]), preserve your natural rights (Locke 1996 [1689]), or provide you with meaning (Rousseau 1920 [1762]).

The contract perspective is a useful framework for understanding our relationship with engagement algorithms. The Internet presents us with a vast, unlimited field of information and cultural content. This expansiveness is prone to making us anxious. To relieve our anxiety, we cede our curational autonomy to the algorithm. By allowing the algorithm to curate our information/cultural environments, we get relief from the ‘anxiety of choice’. In exchange for a condition of curated information abundance and expanded access to the tools of voice, we allow platforms and other companies to extract our data and use it in this process of algorithmic curation. Through this data extraction, algorithms more narrowly curate our information environments and group us into consumer clusters for marketing purposes. These ‘new identities’ we formulate through the algorithm promise to make sense of a complex, contingent world by narrowing the scope and making it appear more certain. By eliminating content that we find dissonant or uncomfortable or packaging the content in ways that allow us to mock or shame dissonant content, we get an information/cultural environment which feels more cognitively comfortable and less anxiety provoking. In addition, we get tools like Ring cameras, which can give us the illusion of safety by making us dependent on ‘personal anomaly detectors’ which scan our environment for ‘threatening’ anomalies.

As with the Lockean and Hobbesian contracts, no one ‘signs’ a contract, but the thought experiment provides a way of understanding the relationship between the subject and the state. Similarly, the algorithmic contract is an effort to theorize our acquiescence to algorithmic governance in this new age. This contract comes with unintended consequences. The Internet was sold to us as a revolutionary platform that would break down barriers to knowledge, communication, and self- expression. Early blogging platforms and Internet culture produced a brilliant explosion of creative output. The web of the early 2010s was heralded as the vehicle to express democracy and freedom in the face of tyranny and oppression. If anything, social media liberates rather than inhibits, so the thought went. But this ability to express ourselves came with a predicament. Our desire to ‘be heard’ and to express ourselves is an indelible fact of our humanity – the desire to search for our authentic self is the same thing that compels us to produce data. But to express ourselves in an attention economy requires that we modify our voice if we want to ‘be heard’. This changes expression from something that emerges from a fixed self that seeks to express, to a malleable identity that adjusts oneself to be heard.

In the last decade, the rapid acceleration of wireless data transmission capacity, the capacity for data storage, and increased processor speeds have produced an analysis revolution. Entire fields of business intelligence, cybersecurity, and so on have grown exponentially in the last decade. By the mid- 2010s it was starting to become obvious that the vast trough of user posts generated by hundreds of millions of social media users could be beneficial as marketing intelligence. The monetization insight was that human expression could be tokenized and placed into models that add predictive value to marketers. This puts individual opinion into the world of supply and demand. If opinion is a commodity, then a lack of it poses a problem. If users are not on their devices enough, then there is a supply challenge. To remedy this, the priority for platforms is to make the production of opinion and engagement habitual.

At that point, encouraging users to stay on the platform to render opinions/produce content generates more material that can be ‘datafied’ and used for machine learning prediction and AI (artificial intelligence) training. The effect of the commodification of Internet engagement was that the market incentives became driven by emphasizing ‘being heard’ (voice) at the expense of expressing oneself (thought). Self- expression requires reflection to know what one wants or needs to say. Speaking (voice) does not require thought. Voice can be immediate, instinctive, and reactionary, generated through simple appeals to affect and desire. Reflection is not as easy to monetize. An abundance of the expression of voice seems like an endless resource. If we had listening devices attached to us all day, the well of expression would not run dry. While studies vary, the average person in Western societies speak about 16,000 words a day (Mehl et al 2007). If the market incentives are to extract voice from us, it does not really matter with whom we are talking. A recent estimate from a former United States intelligence official estimates that upwards of 80 percent of accounts on X (formerly Twitter) were bot accounts (Woods 2022).

The outlier problem in machine learning

The role of a quantitative social science that uses statistical modelling is to close the gap between the reality being studied and the abstraction of the model. In graduate school, I was trained to believe that there was a distinction between the world as it is and our need to abstract it or to make sense of it. If an econometric model abstracts too much (has too few variables/parameters), the resulting model might be overly simplistic, missing the subtleties in the broader population it seeks to understand. On the other hand, if the model is not abstract enough (has too many parameters), the model may become excessively complex, making it challenging to understand the underlying dynamics, thereby violating the law that models should be parsimonious to maximize explanatory power. Simple models that explain a great deal are considered to be ‘elegant’ (that is, they explain a lot with few variables). There is humility to this approach. It recognizes that models are abstractions of the real. This was a necessity because in the world before machine learning and AI, we did not have endless resources to collect data on entire populations of individuals. Samples carefully selected allowed us to infer from models we create about a population at large. It is impossible to explain any phenomena fully because there are always cases that violate expectations. In statistics, we call these outliers: confounding cases not explained by the model in its current state. Advances in science come from adding explanatory variables (understanding) to a model that can increase its power. Additionally, throwing too many variables into a model can produce a ‘degrees of freedom’ problem, where you have more variables than cases.

The data science/AI revolution has challenged that perceived limitation. There exists a push to blur the distinction between the world as it is and the world of the algorithm. Tech companies hire data engineers to push the limits of model optimization. How can we improve ‘model fit’? Data scientists can push the boundaries of ‘unknowability’ by increasing the number of parameters/variables in the model or adding layers in a neural network. Parsimony is not a core concern in machine learning. With massive computational power, you can have billions of cases and hence billions of parameters. The space between model abstraction and reality narrows. The need to develop theory is lessened since explanation is not the point. The point is prediction.

Humans, however, are slippery subjects. A model that might explain behaviour at one point or in one context may not be as effective in another. The truth of human contingency makes prediction challenging. Humans are creatures of habit and pattern, but we also reflect on our habits and patterns and change them with changing circumstances. What makes us ‘tick’ is immanently complex and difficult to understand. The neuroscientist Jeff Lichtman describes the intricacies of how the brain works:

If you can’t understand New York City, it’s not because you can’t get access to the data. It’s just that there’s so much going on at the same time. That’s what the human brain is. It’s millions of things happening simultaneously among different types of cells, neuromodulators, genetic components, things from the outside. There’s no point when you can suddenly say, ‘I now understand the brain,’ just as you wouldn’t say, ‘I now get New York City.’ (Lichtman in Guitchounts 2021)

Much of modern machine learning is modelled upon the workings of the human brain (neural networks). In a neural network, each layer comprises a set of attributes (features) and each neuron in the network is given a particular weight based on the predictive importance of the feature. There are endless ways by which we can be classified. We can be broken down into tens of thousands of attributes (eye colour, hair colour or waist circumferences as examples), and ranked or clustered on them based on the objective. This constellation of features and weights is what makes us distinctive, but, in a neural network, we are broken down into component parts. The task is to determine which of our features improves prediction or classification. One popular machine learning algorithm, ‘gradient descent’, adjusts the ‘weights’ for the different parameters in the model until the model arrives at a ‘local minima’ that optimizes the model’s predictive value (reduces the cost function).

The magic of neural networks is that the allocation of weights is not driven by theory. Through ‘backpropagation’, the model moves back through the layers of the neural network, adjusting the magnitude and direction of weights to produce the ‘right answer’ after it has compared one training case to the output in the final layer. If the output of a particular case looks like ‘random noise’ or does poorly at prediction (that is, has a high cost function) then it will ‘propagate back’ to the previous layer adjusting the direction in the magnitude of the weights to minimize the cost function (that is, get closer to the right answer).

This is what I refer to in this book as the algorithmic problem. To be an outlier, not behaving in ways that are predictable to the algorithm, impacts ‘model fit’. In the language of machine learning, a ‘difficult to classify’ case is one that makes it more challenging to ‘reduce the cost function’ of the model (that is, improve the ‘fit’ of the model in predicting the data).

We know little about what it means for us to be broken down into a set of potentially thousands of features that are weighted to give whoever is conducting the analysis powerful clues as to how any one of us behaves. If the models are limited in their effectiveness, one of two things is possible: a) there is not enough data to accurately predict, or b) we are too variable and unpredictable to ever become a problem that can be solved algorithmically. If ‘a’ is the case, the solution is to collect more data. If ‘b’ is closer to reality, what is the answer?

With limitations as to the extent to which data and algorithms can predict us, the answer to the optimization problem would require changing the behaviour of the subject so they behave in ways that are more consistent with the model.

Put into the language of machine learning: there is an increasing danger that instead of algorithms predicting us better, we are encouraged to behave in accordance with the algorithmic recommendation. We develop what Airoldi (2021) calls a ‘machine habitus’, sorting and classifying the world around us in ways that move us towards our algorithmically determined self. The model classifies us, but we also learn to ‘classify ourselves’, or, in the language of gradient descent, we ‘move towards the local minima’.