A Reality Check (-List) For Digital Methods: Big Data, Big Infrastructures
A Reality Check (-List) For Digital Methods: Big Data, Big Infrastructures
A Reality Check (-List) For Digital Methods: Big Data, Big Infrastructures
Tommaso Venturini
University of Lyon, Lyon, France
Liliana Bounegru
Ghent University, Belgium; University of Groningen, The Netherlands
Jonathan Gray
King’s College London, UK
Richard Rogers
University of Amsterdam, The Netherland
How to cite:
Venturini, T., Bounegru, L., Gray, J., & Rogers, R. (2018). A reality check(list) for digital methods. New
Media & Society, (forthcoming), 146144481876923. http://doi.org/10.1177/1461444818769236
3
A medium is any technical infrastructure that allows the organisation and extension of collective actions in space
or time. The printing press, television, telephone and Web are media in that they allow social actors to interact
without being in the physical presence of each other. One should be not fooled by this simple definition into
taking media infrastructures for granted. Classic works in media studies emphasise how media are not neutral
agents and instead play an active role in the articulation of meaning and communications. For example, Howard
Innis’s pioneering studies (1986, 2008) highlighted what he called the particular characteristics or “biases” of
media and how they enabled different social institutions of law, religion, culture and commerce. Drawing on
Innis’s work, Marshall McLuhan contributed to the recognition of media systems as objects and sites of study.
James Carey recognises Innis and McLuhan’s role in establishing the study of media as “not merely [...]
appurtenances to society but as crucial determinants to the social fabric” (1967, pp. 270-271). He underlines the
limits of models focussing on “transportation and transmission” and instead proposed to consider media as
processes through which “reality is produced, maintained, repaired, and transformed” (2009, p. 19).
A platform is a specific way of organising a media infrastructure, constraining the way in which the medium can
be employed but also facilitating its exploitation. Recent research has focused both on the rhetorical aspects of
platforms (Gillespie, 2010), as well as their “material-technical” characteristics (Helmond, 2015). Facebook is an
interesting example of how a limited repertoire of “sentiments” – including “likes” and a number of ‘reaction’
emoticons – are facilitated through a centralised social media company, and then extended in relation to almost any
digital media.
A scientific inscription is any piece of information that is materialized through the use of a technical device for
the purposes of research. Inscriptions are the foundation of any scientific enterprise for they allow to imprint
knowledge on materials which can be stored, transformed and transmitted (Latour & Woolgar, 1979 and Latour
1985).
A digital trace is any inscription produced by a digital medium in its mediation of collective actions – for
instance, a post published on a blog, a hyperlink connecting two websites or the log of an e-commerce
transaction. We call this particular type of inscriptions ‘traces’ as a reminder that they are (most often) generated
by purposes other than academic research. Some of these inscriptions are ‘native’ to digital media while others
are originally analogue and digitized a later stage.
A corpus is an ensemble of inscriptions or traces that have undergone the process of selection, cleaning and
refining necessary to prepare them for scientific analysis. For instance, hyperlinks are a classic example of digital
traces, but they only become a research corpus when they are translated into constricted lists or into arcs
connecting a network of websites.
The notion of digital methods was introduced in 2007 as a counterpoint to virtual methods, which sought to
introduce the social scientific instrumentarium to digital research (Rogers, 2009). Virtual methods, it was claimed,
consisted in the digitisation of traditional research methods such (e.g. in online surveys or online ethnography).
Rooted in media studies and the so-called computational turn in the humanities and social sciences, digital
methods sought instead to learn from the methods of the medium and repurpose them for social and cultural
research. Reflecting on ‘natively digital’ methods sensitised the researcher to the specificities of the then ‘new
media’, to their effects, platform vernaculars and user cultures. ‘Following the medium’ also would offer the
researcher a strategy to cope with the ephemerality and instability of the Web, where a new feature, a changed
setting or the shutting down of an API could stymie longitudinal studies. Whilst remaining critical of the
implications of such changes, digital methods would ask which kind of research the platform affords. Digital
methods thus may be defined as techniques for the ongoing research on the affordances of online media.
4
from which digital data is collected (Twitter, Facebook, a website, a mobile app), and what kinds of technical skills,
capacities and networks they have available to them (cf. Hargittai & Hsieh, 2013).
Say, for example, that you are investigating data collected through the Steam gaming platform (steampowered.com).
Different cautions will be needed depending on the ambition of your research: do you plan to describe the gaming
habits of Steam’s users? Or are you interested in online gaming trends? Or do you want to inspect the cognitive
effects of computer games? Or question the social role of game playing in general? If you are studying the practices
of specific platform (eg, the habits of Steam gamers), then the inscriptions produced by that platform constitute the
primary traces of the phenomenon you are after. But if you use those particular activities as an example of a more
general phenomenon (e.g. collective game playing), then your traces will only offer you a partial observation.
As should be clear from the example, the distinction is neither binary nor written in stone. It depends on how you
define the scope your investigation and can change as your research evolves. It should also be noted that working
with partial traces is not necessarily unsurmountable problem. If it is true that the larger is the coverage of your
study object, the easier it will be to generalise your findings, it is also true that the more phenomena and media
coincide and the harder it is to separate them analytically. In the paragraphs above, we have used the expression “to
take place in”. While this expression conveniently describes the way in which actions happen within or beyond a
specific medium, it has the disadvantage of artificially separating collective actions from the medium that supports
them. Media are not only ‘places’, ‘spaces’ or ‘contexts’ but actors themselves whose actions interfere and transform
the behaviour of their users (Castells, 2009). These ‘media effects’ should be taken into consideration to understand
that the phenomena we observe are not just hosted and traced by the media in which they occur, but also deeply
shaped by them.
1.1. How much of your study object occurs in the medium you are studying?
In its simplest definition a collective phenomenon can be defined as a network of interfering actions (Latour, 2005).
These actions can be of very different kinds, varying from an occasional intervention of an individual actor (e.g.
when a customer makes a bid in an online auction) to a longstanding configuration of socio-technical forces (e.g. the
legal constraints implemented in the mechanism of the bidding interface). What counts here is the extent to which
the actions that comprise the phenomenon you wish to observe are mediated by – and therefore leave traces in – the
medium that you are repurposing.
If you are studying learning practices in Massive Open Online Courses (or MOOCs), you may for example safely
assume that most interactions that you investigate may take place through the MOOC platform and therefore be
recorded by it. But if you are studying the life of a university through the records of its administrative systems, you
should be aware that most of the informal face-to-face exchanges that constitute a crucial part of college experience
will not show up in your dataset.
A good example of a close alignment between the research object and the medium is a 2016 study of conflict
resolution practices in Wikipedia (Weltevrede & Borra, 2016). This study draws on a project which was initially
meant to use Wikipedia’s traces to identify emerging societal controversies (contropedia.net). Soon, however, it
became clear that, while tensions coming from external debate are often mirrored in the online encyclopaedia, such
conflicts are hard to distinguish from the internal quarrels around the platform’s architecture, policies and guidelines.
Acknowledging this difficulty, the study shifted the focus of its inquiry to examine practices of coordination specific
to the platform and the distinctive ways in which they facilitate collaboration and defuse conflict.
5
Figure 1. Two screenshots of the Contropedia.net interface. Controversial wiki-links are highlighted in red on the original page
and the full evolution of the discussion surrounding them is displayed below (original figure in Weltevrede & Borra, 2016).
Other times, the partiality of medium coverage with respect to the phenomenon may be used strategically. Drawing
on James Gibson’s theory of visual perception (1986) Anders Koed Madsen (2012 and 2015) introduced the term
“web-vision analysis” precisely to point at the way in which researchers can use different media and filtering
parameters to compare different angles on the same phenomenon:
Web-visions are cases that result from deliberate combinations of devices and tools, and the mode of
seeing that results from these combinations is the basis of their potential relevance… the researcher
is left with an arsenal of variables that can be used to manipulate the construction of the web-visions
in a quasi-experimental fashion. The mode of seeing can, for example, be tweaked by altering the
logic of filtering in the delineation device, the country of origin of the device, the language used to
query the device or the settings of the web-crawler used to construct the visualization (Madsen, 2012,
p. 62)
Partiality, in other words, is not always a liability. Purposefully moving away from the main site where the
phenomenon occurs and where it is typically studied may offer fresh angles and perspectives.
6
online constitute the very object of the study, in the second they are the proxies of other actions (walking, standing,
shouting…) taking place outside the medium. Indeed, digital methods takes the explicit stance of using digital traces
to study not only online phenomena but culture and society in general (Rogers, 2013, 2017). Repurposing the media
means using digital traces as proxies for phenomena that extend beyond them.
In an exploratory project, for example, a group of researchers compared the Google Web Search results for the
query “rights” in a number of languages, to highlight the specific ways in which cultures conceive the question of
human rights (Bekema et al., 2009 - https://www.digitalmethods.net/Dmi/NationalityofIssues).
Figure 2. A visual representation of the different human rights as appearing in the results of Google Search for different
countries and languages (original figure in Bekema et al., 2009).
7
Even within a single platform, different operational definitions of the same research object are often possible. Take
the case of the investigation of controversies in Wikipedia. Because of the way in which MediaWiki (the software
that supports the famous collaborative encyclopaedia) stores information, 'controversiality' can be operationalized at
the article level (to highlight which topics are disputed), but also at the level of smaller elements such as the links
within the articles (to reveal, for instance, which references are most contested). In addition to this, multiple
measures of controversiality may be defined, from the volume of edit histories, to the depth of discussions in
associated talk pages (Borra et al, 2014, Weltevrede and Borra, 2016). Each of these operationalizations lead to a
different appraisal of what constitutes a matter of concern or an expression of disagreement.
8
This care is particularly crucial as 'query design' is concerned (Rogers, 2017). The ways in which actors label the
phenomena in which they are engaged can be subtle and complicated. For example, one may note that climate
'scepticism' is the self-description preferred by those who doubt the human causes of climate change, while climate
change 'denial' is the notion used by their opponents (Niederer 2013). Understanding the nuances of emic language
can help you capture different sides of your object and the competitive ways in which different groups frame the
same phenomena (see the concept of 'equivalence framing' in Cacciatore et al., 2016). It also allows you to generate
better and more precise queries. A recent study of 'mental illness' on Tumblr, for example, started from the generic
query #mentalillness (Sanchez-Querubin et al. 2016). Soon, however, the co-hashtag network around this query (the
hashtags most often used alongside #mentalillness) revealed that the concept of #recovery characterizes the most
significant practices associated with mental illness on Tumblr, and thus became the focus of the study.
9
Figure 3. Spread and debunk of the fake story according to which the Pope would have endorsed Donald Trump. The nodes
represent the web pages in which the story has circulated and the lines the different ways in which they mention each other
(original figure in ANONYMIZED).
3.2. Do you use different but comparable operationalizations for different media?
While ‘cross-platform’ research can be useful and sometimes indispensable, it also entails additional difficulties due
to the necessity of developing multiple operational definitions of the entities under consideration. Each of these
definitions should be attuned to the specific medium in which they are used, but also sufficiently consistent to allow
comparisons.
Earlier, we considered the investigation of how actors associate online. The most straightforward choice would be to
operationalize their connections as the hyperlinks connecting their different online personae (their website, their
Facebook page, their Twitter account…), as the notion of hyperlink is defined at a low layer of Web protocols and
does not depend on the specific platform implementation. Such choice, however, would fail to recognise that
different platforms and practices. Hyperlinks among websites may better be translated by ‘friendships’ or ‘likes’ in
Facebook and by ‘retweets’ and ‘mentions’ in Twitter. In cross-platform approaches the trade-off between
attunement and comparability is always problematic and one should find specific solutions coherent with the aims
and the constraints of the research.
10
In an ongoing study, we compared different media to reveal competing framing of open data politics
(ANONYMIZED). On Twitter, many actors seemed to cluster around topics related to business opportunities
(such as #startup, #smartcity or #innovation) as well as transparency and open government (#ogd,
#opengovernment, #transparencia). By contrast, by analysing the Wikipedia pages connected to the theme of open
data, we observed topics such as “open source software”, “free software movement”, “open access”, “free culture
movement” and “Creative Commons” – indicating how open data is articulated less as a policy or economic issue,
and more as part of the “digital commons” movement. Finally, newspaper analysis suggests that open data is
frequently discussed in relation to international development.
4.2. Are you accounting for the ways in which data are ‘given’ by the media?
Much has been written about the unfortunate etymology of the word ‘data’, which conveys the false impression that
information is objectively given and not constructed not only by the researchers but also by the technical infrastructures
11
that have generated those data, their users and the companies and institutions that own those infrastructures
(Bowker, 2013; Drucker, 2011). Acknowledging that we receive our information from someone else (data are ‘given’
at least in this sense) brings our attentions to the conditions of such delivery.
The sources from which we derive our inscriptions and the instruments through which we acquire them have
consequences on the quality of our observations. When observed through the traces that it leaves on Twitter, public
debate often appears as a chaotic flux of conversations ephemerally agglutinating around emerging ideas, while
struggles between overarching world visions and systems of forces, become almost invisible. As the saying goes in
digital methods community, “when all you have is a Twitter feed, everything looks like a hashtag”. Electronic media
do not merely record the interactions that they mediate – not unlike social researchers, they also measure and analyse
(Marres, 2012; Gray et al., forthcoming). They count them beside making them countable (Agre, 1994, Gerlitz and
Rieder, forthcoming).
Investigating climate debate on Twitter, Marres and Gerlitz (2015) noted that the platform relies on “frequency of
mentions” to identify and promote trending topics. Such focus encourages specific practices among the users (e.g.
re-tweeting as way of having messages picked up by the system) and is transmitted to most Twitter analytic tools.
This ends up privileging hashtags referring to events or campaigns (e.g. #cop16, #auspol, #savethearctic) that are
subject to hype-like dynamics. In order to detect more substantial issues, the researchers then moved from
frequency measures to “associationist measures” (not how many times a hashtag is mentioned, but how many other
hashtags co-occur with it), which allowed them to identify tags such as #economics, #flood, #co2, #health,
#environment, and #drought.
Figure 4. Comparison of the most mentioned and most connected hashtags connected to climate change debate in Twitter
(original figure in Marres & Gerlitz, 2015).
12
Not only different digital traces are infused with the technical, commercial and ideological premises of the platforms
that generate them (cf. Srnicek, 2017, Havens & Lotz, 2012, Mandiberg, 2012), but our datasets depends on our
entry point to digital inscriptions. For example, most digital platforms provide an API (Application Programming
Interface) that structures what and how much information may be accessed, as well as by whom and with which
restrictions. The information accessed through such ‘pipelines’ is often significantly different in detail and
completeness from that displayed on the interface of the same platforms (as a result of operations of aggregation,
anonymization or normalisation). Sometimes important portions of digital traces are excluded from APIs – the
Facebook API, for example, recently withdrew all information on personal profiles due to privacy requirements,
even though such profiles constitute the bulk of Facebook’s inscriptions (Rieder, 2013). The possibility remains, of
course, to ‘scrape’ information directly from the publicly accessible interfaces, services and applications, but even in
this way traces bring with them the mark of their origin (Marres & Weltevrede, 2013).
As a conclusion
Instead of concluding with a theoretical discussion, we prefer to remain faithful to the practical approach of this
paper and provide a summary of the eight questions discussed above. This summary is offered in the form of an
aide-mémoire that researchers embarking upon digital method projects can keep with them as a reality-check list of
their findings and interpretations.
References
Bekema, V, Liliana Bounegru, Andrea Fiore, Anne Helmond, Simon Marschall, Sabine Niederer, Bram Nijhof, Richard Rogers
and Elena Tiis. Nationality of Issues. Rights Types. Digital Methods Summer School 2009.
https://www.digitalmethods.net/Dmi/NationalityofIssues
13
Bodle, Robert (2011). Regimes of sharing. Information, Communication & Society, 14(3),
320–337. http://dx.doi.org/10.1080/1369118X.2010.542825
Bollier, D., & Firestone, C. (2010). The promise and peril of big data. Washington: The Aspen Institute. Retrieved from
http://www.ilmresource.com/collateral/analyst-reports/10334-ar-promise-peril-of-big-data.pdf
Boullier, Dominique. 2013. “Le « Hard » Du « Soft » : La Matérialité Du Réseau Des Réseaux.” CERISCOPE Puissance, 1–15.
Bowker, G. C. (2013). Data Flakes: An Afterword to “Raw Data” Is an Oxymoron. In L. Gitelman (Ed.), “Raw Data” Is an
Oxymoron (pp. 167–171). Cambridge Mass.: MIT Press.
Boyd, D., & Crawford, K. (2011). Six Provocations for Big Data. In A Decade in Internet Time: Symposium on the Dynamics of the
Internet and Society. http://doi.org/10.2139/ssrn.1926431
Boyd, D., & Crawford, K. (2012). Critical Questions for Big Data. Information, Communication and Society, 15(5), 662–679.
Bucher, T., & Helmond, A. (2017). The Affordances of Social Media Platforms. In J. Burgess, T. Poell, & A. Marwick (Eds.),
The SAGE Handbook of Social Media. London: Sage.
Burrows, R., & Savage, M. (2014). After the crisis? Big Data and the methodological challenges of empirical sociology. Big Data
& Society, 1(1), 205395171454028. doi:10.1177/2053951714540280
Cacciatore, M. A., Scheufele, D. A., & Iyengar, S. (2016). The End of Framing as we Know it … and the Future of Media
Effects. Mass Communication and Society, 19(1), 7–23. http://doi.org/10.1080/15205436.2015.1068811
Castells, M. (2009). Communication Power. Oxford: Oxford University Press.
Borra, E., Weltevrede, E., Ciuccarelli, P., Kaltenbrunner, A., Laniado, D., Magni, G., Mauri, M., Rogers, R. and Venturini, T.
(2014). Contropedia - the analysis and visualization of controversies in Wikipedia articles. In Proceedings of The International
Symposium on Open Collaboration (OpenSym '14). ACM, New York, NY, USA,. DOI=10.1145/2641580.2641622
http://doi.acm.org/10.1145/2641580.2641622
Carey, J. W. (1967). “Harold Adams Innis and Marshall McLuhan.” Antioch Review, 27(2) (Spring).
Carey, J., W. (2009). Communication as Culture: Essays on Media and Society. New York: Routledge.
Drucker, J. (2011). Humanities Approaches to Graphical Display. DHQ: Digital Humanities Quarterly, 5(1), 1–20.
Frischmann, B. (2001). Privatization and Commercialization of the Internet Infrastructure. The Columbia Science and Technology
Review, 2.
Gerbaudo, P. (2012). Tweets and the Streets: Social Media and Contemporary Activism. London: Pluto Books.
Gerlitz, C. and Rieder, B. (forthcoming). Tweets are not created equal. A platform perspective on social media
Metrics.
Gibson, J. J. (1986). The Ecological Approach To Visual Perception. New York: Psychology Press. Retrieved from
http://www.amazon.com/Ecological-Approach-Visual-Perception/dp/0898599598
Gillespie, T. (2010). The Politics of ‘platforms.’ New Media & Society, 12(3), 347–364.
https://doi.org/10.1177/1461444809342738
Gillespie, T., Boczkowski, P. J., & Foot, K. A. (2014). Media Technologies. Essays on Communication, Materiality and Society. Cambridge
Mass.: MIT Press. http://doi.org/10.7551/mitpress/9780262525374.001.0001
Gray, Jonathan and Bounegru, Liliana and Milan, Stefania and Ciuccarelli, Paolo, Ways of Seeing Data: Towards a Critical
Literacy for Data Visualizations as Research Objects and Research Devices. In Sebastian Kubitschko and Anne Kaun (eds.)
(2016) Innovative Methods in Media and Communication Research. London: Palgrave Macmillan.
14
Hargittai, E. & Hsieh, Y.P. (2013). Digital Inequality. In Oxford Handbook of Internet Studies. Edited by William H. Dutton.
Oxford, UK: Oxford University Press: 129-150.
Havens, T., & Lotz, A. D. (2012). Understanding Media Industries. Oxford: Oxford University Press.
Helmond, A. (2015). The Platformization of the Web : Making Web Data Platform Ready. Social Media + Society, 1(2), 1–11.
http://doi.org/10.1177/2056305115603080
Kooti, F., Gummadi, K. P., & Mason, W. a. (2012). The Emergence of Conventions in Online Social Networks. In Proceedings of
the Sixth International AAAI Conference on Weblogs and Social Media (ICWSM) (AAAI Press, Palo Alto, California).
Latour, B. (1985). Visualisation and Cognition: Drawing Things Together. In H. Kuklick (Ed.), Knowledge and Society Studies in the
Sociology of Culture Past and Present (pp. 1–40). Jai Press.
Latour, B. (1998). Thought experiments in social science: from the social contract to virtual society. In 1st Virtual Society? Annual
Public Lecture. Brunel University, London, 1 April.
Latour, B. (2005). Reassembling the Social An Introduction to Actor-Network Theory. Oxford: University Press.
Latour, B., & Woolgar, S. (1979). Laboratory Life. The Construction of Scientific Facts. Los Angeles: Sage.
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A.-L., Brewer, D., … Van Alstyne, M. (2009). Computational social
science. Science (New York, N.Y.), 323(5915), 721–3.
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). Big data. The parable of Google Flu: traps in big data analysis. Science
(New York, N.Y.), 343(6176), 1203–5. http://doi.org/10.1126/science.1248506
Lury, C., & Wakeford, N. (2012). Inventive Methods: The Happening of the Social. New York: Routledge.
http://doi.org/10.4324/9780203854921
Madsen, A. K. (2012). Web-Visions as Controversy-Lenses. Interdisciplinary Science Reviews, 37(1), 51–68.
http://doi.org/10.1179/0308018812Z.0000000004
Madsen, A. K. (2015). Tracing Data -Paying Attention. Interpreting Methods Through Valuation Studies and Gibson’s Theory
of Perception. In M. Kornberger, L. Justesen, A. K. Madsen, & J. Mouritsen (Eds.), Making Things Valuable (pp. 257–278).
Mandiberg, M. (Ed.). (2012). The social media reader. New York: New York University Press.
Marcus, G., & Davis, E. (2014). Eight (No, Nine!) Problems With Big Data. The New York Times. Retrieved from
http://www.nytimes.com/2014/04/07/opinion/eight-no-nine-problems-with-big-data.html
Marres N and Gerlitz C (2016) Interface methods: Renegotiating relations between digital social research, STS and sociology.
The Sociological Review 64(1): 21-46.
Marres, N. (2012). The redistribution of methods: on intervention in digital social research, broadly conceived. The Sociological
Review, 60, 139–165. http://doi.org/10.1111/j.1467-954X.2012.02121.x
Marres, N. (2017). Digital Sociology: The Reinvention of Social Research. London: Polity Press.
Marres, N., & Weltevrede, E. (2013). Scraping the Social? Issues in real-time social research. Journal of Cultural Economy, 6(3),
313–335. http://doi.org/10.1080/17530350.2013.772070
Mayer-Schönberger, V., & Cukier, K. (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think. Boston:
Eamon Dolan. http://doi.org/10.2501/IJA-33-1-181-183
15
McCormick, T. H., Lee, H., Cesare, N., Shojaie, A., & Spiro, E. S. (2015). Using Twitter for Demographic and Social Science
Research: Tools for Data Collection and Processing. Sociological Methods & Research, 49124115605339.
https://doi.org/10.1177/0049124115605339
McLuhan, M., & Fiore, Q. (1967). The Medium is the Massage: An Inventory of Effects. Gingko Press.
McLuhan, M. (1964). Understanding Media: The Extensions of Man. New York: McGraw-Hill.
Monod, J. (1970). Hasard et la Nécessité. Paris: Seuil.
Moretti, F. (2013). Operationalizing”: or, the function of measurement in modern literary theory. Stanford Literary Lab.
Pamphlet 6.
Munk, A. K. (2013). Techno-Anthropology and the Digital Natives. In L. Botin & T. Borsen (Eds.), What is Techno-Anthropology?
Allborg University Press.
Niederer, S. (2013). Global warming is not a crisis! Studying climate change skepticism on the web. Necsus, 3(Spring). Retrieved
from www.necsus-ejms.org/global-warming-is-not-a-crisis-studying-climate-change-skepticism-on-the-web
O’Neil, C., & Schutt, R. (2013). Doing Data Science. Sebastopol, Ca.: O’Reilly Med
Paßmann, J. & Gerlitz, C., 2014. ‚Good‘ platform-political reasons for ‚bad‘ platform-data. Zur sozio-technischen Geschichte
der Plattformaktivitäten Fav , Retweet und Like. Mediale Kontrolle unter Beobachtung, 3(1), pp.1–40.
Peter, J. D. (2015). The Marvelous Clouds: Toward a Philosophy of Elemental Media. Chicago: University of Chicago Press.
Rieder, B. (2013). Studying Facebook via Data Extraction: The Netvizz Application. Proceedings of WebSci ’13, the 5th Annual
ACM Web Science Conference, 346–355. http://doi.org/10.1145/2464464.2464475
Rieder, B., & Röhle, T. (2012). Digital Methods: Five Challenges. In D. M. Berry (Ed.), Understanding Digital Humanities (pp. 67–
84). Houndmills: Palgrave Macmillan.
Rogers, R. (2009). The End of the Virtual: Digital Methods. Amsterdam: Amsterdam University Press.
Rogers, R. (2013). Digital Methods. Cambridge Mass.: MIT Press.
Rogers, R., & Marres, N. (2002). French scandals on the Web, and on the streets: A small experiment in stretching the limits of
reported reality. Asian Journal of Social Science, 66, 339–353.
Rogers, R. (2017), “Foundations of digital methods: Query Design.” In The Datafied Society: Studying Culture through Data. Mirko
Tobias Schäfer and Karin van Es (eds.). pp. 75-94. Amsterdam University Press.
Ruppert, E., Law, J., & Savage, M. (2013). Reassembling Social Science Methods: The Challenge of Digital Devices. Theory,
Culture & Society, 30(4), 22–46. https://doi.org/10.1177/0263276413484941
Sanchez-Querubin, N., Jelena Schuhmacher, Giacomo Traldi, Carla McCutchin, and Sheena Manabat. #Recovery: Recovering
in and with Tumblr. Digital Methods Summer School 2016.
https://wiki.digitalmethods.net/Dmi/SummerSchool2016RecoverySelfie
Savage, M., & Burrows, R. (2007). The Coming Crisis of Empirical Sociology. Sociology, 41(5), 885–899.
http://doi.org/10.1177/0038038507080443
Sloan, L., Morgan, J., Burnap, P., & Williams, M. (2015). Who Tweets? Deriving the Demographic Characteristics of Age,
Occupation and Social Class from Twitter User Meta-Data. PLOS ONE, 10(3), e0115545.
https://doi.org/10.1371/journal.pone.0115545
Srnicek, N. (2017). Platform Capitalism. Cambridge: Polity Press.
Staiger, J. (2005). Media Reception Studies. New York: University Press.
Venturini, T., Jacomy, M., Meunier, A., & Latour, B. (2017). An unexpected journey: A few lessons from sciences Po médialab’s
experience. Big Data & Society, 4(2), 205395171772094. http://doi.org/10.1177/2053951717720949
16
Venturini, T., Baya Laffite, N., Cointet, J.-P., Gray, I., Zabban, V., & De Pryck, K. (2014). Three maps and three
misunderstandings: A digital mapping of climate diplomacy. Big Data & Society, 1(2).
http://doi.org/10.1177/2053951714543804
Venturini, T., Jensen, P., & Latour, B. (2015). Fill in the Gap: A New Alliance for Social and Natural Sciences. Journal of Artificial
Societies and Social Simulation, 18(2), 11. (http://jasss.soc.surrey.ac.uk/18/2/11.html)
Venturini, T., Ricci, D., Mauri, M., Kimbell, L., & Meunier, A. (2015). Designing Controversies and Their Publics. Design Issues,
31(3), 74–87. http://doi.org/10.1162/DESI_a_00340
Venturini, T., Jacomy, M., Meunier, A., & Latour, B. (2017). An unexpected journey: A few lessons from sciences Po médialab’s
experience. Big Data & Society, 4(2), 205395171772094. http://doi.org/10.1177/2053951717720949
Weltevrede E, & Borra, E (2016). “Platform affordances and data practices: The value of dispute on Wikipedia.” Big Data &
Society 3(1).
Weltevrede, E. (2016). Repurposing Digital Methods: The Research Affordances of Platforms and Engines. Unpublished ms.
https://wiki.digitalmethods.net/pub/Dmi/RepurposingDigitalMethods/Weltevrede_RepurposingDigitalMethods.pdf
17