In this paper we suggest a comprehensive account of operationalization. The goal is to define a workflow which specifies the normative components that ensure that the results match the research question. The focus here lies on epistemic norms – as opposed to social norms – which can be understood as rules that govern processes, thoughts, and actions that ought to be followed in order to produce the intended result (cf. Wedgwood 2018). This workflow is based on a measurement workflow, which is based on input-process-output components (cf. Mari, Wilson, and Maul 2023). In order to make the workflow implementable in Digital Humanities research, we need to identify the components of a generalized method for getting from a research question to its answer. While some of the components have already been discussed in Digital Humanities’ research on operationalization, we propose to put more focus on explication, measurement, and validation. In our view, measurement and validation are underdeveloped in the Digital Humanities. This becomes apparent when looking at discussions on operationalization. Before doing that we briefly line out our understanding of central terms in the discussion In order to develop our account, we will make the following assumptions: The leading question for scientific workflows involving the analysis of data is this: In the case of text based digital humanities, we can specify the scope of theory and data further. Theory and text depend on the disciplinary background of the research question. For example, in Computational Literary Studies approaches, theory means literary theory and empirical data respectively text. A first approach to address this question attempted by Franco Moretti, who introduced operationalization to the discourse of the digital humanities with his seminal paper “Operationalizing or, the Function of Measurement in Modern Literary Theory” (Moretti 2013). In this paper, Moretti goes back to Bridgman’s ([1927] 1958) original conception of operationalization and applies it to literary concepts. His answer to question Q’ famously is: We must imagine a bridge that is built by operations, which take us from the (qualitative) concepts of literary theory to (quantitative) measurement results. This implies existence of a simple and easy way to connect a concept C to an indicator I, resulting in a straightforward measure, which allows inferences, like the more I, the more C. Finally numbers are assigned to produce a measurement metric, be it by counting words or by using more elaborate techniques, like network analysis. In his paper, Moretti gives examples to show how this can be done, but he does not provide a detailed and generalized method that can be followed in different contexts. We therefore specify our question even further: In order to tackle this question, we have to address at least three ambivalent aspects of operations that are at least implicitly underlying understandings meaning of operationalization: In digital humanities research operationalization often only means (a), i.e. that in the research design some kind of workflow or pipeline is developed, that is used to automatically extract some features from the empirical material (text). Often literary concepts are adopted uncritically, when, in fact, critical reflection on the usage of the concepts in the sense of (b) is needed to guarantee that the workflow returns results that (c) are actually valid. The last point holds even then, when statistical validity metrics are incorporated in the workflow, since those tests can only answer how good the machine learning model or the algorithm worked, but not, if the quantified results translate back to literary concepts. This objection follows the main line of argumentation of Pichler and Reiter (2022), who argue for the importance of validity while maintaining its untenability for many real-world scenarios in the digital humanities. Overall, we understand operationalization as a workflow or process in the sense of (c), specifically in terms of the scientific input-process-output workflow (Mari, Wilson, and Maul 2023, see Fig. 1). In the following, we will introduce explication, measurement, and validity as components that make this workflow consistent and appropriate for the respective scientific context. The starting point of operationalization in our account are concepts which are transformed to empirical mini-theories that fit the research task at hand. To reach this goal, we resort to Carnapian explication (cf. Carnap 1950; Brun 2016) which allows us to accommodate various epistemic goals and permits different concept variations for different purposes. Explications involve two areas, the one of the explicandum and the one of the explicatum. These domains are sometimes viewed as pre-scientific and scientific uses of the term, but we can assume that explications can occur in any area. This construes a more exact explication in a more restricted or specialized area. For instance, moving from chemistry to molecular quantum chemistry or literary theory to computational literary theory (CLS). Measurement can be thought of as a process that takes some inputs and returns some outputs, where the input is some kind of empirical data and the output a value, see Fig 1a. But not any arbitrary process is suitable for measurement, it is necessary to specify the properties of the process, which will result in a procedure which is designed for that specific task (Mari, Wilson, and Maul 2023, 27), see Fig. 1b. Validation A common conception of validation is the following: A e.g. thermometer is intended to measure the actual temperature of an object, where temperature is a property of said object. This notion seems to be sufficiently clear, but on closer inspection it turns out that the terms in (M) are underspecified. How do we determine ‘measurement’, ‘measure’, and ‘intended’? What exactly are we referring to, when we use those terms? ‘Measurement’ is ambiguous at least respective its target (Mari, Wilson, and Maul 2023; Maul, Mari, and Wilson 2019). Measurement may either denote a measuring process or the result of such process. To avoid misunderstandings, we will adopt a strict terminology which discriminates measurement processes and measurement results. Where process means a set of operations which are implemented as an instrument, that can range from physical artefacts, like thermometers, to computational devices, like machine learning algorithms. The result is an assigned value on an appropriate scale. Calibration Calibration denotes the inferential activity of assigning values to the measurement instrument indications and thus producing the measurement results, where: Calibration links the outcomes of the measurement process to the epistemic claims about the object that is measured, this step is necessary since the indications of the measurement instrument are not of the same kind as the properties of the object, see Fig. 1c. The scale of a thermometer e.g. presents its indications visually a further step is needed to ensure that those values can be used to represent temperature. The same applies to literary concepts, like, in Moretti’s case character space. Network analysis or word counts provide indicators that need to be mapped to the possible values of the intended property being measured. We can now revise Moretti’s picture of operationalization as a simple measurement process, that connects theory to world directly (see Fig. 2a) and replace it with the refined scheme developed above (Fig. 2b). For our account of operationalization we consider operations as activities that are performed on some type of input, which return some type of outputs. The kind of activity performed in operations can vary wildly, ranging from simple reasoning about concepts over doing physical activity to actual computation. In constructing an operationalization workflow we denote operations as connected boxes: We are finally able to put all the pieces together and construct a workflow that starts with concepts and ends with measurement results. The operationalization workflow has two sources: (i) The concept(s) originating in the source domain (humanities, literary theory, CLS) and (ii) the empirical data / text. Starting with the concepts, the workflow follows these steps: <note xmlns="http://www.tei-c.org/ns/1.0" rend="footnote text" xml:id="ftn1" n="1"> Mini-theories could be considered as models, but since the term model is also used in other contexts, we prefer this term to mark a distinction between the various usages. </note> <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="ref-Bridgman_1958_Logic_Modern_Physics" /> <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="refs" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Bridgman, Percy Williams</hi>. (1927) 1958. <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">The Logic of Modern Physics</hi>. New York: Macmillan. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="Xae5569d0ad0e0a2a8c38ee82b846e09c640a1e8" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Brun, Georg</hi>. 2016. “Explication as a Method of Conceptual Re-engineering.” <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Erkenntnis</hi> 81 (6): 1211–41. <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://doi.org/10.1007/s10670-015-9791-5">https://doi.org/10.1007/s10670-015-9791-5</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="X8b51332d2369e9a8e1252f5e533c8bedd5e40d7" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Carnap, Rudolf</hi>. 1950. <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Logical Foundations of Probability</hi>. Chicago: Chicago University Press. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="X81af4c763527e24800142a4aad3ceb344977f52" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Hühn, Peter, Jan Christoph Meister, John Pier, and Wolf Schmid</hi>, eds. 2021. “The Living Handbook of Narratology.” <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://www-archiv.fdm.uni-hamburg.de/lhn/index.html">https://www-archiv.fdm.uni-hamburg.de/lhn/index.html</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="X0e382a80813bc946da985b45cd21b80504b4645" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Mari, Luca, Mark Wilson, and Andrew Maul</hi>. 2023. <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Measurement Across the Sciences: Developing a Shared Concept System for Measurement</hi>. Springer Series in Measurement Science and Technology. Cham: Springer International Publishing. <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://doi.org/10.1007/978-3-031-22448-5">https://doi.org/10.1007/978-3-031-22448-5</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="X409be5d194d72d2f44171e62ce46a60da5b824c" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Maul, Andrew, Luca Mari, and Mark Wilson</hi>. 2019. “Intersubjectivity of Measurement Across the Sciences.” <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Measurement</hi> 131 (January): 764–70. <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://doi.org/10.1016/j.measurement.2018.08.068">https://doi.org/10.1016/j.measurement.2018.08.068</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="X832125dac9b805b8af61666e965aaada89018a3" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Moretti, Franco</hi>. 2013. “Operationalizing or, the Function of Measurement in Modern Literary Theory.” <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Pamphlets of the Stanford Literary Lab</hi>, no. 6 (December): 1–13. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="ref-NASA_1972_Project_Cyclops_Design" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">NASA</hi>. 1972. “Project Cyclops: A Design Study of a System for Detecting Extraterrestrial Intelligent Life,” no. NASA-CR-133846 (January). <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://ntrs.nasa.gov/citations/19730010095">https://ntrs.nasa.gov/citations/19730010095</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="X7bd28d0f0243e9b899dbbab17be1ddcdb12c558" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Pichler, Axel, and Nils Reiter</hi>. 2022. “From Concepts to Texts and Back: Operationalization as a Core Activity of Digital Humanities.” <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Journal of Cultural Analytics</hi> 7 (4). <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://doi.org/10.22148/001c.57195">https://doi.org/10.22148/001c.57195</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="Xbfd7b83937a59d6b1f324b1cc680f0cb7eaf5ad" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Tal, Eran</hi>. 2017. “Calibration: Modelling the Measurement Process.” <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">Studies in History and Philosophy of Science Part A</hi>, The Making of Measurement, 65–66 (October): 33–45. <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://doi.org/10.1016/j.shpsa.2017.09.001">https://doi.org/10.1016/j.shpsa.2017.09.001</ref>. <anchor xmlns="http://www.tei-c.org/ns/1.0" xml:id="ref-Wedgwood_2018_Unity_Normativity" /><hi xmlns="http://www.tei-c.org/ns/1.0" rend="bold">Wedgwood, Ralph</hi>. 2018. <hi xmlns="http://www.tei-c.org/ns/1.0" rend="italic">The Unity of Normativity</hi>. Edited by Daniel Star. Vol. 1. Oxford University Press. <ref xmlns="http://www.tei-c.org/ns/1.0" target="https://doi.org/10.1093/oxfordhb/9780199657889.013.2">https://doi.org/10.1093/oxfordhb/9780199657889.013.2</ref>. 