Designing Knowledge. Intuition, Intuitivity and Cognition

In this talk I will reflect on 1) the way people perceive and experience knowledge representations in different media formats and genres, 2) on the border between the visible and the unrepresented, and 3) on intuitively conceivable and cognitively plausible. The content of the digital media (transformed into knowledge by the power of transformative redundancy) is usually represented through the combination of different semiotic modes (picture, video, language, animation, sound etc.). The way the modes interact depends on the given discourse and dispositif/apparatus, i.e. on the communication and programmatic tradition and its technical means (cf. Agamben 2009, Kramer 2014). In the social semiotics the set of rules according to which these modes are combined is called design, and the text in which the knowledge is constructed or through which it is transmitted is called multimodal composition (Kress and van Leeuwen 2006).

Usually the representation and the cognitive mechanisms are considered in the studies of multimodal interaction (cf. López-Varela 2015). I will however concentrate on the use of intuition and the intuitivity as its corresponding mechanism inherent to the multimodal composition and defined through the design. We need to understand the interplay of these three factors as a part of representation and its experience in order to describe their role in the global regime(s) of knowledge dissemination within the digital realm. Therefore, my task is to (re-)define intuition, intuitivity and cognition in the context of different media formats. I will discuss three examples of the visual representation regimes from everyday life: 1) online news aggregation, 2) an in-vehicle-infotainment media format, and 3) a smartphone. These will cover differences on all levels of their functionality such as physical object, a virtual environment, life enhancement etc. The reflections will contribute to the creation of evaluation criteria for multimodal knowledge representation.

Keywords: media studies, multimodal analysis, social semiotics, intuition, intuitivity, design, epistemology of digital representation, UX design, cognitive research, dispositif/apparatus analysis

References:

Agamben, Giorgio (2009): What is an Apparatus? and Other Essays. Stanford: Stanford University Press.

Kramer, Stefan (2014): „Globale Medienangebote und lokale Programme. Der Fall des chinesischen Buchdrucks.“ In: Paech, Joachim/Mersch, Dieter (Hgg.): Programm(e). Medienwissenschaftliche Symposien der DFG. Zürich: diaphenes Verlag, pp. 43–64.

Kress, Gunther/Van Leeuwen, Theo (2006): Reading Images. The Grammar of Visual Design. London: Routledge. 

López-Varela, Asunción (2015): Intermedial cognitive semiotics: Some examples of multimodal cueing in virtual environments. In: María Jesús Pinar Sanz (Ed.) Multimodality and Cognitive Linguistics, Amsterdam, John Benjamins, pp. 167–180