Neurophysiological assessments of low-level and high-level interdependencies between auditory and visual systems in the human brain

This dissertation investigates the functional interplay between visual and auditory systems and its degree of experience-dependent plasticity. To function efficiently in everyday life, we must rely on our senses, building complex hierarchical representations about the environment. Early sensory deprivation, congenital (from birth) or within the first year of life, is a key model to study sensory experience and the degree of compensatory reorganizations (i.e., neuroplasticity). Neuroplasticity can be intramodal (within the sensory system) and crossmodal (the recruitment of deprived cortical areas for remaining senses). However, the exact role of early sensory experience and the mechanisms guiding experience-driven plasticity need further investigation. To this aim, we performed three electroencephalographic studies, considering the aspects: 1) sensory modality (auditory/visual), 2) hierarchy of the brain functional organization (low-/high-level), and 3) sensory deprivation (deprived/non-deprived cortices). The first study explored how early auditory experience affects low-level visual processing, using time- frequency analysis on the data of early deaf individuals and their hearing counterparts. The second study investigated experience- dependent plasticity in hierarchically organized face processing, applying fast periodic visual stimulation in congenitally deaf signers and their hearing controls. The third study assessed neural responses of blindfolded participants, using naturalistic stimuli together with temporal response function, and evaluated neural tracking in hierarchically organized speech processing when retinal input is absent, focusing on the role of the visual cortex. The results demonstrate the importance of atypical early sensory experience in shaping (via intra- and crossmodal changes) the brain organization at various hierarchical stages of sensory processing but also support the idea that some crossmodal effects emerge even with typical experience. This dissertation provides new insights into understanding the functional interplay between visual and auditory systems and the related mechanisms driving experience-dependent plasticity and may contribute to the development of sensory restoration tools and rehabilitation strategies for sensory-typical and sensory-deprived populations.