The MuseIT EU project aims to co-design, develop, and co-evaluate a multisensory, user-centred platform for enriched and inclusive engagement with cultural assets. It follows equal opportunity for all as its core principle. There is a market niche for an improved understanding of disability and diversity of needs to facilitate and broaden accessibility when developing technological systems for the digital transformation of the cultural heritage museums. 

MuseIT explores these needs in close contact with people with disabilities in co-design and co-research processes. The project focuses on the development of digital technologies to represent cultural heritage artefacts in multiple modalities by focusing on detecting semantic content about cultural heritage objects and translating their semantic content to knowledge graphs that are further used to create multisensory representations of them. 

These developments include the idea of a haptogram vocabulary with static vs. dynamic patterns carrying word vs. sentence meaning mapped to the body. One version of the vocabulary rethinks patterns from Social Haptic Communication (SHC), employed in co-design with users by our tool called HaptoDesigner. The other version automatically converts a high number of precomputed word embeddings to haptic signals for the Unitouch pattern design studio. While holding the promise to translate cultural heritage objects to the tactile domain, work in progress is also exploring conceptual vs. technological bottlenecks to enable their presentation in the context of a virtual museum.

MuseIT also explores the options of cultural objects in other modes, e.g. converting an image to sound, presenting 3D tactile representations and including these possibilities into virtual inclusive museum exhibitions.

Motifs in folktales and myths have been identified and articulated by scholars, and the computational identification and discovery of such motifs is an area of ongoing research. Achieving this goal means meeting scientific requirements (that methods be comparable and replicable) and requirements for collaboration (that multi-disciplinary teams can reliably access data). To support those requirements, access to consistent reference datasets is needed. Unfortunately, these datasets are not openly available in a format that supports their use in data science. Here we report work in progress toward this goal, having converted the Ashliman Folktexts collection into a public dataset of annotated tale texts. The data can be accessed at doi.org/10.5281/zenodo.6575263.

To help close the gap between folksonomic knowledge vs. digital classical philology, based on a perceived analogy between Newtonian mechanics and evolving semantic spaces, we tested a new conceptual framework in a specific domain, the Ontology of Greek Mythology (OGM). The underlying Wikidata-based public dataset has 5377 entities with 289 types of relations, out of which 34 were used for its construction. To visualize the influence structure of a subset of 771 divine actors by other means than the force-directed placement of graph nodes, we expressed the combination of semantic relatedness plus objective vs. relative importance of these entities by their gravitational behaviour. To that end, the metaphoric equivalents of distance, mass, force, gravitational potential, and gravitational potential energy were applied, with the latter interpreted as the structuration capacity of nodes. The results were meaningful to the trained eye, but, given the very high number of contour maps and heatmaps available by our public tool, their systematic evaluation lies ahead.

By means of a proof-of-concept prototype, which is work in progress, we adopted a multidisciplinary approach to develop a smart-textile-based communication system for use by people with deafblindness. In this system, sensor technologies and computer vision are used to detect environmental cues such as presence of obstacles, faces, objects, etc. Focusing on the communication module here, a new ontology connects visual analytics with the user to label detected semantic content about objects, persons and situations for navigation and situational awareness. Such labelled content is then translated to a haptogram vocabulary with static vs. dynamic patterns, which are mapped to the body. A haptogram denotes a tactile symbol composed over a touchscreen, its dynamic nature referring to the act of writing or drawing. A vest made of smart textile, in the current variant equipped with a 4 x 4 grid of vibrotactile actuators, is used to transmit haptograms on the user’s back. Thereby system messages of different complexity -- both alerts and short sentences -- can be received by the user, who then has the option to respond by pre-coded questions and messages. By means of grids with more actuators, displays with higher resolution can be implemented and tested, paving the way for an extended haptogram vocabulary, covering more detailed ontology content.

Based on the ontology developed in the ongoing SUITCEYES EU-funded project to bridge visual analytics for situational awareness and navigation with semantic labelling of environmental cues, we designed a set of static and dynamic haptograms to represent concepts for two-way communication between deafblind and non-deafblind users. A haptogram corresponds to a tactile symbol drawn over a touchscreen, its dynamic nature referring to the act of writing or drawing, where the touchscreen can take several forms, including a smart textile screen designated for specific areas on the body. In its current version, our haptogram set is generated over a 4 x 4 matrix of cells and is displayed on the back of the user, tested for robustness at the receiving end. The concepts and concept sequences simulating simple questions and answers represented by haptograms are focused on ontology content for now but can be scaled up.

Education and training of morphology for medical students, and professionals specializing in pediatric cardiology and surgery has traditionally been based on hands-on encounter with congenitally malformed cardiac specimens. Large international archives are no longer widely available due to stricter data protection rules, a reduced number of autopsies, attrition rate of existing specimens, and most importantly due to a higher survival rate of patients. Our Cardiac Archive houses about 400 cardiac specimens with congenital heart disease. The collection spans almost 60 years and thus goes back to pre-surgical era. Unfortunately, attrition rate due to desiccation has led to an increased natural decay in recent years. The present multi-institutional project focuses on saving the collection by digitization. Specimens are scanned by high-resolution micro-CT/MRI. Virtual 3D-models are segmented and a comprehensive database is built. We now report an initial feasibility study with six test specimens that provided promising results, however, adequate presentation of the intracardiac anatomy, including septa and cardiac valves requires further refinements. Computer assisted design methods are necessary to overcome consequences of pathological examination, shrinkage and/or distortion of the specimens. For a next step, we anticipate an expandable web-based virtual museum with interactive reference and training tools. Web access for professional third parties will be provided by registration/subscription. In a future phase, segmental wall motion data could be added to virtual models. 3D-printed models may replace actual specimens and serve as hands-on surgical training to elucidate complex morphologies, promote surgical emulation, and extract more accurate procedural knowledge based on such a collection.

In accessibility tests for digital preservation, over time we experience drifts of localized and labelled content in statistical models of evolving semantics represented as a vector field. This articulates the need to detect, measure, interpret and model outcomes of knowledge dynamics. To this end we employ a high-performance machine learning algorithm for the training of extremely large emergent self-organizing maps for exploratory data analysis. The working hypothesis we present here is that the dynamics of semantic drifts can be modeled on a relaxed version of Newtonian mechanics called social mechanics. By using term distances as a measure of semantic relatedness vs. their PageRank values indicating social importance and applied as variable ‘term mass’, gravitation as a metaphor to express changes in the semantic content of a vector field lends a new perspective for experimentation. From ‘term gravitation’ over time, one can compute its generating potential whose fluctuations manifest modifications in pairwise term similarity vs. social importance, thereby updating Osgood’s semantic differential. The dataset examined is the public catalog metadata of Tate Galleries, London.

The current deliverable summarises the work conducted within task T4.5 of WP4, presenting our proposed approaches for contextualised content interpretation, aimed at gaining insightful contextualised views on content semantics. This is achieved through the adoption of appropriate context-aware semantic models developed within the project, and via enriching the semantic descriptions with background knowledge, deriving thus higher level contextualised content interpretations that are closer to human perception and appraisal needs. More specifically, the main contributions of the deliverable are the following: A theoretical framework using physics as a metaphor to develop different models of evolving semantic content. A set of proof-of-concept models for semantic drifts due to field dynamics, introducing two methods to identify quantum-like (QL) patterns in evolving information searching behaviour, and a QL model akin to particle-wave duality for semantic content classification. Integration of two specific tools, Somoclu for drift detection and Ncpol2spda for entanglement detection. An “energetic” hypothesis accounting for contextualized evolving semantic structures over time. A proposed semantic interpretation framework, integrating (a) an ontological inference scheme based on Description Logics (DL), (b) a rule-based reasoning layer built on SPARQL Inference Notation (SPIN), (c) an uncertainty management framework based on non-monotonic logics. A novel scheme for contextualized reasoning on semantic drift, based on LRM dependencies and OWL’s punning mechanism. An implementation of SPIN rules for policy and ecosystem change management, with the adoption of LRM preconditions and impacts. Specific use case scenarios demonstrate the context under development and the efficiency of the approach. Respective open-source implementations and experimental results that validate all the above.All these contributions are tightly interlinked with the other PERICLES work packages: WP2 supplies the use cases and sample datasets for validating our proposed approaches, WP3 provides the models (LRM and Digital Ecosystem models) that form the basis for our semantic representations of content and context, WP5 provides the practical application of the technologies developed to preservation processes, while the tools and algorithms presented in this deliverable can be deployed in combination with test scenarios, which will be part of the WP6 test beds.

Management of change is essential to ensure the long-term reusabilityof digital assets. Change can be brought about in many ways, includingthrough technological, user community and policy factors. Motivated by casestudies in space science and time-based media, we consider the impact ofchange on complex digital objects comprising multiple interdependent entities,such as files, software and documentation. Our approach is based on modellingof digital ecosystems, in which abstract representations are used to assess risksto sustainability and support tasks such as appraisal. The paper is based onwork of the EU FP7 PERICLES project on digital preservation, and presentssome general concepts as well as a description of selected research areas underinvestigation by the project.

The current deliverable summarises the work conducted within task T4.3 of WP4, focusing on the extraction and the subsequent analysis of semantic information from digital content, which is imperative for its preservability. More specifically, the deliverable defines content semantic information from a visual and textual perspective, explains how this information can be exploited in long-term digital preservation and proposes novel approaches for extracting this information in a scalable manner. Additionally, the deliverable discusses novel techniques for retrieving and analysing the context of use of digital objects. Although this topic has not been extensively studied by existing literature, we believe use context is vital in augmenting the semantic information and maintaining the usability and preservability of the digital objects, as well as their ability to be accurately interpreted as initially intended.