Self-touch: an immersive interaction-technique to enhance body awareness

1 Introduction

Impairments in body representation are symptoms of various mental disorders such as eating and weight disorders or depression [1]. They either affect one aspect of body representation, such as sensorimotor representation (body schemata), visuospatial and semantic representations (combined: body image) [2], or a combination of several aspects [3]. One approach to manage impairments in body representation are so-called mind-body therapies. They combine physical and mental exercises and aim to strengthen the perception of internal body signals (body awareness) [1], a crucial element in the formation of body representations. While these exercises range widely in their execution, particularly the use of mindful self-touch to re-evaluate body representation impacts both body awareness and body image [4, 5].

Over the last decades, there has been a great development toward digitally supported mind-body exercises [6]. In particular, a variety of new designs for virtual reality (VR) assisted meditation were presented [7]. However, there has been no research on the impact of physical touch in VR-based mind-body exercises. In this work, we present the concept of an VR-based intervention to increase body awareness which combines the idea of mindful self-touch and virtually altered visual feedback. In a heuristic evaluation, we presented a prototype of the intervention to experts in User Experience (UX) design, Vr development, and obesity therapy to define the strengths and weaknesses of the current design and highlight the potential of virtually enhanced self-touch interactions for mind-body therapies.

2 Related work

Current developments in Virtual Reality (VR), Augmented Reality (AR), or Mixed Reality (MR) (short: XR), allow for an increased application of those technologies to a variety of therapeutic settings.

Recent reviews on XR based interventions for impairments in body representation, reveal the potential of including virtual bodies (avatars) in the treatment [8]. Further, recent reviews of XR based mindfulness and mind-body interventions [7, 9] point out the potential of including avatars into the design of interventions that rather focus on internal body signals than on body schemata or body image. Virtual avatars can be matched to the user in their appearance, movements, or behavior. When embodying an avatar, multi-sensory integration leads to a sense of embodiment (SOE), the “sense that emerges when [the avatar’s] properties are processed as if they were the properties of one’s own biological body” [10]. Embodying a visually modified avatar or working with visual feedback on an embodied avatar can impact the user’s behavior [11] as well as the perception of the own body [12]. Regarding body perception, it has been shown that avatars with different body shapes have an impact on visuospatial body representation [13, 14], namely body weight perception, and on semantic body representations [8], namely the affect towards the body. Regarding the impact of SOE on body awareness, Filipetti et al. [15] showed, that visuo-tactile congruence, correspondence of visual stimuli and tactile stimuli when embodying a rubber hand led to an increase in body awareness as well as in SOE. In a VR-based movement task with photorealistic avatars, Döllinger et al. [16] revealed a positive relation between SOE and body awareness. A project which investigates the effects of mind-body interactions and interactive manipulation is the ViTraS project [17]. It combines conservative therapy approaches to treat eating disorders such as sketching the own body or mirror exposure with the potentials of XR, such as the embodiment of a visually modified personalized avatar. In the ViTraS project classical therapy methods are combined with the potentials of XR to treat eating disorders. A mirror exposure was implemented which allows the visual modification of a personalized avatar. Another interaction technique within the ViTras project is Self-Sketch [18]. This interaction technique was developed to enable patients to visualize deviations of the mental representation of their body schema from their real body shape. The interaction technique enables users to draw the mental representation of their body as a silhouette, then place themselves in this silhouette and compare it with their own body. For this purpose, a real avatar of the user is first created with the help of a detailed 3D scan. Deviations of the drawn shape from the real body shape can thus be clearly perceived and discussed with the therapist. An informal evaluation of this application with four people showed that users were able to draw relatively accurate silhouettes of their bodies. However, sketching the lower parts of the body was found to be tiring; these were also drawn inaccurately. Therefore, a lift was implemented in the application that raises the silhouette by about one m. Lower body parts can now be drawn more comfortably without having to get into a crouched position (see Figure 1).

Figure 1:

Self-sketch application with virtual avatars. A lift-button on the left side provides a comfortable solution to draw the lower body without the need to crouch.

However, while past research has investigated whether generic or photorealistic avatars have an impact on body representation and body awareness, and to what extent the SOE is involved in this relation, and whether different interactions in VR are useful to extend body image interventions, most of the related work relies on the mere embodiment of an avatar, including some movement tasks and eventually modifications of the body shape or objects outside the avatar. So far, there has been no research on the impact of the avatars texture during virtual tasks and on the possibility of investigating visual feedback on the avatar during virtual mind-body exercises. In this paper, we introduce a new concept in which we combine self-touch, a mind-body task which is part of some interventions for impairments in body representation, with texture-based feedback on an avatar in VR. In a simple virtual environment, users touch themselves and receive colored feedback on their interaction.

3 Concept of self-touch

In the development of our concept, we limited ourselves to self-touch exercises in the therapeutic context of working through body image disorders. In collaboration with an expert in the field of body image and obesity therapy, we analyzed how self-touch can work in this context. The following elementary sub-tasks were defined:

1. The patient is asked to draw their perceived body silhouette, similarly to the task presented in Figure 1.

2. The patient is then asked to touch their body parts according to a systematic sequence.

3. The patient is instructed to concentrate on how the touch of the hand feels on their body parts and on the shape of the respective body part in their hand. On the one hand, the aim of this task is to generate body awareness and comforting body stimulation. On the other hand, it aims at a realistic assessment of the bodies’ shapes. In this task sequence, the patient is allowed to close their eyes if they wish or keep their eyes open if they are more comfortable that way.

4. After this task, the patient again is asked to draw their perceived body silhouette and to compare it to their first drawing.

We take up this process in our concept and extend it with a visual feedback. The feedback aims to support the sensation of touch on the body parts for patients that feel uncomfortable with closing their eyes but have difficulties focusing on the sensation when their eyes stay open.

In our concept the user controls a virtual avatar from a first-person perspective. The hands and the body of the avatar follow the real hand and body movements of the user to evoke a sense of embodiment towards the avatar. When touching their own body with their hands, similarly to a ”conventional” self-touch movement, the user receives visual information about the contact area between the virtual hand and the virtual body parts, as well as tactile and proprioceptive information about the location and contact of their real body parts. To further highlight the touching sensation on body parts, we provide additional visual feedback on the texture of the touched virtual body parts by colored highlighting. The more often the corresponding area is touched, the higher the color intensity increases. Similar to non-virtual self-touch interventions, this interaction aims to increase the awareness towards one’s body and to reduce impairments in the perceived body image. The colored highlights can be used either as guidance to show which parts of the body may be neglected or as a visual aid to sensitize to the corresponding body part [19].

4 System description

The development of the first self-touch prototype was done with the game engine Unity3d. For the embodiment, we use a freely available avatar from the Rocketbox Library [20]. The implementation consists of three parts: collision detection, tracking and visualization, which will be explained in detail be-low.

4.1 Body-collision

The presence of a collision model is required as a prerequisite for self-touch. In our prototype, the collision occurs between the hands and the rest of the body. We use Unity3d′s built-in physics engine to detect collisions. Unity3d provides templates for collision detection depending on the object shapes, e.g. cubes, spheres, or cylinders. For more complex objects, the mesh-collider-component provided by Unity is needed, which creates colliders according the object’s 3d geometry. A mesh-collider applied to the virtual avatar would result in an exact collider representation corresponding to the avatar surface. Unfortunately, mesh-colliders are static and remain in their initial position once the avatar performs movements. As a result, the virtual avatar and the collider representation do no longer match. To fix this issue, we use the Unity asset RASCAL which determines the skeletal structure of the avatar and adds mesh-colliders to the individual avatar-bones. This ultimately creates a collider representation which moves dynamical according to the avatar movements as shown in Figure 2. For self-touch the reliable detection of a collision between the hands and the rest of the body is a prerequisite. However, the avatar hands are not represented by mesh-colliders because Unity3d does not recognize collisions between two mesh-colliders. Therefore, a hierarchy of primitive cylinder colliders modeled on the hand skeleton is automatically constructed and attached to the hand. Therefore, we create a hierarchy of several primitive collider objects at the hand and finger joints which can be seen in Figure 3. The collider objects are automatically created according to the anatomy of the hand and represent a separate entity in the scene, detached from the visually visible hand. This entity, called Physical Hand in the following, contains besides the collider objects also a Rigid-body to enable collision detection. The Physical Hand has no visual representation and is invisible to the user. The physical hand follows the tracked user hand.

Figure 2:

The mesh collider is visualized here as a green mesh grid that corresponds to the avatar’s surface. Unlike a static mesh collider, our used mesh collider is fully animatable and changes according to the movement of the avatar. The hands are excluded from this, as they must be considered as separate entities for collision detection.

Figure 3:

The hand gets a composite of several primitive colliders that approximate the hand anatomy. In this example we use spheres at the finger joints and a box collider for the hand palm. Other composites of primitive colliders would also be possible.

4.3 Visualization

To give the avatars a realistic appearance, they are covered with a texture, i.e. a two-dimensional image. The texture contains an unfolded image that determines the appearance of the avatar. In our example, the avatar has two textures, one for the face and one for the body, as seen in Figure 4. In order to correctly place the corresponding section of the texture on the 3D geometry, UV coordinates are needed. UV coordinates are two-dimensional values that are in the range of [0; 1]. Each point on the texture can be addressed by a UV coordinate. For a correct mapping between 2D image space and 3D geometry, each vertex in the 3D geometry has a UV coordinate to define a point in the texture. UV coordinates can be set either automatically by software or manually by an artist. If the user’s hand touches the avatar body, the corresponding UV-coordinate of the contact point is deter-mined. This UV-coordinate is used to determine the texture-coordinates in a special texture which is used as a color-mask. The color-mask is a gray scale image in which dark areas are interpreted as transparent. Lighter areas are less transparent and appear as a colorization on the avatar. An example is shown in Figure 4. In a spherical area around the determined texture-coordinates the color-values are increased by a small amount of white. Multiple touches on the corresponding body part increase the amount of white in the texture area, which leads to a reduction in transparency. The color representation can vary and be either opaque or slightly trans-parent, as shown in Figure 5. The texture mask is processed in a self-developed shader. The shader is set on a new render material, which is called self-touch material in the following. This self-touch material is added to the existing render component of the avatar. The 3d geometry can consist of either one overall mesh or several sub-meshes. Each sub-mesh has its own render material representing the visual representation. Usually there are as many materials as there are sub-meshes. The sub-meshes get their rendering information from the corresponding material which are ordered by the order of the sub-meshes. By adding the self-touch material, more materials exist as sub-meshes. As a result, the last material overwrites the surface rendering of the second last material. To ensure that the correct material is overwritten in the case of avatars that consist of multiple partial meshes, the internal order of sub-meshes and their materials is changed so that the sub-mesh with the most vertices and their corresponding material is placed as second last element.

Figure 4:

The self-touch visualization was realized using a multi-texture approach. (a) Unfolded body-texture of the example avatar (b) contact points on the body are continuously drawn in its own texture which works as a color mask (c) the color mask works as an overlay to the body-texture. The painted areas will be replaced by a specified color. (d) Shows the results of the touched body-parts visualized on the 3d-avatar.

Figure 5:

Self-touch visualization from an allocentric (left) and egocentric (right) perspective. Touched parts are colorized.

5 Evaluation

To include a preliminary evaluation of the interaction concept of virtual self-touch, we conducted heuristic evaluation. Due to COVID-19 the evaluation was carried out remotely via zoom. Our goal is to present our interaction technique to UX experts in order to address potential usability problems and get new design ideas. Additionally, we want to determine the suitability of our interaction technique in a therapeutic context. For this purpose, we asked experts from the therapeutic domain to validate our interaction concept and to evaluate it heuristically. The results should help to improve our interaction technique in order to validate the therapeutic effect in a future study.

6 Outlook

The results from the heuristic evaluation have shown that while the idea of virtual self-touch interaction has excellent potential to augment mind-body therapy, there is still room for improvement. However, the opinions of the experts show that our idea could help in the therapeutic environment to increase the attention to one’s own body. The idea of our interaction technique was compared to a metaphor like applying lotion on the body. However, the opinions of the clinical experts also showed us that the interaction technique should be used by affected patients only in supervision with a therapist, in order to avoid negative emotions and a counteracting therapeutic effect. In order to improve body tracking precision, we are planning to use trackers of a newer generation. Furthermore, the incoming tracking data should be smoothed in order to filter possible outliers. Through an improved physical model, we want to prevent the hands from penetrating the body. To prevent cases in which the physical hand is touching the real body but the virtual hand is still above the avatar surface, a collision offset could be considered. The collision would then already be detected even if there is no contact between the virtual hand and the body. In order to make the collision appear more believable, the hand could snap to the body surface. It is also planned to employ individual finger tracking by means of Leap-Motion sensors which can be attached to the Head Mounted Display. Finally, we will design and evaluate a therapeutic session and include the self-touch module in a virtual therapy program for obesity disorders [17].