Towards an All-Day Assignment of a Mobile Service Robot for Elderly Care Homes

Due to the demographic change in many industrial nations, the proportion of the older population is increasing. With this increase, the number of people who are dependent on outpatient or inpatient care is also rising across the board. Against this background, digital assistive systems could play an important role for improving the situation within those sectors. Therefore, the proposed novel approach describes a possible all-day use of a mobile assistive robot within an inpatient geriatric care facility, which should both relieve the staff and provide a therapeutic and entertaining contribution for the residents. The design of the components of the robot platform required for all-day use was carried out in an iterative development process. This process was started by convening a focus group, which first analyzed the requirements and then critically questioned the current status and actual benefits. Additionally, the accompanying occupational therapists and care assistants (N = 6) answered questionnaires after each of the 32 completed assignments, which were intended to draw attention to existing weaknesses and positive aspects. The main focus was to answer the question of how an assistive robot can be used meaningfully within an inpatient geriatric care facility with the means of the current state of science and whether this platform is perceived as support by the groups of people concerned. Due to the predominantly positive response to this question, the concept presented here for all-day use could be realized. Even if the response and operational capability were predominantly positive, there are still wishes from the staff and residents. These demands cannot yet be guaranteed with the current state of science to the required high degree of robustness under real world conditions. Consequently, the components identified as still in development or conceptually conceived require further research in the respective fields.


Introduction
So far, various projects have successfully investigated the use of assistive robots within individual specified interventions of inpatient geriatric care and in providing care within people's homes [1][2][3][4]. In contrast to focusing on single tasks, this publication's novelty lies in the combination of the utilization of therapeutic as well as general supporting tasks within a inpatient geriatric care facility. As outcome, we provide a prototypical full-time deployment schedule for a mobile robot platform with the available technical possibilities and limitations of the current state of the art.
The resulting schedule is primarily oriented towards the needs of the target groups and provides the proposed daily routine within the cooperating care facility with assistance functions.
The selection of the tasks to be performed by the robot was based on surveys with the employees and on the regular meetings of the focus group. Especially the work with the focus group was essential for the applied iterative and user centered design and development process [5].
Additionally, the majority of research regarding the support of therapies by assistive robots focuses mainly on the therapy participant or the resident and care-giving relatives. With the proposed daily routine, residents as well as personnel could benefit from using such a platform.
With the goal to support all peer groups of an inpatient care facility during a complete day, there is the need to be able to navigate independently within the living environment and therefore the possibility to use smaller therapy robots (e.g. Paro [6]) is excluded.

Focus Group
The development of the presented solution was initiated by conducting a focus group, which consisted of representatives of employees, management, residents and their relatives as well as robot experts, technical philosophers and psychologists.
The goal was to formulate ideas for robot deployment that were able to lessen burden off the staff by means of technical assistance and to enrich their work as well as to provide a therapeutic background for the deployment.
The focus group was planned, conducted and evaluated based on standard guidelines [10][11][12]. During the design and implementation process, the three meetings of the focus group were key in assessing global existing and future functionalities. Accompanying these meetings, the user centered development was ensured by regular project meetings and questionnaires. The evaluation of the focus group meetings (exact composition, sampling strategy, duration, number of sessions, and especially evaluation and method of analysis) will be published separately in the future.

Assistive Robot and Technical Requirements
The used robot platform Anna-Constantia of the type Scitos G5 of the company MetraLabs GmbH (see Figure 1) is equipped with a differential drive for indoor use, which enables it to travel at a speed of up to 1.2 m/s. For this purpose, this platform is equipped with two laser range finders, which ensure omnidirectional perception of the environment and serve as means for localizing the robot within the operational environment and detect obstacles and persons. Furthermore, the robot has a movable head, which itself does not carry any sensors but only serves as an interaction point, a touch screen, and a tactile sensor on the foot, which detects collisions. In addition, a mounting rack has been added to the upper part of the robot around the head in order to attach a pan-tilt unit with a Microsoft Kinect One, additional speakers for better acoustics and two more camera sensors for omnidirectional image processing.
Two of the challenges highlighted by the focus group were the integration of the robot into a therapy for people suffering from dementia and the support of the night shift. Since both main pillars have separate requirements, they should be treated in the following separate Sections 2.5. and 2.6. The deployments of the robot were critically monitored by further meetings of the focus group and questionnaires. In particular, next development steps and possible improvements were discussed.

General Requirements
In all scenarios the robot depends on robust humanmachine interaction. Two main aspects should be highlighted. On the one hand, the robot must be able to perceive people accurately. The detection and recognition of people in camera images is in all cases realized by a deep learning based approach [7] which is trained with publicly available image data-sets of faces [8,9]. Secondly, the most intuitive humanhuman interaction is based on speech-based communication. While a large number of very good variants are available for speech synthesis, there is currently no robust context-free automatic speech recognition system, that goes beyond the recognition of predefined phrases or command words. For this reason, at the current stage of development, most of the robot controls are implemented via touch screens.
The only possible solution of more robustly exceeding a command word based speech control at this point of time would be to incorporate the cloud based voice recognition systems provided by the big tech-companies (e.g. Google, Amazon). Unfortunately, the handling of the data is not transparent in questions like: 1. Where is the data stored? 2. Who listens to it?
3. For what is it used in future scenarios (emotion recognition, speech based person identification)? 4. Is it possible to delete previously recorded voice inputs? Due to these open questions, it would be against the applicable Data Protection and Data Security law of the European Union as well as ethical questionable to use aforementioned systems while working with people suffering from dementia or other cognitive impairments.

Data Security and Data Protection
While working with highly sensible data (e.g. observation of working space, recording medical or personal information within the living quarters) it is of utmost importance to build a foundation of trust towards the assistive robot. Besides information events for staff and residents, that was accomplished by providing booklets and declarations of consent in simple language. Those had to be signed by residents respectively their representatives and staff members to be able to take part in the research project.

Integration in the MAKS-Therapy
The non-drug multi-component group therapy MAKS (stands for Motor stimulation, Activities of daily living, and Cognitive and Social functioning) offers a proven cognitive stabilization and improvement of non-cognitive symptoms for people with mild to moderate dementia as well as for people with mild cognitive impairment [13][14][15][16][17][18].
This group intervention combines the four aspects of motor stimulation, activities of daily living, cognitive and social functioning and summarizes various exercises in a manual. The task of the robot within this therapy is to support the MAKS-therapist in interaction with the residents. While the robot performs an exercise from the MAKS manual, the therapist can focus on individual participants, which need more attention or is able to better observe group dynamics and individual performances of the participants. The robot should also provide information on the participants (biography, health, previous performances) and exercises from the MAKS manual.

Robotic Nightshift Companion
The functionality of the Robotic Nightshift Companion (RoNiSCo [19]) includes patrolling the corridors of a care home to search for residents who have either fallen or are wandering around aimlessly due to disturbed perception of time. If such an incident happens, it is communicated to the staff via a smartphone application which holds an encrypted connection to the robot. After receiving an image and the location of the situation, the nursing staff can now decide, how the robot should react to keep the elderly at the robots current position, so that the nurse can accompany the resident back to her/his room. Furthermore, it is possible to use the smartphone application to call the robot to the nursing staff's location in case of an emergency. For that reason the robot can be equipped with a first aid kit.
To meet this challenge in a robust manner, the robot must be able to use a wide variety of navigation algorithms. For this purpose, the most suitable representatives of their algorithms were selected for use or new procedures were developed. This begins with the creation of an environment model (Google Cartographer [20]) in which the robot must localize itself [21]. Also within this environment model the robot dynamically plans its desired path [22] and avoids obstacles by its reactive navigation behavior [23]. To improve the robustness of the robots behavior, it furthermore has to understand the dynamic of its surroundings [24]. This is related to a car navigational system which chooses its route according to traffic conditions. All the mentioned algorithms were chosen based on their good maintainability and insensitivity to parameter changes.

Realization and Results
From the two main pillars described in the previous Sections 2.5. and 2.6., the large number of suggestions expressed by the focus group as well as the accompaniment of multiple members from the nursing staff during their daily work over several working shifts, it was possible to draw up a prototypical full-day deployment schedule for a robot (see Table 1) to support inpatient care facilities. Since this depends strongly on the technical availability of the robot platform, the pragmatic approach of making the possible applications dependent on the battery capacity was chosen. Theoretically, it would be possible to extend the specified time-frames in terms of duration or scope, but this would not be practically feasible at present. The resulting schedule is specific for the applied assistive robot and therefore can only be a guideline for similar settings. Nevertheless, the technical availability of such a system is essential when planning its usage over longer periods. In order to obtain realistic battery characteristics, the charging and discharging behavior of the robot was determined in practical experiments. This was done by discharging the robot using all the methods previously mentioned in Section 2 (navigation algorithms, head control, image evaluation, voice output) and while charging all modules were deactivated. Both behaviors were linear. In particular the discharge with approximately -12,1% per hour describes the worst-case scenario, since in the rarest cases all modules are really operated at the same time. This means that in real use a higher buffer is available. The charging characteristic is about 15.3% per hour. Figure 2 shows the battery capacity corresponding to the extracted assignments and their time-frames from Table 1.
The suggestions from the focus group also showed that a lot of the exercises and functions used in the MAKS therapy should not only be used in therapy but during the entire fullday deployment. Additional functions like a reminder for the residents to drink water on a regular basis or being a guide inside the facility would then complete the robots role as mobile assistive system that is both therapeutic and entertaining.

MAKS-Therapy
Over a period of two years, the robot-assisted MAKS therapy was tested in a four-stage roll-out. These roll-outs cover a total period of operation of six weeks, during which the robot was used at least three days a week. The therapy sessions were led by 6 different MAKS-therapists (5 female). A session usually took about 2 hours. At the beginning of each session the robot waited in front of the therapy room to greet every participant personally. When everybody had arrived the therapist started with exercises from the social functioning category, like reading the news aloud and singing.
To understand interactions within new domains, roboticists often perform a so-called Wizard-of-Oz experiment [25]. That means a robot operator stays hidden from sight, while people are interacting with the robot. This procedure gives insight into the expected behavior a robot should show within that domain. During the first week of therapy sessions the position of the wizard was taken by a robot expert. However, it turned out that the therapists needed a more direct interaction to reduce the reaction time. Therefore, an easy to use tablet application was designed, which not only encapsulates the functionalities of the robot but also passively documents the therapy exercises performed and the persons present.
In the first roll-out we incrementally increased the number of features the robot could provide to gather more explicit usage information and to avoid overburdening the MAKStherapists and the participants of the therapy. In this stage, the robot was able to read newspaper articles, respond to these articles and initiate conversions about them, tell jokes as well as ask the elderly people for their zodiac sign and read the corresponding horoscope (which was part of the opening ritual within that specific care home and is not part of the MAKS manual). The robot's contribution to the therapy is currently mainly focused on situation dependent speech output as well as providing media (e.g. images, subtitles, videos) via a video projector, which is either connected to the robot's wireless network or attached to the robot itself.
The following iterations included a much wanted karaoke functionality, which not only plays subtitled music but also offers conversation starters about the singer or band or the time-frame it was released. Besides this function, the robot reads famous illustrated fairy-tales, which a lot of elderly could recite while the robot was speaking. The therapeutic aspect from this could be a cognitive stimulating game in which the participants have to complete sentences of those fairy-tales. Additionally the robot observes all therapy activities and documents them in a format which is used in the care home. Most of those functions had to be triggered manually by the therapists, which was quite a lot of work.
The user interface of the tablet application can be seen in Figure 3. From the main menu the therapist can access all 4 main categories from the MAKS manual. A collection of corresponding exercises can be found under each category. Exercises that are yet unsupported by the robot show textual instructions from the manual. In addition a detailed statistical overview of the current documentation and all previous sessions can be viewed to get a wanted distribution of exercise types. The menu on the top enables the therapy leader to control basic functions of the robot like speech speed and the degree of autonomous interaction behavior. As an example for the user interface of an exercise Figure 4 shows the news reporter function. First the therapist picks an article from a picture carousel. After that he or she will be able to play the entire news article, project relevant images and ask for feedback. Sentences can be accessed individually and thus be repeated if necessary.   In the current development cycle, the robot is able to recognize the identity of people and knows their biography. This enables it to automatically initiate different functions like asking for the zodiac sign or speak directly about different live aspects of the participants as a form of memory maintenance and initiation of group discussions. The possibility of this memory maintenance supported by own or free available fitting images is a promising way to train cognitive and social skills. The provided images present a variety of impressions from previous work places, holiday travels or the previous living environment and relatives of the residents. Furthermore, with the robots database, the therapist is able to conveniently access all information about the participant directly, which greatly simplifies such tasks of biography work. Matching algorithms promote context relevant information from the biography database to the therapy leader e.g. if a newspaper article is related to the birthplace or hobbies of one of the residents in the session. By automatically adding the attending residents to the therapy documentation, the robots people identification system also helps the therapist to keep track of the therapy session.
Currently under development is a therapeutic collaborative game, which was already tested during the last roll-out. The main idea behind this game is to utilize a microphone array to detect the direction of noises from the participants who are sitting in a U-shape around a projected game-board on the floor. The noises can be produced by either clapping or by using musical instruments. The corresponding direction of the loudest noise repels a herd of sheep in the opposite direction. The main goal of the game is to guide all sheep over a meadow back to the barn (see Figure 5). On the meadow several bushes, trees and creeks are used to create a labyrinth-like environment. Depending on the previous performance of the participants, the difficulty increases or decreases. The goal, we tried to achieve, was to create a collaborative game, where the robot is the story teller as well as the mobile projector and all players have to work together to coordinate the best strategy to guide the sheep back. The constructed winning condition is thereby not dependent on the performance of a single player, hence everyone wins when the game finishes and no one feels shame or guilt. Since this game activates cognitive (by coordinating), motor (by generating the sounds) and social (achieving the common goal, talking to each other, experiencing success and fun together) skills, it fits perfectly into the portfolio of MAKS therapy. Also in development is a classical but collaborative multiple choice quiz game, in which the residents will be able to work as a team and use different physical input techniques, like shaped and colored signs, to vote by majority.
During the four roll-outs the performance of the robot was evaluated by the MAKS-therapists. To obtain data on the mindset and value of the therapists concerning the robot as well as to gather suggestions for improvements, the therapists were handed a self-developed feedback questionnaire after every robot deployment. The standardized questionnaire asked if the robot was helpful for the therapists, if the residents liked it and if it was of any use for the therapy. The acceptance by the residents was subjectively assets by the therapists. The questionnaires items had an equidistant fourpoint Likert scale for answers. A neutral response category was omitted to avoid the error of central tendency. To iteratively improve the robots interaction and features the questionnaire also contained two open questions to gather information on " features that worked out well" and to gather suggestions on " what needs to be improved" . Methodological background was provided through experienced researchers and the standard work for test theory and questionnaire design by [26]. In total, 32 questionnaires were given to the 6 therapy leaders. The results to the closed questions are shown in Figure 6. As the six therapists could not equally participate in the sessions due to their tight work schedules and the high staff turnover the median answer was calculated for each therapist to better visualize their opinions. In terms of the first question: "being helpful to the therapist" and the third question: "being an useful addition to the therapy itself", 5 out of 6 therapists stated that this was rather the case than not. In case of the second question: "acceptance by the residents" 6 out of 6 therapists stated that the residents liked the robot quite a lot. All responses were positive and none of the therapists thought that the residents disliked the robot. There is no indication on a time component in the given answers, i.e. the answers given by a single therapist were consistent over time.
The feedback of the open questions addressed smaller quality issues concerning speech output and the user interface. The reduced need to read stories and articles aloud while retaining a sort of personal interaction between the reader and the audience was highlighted very positively.

RoNiSCo
So far, two successful prototype runs have been carried out during night shifts. After the evaluation of the gained knowledge within the new operational environment, the optics and the usability of the smartphone application as well as the algorithms for navigation and detection of persons will now be integrated in a form that makes it possible to use the robot in a practical way in everyday work.
During the development of the app, for example, an operating concept was created which enables the nursing staff to call the robot with only one hand. The background is that the mobile phone should be operated while an injured resident is being cared for or comforted. The robot, equipped with a first aid kit, can thus be called to the scene of the accident without any complications or leaving the resident alone.

Development State
The current development state of the robot with its main and sub functions is summarized in Table 2. The matrix of features shown, gives an overview of already available features as well as those that still need further improvement based on the surveys conducted. In addition, conceptually conceived open wishes of the focus group were included. This consolidation is kept short to give an overview, going into details would highly exceed the scope of this publication.

Discussion
During our six-week assignments within the stationary geriatric care facility we observed an increasing positive personalized perception of the robot. It turned out to be very astonishing with how much curiosity and technical understanding the residents encountered the robot. In addition, we were able to observe that the occupational therapists' work was made easier and more enriching. For example, they were relieved of more strenuous activities such as reading stories and articles aloud, which not only led to less stress but also helped them to concentrate more on their patients.
However, there is a great need for improvement, especially in speech-based communication, both in automatic speech recognition and speech synthesis. Speech recognition suffers from the problem that currently no system exists, which is able to robustly recognize spoken statements under real world conditions (e.g. ambient noise, dialects). This problem is aggravated in the context of care for the elderly, since here one has to reckon with people who are no longer able to express themselves clearly to the required extent due to various handicaps. The main problem of speech synthesis mentioned, is the lack of emotion in prosody. Both residents and staff repeatedly asked for a more distinctive emotional component in speech output.

Conclusion and Outlook
In many industrialized countries, the effects of demographic change and a severe shortage of skilled workers are already being felt in both the care of the elderly and medical care. Therefore our main criteria included the investigation of a technical assistive platform, which can be realized on the one hand by an availability as soon as possible and on the other hand by a high acceptance.
The development completed so far has shown that it is possible to introduce a robot platform into the everyday work of these supply fields and to make a contribution through recognizable and communicable benefits, which can be appreciated by all groups of people involved.
However, in future work, a number of other outstanding issues still need to be resolved to robustly deploy an assistive robot without the present of an expert. Additionally, the present functions have to be completed or extended to guarantee an higher degree of autonomy of the robot to truly reduce the workload for the care-giving staff and enhance the living quality of the residents. From a financial perspective, an inpatient care facility must be able to afford an assistive robot. In order for this to be possible, a wide range of tasks is required, with which a potential purchase can be amortized. The proposed prototypical all-day schedule therefore should stand as a guideline for similar settings..

Ethical Statement
All human studies described have been conducted with the approval of the responsible Ethics Committee, in accordance with national law and in accordance with the Helsinki Declaration of 1975 (as amended). A declaration of consent has been obtained from all persons involved.