Designs of Interactive Sound Systems for the Visually Impaired to Enjoy Digital Art by Recognition of Hand Gestures

Somatosensory interaction is a kind of man-machine interfacing technique for information acquisition through human postures, which are widely used in digital art nowadays. To create opportunities for the visually impaired to enjoy digital art, two sound art systems named Dynamic Sound and Concrete Sound, which are based on somatosensory technology, were designed in this study for the visually impaired to appreciate with pleasure. The former system emphasizes resonances between humans and sound, allowing the visually impaired user to trigger different sounds by hand gestures which promote the user’s physio-pleasure and ideo-pleasure. The latter system, also being controlled by hand gestures, combines sounds with three-state physical phenomena as feedbacks which are explained orally by an accompanying person to the visually impaired user, creating an inter-person communication that promotes the user’s socio-pleasure. By public exhibitions, users’ feedbacks were acquired via interviews, and evaluated to prove the effectiveness of the proposed systems with the following findings: 1) interactions by hand gestures offers the visually impaired with opportunities to enjoy digital art; 2) sound art provided by the systems allows the visually impaired to gain pleasure via man-machine interactions; 3) the systems innovatively integrate dynamic visual performances with auditory feedbacks in the interaction process; and 4) through the development of gesture operations, more performance techniques can be devised for sound art, allowing gesture motions to replace control interfaces in future designs.


Motivation
Visually impaired people contact the outside world through hearing and touch to compensate for the loss of visual perception, and through long-term training they can have sufficiently good abilities for spatial and object judgment. However, with the advance of digital technology, more and more diverse and complex sensory stimuli that humans must face in the daily life have been generated, especially those found in digital products or artworks with various types of visual media interfaces [1][2][3][4]. Facing devices of such vision-based objects, the visually impaired are in a relatively weak position. It is desirable that digital technology can be properly used in designing such devices to reduce the gap between visually impaired people and those with normal vision, making up for the psychologically difficult situation faced by the visually impaired to improve their self-confidence in life.
Music plays a role for humans to exert their creativity. If a person can get an opportunity to create music when leaning it, they are able to express themselves to show their potential for art; and in this process, they not only can satisfy the imagination but also can get achievements to identify one's own value [1]. Therefore, how to help visually impaired people to regain control of the sound with the assistance of digital technology and obtain the affirmation of others through the performance of sound art is a question worth pondering.
Furthermore, the fast development of digital technology has created many good ways for human-machine interactions, which may be utilized to design good interfaces of sound-based artworks for the visually impaired. Accordingly, visually impaired people can experience the charm of sound performances and get pleasant feelings through simple interactive operations, eliminating the barriers of high-tech products. This means that suitable usages of sound-base man-machine interfacing techniques in the design of digital artworks are worth study for the welfare of the visually impaired.

Types of Visually Impaired
There are three types of visually impaired, namely, congenital blindness, adventitious blindness, and amblyopic vision. People of the first type refers to those losing vision before 14 years old and have few visual impressions. Due to earlier blindness, they rely on hearing and touch abilities to handle daily living activities before their mind comes mature. Therefore, they can use hands to recognize things. Besides, the time they spent on Braille training is much longer. Compared with the other two types, they have better tactile recognition ability, even better than normal people [5]. However, because of their earlier blindness, they do not have longer visual impressions and have difficulty describing "exterior" images or writing words.
People who have adventitious blindness have deeper impressions about outside images because they developed blindness later in life. However, the way they recognize and touch pictures and symbols are different, causing them to spend more time to get familiar with tactile recognition and Braille. Finally, a person with amblyopia vision has a worse visual capability than common people, and he/she can still live a normal life but with a certain degree of inconvenience in seeing surrounding things. In this study, the visually impaired are assumed to those people of the first two types.
The blindness brings the visually disabled many inconveniences in daily activities involving food, clothing, housing, transportation, education, and entertainment. It also forces them to face enormous stress in their occupations. People who suffer from adventitious blindness will face psychological pressures when they lose vision. Besides the in-mind pressure that they are struggled with by themselves, the pressure from the outside also count for a big part.

Purposes of This Study and Limitations
The goal of this study is to design of interactive sound systems that allows visually impaired people to interact through operations of hand gestures, enhancing their interests in interactive art, giving them feelings of pleasure, and increasing self-confidence in their lives.
More specifically, by comparing different types of sound artworks, it is hoped that the following purposes can be achieved in the design of interactive sound systems that are appropriate for the visually impaired: (1) exploring the difficulties that the visually impaired face in digital art; (2) collecting and analyzing related works to create a set of principles for designing the desired systems; (3) constructing accordingly the desired systems that are based on the use of hand gestures [6] and can bring the visually impaired people feelings of confidence and pleasure; and (4) encouraging the visually impaired to use the systems to increase their willingness to use other types of interactive artworks. Because the systems designed in this study is mainly for use by the visually impaired, the issues of research in this study are related to those satisfying the needs of them and bringing them pleasures and confidence in life. Also, convenience for use by the visually impaired is emphasized, so that the spatial range for using each designed system should be limited to the 3D space right before the system, so that the effort made by a visually impaired user to search the space while using the systems can be reduced. Finally, as mentioned previously, the proposed systems are to be used by the people of the first two types of visual impairment, not including those with amblyopic vision.

Literature Review
The main purpose of this study is to design interactive sound-art systems for uses by the visually impaired, which can promote their feelings of pleasure after performing the systems. Therefore, the first stage in this study is to conduct an extensive literature review that includes the aspects of behaviors and psychology of the visually impaired, interactive and somatosensory techniques for man-machine interfacing, existing works of sound art, and pleasure in product uses by the visually impaired, aiming at collecting ideas and principles for the prototype development and system design in the subsequent stages of this study.

The Visually Impaired and Digital Art
Eyesight is necessary for human beings in many entertainment and artistic activities. Normal people can enjoy movies, paintings, books, and so on, by their visual capabilities. A visually impaired person, no matter whether they have adventitious or congenital blindness, must rely on his/her tactile and hearing senses to appreciate things, losing opportunities to enjoy digital art that utilizes visual interfaces. Unfortunately, most digit art devices are based on visual interfacing and so keep the visually impaired away from interactions with such devices. For example, as said by Lai [7] in a digital art festival, blind people have never been real visitors to an art exhibition because most interdisciplinary art creations nowadays cannot exclude the expression of the art content by visual images; when the blind people step into the exhibition, the only artwork they could appreciate was "Years," an artwork like a record player created by Traubeck [8] which plays "slices of wood," as shown in Figure 1. When the blind people touch the tree ring on the turntable of this work, the voices played by the turntable allow them to feel the idea expressed by the digital art. That is, this work is played by hand touch. This implies that it necessary to study the behavior of hand exploration in order to design the interface of the sound device for the visually impaired to use interactively, as conducted in this study. Figure 1. A work of sound art "Years" designed by Traubeck [8] mentioned by Lai [7]. (a) The appearance of the work. (b) The tree-ring turntable of the work.

Behaviors of Hand Exploration and Interface Design for the Visually Impaired
For a congenitally and adventitiously blind people, under the situation of losing the visual sense, hearing and touching become their ways of perceptions to communicate with the 'exterior' object. How to read and recognize things by hand is a problem in their daily activities. Hand exploration requires touching to know materials of things. Especially, Braille that the visually impaired must learn is a system that requires hands to complete the action of reading. Most visually impaired people send or receive messages by hearing, Braille, or object materials [9]. There are two types of touching on fingers: active touch and passive touch. In an article by Gibson [10], the action of active touch refers to touching things by hand by one's own will, while that of passive touch refers to fastening the position of the hands and allowing things to touch the hands. These two types of touch get different effects under different operations as described in Table 1. Specifically, the action of active touch allows people to decide the touch scope, path, and time at their will so that more of the finger muscles and the fingertip tactile perceptions are used with more information being acquired, while the action of passive touch requires the uses of fingertips, palms, and bigger ranges of tactile perception to recognize surrounding objects. Computers for the blind and refreshable Braille displays nowadays are based on combined uses of active-touch and passive-touch operations. Through active-touch keyboards, people can do typing and the computer will yield the corresponding words for the visually impaired to read.
A visually impaired person often takes himself/herself as a point in space and reaches out to explore the surrounding environment. In a training of orientation ability that they take, they must rely on other perceptions to collect information of the surroundings to measure their own positions and objects in the space so as to build the concept of space map and reach the destination safely. Therefore, the more familiar they are with hand exploration, the better their ability to recognize the environment. There exist quite a few studies on hand gestures for sound control [11][12][13][14][15], which, however, were not applied to the welfare of the visually impaired. Withagen et al. [16] think that the visually disabled can built three-dimensional images in their mind in the same way as normal people do. Although spatial structure is easy to define with visual perception, the visually impaired can even built three-dimension images in their mind after they take professional trainings. Thus, the visually impaired are not absolutely unable to recognize space and images though they have faulty vision. That is, they still can build their own world in their mind through their touch perception abilities.
In a book about visual impairment education, it is said that the visually disabled cannot develop cognition on depth, complexity, and comprehensiveness [17]. However, this is not true if they rely on their touch perception abilities. Therefore, when people explore the characteristics of interface operations of technology products for the visually impaired, they should notice the influences brought by the touch perception and orientation ability of the visually impaired.
Under the situation of losing vision, it takes the visually impaired more time of hand exploration and orientation training to recognize objects in a space. Especially, for people who suffer from congenital blindness, they do not have visual memory, which cause them difficulty to build mental models for interactive interfaces when facing too big or too small interfaces. They need more time and concentration to confirm the position and the purpose of the interface in the space. In contrast, people suffering from adventitious blindness have visual memory. Besides depending on other perceptions to receive information, residual visual memory can also help them build accurate spatial concepts and get used to interactive processes. Therefore, the time they spend is less than that required by congenital blind people.
As a summary, when designing the interface from the viewpoint of the visually impaired, it must be known that different types of visually impaired have different needs, requiring especially different visual aids. With no doubt, the visually impaired have quite big needs for haptic interfaces. When designing the interface, one should consider the details in the interface carefully, such as how to make tactile exploration easy for them to operate efficiently, and how to reduce the time for them to use the digital product via the interface. In this study, it is desired to construct interactive sound systems with appropriate interfaces, helping the visually impaired to rebuild confidences and get pleasant feelings when they use the systems by themselves.

Developments of Somatosensory Technology
New interactive technology paves the roads between humans and machines. However, the goodness of the interactive device does not lie in the efficiency and functions of the product, but in the effectiveness for enjoyment and interaction [18]. In recent years, the contents and visual effects in games have been gradually improved, and the developments of technology and creativity also reform the operational interfaces in games greatly. Such advances of games not only represent great progresses in human-machine interfacing, but also change people's concepts for games and the forms of game playing, like the uses of somatosensory techniques. 3 Applications of somatosensory technology are diversified, including gaming, art, teaching, science, consumer electronics, medicine, etc. Somatosensory technology integrates many different fields, and due to its uses of intuitive and direct interfacing operations, such technology makes human-machine interaction much more humanizing. It is desired in this study to explore the enjoyment felt by the visually impaired when they use machines; therefore, somatosensory devices designed for them must be considered from the viewpoint of visual impairment. Especially, it is difficult for the visually impaired to recognize images and space; therefore, devices with interfaces of tactile sensing and hearing natures should be the first choices. Besides, as mentioned previously, "technique" is not the top priority for the visually impaired; instead, "interactivity" is the main consideration.
Some common-seen somatosensory devices in the market are illustrated in Table 2 with their sensors, sensing techniques, and applications listed. The table content is arranged in the chronological order, from which it can be found that the sensors used in the designs of somatosensory interaction devices gradually change from physical sensors to visual sensors and biosensors, where the visual sensors are capable of sensing hand gestures or detecting humans. Physical sensors rely on electronic and infrared components to detect the three axes in the 3D space and calculate the object position. Though physical sensors are more suitable for detecting large body movements, yet they are not suitable for sensing small body dynamics. On the contrary, visual sensors can be used to detect both body movements and tiny finger motions. Therefore, the visual sensor is regarded in this study as more appropriate for the interface to be used by the visually impaired. Among the visual sensors mentioned in Table 2, the Leap Motion controller is the most properly-used one for the visually impaired because of its functions of sensing hand and finger motions in small ranges. Therefore, it is adopted in this study as the sensor for use in designing the interface of the desired interactive sound systems.

Applications of Somatosensory Technology to Sound Art
The evolution of technology has brought people to digital times, and innovations in digital media makes sound art more diversified in type, allowing artists to create value-reflecting performances by digital software. Combining with art performance in different fields, sound art has jumped out of old principles and become an independent art field in recent years. Through calculating sound changes by computers and mixing interactions, it becomes possible to use sounds to reflect a user's body motion immediately via somatosensory interactive devices. Humans' gestures, body motions, and dynamic tracks can be turned into media signals for conducting music performances. A good example is the work, "This is Beatjazz," unveiled by Ashanti on TED in 2019 [25]. By wearing handmade sensors and conducting body movements, a user can control the issue of sound to perform a jazz dance show, as shown in Figure 2. More examples of existing interactive sound artworks are shown in Table 3 with the interactive interfaces, sound feedbacks, and visual feedbacks of the works listed in the table.
As can be seen from Table 3, the combined use of sound art and interactive technology has produced many excellent works. Among them, somatosensory interaction has gradually become the trend of adopted interactive technology which, however, is not always the first choice for the creator to use in sound art; the use of traditional electronic components sometimes can create outstanding interaction effects as well.

Pleasure in Product Uses by the Visually Impaired
Jordan [33] defines the pleasure and displeasure in product use as the emotional and hedonic benefits and penalties, respectively, associated with product use, and pointed out eight emotional factors which affect the pleasure of a person using the product, namely, security, confidence, pride, excitement, satisfaction, entertainment, freedom, and nostalgia. The pleasure obtained from using a single product may come from several of these emotional factors.
Product attachment has been defined as the emotional bond a consumer experiences with a product, which focuses on the direct relationship between the user and the product [34]. Norman [35] indicates that the attachment between the user and a product can be formed through three levels of involvement: appearance and aesthetic appeal to the user; effectiveness and pleasure of use; and personal satisfaction of using the product. Furthermore, in a book entitled Designing Pleasureable Products, Jordan [36] mentioned four types of pleasure triggered by humans' different external activities or internal feelings, namely, physio-pleasure, socio-pleasure, psycho-pleasure, and ideo-pleasure, as elaborated in Table 4.
When the four types of pleasure listed in Table 4 are to be applied to the visually impaired, the first problem to be solved is how to acquire physio-pleasure, followed by the acquisitions of socio-pleasure and psycho-pleasure, and finally, the problem of acquiring ideo-pleasure. In this study, it is necessary to design good interfaces for the proposed systems for the visually impaired to acquire physio-pleasure because such pleasure comes from the user's feeling of the device in the system through his/her senses and body, like the use of the hand (s). In addition, the loss of vision makes it impossible for the visually impaired to have ideo-pleasure coming from visual appreciation of art or reading of books. However, they still can have such ideo-pleasure given by appreciation of sound and music through the exertion of their hearing sense. One more type of pleasure that can be enjoyed by the visually impaired is socio-pleasure which comes from communication between them and bystanders, because with no vision the visually impaired can still conduct oral dialogues on common topics with bystanders around them. Table 4. Four types of pleasure proposed by Jordan [36].

Type of pleasure Explanation
Physio-pleasure Physio-pleasure is felt through humans' senses and body; touch, taste, and smell will all have an impact on the feeling of such pleasure.
Socio-pleasure Socio-pleasure is a kind of social interaction, which is related to the group communication between individuals and individuals or among multiple people.
Psycho-pleasure Psycho-pleasure involves the cognitive and emotional responses to a user's experience; when the user achieves a goal with a device quickly, the resulting satisfaction of needs will increase the sense of such pleasure.
Ideo-pleasure Ideo-pleasure is closely related to the human's view of aesthetics; when a person feels the same value as his/hers in the process of book reading, music listening or art appreciation, he/she can get such pleasure.
The above observations about the three types of pleasure − physio-pleasure, socio-pleasure, and ideo-pleasure − which may be acquired by the visually impaired give hints for designing digital artworks or systems which may be appreciated by them, including good interfaces based on uses of hands, creations of sounds or music, and oral communication with bystanders, etc. These hints are followed in this study as principles for designing sound artworks for the visually impaired to appreciate.

Summary of the Literature Review
The literature review above helps collecting principles for prototyping development in this study, which are described in the following: (1) losing vision to see directly the outside world, the visually impaired can conduct hand exploration to use tangible interfaces and recognize objects' positions in the surrounding space; (2) the visual sensors in somatosensory devices can detect slight body motions better than physical sensors does; and especially, the Leap Motion controller can be adopted to detect the user's dynamic hand gestures effectively; (3) sound-art devices include diverse technologies of different fields; from the related cases that are surveyed above, uses of rhythmic sounds and switches of electronic components are more commonly seen; (4) offering various types of pleasure like physio-pleasure, socio-pleasure, and ideo-pleasure is achievable and should be considered in the design of digital-art systems for the visually impaired.

Research Methods
The research methods adopted in this study include prototype development and user interview, which are described respectively in the following.

Prototype Development
The procedure of prototype development in this study is divided into four stages as shown in Figure 3 and described in the following, by which two different prototype systems named "Dynamic Sound" and" Concrete Sound" were designed: (1) confirmation of needs  according to the result of the literature review, it was confirmed that the somatosensory technique was to be used to design the sound-based physical interfaces of the prototypes and to develop accordingly the interface operations for man-machine interaction; (2) construction of prototypes  the construction of a prototype system includes four steps, namely, sound design, device design, interface design, and visual design (i.e., the design of visual displays); (3) collection of users' opinions  the prototype systems were exhibited in which the users' performance processes were observed by invited experts when they were performing the prototype systems, followed by interviews with the experts and users to collect their opinions about their observation results or system performance experiences; and (4) refinement of prototypes  the collected experts' and users' opinions were analyzed with conclusions used for refining the prototype systems for the ultimate goal of constructing appropriate interactive sound artworks for uses by the visually impaired to gain confidence and pleasure in daily life. (5) The design concepts behind the two prototype systems, "Dynamic Sound" and" Concrete Sound," constructed in this study are described in Table 5. The first interactive sound system, "Dynamic Sound," was designed in this study with an aim at promoting the relationship between the user and the system; through interactions with the proposed system by hand gestures in space, the user can control the sounds issued by the system dynamically and experience the sound changes to enjoy physio-pleasure and ideo-pleasure. The system is designed mainly for uses by visually impaired people of congenital blindness and adventitious blindness. The second interactive sound system, "Concrete Sound," was designed mainly for promoting the communication between people with normal vision and those with visual impairment. A user, either with normal vision or not, can present "concrete" visual contents of physical phenomena via sounds issued by the system to show the state of his/her mind to the people standing around, resonating with them by oral dialogues on common topics through the visual contents presented by the system.

Interview Method
The interview method is a qualitative approach adopted by many research domains in which the dialogue between an interviewer and an interviewee is utilized to find the ideas, motives, and attitudes of the two sides [37]. This method was adopted in this study to evaluate the effectiveness of the designed systems. Specifically, the prototypes of the two systems "Dynamic Sound" and "Concrete Sound" have been exhibited in a public space for a period of one week, and visitors were invited to experience the two systems. Some of the users were randomly chosen and interviewed by the researchers of this study to collect their opinions about using the systems. Totally, four general users and two visually disabled ones were selected; they were interviewed one by one, respectively, after using the systems. Questions for the interviews were prepared in advance, whose scope was focused on the issues of opinion about device operations, feeling of interactive feedbacks, and pleasure from the interaction, with more details shown in Table 6. Table 6. The outline of interviews with users.

Issue of Interview Content of Interview
Opinion about device operations Asking the users about their opinions about the operations of the proposed systems, understanding of the assistance for the operations, and problems encountered during the performance.

Feeling of interactive feedbacks
Asking the users about their feelings on feedbacks of somatosensory tangible operations conducted during the interaction process.

Pleasure from the interaction
Asking the users about their feelings of pleasure they obtained from performing the systems and oral communication with bystanders, including physio-pleasure, ideo-pleasure and socio-pleasure.

Design Idea of the Proposed System
The word "dynamic" in the title of the first proposed system "Dynamic Sound" is used to contrast with the word "inactive," meaning to get rid of the stationary status and emphasize the action of "initiating." In turn, the word "dynamic" implies not only the transfer of the physical strength but also the feeling of the emotion.
The environment set up for the proposed system is shown in Figure 4. The dynamic sound plays of the proposed system is realized by the exertion of hand gestures recognized by the Leap Motion controller. The gestures include various ways of motion or bending of the fingers and palm, such as the three examples shown in Figure 5. The resulting interaction process is not only a combination of various sound plays but also a sequence of aesthetic actions expressed by the hand gestures.

Design Idea of the Proposed System
By the somatosensory functions of the Leap Motion controller [38] as shown in Figure 6(a), different types of hand gestures like those shown in Figure 5 can be recognized by the proposed system "Dynamic Sound" to generate signals which then are transformed, by use of the visual language of Pure Data [39] like the example shown in Figure 6(b), to be instructions of different meanings to adjust the properties of the sound played by the proposed system. Furthermore, the proposed system was designed to include three pairs of knobs on a controller box as shown in Figure 6(c), which can be turned to adjust the properties of the played sounds as well. That is, in addition to hand-gesture recognition, a second way of sound-play adjustment in the proposed system has been devised, namely, knob turning.
More specifically, the three pairs of knobs were set up on the controller box of the proposed system, each being used to adjust a property of the played sound, namely, tempo, tone, or strength, of the played sound, while hand gestures are used to change the frequency, tone, and strength of the played sound.
Possible hand gestures, as mentioned previously, include motion or bending of the fingers and palm, like translation, open and close, and bending, in the space. More detailed descriptions about the sound play of the proposed system will be given after the architectures of the proposed systems are introduced in the next section.
The proposed system "Dynamic Sound" is expected to be not only a bridge for interactions between the human body and sound but also a tool for the visually impaired to create pleasant feelings, meaning that the system is a digital art that does not belong to people with normal vision only, but also to visually impaired people.

Design of the Architecture and Interactive Interface
The main goal of the design of the architecture of the proposed system "Dynamic Sound" is to provide intuitive tangible interfaces that can be performed conveniently by the visually impaired. In the exterior design of the proposed system, a series of audio speakers are hung in the midair and arranged to surround the user, and the control box mentioned previously is placed on a stand right in front of the user for him/her to operate, as shown in Figure 4. In such an environment, the user can conduct various palm and finger motions freely above the Leap Motion controller on the control box to control the play of sound dynamically. The Leap Motion controller for recognizing the hand gestures is fit into the controller board in the control box whose structure is shown in Figure 7. A remote computer is connected to the Leap Motion controller to process the signal data generated by the controller using the visual programming language of Pure Data [39]. Figure 8 shows a simple illustration of the system process of the proposed system "Dynamic Sound." In the three sets of audio speakers shown in Figures 4, the left one is controlled by the gesture of the left hand as well as by the three left-side knobs on the controller box, and the right set by the gesture of the left hand and the three right-side knobs. The middle set of speakers plays the chord of the sounds played by the left and right speaker sets.

An Introduction to the Leap Motion Controller
The Leap Motion controller developed by Ultraleap, Inc. is a somatosensory device [38] that was used in this study to recognize various hand gestures (including finger and palm motions). With its exterior shown in Figure 6(a), the interior of a Leap Motion controller includes two IR cameras and three IR LED's as shown in Figure 9, whose hand-gesture recognition function is based on the principle of binocular computer vision as illustrated in Figure 10(a). When an object like a hand or a finger is placed in the binocular imaging area, the two sets of 2D image data acquired by the two IR cameras are processed to obtain the 3D surface data of the object, just like the 3D imaging function of the human vision. The range that a Leap Motion controller can detect is from 25 mm to 600 mm, forming an inverted quadrangular prism, as shown in Figure 10(b).

The Interactive Plays of the Proposed System
As mentioned previously, by uses of the Leap Motion controller and the visual programming language of Pure Data, various hand gestures of the user can be transformed into signals to control distinct sound plays. Specifically, the following are the functions of the sound plays implemented in this study by various ways of palm and finger motions and bending via the Leap Motion controller and the Pure Data programs written in this study: 1. waving the palm (s) up and down to change the height (s) of the palm (s)  the frequency of the played sound can be adjusted; 2. moving the two palms to the left or right to change the distance in between  the tone of the played sound can be changed; 3. bending a specific finger  the tone of the sound can be changed; and 4. opening and closing the palm (s)  the strength (volume) of the sound can be changed. In addition, the knobs on the controller box can be used, as mentioned previously, to adjust the properties of the sound played through the speakers via the audio amplifiers in the following way: 1. rotating the left topmost and right topmost knobs  the tempo of the played sound can be adjusted; 2. rotating the left middle and right middle knobs  the tone of the played sound can be adjusted; and 3. rotating the left bottom and right bottom knobs  the strength (volume) of the played sound can be adjusted. While the user rotates the knob by one hand, he/she still can play the resulting sound by the other hand. Table 7 is an "interaction storyboard" illustrating how a user operates the proposed system "Dynamic Sound" to play various types of sound according to the process described by (D1) through (D7) above.
According to the above discussions or the content of Table 7, for example, waving the hand of up and down will boost the sound frequency to become rapid; bending the fingers quickly like playing a melody on a piano will change the tones of the played sound quickly; and so on. By combining various hand gestures skillfully and dynamically, a visually impaired user may play the proposed system to play melody-like sounds, not just to accomplish pure man-machine interactions but also to gain both physio-pleasure and ideo-pleasure resulting from the freedom of sound-controlling hand motions, just like conducting a vivid music performance.   [41]. (b) The interaction area of the controller [22]. Table 7. "Interaction storyboard" of proposed system "Dynamic Sound.".

Label
Illustration Action of the user Function of the system Step into the interaction range with the control box in front of him/her.
(D0) Play background music when the system detects the appearance of one of the user's hands.

D8
(D8) Do nothing while the system plays sound as the feedback corresponding to (D1) to (D7) above.
(D8) Create feedbacks corresponding to the detected motion or bending of fingers and palms.

Design Idea of the Proposed System
The basic idea followed to design the proposed system "Concrete Sound" is to transform different "interactive plays of sound" controlled by hand gestures into "concrete presentations of physical phenomena" showing the three states of material, namely, gas (in the form of rising white smoke), solid (in the form of leaping Styrofoam particles), and liquid (in the form of vibrating water waves), as shown in Figure 11, resulting in an artistic expression of mixed audio and visual effects.
Although a visual impaired user performing this system can only hear the sound changes and is unable to see the visual part of the artistic expression, yet he/she can still imagine the created physical phenomena by listening to an oral explanation of the resulting state given by a bystander with normal vision. This results in a communication between the visually-impaired person and the visually-normal bystander, shortening their distance in perceptual cognition and building resonance between them, from which hopefully the visually impaired person can appreciate the artwork just like a person with normal vision, and get socio-pleasure coming from the oral communication between the them.

Design of the Hand Gestures for Sound Plays and Physical State Generations
The proposed system "Concrete Sound," in addition to being used to create sounds by hand gestures in ways similar to those used by the former system "Discrete Sound," can be performed to generate the physical states of gas, solid, and liquid by the three hand gestures of one-hand fist, one-hand motion, and two-hand motion as shown in Figures 12(a) to 12(c), respectively, as described in the following: 1. (C1a) one-hand fist clenching  generating the gas state in the form of rising white smoke that is intended to mean quietness; 2. (C2a) one-hand up-and-down motion  generating the solid state in the form of leaping Styrofoam particles that are intended to represent action; and 3. (C3a) two-hand horizontal motion  generating the liquid state in the form of vibrating water waves that are intended to mean both gentleness and harshness. These hand gestures are recognized by the Leap Motion controller fit into the control box placed on a table below the hands, as shown in Figure 12 also.  As mentioned previously, though the visually impaired cannot observe dramatic changes of the generated physical phenomena by their eyes, the contrast of hand gestures as well as the accompanying bystander's descriptions of the effects hopefully can make them feel the achievement and confidence and get 1) physio-pleasure resulting from operating the proposed system, 2) ideo-pleasure of appreciating the artistic expression of the system, as well as 3) socio-pleasure mentioned previously coming from communication with the bystander about the resulting visual phenomena.

Design of the Architecture and Interactive Interface
The interface design of "Concrete Sound" is based on the continuous use of the Leap Motion controller to recognize the hand gesture performed by the user as illustrated by the system flow shown in Figure 13. The recognition results of the Leap Motion controller are sent to the computer shown in the figure and transformed into instruction signals by the programs of the visual language of Pure Data written in this study. Besides the audio expression of the sound plays controlled by the user's hand gestures, there is also the visual expression of three-states phenomenon shown respectively as rising white smoke, leaping Styrofoam particles, and vibrating water waves, as mentioned above. These physical phenomena are created in the proposed system by hardware and software means described in the following: 1. (C1b) rising white smoke  created by a fog machine in the smoke box, which is connected to an Arduino Mega 2560 board controlled by the computer; 2. (C2b) leaping Styrofoam particles  created by the vibrating effect of the played sound coming from the hand-gesture control; and 3. (C3b) vibrating water waves  created by the sound-vibrating effect in the same way as that for creating the Styrofoam particle leaping effect described above.

The Interactive Plays of the Proposed System
The way of interaction designed for use in the proposed system "Concrete Sound" is based on the input hand gestures and the yielded physical-phenomena; by distinct hand gestures, three different visual effects can be triggered, as described by the process consisting of the steps (C1a) through (C3a) and (C1b) through (C3b) above. Table 8 is an "interaction storyboard" illustrating how a visually impaired user follow this process to perform the proposed system "Concrete Sound." Table 8. Interaction Storyboard of the proposed system "Concrete Sound".

Label
Illustration Action of the user Function of the system

C0
Step into the interaction range & place one hand on the Motion Leap.
Start the play of sound when the system detects the users' hand, and adjust the tone of the sound as the hand moves around on the Leap Motion.

C1a
Show one-hand fist.
Stop the play of sound, with background noise left.

C1b
Keep showing the one-hand fist.
Turn on the fog machine to produce a smoking effect by letting out white smoke from a hole on the smoke box; stop smoking when the fist is turned into an open palm.

C2a
Wave one hand up and down. Initiate the sound play when the system detects the user's hand; adjust the tempo of the sound according to the height of the hand.

C2b
Keep waving one hand up and down.
Pass sound via the speaker to the Styrofoam particle box to create vibration that produces a particle leaping effect; stop particle leaping as the open palm is turned into a fist.

C3a
Move two hands horizontally to the left and right.
Initiate the sound play when the system detects the user's hands and adjust sound frequency when the distance between the two hands or the number of straightened fingers is changed.

C3b
Keeping waving two hands.
Pass sound via the speaker to the water-wave box to create vibrating water waves; stop water vibration as the open palms are turned into fists.

Exhibitions of the Proposed Systems
The two systems proposed in this study have been exhibited in a public space for a period of one week. The performance process of the system "Dynamic Sound" emphasizes offering of personal experiences. Therefore, audio speakers were arranged to surround the interaction area so as to provide the visually impaired user with free explorations and sound feedbacks in the interaction process. In this way, the user is expected to experience the physio-pleasure brought by the man-machine interaction process.
The system "Concrete Sound" allows the user to explore the free interaction process by triggering three-state visual changes with specific hand gestures. In the exhibition layout, the installation was put in front of the user for him/her to enjoy the three different visual expressions with the accompaniers' helps of oral explanations of the resulting phenomena. The visually impaired user is so expected to enjoy socio-pleasure in addition to physio-pleasure.
Some pictures of the exhibitions and the users performing the interaction processes using the proposed systems are shown in Figure 14.

Interviews with the Users
Fifty users were invited to use both of the two proposed systems in the one-week period of the exhibition, and six of them were selected randomly to accept interviews conducted by the researcher of this study. Among them, four have normal vision and two are visually impaired, whose backgrounds and experiences are listed in Table 9. They were interviewed individually with the dialogues of the whole processes recorded by text by the researcher of this study. For each of the two proposed systems, the interviewees were asked questions of three aspects: operational experience, interactive feedback, and feeling of pleasure. The common opinions in their answers were extracted and summarized in the following.

(A) Analysis of opinions about "Dynamic Sound"
(1) About the operational experiences  1. The somatosensory device can be used to detect the users' hand motion and give feedbacks fluently; however, the overall operation is not easy, taking the user longer time to master it. 2. The tangible operational interface is helpful for man-machine interactions, but it requires the user to recognize the function of every button. 3. The users with normal vision thought that the interfacing part of the system was too small, constraining the exertions of the interactive operations, while the visually impaired users, on the contrary, wanted a smaller interface part including all the buttons so that they can use it more fluently. 4. The users can recognize the sound changes of big motions, but cannot identify sound feedbacks of tiny motions. (2) About the interactive feedbacks  1. The users expected the sound feedbacks to be expressed in a more concrete way so that the differences among the played sounds can be discriminated more easily. 2. The control of the tempo of the played sound is weak so that the user cannot experience the changes of tempo in the interaction process. 3. The sound feedbacks may be designed to be more delicate so that the hand gesture can be used to control plays of smaller sounds. (3) About the feelings of pleasure  1. The users can get physio-pleasure from the ways of operations in the interaction process which is based on hand-gesture recognition. 2. The difference in the operational experiences of performing similar devices in the past decided the degree of physio-pleasure felt by the user while using the system. 3. The user's personal experience has an influence on whether they can accept the content of the proposed system and promote their pleasure in using the system.

(B) Analysis of opinions about "Concrete Sound"
(1) About the operational experiences  1. The users can recognize different correspondences between motions and feedbacks in the interaction processes; however, they thought that such correspondences are few and would better be increased. 2. In the interaction process, some users are unable to relate the hand motions with the feedbacks. 3. The connection between the motion and the audio include physio-pleasure, ideo-pleasure, and socio-pleasure which come respectively from exertions of hand gestures, appreciations of the sound feedbacks, and the oral explanations given by by-standers to them about the visual three-state outputs. Such multiple types of pleasure can enhance the interest of the visually impaired in using interactive systems and appreciating digital art to broaden their views of art fields in daily life.
(3) Three multimedia techniques, namely, recognition of hand gestures, audio plays of sound, and visual presentations of three-state physical phenomena, are innovatively combined in the design of the proposed system "Concrete Sound." These techniques make digital art not only focused on designs of displayed contents but also uses of different media. Specifically, physical effects are immersed in the feedbacks of sound art, which enhances the user's experience of feedbacks in the interaction process. Furthermore, in the three-state visual presentations of the system "Concrete Sound," invisible vibrations of the digital sound are turned into visible effects of Styrofoam particle leaping and water wave vibration. Further studies may be directed to extending these techniques to design more novel digital artworks.
(4) Two types of interaction based on hand-gesture recognition have been proposed, which may be taken as beginning tools for designing more digit artworks for the visually impaired. In the design of the interactive schemes, two types of interaction have been designed, one allowing free explorations of the relationships between hand gestures and played sounds and the other permitting triggering of different sounds to yield distinct visual feedbacks via specific hand gestures. These two interactive interfacing schemes increase the opportunities for the visually impaired to interact with digital art and allow them to get acquainted with somatosensory operations based on hand gestures. This can be a beginning of creations of more types of digital art for uses in daily life, not limited to be used by the visually impaired but also to be appreciated by users with normal vision.

Recommendations
In the future, research works may be directed to designing more gesture detection interfacing techniques for use by visually-impaired people to improve the convenience of their life. In addition, vibration feedbacks can also be added to enhance the interaction effect, and be used in different types of artistic expression so that the visually impaired can have broader living experiences.

Conflicts of Interest
The authors declare no conflict of interest.