Profile

Location-Independent Doppler Sensing System for Device-Free Daily Living Activity Recognition

Abstract – The recent advancements in sensing technology have opened up the possibilities for various services that support daily life, such as energy-saving home appliance control. To realize such services, accurate and cost-effective daily activity recognition in a wide range is essential. To actualize such a system, it is imperative to address the following requirements: the acquisition of sensors entails very high costs (Issue 1), it is hard to achieve precise recognition for location-independent activities like reading a book (Issue 2), a burden of wearing devices from the perspective of residents (Issue 3), and the preservation of residents’ privacy is compromised by using image data from the camera (Issue 4). In this paper, we propose a method for recognizing daily living activities utilizing Doppler sensors in a relatively longer detection range than other motion detection sensors that can be used for dynamic objects. We assess the proposed system by optimizing recognition accuracy, evaluating ensemble methods, and examining sensor reduction’s impact. In the first comparison, the logistic regression achieved the highest accuracy of 65.99% in the leave-one-person-out cross-validation. The second comparison revealed an accuracy of 59.39% for the parallel activity recognition method and 57.24% for the location estimation activity recognition method. In the third comparison, logistic regression achieved a recognition accuracy of 65.26% when four sensor nodes were used: two sensors were placed on both sides of a participant, another was diagonally behind the participant, and the other was installed on the ceiling.

Keywords – Activity recognition, Doppler sensor, machine learning, smart home applications

Smatable: A Vibration-based Sensing Method for Making Ordinary Tables Touch-interfaces

Abstract – In recent years, the equipment that makes up smart homes is required not only to be functional, but also to be integrated with the design and aesthetics of the living space. Among them, interfaces that directly touch the human eye and hands are the key to maintaining design, but there were many issues in terms of integration with design and aesthetics of living spaces. In this paper, we propose an interface system that operates existing furniture by touching it as a new interface that integrates beautifully into the living space. The proposed system detects user operations with four small vibration sensors attached to hidden locations of existing furniture and uses deep learning to learn the vibrations when a person touches the furniture. Using this method, thick materials difficult to achieve with normal capacitive touch sensors can be utilized. In the experiment, a dining table was used as a representative piece of furniture, and the accuracy of detecting the direction in which three participants swiped in four directions on the table was verified. As a result of the experiment, the accuracy was confirmed by Leave-One-Person-Out-Cross-Validation using 3 sessions of swipe data for each individual for 3 participants, and the accuracy was 0.67. Furthermore, we verified the accuracy of a trained model created by adding only one session of evaluation target data to each learning dataset used in the Leave-One-Person-Out-Cross-Validation. As a result, the accuracy reached 0.90, achieving practical precision.

Keywords – Touch interface, Operation recognition, Vibration sensor, Deep learning

Movement Recognition via Channel-Activation-Wise sEMG Attention

Abstract – Context: Surface electromyography (sEMG) signals contain rich information recorded from muscle movements and therefore reflect the user’s intention. sEMG has seen dominant applications in rehabilitation, clinical diagnosis as well as human engineering, etc. However, current feature extraction methods for sEMG signals have been seriously limited by their stochasticity, transiency, and non-stationarity.Objective: Our objective is to combat the difficulties induced by the aforementioned downsides of sEMG and thereby extract representative features for various downstream movement recognition.
Method: We propose a novel 3-axis view of sEMG features composed of temporal, spatial, and channel-wise summary. We leverage the state-of-the-art architecture Transformer to enforce efficient parallel search and to get rid of limitations imposed by previous work in gesture classification. The transformer model is designed on top of an attention-based module, which allows for the extraction of global contextual relevance among channels and the use of this relevance for sEMG recognition.
Results: We compared the proposed method against existing methods on two Ninapro datasets consisting of data from both healthy people and amputees. Experimental results show the proposed method attains the state-of-the-art (SOTA) accuracy on both datasets. We further show that the proposed method enjoys strong generalization ability: a new SOTA is achieved by pretraining the model on a different dataset followed by fine-tuning it on the target dataset.

Keywords – sEMG, Movement recognition, Gestures classification, Transformer

ICMU 2021 Best Paper Award

★ Paper Information ★
Tomokazu Matsui, Shinya Misaki, Yuma Sato, Manato Fujimoto, Hirohiko Suwa, Keiichi Yausmoto, “Multi-Person Daily Activity Recognition with Non-Contact Sensors Based on Activity Co-Occurrence,” The 13th International Conference on Mobile Computing and Ubiquitous Networking (ICMU 2021), pp. 157-164, Tokyo, Japan, Nov. 2021.

An Adaptive Multi-Range-Sensing Method for 3D Localization of Passive RFID Tags

Abstract – The RFID tag system has received a lot of attention for ubiquitous computing. An RFID tag is attached to an object. With the unique ID of the RFID tag, a user identifies the object provided with the RFID tag and derives appropriate information about the object. One important application in the RFID technology is localizing RFID tags, which can be very useful in acquiring the position information concerning the RFID tags. It can be applied to navigation systems and positional detection systems for mobile robots. This paper proposes a new adaptive multi-range-sensing method for 3D localization of passive RFID tags by using a probabilistic approach. In this method, a mobile object (human, robot, etc.) with an RFID reader estimates the positions of RFID tags with multiple communication ranges dynamically. The effectiveness of the proposed method was demonstrated in experiments.

Keywords – RFID (Radio Frequency IDentification), localization, passive RFID tag, transmission power

Localization of Passive RFID Tags by Using Broad-Type Multi-Sensing-Range (B-MSR) Method

Abstract – The radio frequency identification (RFID) system has attracting attention as a new identification source that achieves a ubiquitous environment. Each RFID tag has a unique ID code, and is attached on an object whose information it contains. A user reads the unique ID code using RFID readers and obtains information about the object. One of the important applications of RFID technology is the indoor position estimation of RFID tags. It can be applied to navigation systems for people in complex buildings. In this paper, we propose an effective position estimation method named Broad-type Multi-Sensing-Range (B-MSR) method to improve the estimation error of the conventional methods using sensor model. A new reader antenna with two flexible antenna elements is introduced into B-MSR. The distance between two flexible antenna elements can be adjusted. Thus, two kinds of system parameters can be controlled, the distance between two antenna elements and the transmission power of the RFID reader. In this paper, four sensing ranges are settled by controlling the values of two parameters. The performance evaluation shows four characteristics of B-MSR. Firstly, it reduces the initial estimation error. Secondly, it reduces the moving distance. Thirdly, it reduces the number of different sensing points. Fourthly, it shortens the required estimation time.

Keywords – RFID reader, RFID tag, position estimation, sensor model, broad-type multi-sensing-range method

Swift Communication Range Recognition Method for Quick and Accurate Position Estimation of Passive RFID Tags

Abstract – Radio frequency identification (RFID) system has gained attention as a new identification source that achieves a ubiquitous environment. Each RFID tag has a unique ID and is attached to an object. A user reads the unique ID of an RFID tag by using RFID readers and obtains the information on the object. One of the important technologies that use the RFID systems is the position estimation of RFID tags. Position estimation means estimating the location of the object with the RFID tag. Acquiring the location information of the RFID tag can be very useful. If a user can know the position of the RFID tag, the position estimation can be applied to a navigation system for walkers. In this paper, we propose a new position estimation method named Swift Communication Range Recognition (S-CRR) as an extended improvement on previous CRR that shortens the estimation delay. In this method, the position of an RFID tag is estimated by selecting the communication area model that corresponds to its boundary angles. We evaluated its performance by experiments and simulations of the RFID system. As the results, we found that S-CRR can estimate the position of an RFID tag comparatively accurately and quickly.

Keywords – RFID system, RFID reader, RFID tag, swift communication range recognition (S-CRR), communication area model, boundary angle

Effective Barrier Coverage Constructions for Improving Border Security in Wireless Sensor Networks

Abstract – Recently, the border security systems attract attention as large-scale monitoring system in wireless sensor networks (WSNs). In the border security systems whose aim is the monitoring of illegal immigrants and the information management in long-period, it deploys a lot of sensor nodes that have the communication and sensing functions in the detection area. Hence, the border security systems are necessary to reduce the power consumption of the whole system in order to extend the system lifetime and accurately monitor the track of illegal immigrants. In this paper, we propose two effective barrier coverage construction methods by switch dynamically operation modes of sensor nodes to reduce the operating time of the sensing function that wastes a lot of power consumption. We carry out performance evaluations by computer simulations to show the effectiveness of two proposed methods and show that the proposed methods are suitable for the border security systems.

Keywords – wireless sensor networks, barrier coverage, border security, ZigBee

Array Antennas Comprised of Tetragons for RFID Applications

Abstract – The location estimation capability of a RFID (Radio Frequency Identification) technology is useful for indoor navigation systems on mobile robots. In this paper, we study a simple and small structural antenna at 2.5GHz band that can reduce the beam when installed on a mobile robot. An antenna with small occupation volume by a mechanical rotation is assumed. We used electromagnetic field simulation and experimental results to design a simple structural antenna. Our design culminates in two antennas comprised of Tetragon elements formed with wires. One is called Tetra-4, and is an antenna which has a lateral arrangement of four tetragons. Because the input impedance is considerably high, Tetra-4 is suitable for use in the high impedance system. When driving in 100 Ω balanced system, the gain of Tetra-4 is 8dBi. The half-power band width (HPBW) in the H-plane is 38 degrees, and is 0.44 times that of a dipole with a reflector. Another one is Yagi-Uda antenna comprised of two tetragon elements. When driving in 50 Ω balanced system, the gain is 11dBi. The HPBW is 40 degrees in the H-plane and is 36 degrees in the E-plane.

Keywords – array antenna, tetragon, loop antenna, RFID, location estimation, mobile robot

Array Antennas Comprised of Hexagons for RFID Applications

Abstract – The location estimation capability of radio frequency identification tag is useful for indoor navigation systems on mobile robots. In this paper, we study a simple and small structural antenna at 2.5 GHz band that can reduce the beam when installed on a mobile robot. The antenna with small occupation volume by a mechanical rotation is assumed. We used the electromagnetic field simulation results and the experimental results to design a simple structural antenna. Our design culminates in two antennas comprised of hexagon elements formed with wires. One is called Hexa-4 and is an antenna which has a lateral arrangement of four hexagons. The gain is 8 dBi, with a half-power beam width (HPBW) in the H-plane of 37°. The beam width is 0.43 times that of a half wavelength dipole with a reflector. Another one is Yagi–Uda antenna comprised of two hexagon elements. The gain is 11 dBi, with HPBW in the H-plane of 40° and in the E-plane of 39°. These antenna characteristics were measured in 100 Ω equilibrium system by installing a balun in the measurement system of 50 Ω. The feed networks such as phase shifter are not used except the balun. Understanding the characteristic of the Hexa-n antenna when n is six or more might be worthy.

Keywords – Array antenna, hexagon, loop antenna, RFID, location estimation, mobile robot

A Living Activity Recognition System Based on Power Consumption of Appliances and Energy Harvesting PIR and Door Sensors

Abstract – Progress of ubiquitous computing technology has strong anticipation to realize efficient energy-saving appliance control and elderly monitoring. In order to put those applications into practice, high-accuracy and low-cost in-home living activity recognition is essential. Many researches have tackled living activity recognition so far, but the followings problems remain: (1) privacy exposure due to utilization of cameras and microphones; (2) high deployment and maintenance costs due to many sensors used; (3) burden to force the user to carry the device and (4) wire installation to supply power and communication between sensor node and server; (5) few recognizable activities; (6) low recognition accuracy. In this paper, we propose an in-home living activity recognition method to solve all the problems. To solove the problems (1)-(4), our method only utilizes energy harvesting PIR and door sensors and power meter. The energy harvesting sensor has a solar cell to drive the sensor and wireless communication modules. To solve the problems (5) and (6), we have tackled the following challenges: (i) determining appropriate features for training samples; and (ii) determining the best machine learning algorithm to achieve high recognition accuracy; (iii) complement for the dead zone of PIR sensor semipermanently. We have conducted evaluations with the sensor by five subjects living in a home for 2-3 days each. As a result, the proposed method has achieved F-measure: 68.6% on average.

Keywords – living activities recognition, low privacy-intrusion, PIR sensor, power meter, machine learning

Generating pedestrian maps of disaster areas through ad-hoc deployment of computing resources across a DTN

Abstract – Generating pedestrian maps of disaster areas is an important part of response operations. Maps aid responders in decision-making and show routes that lead evacuees to refuges. However, disasters can damage communication infrastructures, rendering Cloud-based mapping services inaccessible. Responders resort to paper maps, which are difficult to share and cannot recommend routes. In this study, we present a digital pedestrian map generation system for disasters. To realize the system, we addressed these challenges: (1) how to collect the required data and generate the map without Cloud-based computing resources, (2) how to share messages within the system without continuous, end-to-end networks, and (3) how to balance the load of map inference tasks. For (1), GPS traces are collected by responders exploring the area. Then, collected data are sent to Computing Nodes: commodity workstations that are deployed in the disaster area, for processing. For (2), the system establishes a Delay-Tolerant Network that uses Epidemic Routing to communicate across shorter-ranges and uses response vehicles as data ferries to communicate across longer-ranges. For (3), we propose a load balancing heuristic, which uses ferry route timetables and statistical information about the load of Computing Nodes to determine how to offload map inference tasks. We evaluate our system through experiments and simulations and show that it decreases the time needed to generate and deliver pieces of the map by approximately two hours in an extreme case with large quantities of data have to be processed.

Keywords – disaster area mapping, load balancing, delay tolerant networks, data ferries

Implementation and Evaluation of a Participatory Location Sharing System for Cycling Road Race with Reducing Burden for the Athletes

Abstract – Athlete’s location sharing service has already started in various cycling events. However, in such system, the athletes have to bring a smartphone for uploading their position in racing. In this paper, we propose a participatory location sharing system that utilizes supporters’ smartphone for tracking the position of athletes. In our system, the athletes only have a small BLE beacon, and the supporters along the road have a smartphone in which our application is installed. The application uploads and shares its location when it receives a beacon signal. Since there is no existing work that try to track a fast-passing beacon (the speed of bicycle is 30km/h-80km/h), we developed the dedicated beacon and dedicated Android application for experiment. Through various experiments, we clarified the effect of passing speed, the maximum distance for receiving a signal, location accuracy compared with GPS, the effect of landform and shielding. Finally, we conducted an experimental evaluation in the actual cycling events for confirming the availability and effectiveness.

Keywords – bicycle, load cycling race, BLE beacon, smartphone, participatory sensing, location information sharing

Implementation and Evaluation of Analog-PIR-sensor-based Activity Recognition

Abstract – These days, smart home applications such as concierge service for residents, home appliance control, and so on are attracting attention. To realize these applications, we need a system which recognizes various human activities accurately with a low cost device. There are many studies on the activity recognition in a smart home. We also have proposed an activity recognition technique in a smart home by utilizing digital-output-PIR (passive infrared) sensors, door sensors, and power meters. However, the study has an unsolved issue: we cannot distinguish similar activities happening at the same place, for example, “eating” and “reading” while sitting on a sofa. In order to cope with this challenge, we introduce ALPAS: analog-output-PIR-sensor-based activity recognition technique which recognizes the different activities in the same place. Our technique recognizes user's activity by utilizing machine learning with frequency components of the sensor's output as features. However, because the number of features used in ALPAS is 1000 for each analog PIR-sensor, a large capacity memory is required. To reduce the number of features, we select a part of the sensing data. We call the starting point of the selected data as starting frequency (SF) and ending part as ending frequency (EF). We searched SF and EF using a grid search, and evaluated the recognition accuracy. We evaluated the proposed technique in a smarthome testbed. In the evaluation, five participants performed four different activities while sitting on a sofa. As a result, we achieved F-Measure: 63.9% when the EF is 1.4Hz, and F-Measure: 50% or lower when the SF is 9.9Hz or higher.

Keywords – activity recognition, analog PIR sensor, machine learning, feature reduction, grid search

Walking Activity Detection Through a Cane Attached Sensor

Abstract – Walking ability is one of the most important skills for human beings to live healthy in old age. Various rehabilitation programs are provided by care-workers to keep or improve walking ability. However, in a super-aging society, it is likely to be difficult to provide adequate rehabilitation program individually because of a shortage of care walkers. In this paper, we propose the system that can measure and evaluate the walking ability of a senior person who is using a cane and doing rehabilitation. We developed a prototype sensor that can attach to various types of a cane because most of the seniors do not want to wear any sensors on the body. The challenging point of this system is the algorithm to measure a person’s walking ability through the movement of the cane. We collected various walking data from actual elders and developed a method to detect steps from the variation of the accelerometer sensor values. Also, we propose a method to estimate a walking distance by using the multiple regression model. Through the evaluation of 16 participants including elderlies, a hemiplegic patient, and non-handicapped persons, we confirm that our system can detect a walking activity with 95.56% for non-handicapped persons and measure the walking distance with 88.06% accuracy for all participants.

Keywords – walking activity detection, walking distance estimation, cane attached sensor

Beacon-Based Time-Spatial Recognition toward Automatic Daily Care Reporting for Nursing Homes

Abstract – As the world’s population of senior citizens continues to grow, the burden on the professionals who care for them (carers) is also increasing. In nursing homes, carers often write daily reports to improve the resident’s quality of life. However, since each carer needs to simultaneously care for multiple residents, they have difficulty thoroughly recording the activities of residents. In this paper, we address this problem by proposing an automatic daily report generation system that monitors the activities of nursing home residents. The proposed system estimates the multiple locations (areas) at which residents are situated with a BLE beacon, using a variety of methods to analyze the RSSI values of BLE signals, and recognizes the activity of each resident from the estimated area information. The information of the estimated activity of residents is stored in a server with timestamps, and the server automatically generates daily reports based on them. To show the effectiveness of the proposed system, we conducted an experiment for five days with four participants in cooperation with an actual nursing home. We determined the proposed system’s effectiveness with the following four evaluations: (1) comparison of performance of different machine-learning algorithms, (2) comparison of smoothing methods, (3) comparison of time windows, and (4) evaluation of generated daily reports. Our evaluations show the most effective combination pattern among 156 patterns to accurately generate daily reports. We conclude that the proposed system has high effectiveness, high usability, and high flexibility.

Evacuation Center Determination Method Considering Bias of Congestion Degree on Areas

Abstract – The quick and appropriate evacuation of the disaster victims is one of great concern in the disaster situation. Under the disaster situations, it is general that the victims evacuate from the present location to the nearest evacuation center depending on evacuation guidance. However, in case of the situation that many victims are at biased distribution to specific areas, the guidance to the closest evacuation center concentrates many victims to particular evacuation. As a result, the victims who could not enter the closest evacuation center need to evacuate to other evacuation centers, until accepted. Therefore, the problem occurs that the evacuation of victims is delayed significantly. In this paper, to solve this problem, we propose an evacuation center determination method aimed at reducing the evacuation time of the whole victims. Specifically, the proposed method determines an evacuation center so that the evacuation time of all victims becomes short based on the location information of victims and the capacity of evacuation centers. To evaluate the effectiveness of the proposed method, we compared our proposed method with the shortest route selection method and random selection method, by using the simulation study assuming the situation of the Gion festival in Kyoto. As a result, when using the proposed method, we showed that the average evacuation time can be reduced to 35.9% compared with the shortest route selection method. Also, we confirmed that the average evacuation time can be reduced to 8.1% even if victims who follow the proposed method is 20% of the total. From these results, even if victims who follow the proposed method is a part of the total, we showed that our method can reduce the overall evacuation time, and contribute to disaster prevention.

Keywords – decision evacuation location, evacuation induction, simulation, congestion

Investigating effects of interactive signage–based stimulation for promoting behavior change

Abstract – In recent years, many types of research and developments on behavior change have been conducted. The purpose of behavior change is to improve people’s lifestyle pattern or to maintain the improvement for a long time with the aim to achieve a goal such as promoting health condition improvement. To achieve the foundation of a new lifestyle, it is necessary to recognize the daily life patterns of users and give triggers for behavior change to users in their daily life. To realize this, in our research, we develop an interactive signage which is able to identify and actively talk to the passing user and try to induce behavior change by sending visual and auditory stimulation. Then, we record users’ reactions and upload them to the server. In this paper, we report the investigation result on users’ reactions and feelings to the developed interactive signage. As a survey experiment, we set up four interactive signs on a floor of our university and asked 15 participants to carry a name tag with a Bluetooth Low Energy beacon during their daily life. Five kinds of tasks based on dialogue scenarios are posted to the approaching participants. Participants can respond to these tasks through a touchscreen. The period of the experiment was three weeks. To get the data in an ideal environment, during the first week, we asked all the participants to respond to the utterance from the interactive signage whenever they hear the voice message. During the next two weeks, participants were not asked to respond to the task definitely to get the data in the real environment. The result of the experiment showed that our proposed interactive signage could induce behavior change effectively. Based on the result of experiment, we updated our interactive signage system by adding response time (the time passed from showing contents until user respond), record function, and voice feedback function. Furthermore, to collect the data of response time that is considered as a part of users’ reactions, we conducted an additional experiment with the same participants in previous experiment (except for one missing participant) for one week after updating the system. In the additional experiment, the participants were not asked to respond to the utterance definitely. As a result, it is shown that the behavior change by the proposed signage is still effectively induced. We also analyzed the relationship between the day passed and the response rate of each task type. The result shows that the number of ignorance of personal task and check task does not rise even as the time passes.

Keywords –

Effective Application Distribution System for Internet-less Communication during Disasters

Abstract – Smartphone-based disaster communication systems have been the topic of many recent studies. However, existing systems unrealistically assume that prior to a disaster, the required applications have been installed in the smartphones of all users. In disaster areas without Internet access, obtaining the software required for communications systems is difficult because application distribution platforms, such as Google Play, are web-based. In this paper, we present RecurShare, which is a novel application distribution system. The key idea of RecurShare is to be able to distribute applications without the Internet by running a web server that can be accessed using the default web browser and the tethering function of Android smartphones. We evaluated the feasibility of RecurShare experimentally and showed that RecurShare functions without problems on a variety of commercial devices. Also, user feedback suggests that RecurShare sufficiently guides users through the application distribution process in a manner that is easy to understand. Furthermore, we found that RecurShare can distribute itself at an exponential rate and that it can transmit files at speeds that are comparable to those of commonly used wireless communication systems.

Keywords – Internet-less file sharing, recursive application distribution, smartphone, disaster situation

Effective Trilateration-based Indoor Localization Method Utilizing Active Control of Lighting Devices

Abstract – In this paper, we propose a novel trilateration-based indoor localization method utilizing active control of lighting devices. The proposed method estimates the position of a target for measurement by utilizing a trilateration method based on the distances between the target and lighting devices calculated from the illuminance value obtained by active control. The active control involves turning on lighting devices newly installed in the target area one by one. Our method has three key ideas. The first is to remove the effect of light sources other than the lighting equipment used for localization by utilizing only the difference between the illuminance values obtained before and after turning on each lighting device. The second is to remove the influence of a human shadow by utilizing three lighting devices used for localization selected from four lighting devices newly installed in the target area. The third is to accurately estimate the position of a target by introducing a distance-illuminance model that can calculate the distance between an illuminance sensor and a lighting device accurately. To show the effectiveness of the proposed method, we conducted evaluation experiments. It was found that the proposed method can estimate the position of a target with approximately 1 m error on average.

Keywords – indoor localization, trilateration method, active control, distance-illuminance model, lighting device

Strike Activity Detection and Recognition Using Inertial Measurement Unit towards Kendo Skill Improvement Support System

Abstract – In the field of sports, there are increasing opportunities to use inertial measurement units (IMUs) to enhance the training process and improve the performance of athletes. We focus on kendo, a traditional martial art using shinai (bamboo swords) in Japan, and propose methods for detecting and recognizing strike activities using IMUs towards realizing a kendo skill improvement support system. We used a sensor data set of strike activities obtained from 14 participants (seven kendo-experienced and seven inexperienced persons). We attached four IMUs to the participants’ right wrist, waist, and shinai (tsuba and saki-gawa). First, to detect the strike activity, we calculated the dynamic time warping (DTW) distance between the training data and the time series data, and detected the strike activity sections. The proposed method detected strike activities with a high accuracy of 95.0%. Next, to recognize the strike activity, we recognized five types (Center-Men, Right-Men, Left-Men, Dō, and Kote). In the person-dependent (PD) case, we achieved an accuracy of 89.5% using data of the right wrist. In the person-independent (PI) case, we achieved an accuracy of 54.9% using IMUs attached to the three positions. These results clarified the points to be improved in the proposed method to realize the support system.

Keywords – activity recognition, wearable computing, inertial measurement unit, sports support, kendo

On-site Trip Planning Support System Based on Dynamic Information on Tourism Spots

Abstract – Recently, due to the drastic increase in foreign tourists coming to Japan, there has been a demand to provide smart tourism services that enable inbound tourists to comfortably enjoy sightseeing. To provide satisfactory experiences for tourists, it is desirable to provide tourist information in a timely manner by considering dynamic information, which is information that changes over time, such as current congestion information in destination spots and travel route information, in addition to static information, such as the preferences and profiles of tourists. However, in many existing systems, serious problems occur, such as (1) a lack of support for on-site use, (2) a lack of consideration of dynamic information, and (3) heavy burden on tourists. In this paper, we propose a novel system that can provide tourism plans for tourism spots in a timely manner. The proposed system consists of the following two key mechanisms: (A) A mechanism for acquiring preference information from tourists (including preference on dynamic information); (B) a curation mechanism for realizing on-site tourism. To demonstrate the effectiveness of the proposed system, we carried out evaluation experiments utilizing real tourism spots and simulations. As a result, we obtained the following primary findings: (1) On-site tourism spot recommendation is effective for tourists who do not make detailed tourism plans before sightseeing; (2) preference information for participants can be reflected in the tourism spot recommendation while massively reducing the burden on participants; (3) it is possible to obtain a higher satisfaction level than is achieved with model courses, which are often used for sightseeing.

Keywords – on-site tourism recommendation, dynamic information, acquisition of preference information, trip planning

Gamified Participatory Sensing in Tourism: An Experimental Study of the Effects on Tourist Behavior and Satisfaction

Abstract – In the tourism sector, user-generated information and communication among tourists are perceived to be more effective and reliable contents. In addition, the collection of dynamic tourism information with high spatio-temporal resolution is required to provide comfortable tourism in response to the changing tourism style with the advancement of information technology. Participatory sensing, which can collect various types of information is a useful method by which to collect these contents. However, continuous participation of users is essential in participatory sensing, and it is one of the most important points to stimulate participation motivation. In the tourism situation, we also need to pay attention to the total tourist satisfaction of participants. In this paper, we adopt gamification, i.e., the implementation of game design elements in real-world contexts for non-gaming purposes, for participatory sensing as an incentive mechanism to motivate participants with active participation and collect the necessary information efficiently. Within the framework, where points are given when completing the requested sensing task (=mission), two sensing missions with different burdens; Area Mission and Check-in Mission, and three different types of rewarding mechanisms; Fixed, Variable and Dynamic Variable, are designed as a gamification mechanism. We implemented these elements in the proposed participatory sensing platform application and conducted an experimental case study with 33 participants at an actual tourist spot: Kyoto, Japan. Then, we investigate the effects on tourist behavior and satisfaction by analyzing collected sensor data, mission logs, and post-survey answers. As a result, we can conclude the following: (1) the tourist behavior is changed due to the proposed gamification design and the necessary information was collected efficiently; (2) the participants tend to prioritize Check-in Mission over the sightseeing, which can induce a behavior change but might impact sightseeing enjoyment.

Keywords – participatory sensing, gamification, tourism, incentive mechanism, behavior change

SALON: Simplified Sensing System for Activity of Daily Living in Ordinary Home

Abstract – As aging populations continue to grow, primarily in developed countries, there are increasing demands for the system that monitors the activities of elderly people while continuing to allow them to pursue their individual, healthy, and independent lifestyles. Therefore, it is required to develop the activity of daily living (ADL) sensing systems that are based on high-performance sensors and information technologies. However, most of the systems that have been proposed to date have only been investigated and/or evaluated in experimental environments. When considering the spread of such systems to typical homes inhabited by elderly people, it is clear that such sensing systems will need to meet the following five requirements: (1) be inexpensive; (2) provide robustness; (3) protect privacy; (4) be maintenance-free; and, (5) work with a simple user interface. In this paper, we propose a novel senior-friendly ADL sensing system that can fulfill these requirements. More specifically, we achieve an easy collection of ADL data from elderly people while using a proposed system that consists of a small number of inexpensive energy harvesting sensors and simple annotation buttons, without the need for privacy-invasive cameras or microphones. In order to evaluate the practicality of our proposed system, we installed it in ten typical homes with elderly residents and collected the ADL data over a two-month period. We then visualized the collected data and performed activity recognition using a long short-term memory (LSTM) model. From the collected results, we confirmed that our proposed system, which is inexpensive and non-invasive, can correctly collect resident ADL data and could recognize activities from the collected data with a high recall rate of 72.3% on average. This result shows a high potential of our proposed system for application to services for elderly people.

Keywords – energy harvesting sensor, daily activity recognition, machine learning, simple installation sensing system

Evacuation Shelter Decision Method Considering Non-Cooperative Evacuee Behavior to Support the Disaster Weak

Abstract – In disaster situations, special support for the disaster weak are crucial to keep them safe. Common evacuation strategies guide individuals to the shelters closest to their present locations. If evacuees are unevenly distributed across areas, some shelters will not be able to accommodate all arriving evacuees due to the limited capacity of shelters. To tackle this, the existing method decides the destinations for each evacuee considering congestion in disaster areas. However, this method does not consider the disaster weak and can be burdensome for them. Giving that the priority to the disaster weak for shelter decision would be effective to lessen burdens for them, but not all evacuees follow the guidance. When a shelter accepts arriving evacuees unconditionally, some evacuees are rejected, causing a delay in evacuation. If the disaster weak are rejected, the delay will be increased. In this paper, we propose two evacuation shelter decision methods considering the capacity of shelters, the disaster weak, and evacuees’ selfish behavior to realize quick evacuation for the disaster weak: (1) Fixed-rate Reduction Method (FRM), which reduces the assignment number of evacuees less than the capacity at the same percentage to all shelters. (2) Simulation-based Reduction Method (SRM), which reduces the assignment number to shelters that will be crowded based on simulation of an evacuation scenario. Then, these methods decide the destinations for evacuees, with the priority given to the disaster weak. To evaluate the efficiency of the proposed methods, we conducted multi-agent simulation assuming the scenario of evacuation of 30,000 visitors for the Gion Festival including the disaster weak. Through the simulation, we compared our methods with conventional methods including the nearest shelter selection method and the exiting method. As a result, our methods can reduce evacuation time of the disaster weak compared to conventional methods with sufficient cooperation by evacuees.

Keywords – evacuation strategies, disaster weak, shelter decision method, combinatorial optimization problem, multi-agent simulation

Exploring the Impacts of Elaborateness and Indirectness in a Behavior Change Support System

Abstract – Numerous technologies exist for promoting a healthier lifestyle. These technologies collectively referred to as “Behavior Change Support Systems”. However, the majority of existing apps use quantitative data representation. Since it is difficult to understand the meaning behind quantitative data, this approach has been suggested to lower users’ motivation and fail to promote behavior change. Therefore, an interpretation of quantitative data needs to be provided as a supplement. However, different descriptions of the same data may lead to different outcomes. In this paper, we explore the impact of different communication styles for interpretations of quantitative data on behavior change by developing and evaluating Walkeeper – a web-based app that provides interpretations of the users’ daily step counts using different levels of elaborateness and indirectness with the aim of promoting walking. Through the quantitative analysis and results of a user study, we contribute new knowledge on designing such interpretations for quantitative data.

Keywords – behavior change, communication styles, physical activity

IPSJ/ACM Award for Early Career Contributions to Global Research

Recommendation – Dr. Manato Fujimoto received B.E., M.E., and Ph.D. degrees from Kansai University, in 2009, 2011, and 2015, respectively. From April 2015 to Sept. 2021, he was an assistant professor at the Graduate School of Science and Technology, NAIST. From Oct. 2021 to Mar. 2022, he was an associate professor at the Graduate School of Engineering, Osaka City University. He is currently an associate professor at the Graduate School of Informatics, Osaka Metropolitan University. Dr. Fujimoto has consistently worked on research related to “Ubiquitous Computing Systems” for the realization of Society 5.0. He is targeting various fields toward making our world “Hyper-Smart” with a Cyber-Physical System that highly links the real world and cyberspace. Recently, he has achieved achievements both nationally and internationally, including acceptance to Q1 journals and/or top conferences. He has also been active in international collaborative research, especially with the Karlsruhe Institute of Technology (Germany), where he has achieved noticeable achievements, including acceptance to ACM ISWC’18, one of the most difficult international conferences. Furthermore, he has been selected to serve on the organizing committee of various conferences both in Japan and abroad, contributing greatly to the spread of the academic fields covered by IPSJ and ACM.

Wi-CaL: WiFi Sensing and Machine Learning based Device-Free Crowd Counting and Localizat

Abstract – Wireless sensing represented by WiFi channel state information (CSI) is now enabling various fields of applications such as person identification, human activity recognition, occupancy detection, localization, and crowd estimation these days. So far, those fields are mostly considered as separate topics in WiFi CSI-based methods, on the contrary, some camera and vision-based crowd estimation systems intuitively estimate both crowd size and location at the same time. Our work is inspired by the idea that WiFi CSI also may be able to perform the same as the camera does. In this paper, we construct Wi-CaL, a simultaneous crowd counting and localization system by using ESP32 modules for WiFi links. We extract several features that contribute to dynamic state (moving crowd) and static state (location of the crowd) from the CSI bundles, then assess our system by both conventional machine learning (ML) and deep learning (DL). As a result of ML-based evaluation, we achieved 0.35 median absolute error (MAE) of counting and 91.4% of localization accuracy with five people in a small-sized room, and 0.41 MAE of counting and 98.1% of localization accuracy with 10 people in a medium-sized room, by leave-one-session-out cross-validation. We compared our result with percentage of non-zero elements metric (PEM), which is a state-of-the-art metric for crowd counting, and confirmed that our system shows higher performance (0.41 MAE, 81.8% of within-1-person error) than PEM (0.62 MAE, 66.5% of within-1-person error).

Keywords – crowd counting, crowd localization, CSI, machine learning, WiFi sensing

A Decentralized COVID-19 Vaccine Tracking System using Blockchain Technology

Abstract – Coronavirus disease 2019 (COVID-19) vaccines play a crucial role in preventing the spread of the disease. However, the circulation of low-quality and counterfeit vaccines seriously affects human health and the reputation of real vaccine manufacturers (VMs) and increases the amount of fear concerning vaccination. In this study, we address this problem by developing a blockchain-based COVID-19 vaccine tracking system called “Vacchain”. Our Vacchain allows users (USERs) to track and trace the route of vaccines. We propose three mechanisms, namely, a system manager (SYS-MAN), a mutual agreement concerning vaccine ownership, and vaccine passports, to enhance the security and reliability of data recorded in the Vacchain ledger. We develop this system on the Substrate platform with the Rust language. Our implementation, evaluation, and analysis have shown that Vacchain can trace and track vaccines smoothly. In addition, data security and reliability are enhanced by the abovementioned three mechanisms. The proposed system is expected to contribute to preventing the spread of COVID-19.

Keywords – COVID-19 vaccine tracking, blockchain for healthcare, decentralized vaccine management, Substrate platform, cryptography

Inertial measurement unit-sensor-based Short Stick Exercise Tracking to Improve Health of Elderly People

Abstract – Short stick exercises have been attracting attention from the viewpoint of preventing falls and improving the health of elderly people and are generally performed under the guidance of instructors and nursing staff at nursing homes. However, in situations such as the COVID-19 pandemic, where people should refrain from unnecessary outings, it is advisable that individuals perform short stick exercises at home and record their exercise implementation status. In this paper, we propose an inertial measurement unit (IMU)-sensor-based short stick exercise tracking method that can automatically record the types and amounts of exercises performed using a short stick equipped with an IMU sensor. The proposed method extracts time-domain and frequency-domain features from linear acceleration and quaternion time-series data obtained from the IMU sensor and classifies the type of exercise using an inference model based on machine learning algorithms. To evaluate the proposed method, we collected sensor data from 21 young subjects (in their 20s) and 14 elderly subjects (79–95 years old), where the participants performed three sets (10 times per set) of eight basic types of short stick exercises (five types for elderly people). As a result of evaluating the proposed method using this data set, we confirmed that when LightGBM was used as the learning algorithm, it achieved F values of 90.0% and 86.6% for recognizing the type of exercise for young and elderly people, respectively.

Keywords – short stick exercise, machine learning, IMU

Analysis and Visualization of Relationship between Stress and Care Activities toward Reduction of Caregiver Workload

Abstract – In Japan, the demand for nursing homes is increasing owing to the rapid aging of the population, and the shortage of caregivers has become a serious problem. This problem has been recognized as a social issue because it increases the workload per caregiver. In response, we have been developing a platform that can easily collect data on nursing care activities to reduce the workload on the basis of the idea that the mental state (i.e., stress) of caregivers, which changes during care activities, might markedly affect their work efficiency. The objective of this study was to obtain new knowledge for reducing the workload of caregivers by visualizing and analyzing their stress. Specifically, we asked caregivers to wear the device, from which we obtained the objective stress indicators of R-R interval (RRI) and low frequency (LF)/high frequency (HF) ratio. We also obtained subjective stress indicators from questionnaires administered before the start of the experiment, before work, during breaks, and after work. To confirm that the stress level of caregivers depends on their care activity, work shift, and workday, we conducted an empirical experiment in an actual nursing home. We distributed devices and applications to five caregivers, obtained measurement data for a total of 28 days, and analyzed and visualized the changes in psychological states related to care activities, work shift, and workdays. As a result, we were able to collect data on the psychological state for 5 to 15 days per caregiver. Furthermore, the analysis of the data showed that, even though objective stress indicators and subjective evaluations did not necessarily coincide, stress tended to increase with specific care activities, work shift, and workdays. In addition, the subjective and objective stress indicators may change depending on the personality of the caregivers themselves.

Keywords – caregiver, stress level, objective stress, subjective stress

Large-Scale Evacuation Shelter Selection Method Through Iterations of Pedestrian Simulations With Dynamic Congestion Reproduction

Abstract – It is necessary to optimize evacuation guidance to shelters in short evacuation time. The state-of-the-art method based on an idea of combinatorial optimization problems related to evacuees’ locations and the capacities of nearby shelters has been developed, while it cannot mitigate the effect of congestion on roads/streets after evacuation starts. In this study, to cover this problem, we develop a new method that utilizes simulations for estimating the effect of congestion on roads/streets during evacuation and reassigning shelters to evacuees based on the simulation results. By iterating this step, our method derives the congestion-aware solutions for shelter selection that can realize more smooth evacuation. To evaluate our method, we conducted multi-agent simulations assuming a disaster situation in a sightseeing spot. Specifically, we examined a hypothetical case scenario involving the evacuation of 30,000 visitors from the Gion Festival. We compared the proposed method with conventional methods, such as the nearest shelter selection method and our previous method. We found that our proposed method reduced average and total evacuation time and congestion on roads compared to the conventional methods including the nearest shelter selection method and our previous method that only employs combinatorial optimization without estimating congestion. From this result, our idea of simulation-based congestion estimation has an impact of easing congestion during evacuation and preventing overcapacity of shelters at the same time. It shows the possibilities of help in developing congestion-aware evacuation strategies in emergency situations of crowded areas like huge cities or sightseeing spots.

Keywords – combinatorial optimization problem, congestion-aware, disaster management, evacuation
strategies, multi-agent simulation, shelter selection

Manato Fujimoto

Manato Fujimoto

Professional History

Associate Professor

Affiliate Associate Professor

Associate Professor

Visiting Scientist

Assistant Professor

Assistant Professor

Research Fellowship for Young Scientists (DC2)

Education History

Ph.D. in Engineering

Master of Engineering

Bachelor of Engineering