SMART POWER WHEELCHAIR: PROBLEMS AND CHALLENGES OF PRODUCT APPROACH

. This paper focuses on intelligent assistant for power wheelchair (PW) usage in home conditions. Especially in the context of PW intelligent assistant as a consumer product. The main problematic aspects and challenges of smart PW in real application are noted. The approach to formation of system requirements and their classification is offered. The research results proposed and implemented in the ongoing Mobilis project for smart PW. Further prospects of research and development are noted. Also, it is stated that the implementation of smart PW technology opens possibilities to effective integration with new control methods (including brain-computer interfaces).


Introduction
Low birthrate and long span of life tend to rapidly advance, especially in the economically developed countries [12,23]. In such society the demand for the specialized care and mobility products for the disabled persons, based on new technologies, is increasing, not only to support their physical abilities but also to decrease the care complexity. Power wheelchairs (PW), as classic assistive devices, are widely used by elderly persons and disabled people as they require minimal physical efforts for motion [12].
According to the statistics of the World Health Organization (WHO) 15% of the world population suffer from a disability and from 2% to 4% experience considerable problems in practice. Global assessments of the disability increase with the ageing of the world population and improvement of the assessment process and disability measurement [9,14].
Mobility is one of the key components of supporting high quality of life. Elderly people and people with disabilities which hinder walking or using manual wheelchairs are often offered power wheelchairs (PW) to help them move independently in their environment. Decision making, regarding the application of PW is based on certain factors, including the safety of the driver and other persons in the environment, forecast benefits for the driver, possibility of PW access and available funding [4,21].
Assessment of the possibility of the application and PW selection comprises complex monitoring of the numerous factors, influencing the readiness of a separate person to use PW and the selection of the corresponding PW, taking into account the needs and available situation. In the process of decision making and PW selection it is necessary to perform the efficient training of PW driving to make the future user a safe, efficient and polite driver [4,21,22].

Classic power wheelchair driving challenges
for a great number of people, who require long-term care. The examples of the diagnosis which may influence the ability of the person to control safely PW are dementia, cerebral palsy with cognitive disorders, amyotrophic lateral sclerosis, severe traumatic Brain Injury, multiple sclerosis and Parkinson disease, syringomyelia, myasthenia, consequences of cerebral and spinal strokes [4,10,17,21].
In general the PW driving process may be schematically shown as a cyclic sequence of tasks: Perception  Planning  Steering. To provide mobility for more people, the solution has to overcome limitations on each step of the sequence.

Perception limitation
Head movement may be limited due impairments, so users cannot look around to observe all surrounding space. The same applies to visually impaired people.
The importance of this position is justified by the fact that a significant proportion of PW users (especially elderly people) may have problems with vision, hearing, vestibular analyzer which significantly complicates their orientation in space. In particular, coordination disorders due to pathology of the brain and pathology of the vestibular analyzer; diplopia, a significant decrease in visual acuity; neurosensory deafness, etc. This may in various combinations be a manifestation of the above pathology or a consequence of comorbidity [20].

Planning limitation
Some people with cognitive deficits cannot operate wheelchair, as they may have difficulties with prediction and/or planning wheelchair motion.
Safe operation of PW requires a sufficient level of the cognitive functions, ability, including decision making, memory, judgements, self-consciousness. Certain studies show that 60-80% of the patients of Long-Term Care (LTC) have dementia. Diminished attention, poor operation control, loss of memory problems are the known features, connected with different forms of dementia, including Alzheimer disease, which make independent navigation of PW difficult or impossible. Determining the acceptability of PW using, the doctors face the difficult decision, connected with the needs of their patients in the independent mobility, safety of the driver and others in the environment [22]. IAPGOŚ 3/2021 p-ISSN 2083-0157, e-ISSN 2391-6761

Steering limitation
For user with severe motor impairments (like amyotrophic lateral sclerosis or spinal cord injury, cerebral palsy, etc), traditional control interfaces, such as a 2-axis joystick, are not suitable, as they require precise movement control over the limbs, which in their case is not possible.
Usage of a joystick requires prior training for obtaining sufficient operation skills, especially in case of motion in narrow places or in crowded places, in places with high concentration of different objects. That is why, emergency situation caused by the operation errors occur among elderly people not only as a result of sensitivity decrease due to ageing but also as a result of the difficult control of the system according to the report, issued by the Tokyo Metropolitan Police Agency (Japan), 200 cases of dents are connected wild the automobiles and power wheelchairs, happen each year. 25% of these cases belong to the operating errors [12].

Smart power wheelchair as a solution
Taking into account the above-mentioned problems of driving PW the development and implementation of Smart Power Wheelchair (SPW) is required, as a solution. SPW should be able to avoid collisions and support navigation, simplifying control tasks for the user.
Existing scientific research also stated that for the safe motion of PW it is better to use the autopilot-like smart drive assist system, overcoming the limitation of the control during the usage of the joystick, voice commands, etc. [12].
The development of SPW is an important scientific-applied problem, involving large groups of population and has significant social value.
The analysis of the literature sources related to SPW, reveal significant activity during the last 20 years. Many SPW research projects resulted in identification of certain problems, connected with the convenience of usage and spheres for improvement [5,8,7,11,12,14,15,16,21,22].
Highly appreciating the tremendous efforts, made by the numerous researchers in the field of the development of the smart PW, authors state that the development of the intelligent assistant of PW, in particular in the sphere of SPW usage in home conditions is still open problem, especially in the context of making SPW consumer mass product.

Requirement for smart power wheelchair
According to study [19], PW users most of the time (64%) drive indoors (mostly at home). Authors believe that home drive is a most important use case for SPW technology and effective operation of SPW in home conditions is a key to a successful mass product. Thus, specific requirements for home driving need additional research.
However, driving in such a crowded environment as a user's home implies a set of specific requirements that significantly differ from driving in other conditions (e.g. hospitals, airports, research labs etc.). Moreover, home driving in many aspects is more complex than driving in large spaces, so successful implementation of smart home drive assistance system will be helpful to implement other use cases.
When SPW is used in home, these conditions are characterized by the increase system requirements to the accuracy of the driving (decrease of the distance to the obstacle while movingalong the wall, passage across the narrow doorways, etc.), need of the dynamic monitoring of the environment and reconstruction/updating of the 3-D map of the premises (in the process of the object position change, motion of the domestic animals, change of the usual position of the objects after cleaning by the outsiders) peculiarities of the decision making regarding the possibility of the passage with minimal possibility of injuring the person-user due to inaccuracy/error of the intelligent system (passage with the hanging/suspended obstacle, that can injure person-user; rapid acceleration of SPW and injuring while emergency braking, problems with the correct parking of SPW if there is a need to pass to bed or from the bed, etc.), and problem of the absence of the person-assistant for rapid help in the emergency situation (impossibility of letting such person in), etc.

Planning
Planning requirements is lowered from drive path to end goal.

Steering
Steering precision requirements is lowered by 10X. User just provides direction or intent with control interface and system does actual steering. The system should operate with rough direction command and be able to compensate tremor.
Also, modern methods of control should be taken into consideration in the research and development of SPW, in particular Brain Computer Interface (BCI). The combination of SPW technology with BCI basically opens new prospects and functional possibilities. Accuracy and reaction speed of BCI systems are rather low. On the other hand, intelligent assistants for PW decrease the requirements to the perception, planning and steering for PW. Thus, there appears the possibility of efficient integrationthe user shows his intentions by means of the BCI system and real motion control is performed by the control system of SPW [2].
Hence, in the given study the authors focus attention on home use of SPW. Based on some individual research conducted using actual PW driving experience of several people, and interviews with many PW users, a list of requirements that seems reasonable, have been created and refined in view of modern publications [1]. The system requirements for mass market SPW categorized, taking into account the factors, described above.
We believe that a mass product SPW should have following properties (in each section, requirements listed in order from most important to less important).

Steering precision requirements
"The Smart Wheelchair Component System" (AKA SWCS) [18], proposed a list of criteria for an effective wheelchair drive assist system. While the criteria seem reasonable and well developed, we believe that the actual system for indoor use should provide much more precise operation, to be practical for home usage. − System should adapt to change of parameters of wheelchair during lifetime wear (probably with calibration / tuning during operation).

User interface requirements
− System should support regular joystick (or other existing controller) operations (smart functions should assist driving, while direction controlled in familiar way); − System should allow operation with serious mechanical control noise because of user's condition (like tremor, imprecise finger movements, etc.); − System should support touch screen interface with wired touch screen attached to wheelchair and/or wired/wireless connection with user's smartphone/smartwatch; − Ability to lock certain parameters or functions to prevent user to accidently modify them (to allow operation for children and/or patients with affected cognitive abilities); − System should support voice commands; − System should provide voice feedback in case it seriously alters user maneuver (so user will be aware of reason of wheelchair evolution); − System should support integration with other control methods (pneumatic sensors, lip movement stick, head tilt, etc.).

Basic functional requirements
− Collision mitigation (safe stop in front of obstacle); − Collision avoidance (system should try to drive around obstacle, in case free pathway exist); − Walls follow (drive near the wall keeping reasonable safe distance); − Ramp driving assist (drive in center of accessibility ramp); − Door passage assist (driving keeping in center of doorway, taking into account that typical doorway may be 70 … 75 cm wide); − Limit application of emergency braking (try to gracefully slow down when possible); − Stairs detection (upstairs and downstairs); − Unexpected obstacle mitigation / avoidance (if something appears in front of wheelchair it should apply brakes with delay no more than 200 ms); − Sidewalk drive assist (keep direct movement on sidewalk in case its visual borders are clearly visible).

Advanced functional requirements
− Automatic (or semi-automatic) construction of 3D map of user's apartment for future autopilot navigation with space no less than 100 m 2 ; − Provide ability to set predefined destination and desired orientation of wheelchair on constructed 3D map; − System should be able to find its location (automatically or semi-automatically) on existing map after been switched off and moved to new location on known apartment or detect that location is unknown (say, wheelchair transported to new building); − Provide ability to drive to predefined destination automatically in known environment (with digital map exists) as "hand-off" autopilot (with minimum requirement to apartment); − Provide ability to automatic map refresh in case of small interior changes (furniture movement, new random small objects on floor); − Provide ability to drive to predefined destinations automatically in a known environment (with digital map exists) as "eyes-off" autopilot (probably with some additional requirements to apartment).

Sensor requirements
− System should not rely on GPS (or alike) satellite navigation systems (as using these system indoors may not provide enough precision); − Sensors should be located in places, where they have fewer chances to be affected during regular operations (far from legs, lower regions that be subject of impacts, dirt, mud etc.); − Sensor should provide precisions about 1cm on area about 2m×2m in front of wheelchair; − Sensor system should be extendable to support rear sensor for driving backwards; − System should not require apartment interior changes (like putting visual markers or RFID beacons) in most real word operation scenarios. − System should be able to work without relaying to any radio technology ("no antenna") for operation. For example, a system should not be dependent on GPS (as it may not work well indoors) or RFID (as it requires markers pre-placement procedure). (zones where wheelchair driving is not safe for some reasons), so system will not allow drive to these places (zones may be marked ether digitally on map or visually with, say, red strips on floor); − System may implement concept of optional "YELLOW ZONES" (zones where wheelchair driving speed should be limited for some reasons), so system will not allow overspeed drive in these places (zones may be marked ether digitally on map or visually with, say, strips/pattern on floor). Also these zones may be marked as zones restricted for full-auto drive (so wheelchair will drive there only with explicit user command); − System may implement concept of "GREEN ZONES" (zones, where a stationary object may be detected as an obstacle, but the obstacle is passable by wheelchair). For example, the wheelchair technically may be capable of overcoming obstacles with height of 10 cm (for example, door threshold), if driven with caution. By default, wheelchair will threat anything with height > 2.5 cm (configurable) as an obstacle; − System should be configured to apply emergency stop in case the user provides an explicit stop command or removes hand from the joystick (or other controller).

Optional requirements
While these requirements not required to core system operation, it may provide additional benefits: − System may be able to be installed on existing wheelchair (at least on several popular models); − System may provide functionality of uploading constructed 3D indoor map (with or without market navigation points) to the external storage (or cloud), so other wheelchair may be used in same environment (via wired or wireless interface); − System may provide functionality to download ready constructed 3D indoor map (with or without marked navigation points) from the external storage (or cloud), so may easy adapt to new space, like entering new building (via wired or wireless interface); − System should not require usage of odometers, but may use them in case they are available (as installation of odometers virtually impossible on existing wheelchairs); − System may use MEMS accelerometers and gyroscopes to better movement tracking (as they are cheap and can be installed easy on existing wheelchair); − System may support operation on multi floor building; IAPGOŚ 3/2021 p-ISSN 2083-0157, e-ISSN 2391-6761 − System may support extended space operations by using external memory and additional external means. − System may be integrated with external systems (patient state monitor, hospital information system, prescribed movement schedule inside hospital, elevators, tracking report systems etc.).

Smart power wheelchair implementation challenges
Similar to the requirements, authors define the basic problems and challenges in key areas.

Functional challenges
− The user's apartment layout is constantly changing. Furniture may be shifted, floor patterns may be changed (for example, by different carpet installed), so a constant 3D map or pre-recorded path becomes obsolete within the timespan of hours/days. − Moving obstacles may prevent usage of stable (or pre-calculated) route. − Driving around obstacles may require route replanning.
To drive around, the wheelchair should seamlessly switch from "low-level" follow path mode to "hi-level" navigation tasks. − User comfort depends on human tolerance to accelerations.
Acceleration value and direction should be considered.
User Interface challenges − Low Precision. Reasonable goal is to 10X reduction of precision required (user just shows direction). − Noisy physical input. Tremor compensation vs fast response (smart low pass filtration + additional level of DNN to extract intent). − Voice control drive. Operate a wheelchair with simple commands like "forward", "backward", "left", "right", "stop", "slower", "faster", etc. − Auto drive destination marking (autopilot should be able to provide an easy way to specify destinations).

Mechanical challenges
− ~5 cm clearance required to pass through narrow doors (for example, to pass 60 cm wheelchair in 70 cm doorway). − Wheelchair rollers may have mechanical hysteresis (their position depends on previous turn direction). − Occupant pose may affect actual motion platform shape (the occupant pose may be non-constant). − Mechanical properties are not constant (slowly change due to physical wear or battery condition).
Electronics/Software challenges − Wheelchair has a complex shape at ~5 cm detail level. − Route planning algorithms should take into account actual wheelchair shape, as in tight environment approximation of wheelchair with simple shape may not work. − To drive a wheelchair around collision several strategies may compete, as there are many maneures exist that may drive around. − Result wheelchair orientation may be important for future driving.
Safety challenges − Reliability of automatic driving software. For example, the system should apply "safety stop" and brakes in case it detects an operation anomaly. − Predictable (Custom) Limitations. It should be possible to define "restrictions" on the map, so a wheelchair will not drive to said zones automatically. − Emergency stop. User should have a clear ability to issue emergency stop command.

Mobilis project approach for smart power wheelchair
The goal of Mobilis project is to create an autopilot for power wheelchair: Smart AI module that transforms a regular power wheelchair into a smart personal mobility platform. The system will use 3D cameras and regular video cameras to do all the motion tracking and collision avoidance. For automatic driving, 3D map of the user's apartment will be constructed automatically [1].
The goal is to provide several levels of automation: from simple collision avoidance and semi-automatic driving to fully automatic driving indoors. The solution will integrate the wheelchair and user's smartphone (smartwatch) as well.
Mobilis project plans to use 3D cameras with the option to use cameras of several types (stereoscopic cameras, laser pattern projection technology, TOF cameras) for different product versions. Also, regular video cameras will be used as additional visual sensor input (including visual motion tracking and 3D map construction in natural colors).
The primary sensor (stick approximately 10 cm × 1cm size) is planned to be located under the joystick (3D and regular video camera). The sensor is located under the joystick facing 20-60 degrees downward. This allows seeing both the surrounding space and bottom of the wheelchair simultaneously (in both 3D and video). Additional sensors may be located on opposite arm-rest and at the back of the wheelchair (to allow drive in rear direction).
Accelerometers and gyroscopes (as MEMS chips) will be (optionally) used as additional sources of motion data. Odometers are not required for operation, as visual motion tracking will be the primary source of motion data (odometers may be used if available). Because odometers are not required, this radically simplifies installation on existing wheelchairs as it is not required to alter mechanical powertrain.
To achieve goals of precise and stable collision avoidance and route planning an alternative approach (compared to conventional VFF and VFH algorithms) will be used: new Multipath route planning algorithm. Algorithm is performing path prognosis by modeling wheelchair movements in accelerated time. In practice, the wheelchair constantly performs a series of computational experiments to evaluate optimal path.
For autopilot operation, a special 2-level strategy route planning algorithm will be used (selection of intermediate goals, while goal-to-goal transfer will be done by Multipath algorithm).
To implement solution that can be used in mass production and deployment on different wheelchairs, several important issues should be taken into consideration.
Initially the wheelchair parameters are known only approximately. Moreover, even if the wheelchair model is known, a particular wheelchair may have seriously different actual parameters due to manufacturing process instability, wear, battery condition, occupant weight etc. That is, PW itself is not a precise electromechanical system.
Installation of system components may be performed with serious deviations (±20% tolerance). The system should be able to self-calibrate during installation (and fine-tuning during operation).
Many solutions imply long calibration stages to adapt to a particular wheelchair. Mobilis approach is to discover actual parameters by self-learning method.
To fine tune parameters neural network will be used to select appropriate model parameters. To avoid computational instability in case a model tuned by a neural network will operate out of a known dataset, a rough simplified model will be used as source of "reasonable corridor" of calculated evolutions.
Integration with existing wheelchairs planned to be implemented via Mobilis Connector component that acts as a proxy between joystick and motor power controller, providing full "drive by wire" possibility. As a safety fallback mechanism, "pass through" mode may be activated, so the joystick will drive the motor controller directly, without smart module functionality.

Discussion and conclusions
Research on the Mobilis project is developing a platform that combines PW with a user's smartphone and a wireless control system to implement Smart Power Wheelchair. The platform connects intelligent software and hardware modules. Mobilis solution is designed to equip most existing (or new) PW with 2engine layouts with various sets of additional features, including: smartphone control, automatic obstacle avoidance and home autopilot [1,3,13].
During Mobilis Power Wheelchair Autopilot project, test drives and interviews with people with disabilities were conducted that allowed to summarize and prioritize consumer expectations and system requirements. Also, considerable work has been done to identify problems and challenges faced by SPW operating at home conditions. Future research of certain tasks requires a comprehensive combination of methods of artificial intelligence, 3D computer vision, machine learning, mathematical modelling, ergonomics and robotics.