This GovTech team built an eCanvas to help persons with special needs paint

By developing a mobile canvas that automatically dispenses paint, engineers at GovTech hope to lend a helping hand to persons with special needs.

With only her mouth, this 19-year-old painter toggles a joystick positioned next to her face to move around in a wheelchair. Due to circumstances surrounding her premature birth, Ms Elaine Eng was born with cerebral palsy—a lifelong, physical disability that affects both movement and posture.

Despite her limitations, Elaine enjoys painting. She lays an array of colours onto an empty canvas using a paintbrush held in her mouth. At the moment, Elaine receives assistance from volunteers at the Cerebral Palsy Alliance Singapore (CPAS), who help her move the canvas and dispense paint.

In August 2018, a team led by Dr Ong Lay Teen, a Senior System Engineer at the Government Technology Agency of Singapore (GovTech), visited CPAS to discuss how they could apply new technologies to help those with a physical disability. “We were introduced to Elaine and were told that she is a mouth painter,” recalled Dr Ong. “It all began from there when we saw her passion to paint.”

A dream come true for mouth painters

Moved by Elaine’s love for painting, the team decided to build the e-canvas, a voice-controlled device that would provide greater independence for her artistic endeavours.

The eCanvas consists of a motorised aluminium frame that clamps onto a drawing canvas and moves the canvas to within the reach of the painter. A microphone connects to the eCanvas receives voice commands of the painter, either to move the canvas or dispense paint on a palette. A mini-computer installed with a pre-trained neural network engine processes the voice commands. “At the beginning, the idea was to have a canvas that moved up and down,” said Mr Quek Yang Boon, Director of Smart Nation Systems and Solutions, Sensors and Internet of Things at GovTech.

But the project soon evolved into something bigger, with the paint dispenser as an example of the team’s ingenuity. Improvised from simple, off-the-shelf items, the paint dispenser consists of several soap dispensers that had been repurposed to release paint onto a turntable, which acts as the palette.

“Each person involved [in the e-canvas project] had a different skillset that blended very well together,” said Mr Quek. The team specifically designed or sourced for parts of the eCanvas, such as the paintbrush holder and circuit board. “Every single bolt and nut that’s on there was screwed on by us. I believe that is what true engineering is about,” he added.

Perfecting the eCanvas’ design

Even originally planned features, such as the voice command system, had to be revised several times before they could perform well enough. “We imagined that the solution would be this or that,” Dr Ong shared, “but when we tested the eCanvas on-site, we realised that we had to change the design for it to be more applicable to Elaine.”

Instead of a generic system that could recognise many words, the team built a custom system—powered by a convolutional neural network—to recognise only a few keywords for better performance. To further increase the system’s accuracy, Elaine’s voice was added into the mix when training the device.

The voice control system interface was designed to provide a seamless user experience for Elaine. She did not have to navigate through levels of voice commands like in a phone menu, as the options were parallelised onto a single layer. This meant that Elaine would only have to say a specific command for a desired action to occur.

Enabling one for many

The eCanvas project demonstrates how technology can be used to improve lives and showcases the engineering and development capabilities within GovTech. From hardware to software, the Sensors and Internet of Things (SIOT) team has the expertise to potentially make a difference in the assistive technologies space, said Mr Quek. For instance, the ideas and solutions put forth to cater to those with physical disabilities can be adapted to serve other segments of the population as well, such as the elderly, which will comprise a quarter of Singapore’s population by the year 2030.

This aligns with GovTech’s focus on inclusiveness, whereby no individual is left behind as Singapore digitalises and strives to become a smart nation. “As part of the government, we tend to always look at the big picture and to take on a lot of big projects,” said Mr Quek. “What I think the eCanvas shows is that we can also look at the value that technology can create for smaller segments of society who may be overlooked.”