History of FORPHEUS's growth ~ From a rallying ping pong robot to entity growing together with humans with AI ~

[part 3 of 3] Humans and machines can grow together with AI (artificial intelligence)

FORPHEUS evolving year by year as the state-of-the-art technology embodying " the future when mankind and machines are reconciled".
"Why was it born?" "How will it evolve this year?" The successive development leaders clarify the whole picture.

The last part introduces the 3rd generation FORPHEUS demonstrated in CEATEC JAPAN 2016.
How has FORPHEUS evolved with deep learning among AI technologies? What is the key to actually introduce deep learning?

Evolution of FORPHEUS and a new relationship between humans and machines never stops.

Interviewing Taku Oya, as a current development leader of FORPHEUS
- The 3rd generation FORPHEUS that makes us foresee a future where humans and machines can grow together. Please tell us the keys and difficulties in its evolution.

FORPHEUS in the past played the role of a coaching machine which brings out human abilities, while OMRON believes that a future will come that "humans and machines grow together" when considering the harmony between humans and machines.

Past FORPHEUS had not incorporated technology where humans raise machines, but it is the key evolution this year.

To be more specific, we have adopted AI (artificial intelligence) technology called deep learning.
We acquired various data from people coming here to Keihanna Innovation Center. Some were good at ping pong and some were novices. We had them play a 3-minute rally per player and eventually we acquired data from more than 100 people and completed the project.

We found out one thing while teaching FORPHEUS.
Those who do not move their head and shoulders much can rally with a fast ball.
This year's FORPHEUS watches not only ball's speed but also "harmony level" with a player by observing his/her facial expressions and skeletal information. We configured FORPHEUS so that it can consider changing its return level in 3 phases, from Easy to Normal and Hard, when the player's skill grows while he/she is enjoying the rally.

Recognizing human skeletal information (retake the still)

For deep learning of FORPHEUS, we asked advice from Mr. Yamashita of Chubu University, who is prominent in the field, on the key of deep learning, and applied the drive monitoring technology with AI developed by OMRON.

Sometimes a rally failed and a ball fell on the floor during a ping pong game. In reality, the most important thing is to remove such data.
If you take data into deep learning while ping pong is not being played, the robot learns wrong data. The task of removing such data and teaching data that are really necessary actually took the longest time.

We are continuing to make improvements from last year on the features to make the machine's thoughts clear and to motivate humans.

OMRON has a technology to recognize facial expressions called OKAO which is used for FORPHEUS so that it can observe player's facial expressions during a rally and understand that "harmony level" gradually rises when he/she smiles often which gives FORPHEUS pleasure and makes the player wish to continue the rally.
We incorporated mechanisms that change the colors of FORPHEUS's head and body and show the accumulated harmony gauge on the Internet to realize communication for mutual motivation.

FORPHEUS changes its facial expression as the harmony level increases

Just continuing a rally does not increase the "harmony level". The robot observes feelings of humans perceived through facial expressions as well as a person's gradual improvement in a rally.

We share the same goal to enhance this "harmony level" for humans and machines.
We believe it should lead to the mutual growth of humans and machines and we will continue development.

We are looking forward to a world where more and more people can experience the realization of "harmonize humans and machines" and that an exciting future which expands human possibilities is within our reach. And we are hoping for cooperation with people in the world to take steps in technological development for solutions to social issues standing in the way to such an exciting future.

FORPHEUS was born not only to play ping pong.
It shows the way to our future that can bring out human abilities and possibilities.
In 10 years or so from now, the relationship between humans and machines will significantly change.

Using FORPHEUS as the key to reflect on the relationship between humans and machines in the future with many people, our challenge for the future never ends.