A "thoughtful" robot that builds a never-seen-before relationship between human and machine OMRON’s latest challenge – the development of a robot that grows together with people

Yoshihisa Ijiri, who earned his doctorate in information science, specializes in image processing and machine learning at OMRON, pursuing areas like facial image processing and image sensing algorithms on production lines.

In recent years, he has tackled developmental challenges in creating robots for the manufacturing floor that anyone can use, building a future where we can produce anything we want, no matter where we are.

Why do we need robots that anyone can easily use? How does he define the kind of "thoughtful" robots in demand on manufacturing floors? What experiences led him to start thinking this way?

We sat down with him and found out.

Simplifying robot controls to support the next generation of manufacturing

As consumer needs diversify, modern manufacturing demands are shifting and the focus is on the ability to be more flexible, such as adding more variations to the products companies make.

Furthermore, the impacts of worker shortages have led to great expectations placed on robots.

At the same time, Ijiri notes that robots up to now "weren't necessarily made to be easy to use on the manufacturing floor."

"For accurate robot operation, you need to design things like robot hands and areas for robots to place objects after working on them. Every design is different, depending on the objects and how they're being worked.

The result of this is that highly functional robots get turned into machines that can only do a limited number of things. That makes flexibility, such as adding a variety of new product types, difficult.

That's why manufacturing has always relied on skilled teams of people, but in a rapidly aging society with a low birth rate, the labor pool is shrinking. We're edging nearer to a dead end.

Even for work that can be robotized, programming the robots can still be challenging for inexperienced people. They need training to handle robots safely, and they need real experience with them in operation.

Configuration on the floor takes a vast amount of time, and a single mistake can lead to significant losses, meaning that handling robots with utmost care becomes essential."

This leads to a situation where robot deployment requires a team of experts, a paradox as robots are supposed to help reduce the need for specialized employees.

There needs to be a way to keep robot operation simple. This realization gave Ijiri an idea.

"If a robot learns the skills it needs for its operations on the floor, people can put those operations together in sequence," he explains. "And if the robot can do its tasks while adapting to changes in the environment, all we'd need to teach people is the 'what' behind the job.

It'd be a huge leap in reducing the amount of 'how' that goes into the instruction, and it would allow non-experts to use the robots. I think that will make it dramatically easier to handle robots."

Using "Growing AI" to let robots adjust to target objects and environments

"If someone trips over something," Ijiri continues, "they reflexively try to regain their balance.

When you hold something, you can instantly gauge the object's shape and distance from you. If you feel you're about to drop it, you can reflexively change your grip. Most of the time, you're able to hang on to it just fine.

That is the kind of mechanism that I'd like to see with robots."

At the same time, forms of manufacturing automation where people lose the power to control and adjust are not preferable.

"No matter how much verification you do on a production line, you'll always run into some kind of trouble." he notes.

"Keeping production going is vital, but if you do need to stop the line, it is important that you can swiftly and smoothly undergo adjustments to get it going again.

That's why it's preferable to have systems that are easy to understand, not overly intricate ones. This is why AI gets shunned in manufacturing if it's an unknown to people.

If we want to leverage AI on the floor, I think it's important that we use it to make 'thoughtful' robots that can improve their behavior based on people's orders.

Is the robot focused on speed or precision? Where is it allowed to grasp objects? What areas of objects can't be touched? Depending on the production process, you always have restrictions in place on the floor, and things need to be adjusted for that.

We often see cases where the AI thinks it has a solution for certain processes on the manufacturing floor, but it winds up not being realistically suited for the operation.

In other words, an 'intelligent' AI, in the general sense of the term, isn't useful in itself. It needs to have the kind of intelligence that lets it flexibly respond to people's demands and improve its behavior.

However, instead of having to program each individual behavior like before, we need something that requires the bare minimum input from people. It needs to have a kind of 'common sense'.

That's the kind of growing AI that I'd like to build.

Something that could improve its operation just by having people around it say 'go faster!' or 'do it more carefully!', that kind of thing.

Those kinds of instructions come naturally to us, but with the current 'stiff' AI we have, it's difficult."

The key to this is an AI that can react and behave "thoughtfully" based on people's orders.

This "thoughtful" AI will form the basis of robots that nimbly respond to manufacturing conditions and user needs, working just as people want them to with ease.

"Learned" AI can be highly precise under the conditions that it is taught, but in an unfamiliar environment, there's no telling what may happen, and adjustment is difficult.

Flexible AI that can continue to "grow" over time will be key in dealing with this.

Ijiri and his team are building the technology to make this happen, starting with flexible, more natural gripping & handling behavior — the basis of many types of robotic operations.

The transition from the laboratory to the manufacturing floor

As Ijiri puts it, brand-new challenges that haven't been tackled before are encountered when ideas come out of the lab and head for the manufacturing floor.

Ijiri works in the Intelligent System Research Center, but he built his experience in a wide number of fields, from product development to sales and customer support.

"What impressed me the most," he recalls, "is how serious-minded the manufacturing technicians were that I encountered, inside and outside of the company, during customer support duties.

Whenever a problem occurred, they had the energy and drive to stay on-site until it was fixed. They treated it as this uncompromising battle that always tested their technical skills.

The experience you build up with this approach creates a kind of quality you just can't imitate.

However, technicians that deal with manufacturing are constantly at the mercy of rising demands for product quality. These people worked long, hard hours, and I almost felt like they were pushing themselves too hard.

Seeing the technicians work, I thought to myself that if we could make things simpler and easier, it would help them out."

Having machines think more autonomously to meet the needs of users and the production floor should help them become simpler.

That, in turn, can create a production floor where machines perform the tasks they can, while humans handle what they can't.

This forms the basis of Ijiri's belief, and what drives him as he undertakes the challenges in the development of a "thoughtful" robot. It is the drive of people such as Ijiri that allow OMRON to continue to innovate and challenge.