Research
Research
With a vision of society 10 years from now, the Center of Innovation (COI) Program was initiated, under the support of the Japan Science and Technology Agency, to realize radical innovation through industry-academia collaboration. In April 2018, Tokyo Tech launched the Research Center for the Earth Inclusive Sensing Empathizing with Silent Voices, which seeks to lend an ear to the "silent voices", or previously unknown modes of signaling, that arise in human interaction, society, and nature. Among their projects, we looked at their work on a monitoring system to improve welfare of cattle, and on new methods to quantify human communication.
In the food-service industry, corporations like Starbucks and McDonald's have begun paying greater attention to animal welfare, which is defined by the World Organisation for Animal Health (OIE) as "the physical and mental state of an animal in relation to the conditions in which it lives and dies." This includes livestock, laboratory animals, pets, and other animals that live close to humans.
Intensive rearing environments are highly stressful for livestock and tend to increase the risk of disease. It goes without saying, then, that improved animal welfare leads to healthy food production. Furthermore, with Japan's aging population, the amount of agricultural land left idle is also increasing. One suggested application for idle farmland is to use it as cattle pasture. Doing so might also contribute to improving animal welfare. An obstacle to this, however, is that Japan is seeing a decrease in its number of farmers, as well as aging of those that still work, and insufficient investment to make pasturing a viable enterprise.
A joint team led by Associate Professor Hiroyuki Ito of the Institute of Innovative Research at Tokyo Tech and Associate Professor Kenichi Takeda of the Faculty of Agriculture at Shinshu University is developing IoT and AI solutions to address these issues. The team developed a cattle monitoring system utilizing state-of-the-art edge AI1 under the Tokyo Tech Center of Innovation (COI) Research Center for Earth Inclusive Sensing Empathizing with Silent Voices. Aiming for practical implementation in 2021, field testing is underway at the Shinshu University Faculty of Agriculture.
©Ryoko Takagi
Manga by Ryoko Takagi, 1st-year master's student, School of Environment and Society
Tokyo Tech Associate Professor Hiroyuki Ito
The edge AI-based cattle monitoring system comprises a computer device worn around the cow's neck that has an accelerometer and connectivity functions. It can collect and analyze data on an individual animal's movements, and then transmit the results to the cloud, a smartphone, or other interface. The system allows ranchers and dairy farmers to monitor the behavior and posture of cattle via smartphone, informing them whether they are drinking, eating, lying down, or walking.
The device uses a high-performance, low-energy consumption development board, called SPRESENSETM 2, produced by Sony Semiconductor Solutions Corporation. To analyze cattle behavior, the board applies an AI algorithm developed by Ito, who specializes in electronics.
The team also plans to employ GPS to detect the location of grazing cattle and send processed data to a cloud computing service (FACERE®3) provided by Information Services International-Dentsu, Ltd. (ISID) for further analysis using AI. This will make it possible to monitor not only individual animals but also the state of the entire herd. It will allow for detection of signs of heat, labor, disease, and other states of health.
Monitoring multiple animals simultaneously in real time
Shinshu University Associate Professor Kenichi Takeda
Takeda explains, "Cattle are highly social and usually live in groups. The system can alert ranchers by smartphone when an animal moves away from the herd or behaves differently, signs that might indicate that the animal is not feeling well or is going into heat. Then ranchers will be able to respond promptly and efficiently, reducing the need for manpower."
In addition, the diet of grazing cattle differs from that of animals raised in a barn, and ranchers are mindful of what plants their livestock are eating. Monitoring behavior will allow ranchers to better understand the diet of their livestock.
Actually, monitoring systems that measure cattle movement with accelerometers, analyze data using AI, and send results via Bluetooth or other short-distance wireless protocols are already commercially available in Japan. However, current systems have short communication ranges, which makes it difficult for use in wide areas such as those in pasturing. They are also capable of sending data to the cloud via 4G network, but the large power consumption necessary means that the sensor battery runs out quickly. While it is easy to charge a smartphone when its battery is running low, charging a sensor collar placed on a cow is not as simple a matter.
The team's new cattle monitoring system improves on existing systems firstly by minimizing the data processed by AI on the edge device. This is where much of Ito's expertise has been applied. The data is then sent via a long-distance wireless technology known as LPWA (low power, wide area). Covering a broader area and requiring less power despite a higher bitrate makes it possible to more accurately monitor grazing cattle for longer periods. This will make it easier for ranchers to monitor a greater number of animals. In fact, the system will provide a monitoring range of approximately 100 km from a single location. If implemented on a village-wide scale, broad coverage by LPWA will reduce the burden per livestock farmer. Working in collaboration with Farmnote Inc., a company that provides IoT solutions for ranchers, the team aims to design the world's first inclusive sensing system4.
Checking a cow's health status via smartphone
Ito outlined current issues in the implementation of the system. "The biggest issue is the creation of training data. To predict cattle behavior and posture accurately using AI, training data to teach the AI what the data means is essential. To make training data, we have to match actual behaviors and postures with data collected through the accelerometer. However, because cattle don't follow instructions, this has not been an easy task. We found ourselves chasing cattle around and recording their behaviors all day for days on end. Still, being out in the field and in contact with animals is unlike anything I've experienced to this point as an electronics researcher. It's a tough, but healing experience."
Reflecting on the joint Tokyo Tech-Shinshu University project, Ito states, "One day I asked myself if my work was really helping society. This prompted me to start looking into real needs by talking with people in different fields. That was when I met Takeda-sensei. He told me about the issues that ranchers were having. Our discussions motivated me to develop a cattle behavior monitoring system."
Tokyo Tech's COI Research Center for the Earth Inclusive Sensing Empathizing with Silent Voices has set a goal of listening to the "silent voices" of nature, plants, animals, and structures, utilizing the IoT and AI to pursue ways to achieve a prosperous coexistence in our shared global environment. Ito and team's objective fit with this goal, and thus the joint project began under the research center.
On the joint project with Tokyo Tech, Takeda explains, "I was thinking the IoT or other state-of-the-art technologies might offer solutions to problems that ranchers around Shinshu University face. However, we did not have the know-how in this area. So I truly appreciate meeting Ito-sensei and the Tokyo Tech team and their work to develop the device."
At the end of the interview, Ito states, "Because Takeda-sensei communicated a clear goal, we have unwaveringly been able to advance development. His familiarity with cattle behavior made the development of these highly practical devices possible. Although Japan has high technical capabilities in IT, especially in the area of semiconductors and telecommunications, most of the IT devices used in the livestock industry are imported. For this reason, we want to develop a 'Made in Japan' cattle monitoring system as soon as possible, and support the livestock industry."
A design in which a sensing device at the system "edge" performs AI processing that would normally be conducted in the cloud.
SPRESENSETM is a trademark of Sony Corporation.
FACERE® is a registered trademark of Information Services International-Dentsu, Ltd.
A system capable of analyzing cattle behavior by applying edge AI to movement (accelerometer) and location (GPS) data. Can also acquire and analyze data on temperature, humidity, and other rearing environment factors that affect cattle behavior.
Tokyo Tech Professor Yoshihiro Miyake
In the social environments of school and the workplace, communication is a vital element. However, though communication is generated between people, it is not something that can be seen with the eyes. Under Tokyo Tech's Center of Innovation (COI) Research Center for Earth Inclusive Sensing Empathizing with Silent Voices, Professor Yoshihiro Miyake of the School of Computing has been working on ways to visualize the degree of communication in groups, based on the Japanese concept of "ba".
Ba, which literally translates as "place," is a concept describing shared contexts where human interactions, be they face-to-face, virtual, or more intangible, lead to creation and transfer of knowledge. Miyake explains, "Particularly in present-day Japan, there is a need to improve intellectual productivity. Improving intellectual productivity is, in essence, improving of creativity. And for that, 'co-creation' through mutual communication is essential."
When we think of "communication," we tend to imagine verbal and written communication. However, non-verbal communication such as facial expressions and gestures accounts for approximately 70% of all interactions. In developing a system to visualize non-verbal communication, Miyake focused on nodding, a very common form of non-verbal communication. He chose nodding in particular because of its strong correlation with degree of empathy. Though visualizing nodding was difficult before the emergence of IoT technology, nowadays, wearable sensors make it easy to quantify movement. Moreover, when you have multiple people wearing the sensors, you can analyze their degree and target of empathy. The results can then be applied to devise more effective ways of communication, thus achieving a ba of greater creativity.
Students in Professor Tamio Nakano's classes at the Tokyo Tech Academy for Leadership and Institute for Liberal Arts were asked to sit at a circular table and hang a smartphone with an accelerometer around their necks to measure their nodding. The data was sent wirelessly to a server for analysis in real time and provided a simple way to visualize degree of communication using figures and graphs. For example, during an interaction involving students A, B, and C, when the degree of empathy was higher between students A and B, their line would be shown in bold. "Looking at the change in students' nodding over time and applying a cross-correlation function showed us at a glance who is and who isn't empathizing," states Miyake.
SyncViewer, the world's first system for visualizing empathy. Accelerometers worn by participants of a discussion measure their movements in real time. Data are then analyzed for mutual tuning in the rhythm of their movements.
Miyake further plans to use AI to evaluate the effectiveness of communication automatically and in real time. The smartphones previously used for sensing will be replaced with an IoT edge device5 built on the SPRESENSETM platform by Sony Semiconductor Solutions Corporation.
"At Tokyo Tech, we are actively promoting student-centered learning. Currently, the only way to gauge effectiveness and satisfaction in the classroom is with questionnaires. But if we can perfect this system, we will be able to quantify effectiveness. This has attracted attention from people both inside and outside the university. We also need to work on how to apply our results in the classroom and improve situations where course satisfaction is low," says Miyake.
IoT is an acronym for Internet of Things, describing a wide range of devices with connectivity functions that enable the transmission of acquired data to a server via the internet for processing. An edge device is a device with sensors and functions that allow it to acquire and immediately process data. Edge computing is a rapidly progressing field of research.
Tokyo Tech's Center of Innovation (COI) Research Center for Earth Inclusive Sensing Empathizing with Silent Voices is building an open, win-win partnership with industry for the purpose of creating a sustainable future society. We interviewed Project Leader Toshiyuki Hiroi (Sony Corporation) and Research Leader Hitoshi Wakabayashi (Tokyo Tech) about the concept and goal of the project.
Project Leader Toshiyuki Hiroi, Sony Corporation
Hiroi: Social challenges form the foundation of the project theme, "empathizing with silent voices." Along with advancing economic growth in Asia, food scarcity and environmental destruction have intensified. With Japan's decreasing birthrate and aging population, labor shortages have become increasingly serious in a wide range of industries, including agriculture, civil engineering, and social welfare. In approaching these issues, prevention is a more effective alternative than reacting to problems after they arise. Sensing silent voices in nature and society is key to reducing resource costs, minimizing waste, and increasing efficiency. Such actions will lead to an environmentally friendly and sustainable future society.
Wakabayashi: We are using state-of-the-art technologies such as IoT and AI to sense those silent voices, transform the data into comprehensible information, then transmit the information to people who need it when they need it.
To this end, an IoT edge device that is small, lightweight, and ultra-low power is essential; one with sensors, connectivity functions, and a processor capable of on-site AI processing. For this, we needed a project leader who is an industry specialist. We asked Hiroi-san to be our project leader because of his technical knowledge and skill in processors, as well as his extensive experiences in practical implementation.
Hiroi: For many years I have been involved in LSI development for game devices at Sony. And about 20 years ago I began growing deeply concerned about environmental issues. LSIs were rapidly becoming smaller, lighter, and cheaper, and I started conducting research on long-range wireless systems. During that time, I began thinking that there must be a way to use these technologies to protect the environment. So when Professor Wakabayashi asked me to join the COI project, I accepted immediately.
Wakabayashi: In this COI project, we have been working to develop an IoT edge device capable of detecting and recognizing light, temperature, and movement (acceleration). For the device platform, we are using the SPRESENSETM, a Low-Power, Smart-Sensing Processor-Equipped Board for IoT developed by Project Leader Hiroi and produced by Sony Semiconductor Solutions Corporation. By loading the device with Tokyo Tech's state-of-art AI technology, and replacing the existing sensors with next-generation ultra-sensitive diamond, graphene, or carbon nanotube sensors, which Tokyo Tech is currently developing, we believe we can create a low-energy, yet high-performance IoT edge device.
Hiroi: One of the applications is a cattle behavior monitoring system being developed by Hiroyuki Ito from Tokyo Tech and Kenichi Takeda from Shinshu University. Another is in the concept of "silent voices in group communication" by Tokyo Tech's Yoshihiro Miyake and Takayuki Nozawa.
Research Leader Hitoshi Wakabayashi, Tokyo Tech
Wakabayashi: I have been involved in many industry-academia collaborations. But unlike the common practice where project members work in isolation, a wonderful aspect of this COI project is that it gathers specialists from a broad range of fields and allows them to work together in an open environment. I am grateful for this opportunity to collaborate with Sony and other innovative companies and universities.
Hiroi: Professor Wakabayashi has been an advocate of the Open Innovation Consortium concept. I believe this matches the purpose of the COI project quite well. I also think Sony would benefit by pursuing more joint research, not only in entertainment and IT, but also in other areas, like agriculture. While it can be difficult for companies to participate in short-term or unprofitable industry-academia projects, the issues that society is facing cannot be solved by a single company or university. Rather than getting stuck in the development practices of the past, it is important to employ flexible and multifaceted approaches to identify the best solutions. I hope, therefore, to have more people from a variety fields to collaborate with us in the project.
Wakabayashi: I totally agree. Following on-going projects, as the cattle behavior monitoring and group communication systems, I look forward to producing our third and fourth solutions. This COI project will finish at the end of March in 2022; but we are planning to start a consortium that will allow us to continue research and development even through FY2022 and beyond.
Hiroi: It has been about 50 years since the first Intel microprocessor was developed, and performance over that time has increased by a factor of 10 billion. The old supercomputers that once filled rooms can now fit in your pocket. And the internet has grown to connect people around the world. As a result, the things that happen on the other side of the world are no longer other people's problems. I believe this will tie into global coexistence and world peace. This is the reason why I believe IT is the effective way to realize the sustainable future society that the COI project is aiming at. The goal of this COI project is not such that we will see results in one or two years. Therefore, using projects like this as a foundation for an open innovation consortium, I look forward to working as a member of that team towards our vision.
The Special Topics component of the Tokyo Tech Website shines a spotlight on recent developments in research and education, achievements of its community members, and special events and news from the Institute.
Past features can be viewed in the Special Topics Gallery.
Published: September 2019
© Tokyo Institute of Technology. All rights reserved.
