Prospective Students
Prospective Students
The new world of design for virtual and real experiences
Tokyo Tech alumna Chiharu Kodama is a designer and art director who believes that the world can be made better with the power of design. School of Engineering Associate Professor Yoshihiro Watanabe comprehends the instants that human eyes cannot perceive and works with control technology on dynamic projection mapping. The two got together to discuss this new era in design in which relationships — humans and computers, the virtual and the real, time and distance — are changing.
(Interview held on June 21, 2021 at the Suzukakedai Campus)
Dynamic projection mapping, proposed by Associate Professor Watanabe, projects images onto the surfaces of even moving objects without latency, can change the appearance of surface material, and even create shadows. Models used in experiments are a figure of a rabbit and a human, which make it easier to view the projection of surface irregularities. The texture and color that stay in place on the object, even when the object is moved, appear as if they are truly of the material. Ordinary television cameras are capable of 30 to 60 fps*. Used here, however, are cameras that can capture at 360 fps and projectors that are among the fastest in the world and can project at a maximum of 947 fps. With these, the lab has achieved latency so low that it is not visually perceptible.
The figure of a rabbit, called the Stanford Bunny, is a test model for computer graphics that was developed at Stanford University in 1994. Because of its geometric complexity and the number of triangular elements when the model is read into a 3D scanner, it is a common choice of test model for many types of 3D graphic data.
Watanabe: You were already interested in and working with design when you were an undergraduate at Tokyo Tech, and that has led to what you are doing today. Becoming a designer after graduating from Tokyo Tech is not a common career path. How do you feel about where you are now?
Kodama: There was no precedent of a Tokyo Tech graduate becoming a designer, and for a while after I graduated, I was just working frantically. Now, my thinking based in the sciences sometimes gives a favorable impression, and at other times, I am told that my comprehension makes it easier to discuss business. I make a conscious effort to distinguish when to use my left-brain thinking and my right-brain thinking.
Watanabe: What do you value when you design?
Kodama: I want to craft the design starting with user experience. I consider myself proficient in UI/UX*1 design. I don't come up with ideas out of the blue, but I keep an inventory of my own experiences and, based on them, think how I want users to feel. I feel that this thinking is important.
Watanabe: With new services, you haven't experienced them yet yourself, right? When you design such new experiences, where do they come from? From daily life? Virtual? Or, do you think of daily life and the virtual as contiguous?
Kodama: Many of my DX*2 projects are about designing services to automate or make efficient the tasks that are being done by salespeople. When digitizing a book, for example, there is an experience to base it on, touching paper. Often, there is a connection to the real world. UI/UX design involves more than just whether it's good to look at. What is required is how to interpret into reality what the client, a company or society, considers good. Because I'm designing what is directly connected to how people behave, I also have to view it from the vantage point of pursuing a profit margin or other figures. I feel from experience that I can make the world better with design that factors in business and not just artistic sense; and I keep my views updated.
Watanabe: As a researcher, I find that, when publishing my research findings in the end, the power of design greatly influences the communication of the findings to people. In the past, I was making scanning technology that would perform 100 times faster than current technology for digital archiving, converting books into images. However, some people insisted that a book is an object that is complete only when the paper and the tactile feel are part of it, meaning that simply making books into images does not qualify as digital archiving. This made me feel that we must consider what the true nature of an object is.
A high-speed, high-definition book scanner system with high-speed machine page flipping, real-time 3D deformation recognition technology, and a high-speed distortion correction algorithm. One book of around 250 pages can be digitized in one minute.
Kodama: What was your conclusion in response to the comment that it was not digital archiving?
Watanabe: I believe that it is not digital archiving unless it makes recreation possible. Preserving the surface of a book is not preserving its true nature. We need to think about whether we would be able to hold it in our hands as an object with the same paper and tactile feel, say, 100 years from now. So, I believe that the issue is how we can reproduce the material that we take in. This background is one reason I started work on projection mapping, which allows me to investigate the true nature of objects from both viewpoints, the real and the virtual. Projection mapping allows the reproduction of real things virtually. For example, a tattoo cannot be easily erased, but projection mapping can change a tattoo's design in an instant. It's also possible to make an everyday object look like a diamond. So, what does that do to the sense of value that we held? I am doing research to ask this question and involve many people in considering it.
Kodama: I love your stance of trying to involve people in the development of technology. There are researchers who love research and want to make new things, and I'm moved to know that there are also those who want to pose a question to the world and design to communicate it to people. What's important in UI/UX design is that there is a person we want to communicate something to, and we try to reach their heart. This thinking has much in common with your work.
Kodama: I've been spending more time at home because of COVID-19, and it's made me notice more changes around me, changes in the chirping of birds, changes in the way the wind blows. I've been thinking recently that it's important to design my surroundings so that I can notice such variables in daily life.
Watanabe: I can see your scientific thinking in how you described changes in society and the seasons as "variables." I'm impressed you're making it a part of your daily life. It must help in organizing your thoughts and laying the circuitry of thought to communicate them to people.
Kodama: Now, I think that even studying factorization was useful. Experience has many variables, and I think that mathematics fostered my thinking on how to break them up into different factors.
Watanabe: I loved sci-fi when I was young and watched lots of it. When I take a practical look at the technology in them, it helps me come up with ideas. I also recall that an artist I met was thinking about things logically, not just with his artistic sense, and it made me realize that logic can give rise to unexpected things that move the heart.
Kodama: Sci-fi can be something like a starter for the imagination, and I find that it gives me hints, too. It's amazing that you are looking at both design and art.
Watanabe: No, no, it's nothing! (Laughs) Although people have the impression that the outcome of our industry is the solutions we provide, 80% of our work is various viewpoints, and I feel what's important is our ability to discover problems.
Kodama: It's like mathematics. The equations are difficult to craft, and after it's been created, all that is left is to prove it.
Watanabe: When I collaborated with artists and dancers, what I found difficult was to negotiate between technology and art what each side could not compromise.
Kodama: People think that what's required in design is artistic sense, but what's really important is to first organize your thoughts and communicate. What's vital in directing is how much we can express in a common language. With less opportunities to talk face-to-face because of COVID-19, I think it means we have to work to get even closer. I'm guessing that's true of technology as well.
Watanabe: There are actually many examples of technology limiting people's behavior without them realizing it. Take a photocopy machine, for example. We hold down the cover when we're scanning a thick book, and that's because there is no technology to make the book flat. The design requires work of people. It's an omission on the part of technology. I think what is becoming necessary in engineering is the ability to find such problems from a design perspective.
A 3D display made more realistic by using the physical materials to be reproduced as the screen and rotating it. By using real yarn to show a panda made of yarn, for example, it improves both the 3D rendering and the texture reproduction.
Kodama: Projection mapping is already used in art, entertainment, and advertising, but I heard that your research is about the more advanced technology, dynamic projection mapping. What is your goal?
Watanabe: The concept is that the appearance of what is in front of you can be changed at will. I want to introduce digitally processed objects into daily life. To do this requires being able to project onto a target in motion. Capturing with cameras, reproducing with modeling, projecting with projectors, and correcting — the software and hardware for all these four steps must be fast. The speed here is one of the advantages of the technology. Further, whether the projection on the target in motion looks real depends on the time it takes to project, and though we are speeding it up, it will never reach the ideal of zero seconds. But humans are not aware of latency if it is within a certain length, so I am aiming to change appearances at will by including aspects of cognitive psychology.
Kodama: What is the latency that human eyes still perceive as real?
Watanabe: Six milliseconds (0.006 seconds). Aiming for this time is one of the tenets in my laboratory, and we replace our cameras and other equipment with those of higher specs each year. Our projectors are among the fastest in the world. As we update the core technology of hardware, we are also expanding our application, little by little.
Kodama: The speed that humans perceive is also important in UI/UX design. With websites, for instance, the loading speed tends to be felt, and I'm also looking at cognitive psychology and using it, like how showing an animation on the load screen that indicates waiting time makes a difference in the psychological burden. I feel that more things are taking such facts into consideration. We're raising the average level. This is good.
Watanabe: When I attend events with research presentations, there are people from many fields, from cognitive psychology to philosophy to statistics. Even regarding the UX of one device, the breadth of consideration is growing, such as ethics and cultural anthropology, and I feel that it has become time to carry out research from many vantage points, not just in one field.
Kodama worked on a redesign of the member-only Work Styling website of Mitsui Fudosan. She was committed to making it a user-first design and created it after much stubbornly frank research and discussion. The resulting design is a comfortable experience for the user.
Kodama: You are continuing your research at a university as you consider multiple perspectives. Where do you see yourself going?
Watanabe: What's important to me is, when someone hears of the research idea, would it, the following day, change that person's view of the world? Would I be able to deliver an experience that makes the person not want to go back to the way things were before? With such ideas in mind, I would like to further my current research and produce scanning that would make it possible to possess tens of thousands of books in a flash and projection mapping that can change the value of things visually without constraints. Additionally, I will try to pioneer futuristic technology with a view of the real and the virtual as contiguous concepts. In this, especially, I think opportunities to work together with designers will be important.
Kodama: A design engineering point of view is necessary for technology to evolve.
Watanabe: I agree. One account of a lack in technology is, when smartphones first became available, iPhone designs for icons and other elements adopted skeuomorphism.*3 The idea was to supplement with design to get people used to the skills of operating smartphones, but eventually, this was replaced with flat design*4 and Material Design.*5 As the designs moved on and trends changed, it made me wish there was something that design needed from technology in return. Have you had any moments like that?
"Find my Tokyo. List" is a new service of Tokyo Metro, and Kodama was the art director and UI designer. A user can just select photos to collect destinations to recommend into a list.
Kodama: What I personally feel as I use devices is that something that supports the tactile feel of objects has potential. Senses such as touch still feel trapped inside the screen.
Watanabe: I think what is called texture is what separates the real from the virtual. When we have something that is not real but has real texture and we have the technology to reproduce it at will, what design is will also be redefined.
Kodama: Just having a three-dimensional sensation on the display surface will change things.
Watanabe: We are speaking about physically unimpaired people here, people that recognize the world with five senses, but when we think about whether we can operate a smartphone while it is in your pocket, we will need technology that will apply to one of the five senses, such as touch. When we design such technologies, we must be sure to incorporate human experience, or they will become things that will not meet our needs.
The digitization of a pamphlet that had been published by Hokuoh, Kurashino Douguten until the end of 2020. It was designed to be read pleasantly in the app, with a feel as close to the published paper as possible.
Kodama: Yes, what the true nature of something is. I agree that pursuing that is important in both design and technology.
Watanabe: Creating a new technology is not all that difficult, but identifying social needs that have not come to the fore can't be done with only researchers and engineers. That is especially why I wish for students, who will be bearing the future, to train their eye for looking at society and to take an interest in a diversity of things. I hope that they can experience many things they can't find answers to or understand and that they can enjoy their university life with an open mind.
Kodama: A combination of engineer thinking and designer viewpoint is needed in society, and I believe that design can play a large role in making society better. That's why I wish for students to take an interest in design with scientific thinking to elucidate the mechanisms and principles of the world, perhaps viewing design as mathematics. It would be wonderful if we could have even one more designer who is a graduate of Tokyo Tech.
Yoshihiro Watanabe
Associate Professor, Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology
Received a Doctor of Information Science and Technology degree in 2007 from the Department of Information Physics & Computing, Graduate School of Information Science and Technology, the University of Tokyo. Became a research fellow at the Graduate School of Information Science and Technology, the University of Tokyo, the same year. Became a Project Assistant Professor in 2008, an Assistant Professor in 2011, a Lecturer in 2014, and at the current post since 2018. Research fields include computer vision, augmented reality, digital archiving, and interaction. Supervising faculty member for the Department of Information and Communications Engineering, graduate major in of Information and Communications Engineering.
Watanabe Lab
Studies augmenting reality with dynamic projection mapping that makes use of advanced visual sensing and high-speed vision. Using speed surpassing human perception as the key, the lab uses high-performance cameras, projectors, and more to comprehend and control the instants that are invisible to the eye to explore the possibility of designing new realities.
Chiharu Kodama
Art director and UI/UX designer
Graduated from the Department of Social Engineering, School of Engineering, Tokyo Institute of Technology Attended a design specialty school while studying at Tokyo Tech and gained experience at design offices and web design companies. Joined Art & Mobile in 2011 as its first employee. After working as a freelancer and at a major advertising company, established the design studio Ium Inc. in August 2017. Provides art direction and consulting centered on UX and UI design for apps, websites, print, and more.
The Special Topics component of the Tokyo Tech Website shines a spotlight on recent developments in research and education, achievements of its community members, and special events and news from the Institute.
Past features can be viewed in the Special Topics Gallery.
Published: 2021
© Tokyo Institute of Technology. All rights reserved.
