Welcome!

I am a researcher at NTT Media Intelligence Laboratories working with Dr. Dan Mikami. From 2019 to 2020, I was staying at Robotics Institute, Carnegie Mellon University as a Visiting Researcher under the supervision of Prof. Matthew O'Toole and Prof. Kris Kitani. I've received my Ph.D. in 2019, at Graduate School of Engineering Science, Osaka University under the supervision of Prof. Daisuke Iwai and Prof. Kosuke Sato.

Research interests: Computer vision (image&video inpainting, learning based image quality assessment, computational imaging)

News

(2020. Jul.) Our new paper is accepted to ECCV2020! More efficient form of NLOS scanning that reduces both acquisition times and computational requirements! Mariko Isogawa, Dorian Chan, Ye Yuan, Kris Kitani, Matthew O'Toole. "Efficient Non-Line-of-Sight Imaging from Transient Sinograms".

(2020. Feb) Our paper is accepted to CVPR2020! First to estimate completely hidden 3D human pose and motion using an optical NLOS imaging system. Mariko Isogawa, Ye Yuan, Matthew O'Toole, and Kris Kitani, "Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation".

Short Bio

2013-current --- Researcher at NTT Media Intelligence Laboratories
2019-2020 --- Visiting Scholar at Robotics Institute, Carnegie Mellon University
2016-2019 --- Ph.D. of Engineering, Osaka University, Japan
2011-2013 --- Master of Engineering, Osaka University, Japan (1st Class Honor in the division of System Science and Applied Informatics, the Graduate School of System Engineering)
2007-2011 --- Bachelor of Engineering, Osaka University, Japan

Research Projects

Efficient Non-Line-of-Sight Imaging from Transient Sinograms (ECCV 2020 (accpetance rate 26.4%)).
[Project page] [Paper] [Video]

Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation (CVPR 2020 (accpetance rate 22.1%), invited at MIRU2020, FIT2021).
[Project page] [Arxiv paper] [Video]

Training Data Generation without any Manual Operations for Learning-Based Preference Order Estimation (IJCV 2018 (IF 6.0), BMVC 2017 (Spotlight)).
[Open Access Journal Paper]
Image inpainting is widely acknowledged as a task whose results are quite difficult to evaluate objectively. Thus, existing learning-based image quality assessment (IQA) methods for inpainting require subjectively annotated data for training, which requires huge annotation cost. To overcome this issue, our proposed framework generates simulated failure results of inpainted images whose subjective qualities are controlled as the training data.

Mask Region Optimization for Image Inpainting (IEEE Access 2018, MIRU 2018 Interactive Session Award).
[Open Access Journal Paper]
In image inpainting, users draw a mask to specify the region. However, it is widely known that users typically need to adjust the mask region by trial and error until they obtain a desired naturally inpainting result, because inpainting quality is significantly affected by even a slight change in the mask. This manual masking takes a great deal of users’ working time and requires considerable input. To reduce such human labor, we propose the method for masked region optimization so that good inpainting results can be automatically obtained.

Learning-to-Rank Based Preference Order Estimation for Inpainted Images (MTAP 2018, IEEE ISMAR 2015 poster).
[Open Access Journal Paper]
This paper proposes an image quality assessment (IQA) method for image inpainting, aiming at selecting the best one from a plurality of results. It is widely known that inpainting results vary largely with the method used for inpainting and the parameters set. Thus, in a typical use case, users need to manually select the inpainting method and the parameters that yield the best result. This manual selection takes a great deal of time and thus there is a great need for a way to automatically estimate the best result. Thus, we propose the method that solves this problem as a learning-based ordering task between inpainted images.

Image and Video Inpainting via feature reduction and compensation (MTAP 2016, IEEE ISMAR 2015 poster, IEICE MVE Award 2014).
[Open Access Journal Paper]
Most existing image inpainting methods fail when similar regions do not exist in undamaged regions or dataset. To overcome this, our approach creates similar regions by projecting a low dimensional space from the original space. The approach comprises three stages. First, input images/videos are converted to a lower dimensional feature space. Second, a damaged region is restored in the converted feature space. Finally, inverse conversion is performed from the lower dimensional space to the original space.

Making Graphical Information Visible in Real Shadows on Interactive Tabletops (IEEE TVCG 2014, Invited at IEEE ISMAR 2014).
[Project Page]
We introduce a shadow-based interface for interactive tabletops. The proposed interface allows a user to browse graphical information by casting the shadow of his/her body, such as a hand, on a tabletop surface. Central to our technique is a new optical design that utilizes polarization in addition to the additive nature of light so that the desired graphical information is displayed only in a shadow area on a tabletop surface. In other words, our technique conceals the graphical information on surfaces other than the shadow area, such as the surface of the occluder and non-shadow areas on the tabletop surface. We combine the proposed shadow-based interface with a multi-touch detection technique to realize a novel interaction technique for interactive tabletops. We implemented a prototype system and conducted two proof-of-concept experiments along with a quantitative evaluation to assess the feasibility of the proposed optical design. Finally, we showed several implemented application systems of the proposed shadow-based interface.

Publications

Journal Papers

  1. Yuta Kageyama, Mariko Isogawa, Daisuke Iwai, and Kosuke Sato, "ProDebNet: projector deblurring using a convolutional neural network", Optics Express, vol. 28, issue 14, pp. 20391-20403, 2020. [Open Access Journal Paper]
  2. Siqi Sun, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Yoshinori Kusachi, "Multi-view video synchronization using motion rhythms of human joints", ITE Transactions on Media Technology and Applicationss, vol. 8, no. 2, pp. 100-110, 2020.
  3. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, Daisuke Iwai, Kosuke Sato, and Hideaki Kimata, "Which is the better inpainted image? Training data generation without any manual operations", International Journal of Computer Vision (IJCV), vol. 127, no. 11, pp. 1751-1766, 2019. (IF 6.0, Invited paper) [Open Access Journal Paper]
  4. Tomoya Kaichi, Shohei Mori, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Yoshinori Kusachi, "Image-based center of mass estimation of the human body via 3D shape and kinematic structure", Sports Engineering, vol. 22, no. 17, 2019. [Paper Link]
  5. Mariko Isogawa, Dan Mikami, Daisuke Iwai, Hideaki Kimata, and Kosuke Sato, "Mask Optimization for Image Inpainting", IEEE Access, vol. 6, pp. 69728-69741, 2018. [Open Access Journal Paper]
  6. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata, "Extrinsic Camera Calibration of Display-Camera System with Cornea Reflections", IEICE ED, vol. E101.D, no. 12, pp.3199-3208, 2018.
  7. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Hideaki Kimata. "Image quality assessment for inpainted images via learning to rank", Springer: Multimedia Tools and Applications (MTAP), vol. 78, no. 2, pp. 1399-1418, 2018. [Open Access Journal Paper]
  8. Shogo Miyata, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Akira Kojima. "Extrinsic Camera Calibration Without Visible Corresponding Points Using Omnidirectional Cameras", IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), vol.28, no.9, pp.2210-2219. 2018.
  9. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Image and video completion via feature reduction and compensation", Springer: Multimedia Tools and Applications (MTAP), vol.76, issue 7, pp. 9443-9462. 2017. [Open Access Journal Paper]
  10. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Akira Kojima. "Extrinsic Camera Calibration with Minimal Con guration using Cornea Model and Equidistance Constraint", IPSJ Transactions on Computer Vision and Applications (CVA), vol.8, pp.20-28. 2016.
  11. Mariko Isogawa, Daisuke Iwai, and Kosuke Sato. "Making Graphical Information Visible in Real Shadows on Interactive Tabletops", IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 20, No. 9, pp. 1293-1302. 2014. (invited at ISMAR2014)
  12. Marina Takeuchi, Mariko Isogawa, Daisuke Iwai, and Kosuke Sato. "Improvement of Operability by Loosing Spatial Relationship with Real Shadow on the Pointing Interface which Uses a Shadow Metaphor", Transactions of the Virtual Reality Society of Japan, vol.19, no.2, pp.207-214. 2014.

International Conference Papers

  1. Ryosuke Hori, Ryo Hachiuma, Hideo Saito, Mariko Isogawa, and Dan Mikami, "SILHOUETTE-BASED SYNTHETIC DATA GENERATION FOR 3D HUMAN POSE ESTIMATION WITH A SINGLE WRIST-MOUNTED 360° CAMERA", IEEE International Conference on Image Processing (ICIP), to appear, 2021.
  2. Mariko Isogawa, Dorian Chan, Ye Yuan, Kris Kitani, and Matthew O'Toole, "Efficient Non-Line-of-Sight Imaging from Transient Sinograms", 16th European Conference on Computer Vision (ECCV), pp. 193-208, 2020. (acceptance rate: 26.4%)
  3. Mariko Isogawa, Ye Yuan, Matthew O'Toole, and Kris Kitani, "Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013-7022. (acceptance rate: 22.1%, invited at MIRU2020, FIT2021)
  4. Yuta Kageyama, Mariko Isogawa, Daisuke Iwai, and Kosuke Sato, "Generative Adversarial Network Based Image Blur Compensation for Projection-Based Mixed Reality", IEEE 8th Global Conference on Consumer Electronics (GCCE), pp. 227-228, 2019.
  5. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, Siqi Sun, and Yoshinori Kusachi, "Easy Extrinsic Calibration of VR System and Multi-Camera based Marker-less Motion Capture System", IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), pp. 83-88, 2019.
  6. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, Yoshinori Kusachi, Naoki Saijo, "VR-based Batter Training System with Motion Sensing and Performance Visualization", IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), Domo, pp. 1353-1354, 2019.
  7. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata. "Human Pose as Calibration Pattern; 3D Human Pose Estimation with Multiple Unsynchronized and Uncalibrated Cameras", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1775-1782, 2018.
  8. Tomoya Kaichi, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata. "Estimation of Center of Mass for Sports Scene Using Weighted Visual hull", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1809-1815, 2018.
  9. Mariko Isogawa, Dan Mikami, Takehiro Fukuda, Naoki Saijo, Kosuke Takahashi, Hideaki Kimata, and Makio Kashino. "What Can VR Syetems Tell Sports Players? Reaction-Based Analysis of Baseball Batters in Virtual and Real Worlds", The 25th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), pp.587-588, 2018.
  10. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Hideaki Kimata. "Which is the better inpainted image? Learning without subjective annotation", British Machine Vision Conference (BMVC), pp. 5.1-5.12, Spotlight (acceptance rate: 8.8%) 2017.
  11. Shogo Miyata, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata. "Ball 3D Trajectory Reconstruction without Preliminary Temporal and Geometrical Camera Calibration", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.164-169, 2017.
  12. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Eye gaze analysis and learning-to-rank to obtain the most preferred result in image inpainting", IEEE International Conference on Image Processing (ICIP), pp.3538-3542, 2016.
  13. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Akira Kojima. "Cornea-reection-based Extrinsic Camera Calibration without a Direct View", International Conference on Computer Vision Theory and Application (VISAPP), pp.15-24, 2016.
  14. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Virtual Omnidirectional Video Synthesis with Multiple Cameras for Sports Training", 3rd International Congress on Sports Sciences Research and Technology Support (icSPORTS), pp.271-275, 2015.
  15. Dan Mikami, Mariko Isogawa, Kosuke Takahashi, Hideaki Takada, and Akira Kojima. "Immersive Previous Experience in VR for Sports Performance Enhancement", 3rd International Congress on Sports Sciences Research and Technology Support (icSPORTS), 2015.
  16. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Content Completion in Lower Dimensional Feature Space through Feature Reduction and Compensation", IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.156-159, 2015.
  17. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Toward Enhancing Robustness of DR System: Ranking Model for Background Inpainting", IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.178-179, 2015.
  18. Dan Mikami, Mariko Isogawa, Kosuke Takahashi, and Akira Kojima. "Automatic Visual Feedback from Multiple Views for Motor Learning", IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.212-213, 2015.
  19. Marina Takeuchi, Mariko Isogawa, Daisuke Iwai, Kosuke Sato. "Weak Perspective Shadow Interface for Seated User's Pointing on Large Wall Display", In Proceedings of IEEE/SICE International Symposium on System Integration (SII), pp.316-321, 2014.
  20. Mariko Miki, Daisuke Iwai, and Kosuke Sato. "Optically Hiding of Information with Polarized Complementary Projection", In Proceedings of International Conference on Arti cial Reality and Telexistence (ICAT), pp.166, 2011.
  21. Mariko Miki, Daisuke Iwai, and Kosuke Sato. "Optically Hiding of Tabletop Information with Polarized Complementary Image Projection - Your Shadow Reveals It!", In Proceedings of ACM International Conference on Interactive Tabletops and Surfaces (ITS), pp.260-261, 2011.

Domestic Conference Papers

Click here to review domestic conference papers

Talks

Media

Award&Honor

Academic Service

Organization Committee

Peer-Reviewer