I am an Assistant Professor at Department of Information and Computer Science, Keio University. Before I joined Keio University, I was working at NTT Laboratories (the supervisor was Prof. Dan Mikami). From 2019 to 2020, I was staying at Robotics Institute, Carnegie Mellon University as a Visiting Researcher under the supervision of Prof. Matthew O'Toole and Prof. Kris Kitani. I've received my Ph.D. in 2019, at Graduate School of Engineering Science, Osaka University under the supervision of Prof. Daisuke Iwai and Prof. Kosuke Sato.

Research interests: Computer vision (image&video inpainting, learning based image quality assessment, computational imaging)

(2022. May.) Our new journal paper is accepted to IEEE Access! Ryosuke Hori, Ryo Hachiuma, Mariko Isogawa, Dan Mikami, and Hideo Saito. "Silhouette-based 3D Human Pose Estimation Using a Single Wrist-mounted 360° Camera".
(2022. Apr.) I joined Department of Information and Computer Science, Keio University as an Assistant Professor!
(2022. Mar.) Our new paper is accepted to CVPR2022! Robust and memory-efficient video magnification! Shoichiro Takeda, Kenta Niwa, Mariko Isogawa, Shinya Shimizu, Kazuki Okami, Yushi Aono. "Bilateral Video Magnification Filter".

2022-current --- Assistant Professor at Department of Information and Computer Science, Keio University
2013-2022 --- Researcher at NTT Laboratories
2019-2020 --- Visiting Scholar at Robotics Institute, Carnegie Mellon University
2016-2019 --- Ph.D. of Engineering, Osaka University, Japan
2011-2013 --- Master of Engineering, Osaka University, Japan (1st Class Honor in the division of System Science and Applied Informatics, the Graduate School of System Engineering)
2007-2011 --- Bachelor of Engineering, Osaka University, Japan

Efficient Non-Line-of-Sight Imaging from Transient Sinograms (ECCV 2020 (accpetance rate 26.4%)).
[Project page] [Paper] [Video]

Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation (CVPR 2020 (accpetance rate 22.1%), invited at MIRU2020, FIT2021).
[Project page] [Arxiv paper] [Video]

Training Data Generation without any Manual Operations for Learning-Based Preference Order Estimation (IJCV 2018 (IF 6.0), BMVC 2017 (Spotlight)).
[Open Access Journal Paper]
Image inpainting is widely acknowledged as a task whose results are quite difficult to evaluate objectively. Thus, existing learning-based image quality assessment (IQA) methods for inpainting require subjectively annotated data for training, which requires huge annotation cost. To overcome this issue, our proposed framework generates simulated failure results of inpainted images whose subjective qualities are controlled as the training data.

Mask Region Optimization for Image Inpainting (IEEE Access 2018, MIRU 2018 Interactive Session Award).
[Open Access Journal Paper]
In image inpainting, users draw a mask to specify the region. However, it is widely known that users typically need to adjust the mask region by trial and error until they obtain a desired naturally inpainting result, because inpainting quality is significantly affected by even a slight change in the mask. This manual masking takes a great deal of users’ working time and requires considerable input. To reduce such human labor, we propose the method for masked region optimization so that good inpainting results can be automatically obtained.

Learning-to-Rank Based Preference Order Estimation for Inpainted Images (MTAP 2018, IEEE ISMAR 2015 poster).
[Open Access Journal Paper]
This paper proposes an image quality assessment (IQA) method for image inpainting, aiming at selecting the best one from a plurality of results. It is widely known that inpainting results vary largely with the method used for inpainting and the parameters set. Thus, in a typical use case, users need to manually select the inpainting method and the parameters that yield the best result. This manual selection takes a great deal of time and thus there is a great need for a way to automatically estimate the best result. Thus, we propose the method that solves this problem as a learning-based ordering task between inpainted images.

Image and Video Inpainting via feature reduction and compensation (MTAP 2016, IEEE ISMAR 2015 poster, IEICE MVE Award 2014).
[Open Access Journal Paper]
Most existing image inpainting methods fail when similar regions do not exist in undamaged regions or dataset. To overcome this, our approach creates similar regions by projecting a low dimensional space from the original space. The approach comprises three stages. First, input images/videos are converted to a lower dimensional feature space. Second, a damaged region is restored in the converted feature space. Finally, inverse conversion is performed from the lower dimensional space to the original space.

Making Graphical Information Visible in Real Shadows on Interactive Tabletops (IEEE TVCG 2014, Invited at IEEE ISMAR 2014).
[Project Page]
We introduce a shadow-based interface for interactive tabletops. The proposed interface allows a user to browse graphical information by casting the shadow of his/her body, such as a hand, on a tabletop surface. Central to our technique is a new optical design that utilizes polarization in addition to the additive nature of light so that the desired graphical information is displayed only in a shadow area on a tabletop surface. In other words, our technique conceals the graphical information on surfaces other than the shadow area, such as the surface of the occluder and non-shadow areas on the tabletop surface. We combine the proposed shadow-based interface with a multi-touch detection technique to realize a novel interaction technique for interactive tabletops. We implemented a prototype system and conducted two proof-of-concept experiments along with a quantitative evaluation to assess the feasibility of the proposed optical design. Finally, we showed several implemented application systems of the proposed shadow-based interface.

Journal Papers

1. Ryosuke Hori, Ryo Hachiuma, Mariko Isogawa, Dan Mikami, and Hideo Saito, "Silhouette-based 3D Human Pose Estimation Using a Single Wrist-mounted 360° Camera", IEEE Access, to appear, 2022.
2. Yuta Kageyama, Mariko Isogawa, Daisuke Iwai, and Kosuke Sato, "ProDebNet: projector deblurring using a convolutional neural network", Optics Express, vol. 28, issue 14, pp. 20391-20403, 2020. [Open Access Journal Paper]
3. Siqi Sun, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Yoshinori Kusachi, "Multi-view video synchronization using motion rhythms of human joints", ITE Transactions on Media Technology and Applicationss, vol. 8, no. 2, pp. 100-110, 2020.
4. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, Daisuke Iwai, Kosuke Sato, and Hideaki Kimata, "Which is the better inpainted image? Training data generation without any manual operations", International Journal of Computer Vision (IJCV), vol. 127, no. 11, pp. 1751-1766, 2019. (IF 6.0, Invited paper) [Open Access Journal Paper]
5. Tomoya Kaichi, Shohei Mori, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Yoshinori Kusachi, "Image-based center of mass estimation of the human body via 3D shape and kinematic structure", Sports Engineering, vol. 22, no. 17, 2019. [Paper Link]
6. Mariko Isogawa, Dan Mikami, Daisuke Iwai, Hideaki Kimata, and Kosuke Sato, "Mask Optimization for Image Inpainting", IEEE Access, vol. 6, pp. 69728-69741, 2018. [Open Access Journal Paper]
7. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata, "Extrinsic Camera Calibration of Display-Camera System with Cornea Reflections", IEICE ED, vol. E101.D, no. 12, pp.3199-3208, 2018.
8. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Hideaki Kimata. "Image quality assessment for inpainted images via learning to rank", Springer: Multimedia Tools and Applications (MTAP), vol. 78, no. 2, pp. 1399-1418, 2018. [Open Access Journal Paper]
9. Shogo Miyata, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Akira Kojima. "Extrinsic Camera Calibration Without Visible Corresponding Points Using Omnidirectional Cameras", IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), vol.28, no.9, pp.2210-2219. 2018.
10. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Image and video completion via feature reduction and compensation", Springer: Multimedia Tools and Applications (MTAP), vol.76, issue 7, pp. 9443-9462. 2017. [Open Access Journal Paper]
11. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Akira Kojima. "Extrinsic Camera Calibration with Minimal Conguration using Cornea Model and Equidistance Constraint", IPSJ Transactions on Computer Vision and Applications (CVA), vol.8, pp.20-28. 2016.
12. Mariko Isogawa, Daisuke Iwai, and Kosuke Sato. "Making Graphical Information Visible in Real Shadows on Interactive Tabletops", IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 20, No. 9, pp. 1293-1302. 2014. (invited at ISMAR2014)
13. Marina Takeuchi, Mariko Isogawa, Daisuke Iwai, and Kosuke Sato. "Improvement of Operability by Loosing Spatial Relationship with Real Shadow on the Pointing Interface which Uses a Shadow Metaphor", Transactions of the Virtual Reality Society of Japan, vol.19, no.2, pp.207-214. 2014.

International Conference Papers

1. Shoichiro Takeda, Kenta Niwa, Mariko Isogawa, Shinya Shimizu, Kazuki Okami, Yushi Aono. "Bilateral Video Magnification Filter", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), to appear. (acceptance rate: 25.33%)
2. Ryosuke Hori, Ryo Hachiuma, Hideo Saito, Mariko Isogawa, and Dan Mikami, "SILHOUETTE-BASED SYNTHETIC DATA GENERATION FOR 3D HUMAN POSE ESTIMATION WITH A SINGLE WRIST-MOUNTED 360° CAMERA", IEEE International Conference on Image Processing (ICIP), pp.1304-1308, 2021.
3. Mariko Isogawa, Dorian Chan, Ye Yuan, Kris Kitani, and Matthew O'Toole, "Efficient Non-Line-of-Sight Imaging from Transient Sinograms", 16th European Conference on Computer Vision (ECCV), pp. 193-208, 2020. (acceptance rate: 26.4%)
4. Mariko Isogawa, Ye Yuan, Matthew O'Toole, and Kris Kitani, "Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013-7022. (acceptance rate: 22.1%, invited at MIRU2020, FIT2021)
5. Yuta Kageyama, Mariko Isogawa, Daisuke Iwai, and Kosuke Sato, "Generative Adversarial Network Based Image Blur Compensation for Projection-Based Mixed Reality", IEEE 8th Global Conference on Consumer Electronics (GCCE), pp. 227-228, 2019.
6. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, Siqi Sun, and Yoshinori Kusachi, "Easy Extrinsic Calibration of VR System and Multi-Camera based Marker-less Motion Capture System", IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), pp. 83-88, 2019.
7. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, Yoshinori Kusachi, Naoki Saijo, "VR-based Batter Training System with Motion Sensing and Performance Visualization", IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), Domo, pp. 1353-1354, 2019.
8. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata. "Human Pose as Calibration Pattern; 3D Human Pose Estimation with Multiple Unsynchronized and Uncalibrated Cameras", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1775-1782, 2018.
9. Tomoya Kaichi, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata. "Estimation of Center of Mass for Sports Scene Using Weighted Visual hull", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1809-1815, 2018.
10. Mariko Isogawa, Dan Mikami, Takehiro Fukuda, Naoki Saijo, Kosuke Takahashi, Hideaki Kimata, and Makio Kashino. "What Can VR Syetems Tell Sports Players? Reaction-Based Analysis of Baseball Batters in Virtual and Real Worlds", The 25th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), pp.587-588, 2018.
11. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Hideaki Kimata. "Which is the better inpainted image? Learning without subjective annotation", British Machine Vision Conference (BMVC), pp. 5.1-5.12, Spotlight (acceptance rate: 8.8%) 2017.
12. Shogo Miyata, Hideo Saito, Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Hideaki Kimata. "Ball 3D Trajectory Reconstruction without Preliminary Temporal and Geometrical Camera Calibration", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.164-169, 2017.
13. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Eye gaze analysis and learning-to-rank to obtain the most preferred result in image inpainting", IEEE International Conference on Image Processing (ICIP), pp.3538-3542, 2016.
14. Kosuke Takahashi, Dan Mikami, Mariko Isogawa, and Akira Kojima. "Cornea-reection-based Extrinsic Camera Calibration without a Direct View", International Conference on Computer Vision Theory and Application (VISAPP), pp.15-24, 2016.
15. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Virtual Omnidirectional Video Synthesis with Multiple Cameras for Sports Training", 3rd International Congress on Sports Sciences Research and Technology Support (icSPORTS), pp.271-275, 2015.
16. Dan Mikami, Mariko Isogawa, Kosuke Takahashi, Hideaki Takada, and Akira Kojima. "Immersive Previous Experience in VR for Sports Performance Enhancement", 3rd International Congress on Sports Sciences Research and Technology Support (icSPORTS), 2015.
17. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Content Completion in Lower Dimensional Feature Space through Feature Reduction and Compensation", IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.156-159, 2015.
18. Mariko Isogawa, Dan Mikami, Kosuke Takahashi, and Akira Kojima. "Toward Enhancing Robustness of DR System: Ranking Model for Background Inpainting", IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.178-179, 2015.
19. Dan Mikami, Mariko Isogawa, Kosuke Takahashi, and Akira Kojima. "Automatic Visual Feedback from Multiple Views for Motor Learning", IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.212-213, 2015.
20. Marina Takeuchi, Mariko Isogawa, Daisuke Iwai, Kosuke Sato. "Weak Perspective Shadow Interface for Seated User's Pointing on Large Wall Display", In Proceedings of IEEE/SICE International Symposium on System Integration (SII), pp.316-321, 2014.
21. Mariko Miki, Daisuke Iwai, and Kosuke Sato. "Optically Hiding of Information with Polarized Complementary Projection", In Proceedings of International Conference on Articial Reality and Telexistence (ICAT), pp.166, 2011.
22. Mariko Miki, Daisuke Iwai, and Kosuke Sato. "Optically Hiding of Tabletop Information with Polarized Complementary Image Projection - Your Shadow Reveals It!", In Proceedings of ACM International Conference on Interactive Tabletops and Surfaces (ITS), pp.260-261, 2011.

Domestic Conference Papers

• FIT2021 'Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation (from CVPR2020)', 2021.
• Keio University '見えないものを見えるようにする，隠蔽された情報の処理技術', 2020.
• MIRU2020 'Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation (from CVPR2020)', 2020.
• Kumamoto University 'インペインティング〜修復および主観品質評価に関する最近の動向〜', 2018.
• Sojo University '機械学習を用いた画像・映像処理技術', 2018.
• IEICE Technical Group on Signal Processing (SIP) 'インペインティング〜修復および主観品質評価に関する最近の動向〜', 2018.
• International workshop on Diminished Reality (IWDR), IEEE ISMAR 'Image Inpainting for Diminished Reality', 2016.
• IEEE TVCG Special Session, IEEE ISMAR 'Making Graphical Information Visible in Real Shadows on Interactive Tabletops', 2014.

• 読売新聞地域面:日本学生科学賞受賞による掲載, 2005年1月19日.
• 京都新聞:京都市立学校文化芸術賞受賞による掲載, 2005年3月1日.
• ITmedia NEWS:「あの人、角を曲がった先でどんな格好してるだろう」を推定する技術　カーネギーメロン大学が開発 [Link], 2020年8月12日

• NTT メディアインテリジェンス研究所 所長表彰 特別賞(社内表彰), 2021.
• MIRU Interactive Session Award (Best Poster Paper Award), The 22st Meeting on Image Recognition and Understanding (MIRU), 共著者の受賞, 2019.
• NTT サービスイノベーション総合研究所 所長表彰 優秀業績賞(社内表彰), 2019.
• NTT メディアインテリジェンス研究所 所長表彰 特別賞(社内表彰), 2019.
• MIRU Interactive Session Award (Best Poster Paper Award), The 21st Meeting on Image Recognition and Understanding (MIRU), 2018.
• NTT President Award(社内表彰), 2017.
• NTT サービスイノベーション総合研究所 研究開発奨励賞(社内表彰), 2015.
• NTT 知的財産センタ所長表彰(社内表彰), 2015.
• MVE Award (Best Presentation Award), IEICE Technical Committee on Media Experience and Virtual Environment (MVE), 2014.
• 1st Class Honor in Master Course, System Innovation Course, Graduate School of Engineering Science, Osaka University, 2013.

Organization Committee

• MVA 2023 (Poster Chair)
• IEEE ISMAR 2022 (Comittee member)
• IEEE VR 2022 (Comittee member)
• ACCV 2020 (Financial Chair)
• MIRU 2018, 2019, 2020 (Area Chair)
• MIRU 2017 (Committee for young researcher's program)

Peer-Reviewer

• CVPR (2022-)
• ICCV (2021-)
• ECCV (2022-)
• IJCAI (2021-)
• IEEE TCSVT (2017-2020)
• ACM SIGGRAPH Asia (2017-2018)
• IEEE Access (2019)
• IEICE Trans. on Information and Systems (2019, 2018, 2017, 2016, 2015)
etc..

Others