Bio

Hi there! This is Qiyu Wu, a Research Scientist on the Multimodal NLP team at Creative AI Lab, Sony. We conduct language-centric multimodal research to enhance content creation in music, film, and games. Feel free to contact for discussion or collaboration!

Before joining Sony, I received my Ph.D. from The University of Tokyo, advised by Yoshimasa Tsuruoka and supported by JSPS DC Fellowship. I have served as an Area Chair for ACL Rolling Review and NeurIPS, and as a program committee member (reviewer) for several top-tier conferences including ACL, EMNLP, NAACL, ICLR, NeurIPS, and ICML. Additionally, I co-organized the GenProCC Workshop at NeurIPS 2025. Here is my CV. Reach out to me by wuqiyu576 [AT] gmail [DOT] com, or LinkedIn.

Research Interests

My research focus lies in Multimodal NLP, mainly encompassing multimodal LLMs as well as better representing textual semantics in both monolingual and multilingual contexts. I have published papers More at conferences such as ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, AAAI, EACL, VLDB, etc.

News

• 2026-10, Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing and Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA are accepted to NeurIPS 2026.
• 2026-08, 3 papers accepted to EMNLP 2026: MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval (Main, Oral), MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations (Main), and Syn-Omni: Structured Specialization and Progressive Collaboration for Omnimodal Embeddings (Findings).
• 2026-07, selected as an ACL 2026 Outstanding Reviewer (top 1.4%).
• 2026-06, gave an AiTech Seminar guest talk at Okazaki Lab, Institute of Science Tokyo.
• 2026-05, DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics is accepted to ICML 2026, and Reinforcement Learning from Bagged Reward is selected for the ICML 2026 Poster Track.
• 2026-05, serve as Area Chair for NeurIPS 2026.
• 2026-04, Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error Annotations is accepted to ACL 2026.
• 2026-01, VIRTUE: Visual-Interactive Text-Image Universal Embedder and LLM2Fx-Tools: Tool Calling For Music Post-Production are accepted to ICLR 2026, and BREAK-THE-BEAT! Controllable MIDI-to-Drum Audio Synthesis is accepted to ICASSP 2026.
• 2025-12, co-organized GenProCC Workshop at NeurIPS 2025.
• 2025-07, DeepResonance: Enhancing Multimodal Music Understanding via Music-centric Multi-way Instruction Tuning is accepted to EMNLP 2025 Main.
• 2025-05, Improving Word Alignment Using Semi-Supervised Learning is accepted to ACL 2025 Findings.
• 2025-04, a paper on Reinforcement Learning from Bagged Reward is accepted to Transactions on Machine Learning Research.
• 2025-02, serve as Area Chair for ACL Rolling Review.
• 2024-10, I join Sony Creative AI Lab, I’ll mainly work on multi-modal representation learning.
• 2024-09, Word Alignment as Preference for Machine Translation is accepted to EMNLP 2024 Main, Oral.
• 2024-04, Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding is accepted to ICML 2024.
• 2024-03, Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment is accepted to NAACL 2024 Findings.
• 2024-01, Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding is accepted to EACL 2024 Main.
• 2023-11, just visited and gave talks at Osaka University and Kyoto University, “Mitigating Reporting Bias in Visual- Language Datasets w/ Large Generative Models”. [Slides]
• 2023-09, I’m fortunate to be selected as Research Fellowships for Young Scientists (DC2), JSPS!
• 2023-08, Blocker and Matcher Can Mutually Benefit: A Co-Learning Framework for Low-Resource Entity Resolution, in collaboration with HKUST, is accepted to VLDB 2024 Research Track.
• 2023-07, attending ACL 2023 in Toronto, will present WSPAlign in-person.
• 2023-06, give a talk at Sony, “Leveraging Unlabeled Text: Data-centric Approaches to Improve NLP Training”. [Slides]
• 2023-06, start my internship at Creative AI Lab, Sony, Tokyo, I will work on NLP for multi-modal training.
• 2023-05, WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction, in collaboration with NTT CS lab, is accepted to ACL 2023 Main.
• 2023-01, start visiting San Diego as a visiting student at UCSD.
• 2022-12, attending EMNLP 2022 in Abu Dhabi to present PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings.
• 2022-10, PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings, in collaboration with Microsoft, is accepted to EMNLP 2022 Main.
• 2021-10, I join Tsuruoka Lab at University of Tokyo as PhD student!
• 2021-10, I am glad to be selected to be supported by JST Support for Pioneering Research Initiated by the Next Generation (SPRING) Program from 2021 to 2024!
• 2021-04, invited talk at TechBeat, “Light language pre-training”. [Slides]
• 2021-01, Taking Notes on the Fly Helps Language Pre-Training, in collaboration with Microsoft Research Asia, is accepted to ICLR 2021.
• 2020-12, Community-Aware Multi-Task Transportation Demand Prediction, in collaboration with Baidu Research, is accepted to AAAI 2021.
• 2020-04, start internship at Machine Learning Group, Microsoft Research Asia, Beijing.

Recent Research

Mixture of Probes

NeurIPS 2026

Learning from training-only privileged modalities through structured probing, improving multimodal LLMs when only one modality is available at inference.

Paper

MCA

EMNLP 2026 Main, Oral

Modality composition awareness reduces modality shortcuts and improves composed multimodal retrieval under distribution shifts.

Paper

MLLMCLIP

EMNLP 2026 Main

Feature-level distillation transfers knowledge from multimodal LLMs to CLIP, improving compositional understanding and vision-language representations.

Paper

MusTBench

Preprint 2026

A music-expert-validated benchmark with five temporal grounding tasks, paired with MusT training to improve temporal understanding in music LLMs.

Paper /Code /Data /Project