Brief Bio

I am a second-year Ph.D. student in Computer Science at Worcester Polytechnic Institute (WPI) advised by Dr. Jacob Whitehill. I am interested in applications in machine learning, and my research focus is about speech processing, multimodal processing and analysis related to them.

I worked as a machine learning engineer at Kuaishou Technology (Beijing) in the group of user profiling from Jul 2021 to Nov 2022. During my time in Kuaishou, I was in charge of the user profession project and live-stream recruiting project.

Before joining Kuaishou, I received my Master of Science in Computer Science degree from WPI in 2021, and was accepted as a member of Upsilon Pi Epsilon (UPE). Before that, I received my bachelor degree from University of Science and Technology of China (USTC) in 2019, majoring in Computer Science and Technology.

Research Experiences

Research Assistant
(WPI, Aug 2023 - present)
  • Working on a feedback-driven HCI system that performs real-time speech recognition and speaker diarization, and utlizing Large Language Models (LLMs) for real-time transcription and speaker correction.
  • Investigating the impact of each modality on multi-modal speech recognition in various conditions, such as different auditory noise levels, raw/abstract modality inputs. And also how the performance changes when incorporating more modalities such as OCR and synchronized lip movements.
  • Combining discrete tokens and LLMs, and perform mix-supervised training methods for multi-modal tasks, such as speech recognition (ASR), speech translation (S2TT, S2ST), image caption, etc.
  • Performed uncertainty-based active learning methods and pseudo-labeling methods to figure out a more efficient approach for finetuning ASR models.
Undergraduate Student Researcher
(USTC, Anhui Province Key Laboratory of Big Data Analysis and Application, Oct 2018 - May 2019)
  • Worked on image processing and image recognition for a plant disease recognition project.

Publications

Work Experiences

Machine learning engineer in Kuaishou Technology (Beijing)
(Kuaishou user profiling group, Jul 2021-Nov 2022)
  • Led the user profession project, and developed the first user profession labeling system in Kuaishou by building a weakly supervised framework for strategy mining. Designed an multi-task learning model with Multi-gate Mixture-of-Experts (MMoE) structure and optimized the performance with transfer learning ideas.
  • Participated in mining the "recruit intent" crowd for Kuaishou live-stream recruiting project, and analysis MRR, posterior recall, and also business metrics in ABtest.
  • Participated in optimizing the performance of several user gender models by deploying knowledge discovery methods, and conducting model ensemble.
  • Took part in basic feature extraction and maintenance works, and business analysis in experiments.

Teaching Experiences

  • Mentor of iSAT High School Internship Program (WPI, Jan 2025 - Apr 2025)
  • Teaching Assistant of Computer Programming Languages in C (USTC, Aug 2018 - Jan 2019)

Professional Activities

  • Reviewer of IEEE International Conference on Multimedia & Expo (ICME) (2025)

Honors and Awards

  • Member of Upsilon Pi Epsilon, the International Honor Society for Computing Science, WPI, 2021.
  • Excellent student bronze award, USTC, 2017.
  • Third-class CGNPC scholarship, USTC, 2016.

Hobbies

I've received qualification grades in playing both the violin and the flute. I was a member of USTC Philharmonic and USTC Student Symphony Orchestra, I gave shows in new year concerts and festivals.
I've learned watercolor painting and a little sketching.
I like traveling, and I really love taking photos of landscapes and delicious foods!

Fun facts about me

Contact Me

Address
Unity Hall 320,
Worcester Polytechnic Institute,
Worcester, Massachusetts.
Email
yguan2@wpi.edu
yiwen.irelia.guan@outlook.com