About me
I'm a master student in Robotics, Cognition, Intelligence (RCI) at Technical University of Munich (TUM). I finished my master thesis in the Center for Information and Language Processing (CIS), LMU in the group of Prof. Dr. Hinrich Schütze, which focus on Large Language Model and Mechanistic Interpretability (MI). My research interests include interpretability for large language model, and multilingual natural language processing. I'm also interested in reinforcement learning and knowledge editing for language model.
Before that, I obtained my bachelor's degree in Mechanical Engineering at Southeast University, Nanjing, China.
Research Interests
- Multilingual NLP
- Interpretability for Large Language Models
- Knowledge-Grounded Text Generation
News
Feb. 2025 - The preprint of my master thesis: "On Relation-Specific Neurons in Large Language Models" is available on arXiv!
Previous Projects
Detecting Multi-Lingual Relation Specific Neurons in LLMs
Ludwig-Maximilians-Universität | 07.2024 - 02.2025 | Master Thesis
Large Language Models, Mechanistic Interpretability, Multilingual NLP
- Identification and characterization of neurons that store explicit knowledge about semantic relations, with a particular focus on multilingual contexts.
- Development of a methodological pipeline for localizing relation-specific neurons using a tailored dataset and validation by controlled generation experiments.
- Detailed analysis of the neuronal activation patterns in the Llama-2 (7B & 13B) model to examine their distribution and inherent properties.
- Achievements: Creation of a comprehensive relation-specific neuron set for Llama-2 model and a planned submission to a top NLP conference.
LLM-based Chat-with-File Chatbot Development
Technical University of Munich | 08.2023 - 04.2024 | HiWi Project
Large Language Models, LangChain, RAG, Streamlit
- Designed and implemented an intelligent chatbot system capable of interacting with user-uploaded files, leveraging Retrieval-Augmented Generation (RAG) and LangChain.
- Developed a robust RAG pipeline encompassing document chunking, embedding, retrieval optimization, and a user-friendly interface built with Streamlit.
- Achievements: Automated document processing, significantly enhancing retrieval efficiency and reducing response time from 20s to 3s.
LLM-assisted Construction of a Corpus of Sustainable Product Reviews
Technical University of Munich | 09.2023 - 04.2024 | Guided Research, Grade: 1.3
Large Language Models, Corpus Creation, LangChain
- Development of a scalable, automated pipeline for creating a domain-specific corpus of sustainable product reviews.
- Implementation and optimization of data collection and processing methods using OpenAI API and LangChain.
- Systematic evaluation and refinement of prompting techniques (zero-shot, few-shot, chain-of-thought) to maximize the quality and consistency of the annotated data.
- Achievements: Increased LLM-assisted annotation accuracy from 20% to 95%, significantly improving the efficiency and reliability of automated data generation.