Rulin Shao

I am a first-year PhD at University of Washington advised by Prof. Pang Wei Koh and Prof. Luke Zettlemoyer . I worked as an applied scientist at AWS, focusing on large-scale pretraining for Amazon Bedrock , from January 2023 to June 2023. I obtained my master in Machine Learning at CMU advised by Prof. Eric Xing . I did my undergraduate in Mathematics at XJTU.

I'm interested in the scaling properties and robustness of retrieval-based LMs, and the efficient system design to make large-scale retrieval more accessible to academics. I am also interested in long-context chatbots and vision-language models.

Google Scholar  /  GitHub  /  Twitter  /  Linkdin  /  Email

• 2023-10 Gave a talk at AWS on Long-context LLMs! [slides]
• 2023-10 Gave a talk at SAMPL Lab on scaling up retrieval-based language models!
• 2023-10 Introducing LightSeq , a better sequence parallelism solution for long-context LLM training! It is highly optimized for decoders, and it does not entangle with the model architecture, allowing you to scale up your sequence length without an upper bound!
• 2023-08 Introducing VisIT-Bench , a new vision-language instruction following benchmark inspired by real-world use cases.
• 2023-06 Introducing our latest work on the long-context models (LongChat) and benchmark (LongEval) in this LMSys blog ! Stay tuned!
• 2023-01 I graduated from CMU and joined AWS as a full-time Applied Scientist!

My boyfriend asked me to put him here :). He is awesome!