Weilin Zhao

selected publications

1. BMInf

   BMInf: An Efficient Toolkit for Big Model Inference and Tuning

   Han, Xu, Zeng, Guoyang, Zhao, Weilin, Liu, Zhiyuan, Zhang, Zhengyan, Zhou, Jie, Zhang, Jun, Chao, Jia, and Sun, Maosong

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations May 2022

   Abs HTML Code

   In recent years, large-scale pre-trained language models (PLMs) containing billions of parameters have achieved promising results on various NLP tasks. Although we can pre-train these big models by stacking computing clusters at any cost, it is impractical to use such huge computing resources to apply big models for each downstream task. To address the computation bottleneck encountered in deploying big models in real-world scenarios, we introduce an open-source toolkit for big model inference and tuning (BMInf), which can support big model inference and tuning at extremely low computation cost. More specifically, at the algorithm level, we introduce model quantization and parameter-efficient tuning for efficient model inference and tuning. At the implementation level, we apply model offloading, model checkpointing, and CPU-GPU scheduling optimization to further reduce the computation and memory cost of big models. Based on above efforts, we can efficiently perform big model inference and tuning with a single GPU (even a consumer-level GPU like GTX 1060) instead of computing clusters, which is difficult for existing distributed learning toolkits for PLMs. BMInf is publicly released at https://github.com/OpenBMB/BMInf.

2. BMCook: A Task-agnostic Compression Toolkit for Big Models

   Zhang, Zhengyan, Gong, Baitao, Chen, Yingfa, Han, Xu, Zeng, Guoyang, Zhao, Weilin, Chen, Yanxu, Liu, Zhiyuan, and Sun, Maosong

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations Dec 2022

   HTML Code
3. OpenPrompt

   OpenPrompt: An Open-source Framework for Prompt-learning

   Ding, Ning, Hu, Shengding, Zhao, Weilin, Chen, Yulin, Liu, Zhiyuan, Zheng, Haitao, and Sun, Maosong

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations May 2022

   Abs HTML Code

   Prompt-learning has become a new paradigm in modern natural language processing, which directly adapts pre-trained language models (PLMs) to cloze-style prediction, autoregressive modeling, or sequence to sequence generation, resulting in promising performances on various tasks. However, no standard implementation framework of prompt-learning is proposed yet, and most existing prompt- learning codebases, often unregulated, only provide limited implementations for specific scenarios. Since there are many details such as templating strategy, initializing strategy, verbalizing strategy, etc., that need to be considered in prompt-learning, practitioners face impediments to quickly adapting the de-sired prompt learning methods to their applications. In this paper, we present Open- Prompt, a unified easy-to-use toolkit to conduct prompt-learning over PLMs. OpenPrompt is a research-friendly framework that is equipped with efficiency, modularity, and extendibility, and its combinability allows the freedom to combine different PLMs, task for- mats, and prompting modules in a unified paradigm. Users could expediently deploy prompt-learning frameworks and evaluate the generalization of them on different NLP tasks without constraints.

4. PTR

   Ptr: Prompt tuning with rules for text classification

   Han, Xu, Zhao, Weilin, Ding, Ning, Liu, Zhiyuan, and Sun, Maosong

   arXiv preprint arXiv:2105.11259 May 2021