BitDelta: Your Fine-Tune May Only Be Worth One Bit (2402.10193v3)

Abstract: LLMs are typically trained in two phases: pre-training on large internet-scale datasets, and fine-tuning for downstream tasks. Given the higher computational demand of pre-training, it's intuitive to assume that fine-tuning adds less new information to the model, and is thus more compressible. We explore this assumption by decomposing the weights of fine-tuned models into their pre-trained components and an additional delta. We introduce a simple method, BitDelta, which successfully quantizes this delta down to 1 bit without compromising performance. This interesting finding not only highlights the potential redundancy of information added during fine-tuning, but also has significant implications for the multi-tenant serving and multi-tenant storage of fine-tuned models. By enabling the use of a single high-precision base model accompanied by multiple 1-bit deltas, BitDelta dramatically reduces GPU memory requirements by more than 10x, which can also be translated to enhanced generation latency in multi-tenant settings. We validate BitDelta through experiments across Llama-2 and Mistral model families, and on models up to 70B parameters, showcasing minimal performance degradation over all tested settings.

• The paper presents BitDelta, a novel approach that compresses fine-tuning deltas to 1-bit using scale distillation.
• It employs a two-step method combining binary quantization with high-precision scaling to maintain performance across large models.
• It significantly reduces GPU memory and storage overhead, enabling scalable multi-tenant model serving.

BitDelta: Efficient 1-Bit Quantization of Fine-Tuned Model Deltas

Introduction

In the field of LLMs, fine-tuning has emerged as a quintessential phase following the extensive pre-training stage. It fine-tunes these behemoths to specific tasks or aligns them with personalized preferences. However, with the burgeoning costs associated with storing and serving a vast number of uniquely fine-tuned models, the need for efficient solutions has become increasingly apparent. Enter BitDelta: an innovative approach that benchmarks the proposition of compressing the delta (the differential update during fine-tuning) to a mere 1-bit without a noticeable dip in performance. This technique not only presages significant reductions in storage and GPU memory demands but also holds promise for enhancing multi-tenant model serving via remarkable compression rates and generation latency improvements.

BitDelta Methodology

The BitDelta method ingeniously quantizes the weight adjustments (delta) resulting from fine-tuning into 1-bit representations while retaining a high precision scale factor for each weight matrix. This two-pronged strategy involves:

1. Quantizing the Delta: The method initially quantizes the delta of the weight matrix to a binary matrix sign representation multiplied by a scaling factor. This significantly compresses the model size while minimizing the L2 approximation error.
2. Scale Distillation: Subsequently, BitDelta employs a model distillation technique over a small dataset to refine the scaling factors further, thereby enhancing model fidelity post-quantization.

Empirical validations demonstrate that BitDelta operates across various LLMs (up to 70B parameters) with minimal degradation in performance, a testament to its robustness and utility.

Theoretical and Practical Implications

The implications of BitDelta are far-reaching, from theoretical considerations to practical applications:

• Multi-tenant Model Serving: The method introduces an efficient paradigm for serving multiple fine-tuned models on shared infrastructure. It significantly reduces the GPU memory footprint by over 10x, paving the way for a scalable, multi-tenant model serving environment.
• Parameter-Efficient Fine-Tuning (PEFT): BitDelta complements existing PEFT methods by offering an alternative solution focused on post-training compression, which could harmonize with techniques like LoRA for even greater efficiency.
• Storage and Computational Efficiency: Reducing the fine-tune delta to 1-bit representations without sacrificing performance naturally translates to lower storage costs and faster, more efficient model serving, particularly in memory-bound inference scenarios.

Future Directions

While BitDelta stands as a significant advancement in LLM efficiency, future explorations could explore:

• Extending the quantization techniques to different components of the neural network architecture.
• Optimizing the calibration dataset and distillation process for enhanced performance.
• Integrating BitDelta with existing parameter-efficient fine-tuning methodologies to explore compounded benefits.

Conclusion

BitDelta epitomizes a leap forward in our continuous quest for scalable, efficient AI technologies. By demonstrating that the fine-tune delta of LLMs can be compressed to 1-bit with negligible performance loss, it unlocks new possibilities for model serving and management. Furthermore, this research serves as a foundation for future endeavors aimed at enhancing the lifecycle efficiency of AI models from training through to deployment.