feat(quantization): add uniform uint4 reformer/quantizer - #698
Open
JalinWang wants to merge 3 commits into
Open
Conversation
JalinWang
marked this pull request as ready for review
August 26, 2026 09:15
JalinWang
requested review from
chinaux,
iaojnh and
richyreachy
as code owners
August 26, 2026 09:15
JalinWang
force-pushed
the
feat/uniform-uint4-main
branch
from
August 26, 2026 11:20
077333b to
ed8c3a3
Compare
richyreachy
reviewed
Aug 27, 2026
| const auto source_type = source_meta.data_type(); | ||
| const bool is_fp32 = source_type == IndexMeta::DataType::DT_FP32; | ||
| const bool is_native_flat = | ||
| accept_native_flat && source_type == IndexMeta::DataType::DT_FP16; |
Collaborator
There was a problem hiding this comment.
same wrapper for fp16 as fp32 does?
richyreachy
reviewed
Aug 27, 2026
| } | ||
| row = decoded.data(); | ||
| } else { | ||
| const auto *input = static_cast<const uint8_t *>(source_row); |
Collaborator
There was a problem hiding this comment.
这段代码执行不到，上面限定了fp32和fp16，可以先删除，如果限定放开，再添加
richyreachy
reviewed
Aug 27, 2026
|
|
||
| UniformQuantizeFunc get_uniform_quantize_func(DataType data_type) { | ||
| if (data_type == DataType::kInt8) { | ||
| if (data_type == DataType::kUint7) { |
Collaborator
There was a problem hiding this comment.
pass out：
if (zvec::ailego::internal::CpuFeatures::static_flags_.AVX512_VNNI) {
return avx512_vnni::uniform_uint7_quantize;
}
richyreachy
reviewed
Aug 27, 2026
|
|
||
| #include "avx512_vnni/uniform_uint4/quantize.h" | ||
|
|
||
| #if defined(__AVX512F__) || (defined(_MSC_VER) && defined(__AVX512F__)) |
Collaborator
There was a problem hiding this comment.
the second part is redundant. can remove it.
richyreachy
reviewed
Aug 27, 2026
| ~UniformUint4Converter() override = default; | ||
|
|
||
| int init(const IndexMeta &index_meta, const ailego::Params ¶ms) override { | ||
| if (index_meta.data_type() != IndexMeta::DataType::DT_FP32 || |
Collaborator
There was a problem hiding this comment.
in uint8 uniform, there is a valid check statement as follows:
if (original_dimension_ == 0 || original_dimension_ > MAX_DIMENSION) {
LOG_ERROR("UniformUint8Converter: dimension=%zu must be in [1, %d]",
original_dimension_, MAX_DIMENSION);
return IndexError_InvalidArgument;
}
better check it overall in this pr to keep consistency:
- return value: IndexError_InvalidArgument
- MAX DIMENSION check
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Followup of #608. Provide better precision & performance trade-off on some datasets along with refining.