Task 860: .ZST File Format
Task 860: .ZST File Format
Zstandard (.zst) File Format Specifications
The .zst file format is associated with Zstandard (zstd), a lossless compression algorithm developed by Facebook. It supports real-time compression and decompression, with files consisting of one or more concatenated frames. Each frame is either a Zstandard frame (containing compressed data) or a skippable frame (for metadata). The format is platform-independent and does not include archival or encryption features. The specifications are derived from the official documentation available in the Zstandard repository.
1. List of All Properties Intrinsic to the .zst File Format
The properties listed below represent the structural elements and fields inherent to the .zst format, including headers, descriptors, identifiers, checksums, and block details. These are organized by frame type, as a .zst file may contain multiple frames in sequence. Properties are binary-encoded, primarily in little-endian format, and include variable-length fields based on flags.
General File Structure Properties:
- Concatenation of one or more frames (Zstandard or skippable).
- No overall file header or footer; frames are self-contained.
- Support for streaming and independent decompression of frames.
Zstandard Frame Properties:
- Magic Number: 4 bytes, fixed value 0xFD2FB528 (little-endian). Identifies the start of a Zstandard frame.
- Frame Header (2–14 bytes total):
- Frame_Header_Descriptor: 1 byte.
- Bits 7–6: Frame_Content_Size_flag (0–3, determines Frame_Content_Size field size: 0, 1, 2, 4, or 8 bytes).
- Bit 5: Single_Segment_flag (if set, omits Window_Descriptor and implies Window_Size equals Frame_Content_Size).
- Bit 4: Unused_bit (must be 0).
- Bit 3: Reserved_bit (must be 0).
- Bit 2: Content_Checksum_flag (if set, includes a 4-byte content checksum at the frame's end).
- Bits 1–0: Dictionary_ID_flag (0–3, determines Dictionary_ID field size: 0, 1, 2, or 4 bytes).
- Window_Descriptor: 0 or 1 byte (present if Single_Segment_flag is unset).
- Bits 7–3: Exponent (for calculating Window_Size).
- Bits 2–0: Mantissa (for calculating Window_Size).
- Window_Size formula: (1 << (10 + Exponent)) + ((1 << (10 + Exponent)) / 8) * Mantissa (range: 1 KB to ~3.75 TB).
- Dictionary_ID: 0–4 bytes (little-endian, if flag set; 0 indicates no dictionary).
- Frame_Content_Size: 0–8 bytes (little-endian, if flag set; represents decompressed content size, with adjustments for small values).
- Data Blocks (one or more, until Last_Block flag):
- Block_Header: 3 bytes (little-endian).
- Bit 0: Last_Block (set if this is the final block in the frame).
- Bits 1–2: Block_Type (0: Raw_Block, 1: RLE_Block, 2: Compressed_Block, 3: Reserved/invalid).
- Bits 3–23: Block_Size (21-bit unsigned, maximum min(Window_Size, 128 KiB)).
- Block_Content: Variable length (equals Block_Size).
- For Raw_Block (Type 0): Uncompressed raw bytes.
- For RLE_Block (Type 1): Single byte repeated Block_Size times.
- For Compressed_Block (Type 2):
- Literals Section:
- Literals_Section_Header: 1–5 bytes.
- Bits 7–6: Literals_Block_Type (0: Raw, 1: RLE, 2: Compressed with Huffman tree, 3: Treeless/reuse prior tree).
- Size_Format (1–2 bits): Determines encoding of Regenerated_Size and Compressed_Size.
- Regenerated_Size: Decompressed literals size (10–20 bits, range 0–262143).
- Compressed_Size: Compressed literals size (if applicable, 10–18 bits).
- Huffman_Tree_Description: Variable (present if Literals_Block_Type=2; describes weights for 2–256 symbols).
- Jump Table: 6 bytes (present if 4 streams; 3 × 2-byte stream sizes).
- Streams: 1 or 4 bitstreams (Huffman-coded literals).
- Sequences Section:
- Sequences_Section_Header: 1–4 bytes.
- Number_of_Sequences: 1–3 bytes (0–0x7F00 + extra).
- Symbol Compression Modes: 1 byte (2 bits each for Literals_Lengths_Mode, Offsets_Mode, Match_Lengths_Mode; modes: 0=Predefined, 1=RLE, 2=FSE_Compressed, 3=Repeat).
- FSE Tables (variable, if FSE_Compressed_Mode or Repeat_Mode): For Literals Length Codes (0–35), Offset Codes (0–31+), Match Length Codes (0–52).
- bitStream: Remaining bytes (FSE-compressed sequences, read backwards).
- Content Checksum: 0 or 4 bytes (32-bit xxHash-64 of decompressed content, little-endian, if Content_Checksum_flag set).
Skippable Frame Properties:
- Magic Number: 4 bytes, values 0x184D2A50 to 0x184D2A5F (little-endian; 16 possible variants).
- Frame_Size: 4 bytes (little-endian unsigned 32-bit; size of User_Data, max 2^32 - 1 bytes).
- User_Data: Variable (arbitrary bytes; ignored by decoders, used for metadata).
These properties define the binary layout and ensure compatibility, integrity, and efficient processing.
2. Two Direct Download Links for .zst Files
- https://downloads.sourceforge.net/project/lzmautils/xz-5.2.7.tar.zst (Sample compressed tar archive of XZ Utils source code).
- https://geo.mirror.pkgbuild.com/core/os/x86_64/zstd-1.5.6-1-x86_64.pkg.tar.zst (Sample Arch Linux package archive compressed with Zstandard).
3. Ghost Blog Embedded HTML/JavaScript for Drag-and-Drop .zst File Property Dump
The following is an embeddable HTML snippet with JavaScript suitable for a Ghost blog post (using the HTML card). It enables users to drag and drop a .zst file, parses the file structure, and displays all intrinsic properties on the screen. Parsing is implemented manually without external libraries, focusing on header extraction (full decompression of compressed blocks is not performed, as it requires the Zstandard algorithm implementation).
Drag and drop a .zst file here
4. Python Class for .zst File Handling
The following Python class parses a .zst file, extracts and prints all properties, and includes methods for reading (decoding headers) and writing a simple Zstandard frame (e.g., a raw block frame without compression for demonstration, as full compression implementation is beyond scope).
import struct
import os
class ZstFileHandler:
def __init__(self, filepath):
self.filepath = filepath
self.properties = {}
def read_and_decode(self):
with open(self.filepath, 'rb') as f:
data = f.read()
view = memoryview(data)
pos = 0
self.properties = {'frames': []}
while pos < len(view):
magic, = struct.unpack_from('<I', view, pos)
pos += 4
if magic == 0xFD2FB528:
frame = {'type': 'Zstandard', 'properties': {}}
fhd = view[pos]
pos += 1
fcs_flag = (fhd >> 6) & 0x03
single_seg = (fhd >> 5) & 0x01
checksum_flag = (fhd >> 2) & 0x01
dict_flag = fhd & 0x03
frame['properties']['Frame_Header_Descriptor'] = {
'FCS_Flag': fcs_flag, 'Single_Segment': single_seg,
'Checksum_Flag': checksum_flag, 'Dict_Flag': dict_flag
}
window_size = 0
if not single_seg:
wd = view[pos]
pos += 1
exp = (wd >> 3) & 0x1F
mant = wd & 0x07
window_size = (1 << (10 + exp)) + (mant * (1 << (10 + exp)) // 8)
frame['properties']['Window_Descriptor'] = {'Exponent': exp, 'Mantissa': mant, 'Size': window_size}
dict_id = 0
if dict_flag:
fmt = {1: '<B', 2: '<H', 4: '<I'}[dict_flag if dict_flag < 3 else 3]
dict_id, = struct.unpack_from(fmt, view, pos)
pos += [0,1,2,4][dict_flag]
frame['properties']['Dictionary_ID'] = dict_id
content_size = 0
if fcs_flag:
sizes = [0,1,2,4,8]
fcs_size = sizes[fcs_flag if fcs_flag <= 2 else 3 if fcs_flag == 3 else 4]
fmt = ['','<B','<H','<I','<Q'][fcs_size]
content_size, = struct.unpack_from(fmt, view, pos)
if fcs_size == 2: content_size += 256
pos += fcs_size
frame['properties']['Frame_Content_Size'] = content_size
if single_seg: window_size = content_size
frame['properties']['Data_Blocks'] = []
last_block = False
while not last_block and pos < len(view):
bh, = struct.unpack_from('<I', view, pos)
bh &= 0xFFFFFF # 3 bytes
pos += 3
last_block = bh & 0x01
block_type = (bh >> 1) & 0x03
block_size = bh >> 3
block = {'Last': last_block, 'Type': ['Raw', 'RLE', 'Compressed', 'Reserved'][block_type], 'Size': block_size}
if block_type == 2:
block['Compressed_Properties'] = 'Header parsing stub - full decompression not implemented'
frame['properties']['Data_Blocks'].append(block)
pos += block_size
if checksum_flag:
checksum, = struct.unpack_from('<I', view, pos)
pos += 4
frame['properties']['Content_Checksum'] = checksum
self.properties['frames'].append(frame)
elif 0x184D2A50 <= magic <= 0x184D2A5F:
frame = {'type': 'Skippable', 'properties': {}}
frame_size, = struct.unpack_from('<I', view, pos)
pos += 4
frame['properties']['Magic_Number'] = magic
frame['properties']['Frame_Size'] = frame_size
frame['properties']['User_Data'] = 'Skipped - arbitrary data'
pos += frame_size
self.properties['frames'].append(frame)
else:
raise ValueError('Invalid frame magic number')
def print_properties(self):
print('Zstandard (.zst) File Properties:')
for frame in self.properties['frames']:
print(f"\n{frame['type']} Frame:")
for key, value in frame['properties'].items():
if isinstance(value, list):
print(f" {key}:")
for item in value:
print(f" {item}")
else:
print(f" {key}: {value}")
def write_simple_frame(self, output_path, raw_data=b'Hello, Zstandard!'):
# Writes a simple single raw block frame (no compression)
with open(output_path, 'wb') as f:
# Magic
f.write(struct.pack('<I', 0xFD2FB528))
# FHD: no FCS, single seg, no checksum, no dict
fhd = 0x20 # Single_Segment_flag set
f.write(struct.pack('<B', fhd))
# Content_Size (2 bytes, +256)
content_size = len(raw_data)
f.write(struct.pack('<H', content_size - 256))
# Single block: raw, last, size
bh = (1 << 0) | (0 << 1) | (len(raw_data) << 3)
f.write(struct.pack('<I', bh)[:3])
# Content
f.write(raw_data)
# Example usage:
# handler = ZstFileHandler('example.zst')
# handler.read_and_decode()
# handler.print_properties()
# handler.write_simple_frame('output.zst')
5. Java Class for .zst File Handling
The following Java class parses a .zst file, extracts and prints all properties, and includes methods for reading (decoding headers) and writing a simple Zstandard frame.
import java.io.*;
import java.nio.*;
import java.nio.file.*;
import java.util.*;
public class ZstFileHandler {
private String filepath;
private Map<String, Object> properties;
public ZstFileHandler(String filepath) {
this.filepath = filepath;
this.properties = new HashMap<>();
}
public void readAndDecode() throws IOException {
byte[] data = Files.readAllBytes(Paths.get(filepath));
ByteBuffer buffer = ByteBuffer.wrap(data).order(ByteOrder.LITTLE_ENDIAN);
List<Map<String, Object>> frames = new ArrayList<>();
properties.put("frames", frames);
while (buffer.hasRemaining()) {
int magic = buffer.getInt();
if (magic == 0xFD2FB528) {
Map<String, Object> frame = new HashMap<>();
frame.put("type", "Zstandard");
Map<String, Object> frameProps = new HashMap<>();
frame.put("properties", frameProps);
byte fhd = buffer.get();
int fcsFlag = (fhd >> 6) & 0x03;
int singleSeg = (fhd >> 5) & 0x01;
int checksumFlag = (fhd >> 2) & 0x01;
int dictFlag = fhd & 0x03;
Map<String, Integer> fhdMap = new HashMap<>();
fhdMap.put("FCS_Flag", fcsFlag);
fhdMap.put("Single_Segment", singleSeg);
fhdMap.put("Checksum_Flag", checksumFlag);
fhdMap.put("Dict_Flag", dictFlag);
frameProps.put("Frame_Header_Descriptor", fhdMap);
long windowSize = 0;
if (singleSeg == 0) {
byte wd = buffer.get();
int exp = (wd >> 3) & 0x1F;
int mant = wd & 0x07;
windowSize = (1L << (10 + exp)) + (mant * (1L << (10 + exp)) / 8);
Map<String, Integer> wdMap = new HashMap<>();
wdMap.put("Exponent", exp);
wdMap.put("Mantissa", mant);
wdMap.put("Size", (int) windowSize); // Cast for simplicity
frameProps.put("Window_Descriptor", wdMap);
}
long dictId = 0;
if (dictFlag > 0) {
if (dictFlag == 1) dictId = buffer.get() & 0xFF;
else if (dictFlag == 2) dictId = buffer.getShort() & 0xFFFF;
else dictId = buffer.getInt() & 0xFFFFFFFFL;
frameProps.put("Dictionary_ID", dictId);
}
long contentSize = 0;
if (fcsFlag > 0) {
int fcsSize = new int[]{0,1,2,4,8}[fcsFlag > 2 ? 3 : fcsFlag];
if (fcsSize == 1) contentSize = buffer.get() & 0xFF;
else if (fcsSize == 2) contentSize = (buffer.getShort() & 0xFFFF) + 256;
else if (fcsSize == 4) contentSize = buffer.getInt() & 0xFFFFFFFFL;
else contentSize = buffer.getLong();
frameProps.put("Frame_Content_Size", contentSize);
}
if (singleSeg == 1) windowSize = contentSize;
List<Map<String, Object>> blocks = new ArrayList<>();
frameProps.put("Data_Blocks", blocks);
boolean lastBlock = false;
while (!lastBlock && buffer.hasRemaining()) {
int bh = buffer.getInt() & 0xFFFFFF;
lastBlock = (bh & 0x01) == 1;
int blockType = (bh >> 1) & 0x03;
int blockSize = bh >> 3;
Map<String, Object> block = new HashMap<>();
block.put("Last", lastBlock);
block.put("Type", new String[]{"Raw", "RLE", "Compressed", "Reserved"}[blockType]);
block.put("Size", blockSize);
if (blockType == 2) {
block.put("Compressed_Properties", "Header parsing stub - full decompression not implemented");
}
blocks.add(block);
buffer.position(buffer.position() + blockSize);
}
if (checksumFlag == 1) {
int checksum = buffer.getInt();
frameProps.put("Content_Checksum", checksum);
}
frames.add(frame);
} else if (magic >= 0x184D2A50 && magic <= 0x184D2A5F) {
Map<String, Object> frame = new HashMap<>();
frame.put("type", "Skippable");
Map<String, Object> frameProps = new HashMap<>();
frame.put("properties", frameProps);
int frameSize = buffer.getInt();
frameProps.put("Magic_Number", magic);
frameProps.put("Frame_Size", frameSize);
frameProps.put("User_Data", "Skipped - arbitrary data");
buffer.position(buffer.position() + frameSize);
frames.add(frame);
} else {
throw new IllegalArgumentException("Invalid frame magic number");
}
}
}
public void printProperties() {
System.out.println("Zstandard (.zst) File Properties:");
List<Map<String, Object>> frames = (List<Map<String, Object>>) properties.get("frames");
for (Map<String, Object> frame : frames) {
System.out.println("\n" + frame.get("type") + " Frame:");
Map<String, Object> frameProps = (Map<String, Object>) frame.get("properties");
for (Map.Entry<String, Object> entry : frameProps.entrySet()) {
if (entry.getValue() instanceof List) {
System.out.println(" " + entry.getKey() + ":");
List<Map<String, Object>> items = (List<Map<String, Object>>) entry.getValue();
for (Map<String, Object> item : items) {
System.out.println(" " + item);
}
} else if (entry.getValue() instanceof Map) {
System.out.println(" " + entry.getKey() + ": " + entry.getValue());
} else {
System.out.println(" " + entry.getKey() + ": " + entry.getValue());
}
}
}
}
public void writeSimpleFrame(String outputPath, byte[] rawData) throws IOException {
// Writes a simple single raw block frame
try (FileOutputStream fos = new FileOutputStream(outputPath);
DataOutputStream dos = new DataOutputStream(fos)) {
dos.writeInt(0xFD2FB528); // Magic (LE)
dos.writeByte(0x20); // FHD: single seg
int contentSize = rawData.length;
dos.writeShort(contentSize - 256); // FCS 2 bytes
int bh = (1 << 0) | (0 << 1) | (contentSize << 3);
dos.write(bh & 0xFF);
dos.write((bh >> 8) & 0xFF);
dos.write((bh >> 16) & 0xFF);
dos.write(rawData);
}
}
// Example usage:
// public static void main(String[] args) throws IOException {
// ZstFileHandler handler = new ZstFileHandler("example.zst");
// handler.readAndDecode();
// handler.printProperties();
// handler.writeSimpleFrame("output.zst", "Hello, Zstandard!".getBytes());
// }
}
6. JavaScript Class for .zst File Handling
The following JavaScript class (Node.js compatible, using fs) parses a .zst file, extracts and prints all properties to console, and includes methods for reading (decoding headers) and writing a simple Zstandard frame.
const fs = require('fs');
class ZstFileHandler {
constructor(filepath) {
this.filepath = filepath;
this.properties = {};
}
readAndDecode() {
const data = fs.readFileSync(this.filepath);
const view = new DataView(data.buffer);
let pos = 0;
this.properties.frames = [];
while (pos < data.length) {
const magic = view.getUint32(pos, true);
pos += 4;
if (magic === 0xFD2FB528) {
const frame = { type: 'Zstandard', properties: {} };
const fhd = view.getUint8(pos);
pos += 1;
const fcsFlag = (fhd >> 6) & 0x03;
const singleSeg = (fhd >> 5) & 0x01;
const checksumFlag = (fhd >> 2) & 0x01;
const dictFlag = fhd & 0x03;
frame.properties.Frame_Header_Descriptor = {
FCS_Flag: fcsFlag,
Single_Segment: singleSeg,
Checksum_Flag: checksumFlag,
Dict_Flag: dictFlag
};
let windowSize = 0;
if (!singleSeg) {
const wd = view.getUint8(pos);
pos += 1;
const exp = (wd >> 3) & 0x1F;
const mant = wd & 0x07;
windowSize = (1 << (10 + exp)) + (mant * (1 << (10 + exp)) / 8);
frame.properties.Window_Descriptor = { Exponent: exp, Mantissa: mant, Size: windowSize };
}
let dictId = 0;
if (dictFlag) {
dictId = this.getLittleEndian(view, pos, [0,1,2,4][dictFlag]);
pos += [0,1,2,4][dictFlag];
frame.properties.Dictionary_ID = dictId;
}
let contentSize = 0;
if (fcsFlag) {
const fcsSize = [0,1,2,4,8][fcsFlag > 2 ? 3 : fcsFlag];
contentSize = this.getLittleEndian(view, pos, fcsSize);
if (fcsSize === 2) contentSize += 256;
pos += fcsSize;
frame.properties.Frame_Content_Size = contentSize;
}
if (singleSeg) windowSize = contentSize;
frame.properties.Data_Blocks = [];
let lastBlock = false;
while (!lastBlock && pos < data.length) {
let bh = view.getUint32(pos, true) & 0xFFFFFF;
pos += 3;
lastBlock = bh & 0x01;
const blockType = (bh >> 1) & 0x03;
const blockSize = bh >> 3;
const block = { Last: lastBlock, Type: ['Raw','RLE','Compressed','Reserved'][blockType], Size: blockSize };
if (blockType === 2) {
block.Compressed_Properties = 'Header parsing stub - full decompression not implemented';
}
frame.properties.Data_Blocks.push(block);
pos += blockSize;
}
if (checksumFlag) {
const checksum = view.getUint32(pos, true);
pos += 4;
frame.properties.Content_Checksum = checksum;
}
this.properties.frames.push(frame);
} else if (magic >= 0x184D2A50 && magic <= 0x184D2A5F) {
const frame = { type: 'Skippable', properties: {} };
const frameSize = view.getUint32(pos, true);
pos += 4;
frame.properties.Magic_Number = magic;
frame.properties.Frame_Size = frameSize;
frame.properties.User_Data = 'Skipped - arbitrary data';
pos += frameSize;
this.properties.frames.push(frame);
} else {
throw new Error('Invalid frame magic number');
}
}
}
printProperties() {
console.log('Zstandard (.zst) File Properties:');
this.properties.frames.forEach(frame => {
console.log(`\n${frame.type} Frame:`);
Object.entries(frame.properties).forEach(([key, value]) => {
if (Array.isArray(value)) {
console.log(` ${key}:`);
value.forEach(item => console.log(` ${JSON.stringify(item)}`));
} else {
console.log(` ${key}: ${JSON.stringify(value)}`);
}
});
});
}
writeSimpleFrame(outputPath, rawData = Buffer.from('Hello, Zstandard!')) {
// Writes a simple single raw block frame
const buffer = Buffer.alloc(4 + 1 + 2 + 3 + rawData.length);
let pos = 0;
buffer.writeUInt32LE(0xFD2FB528, pos); pos += 4;
buffer.writeUInt8(0x20, pos); pos += 1; // FHD
const contentSize = rawData.length;
buffer.writeUInt16LE(contentSize - 256, pos); pos += 2;
const bh = (1 << 0) | (0 << 1) | (contentSize << 3);
buffer.writeUInt8(bh & 0xFF, pos); pos += 1;
buffer.writeUInt8((bh >> 8) & 0xFF, pos); pos += 1;
buffer.writeUInt8((bh >> 16) & 0xFF, pos); pos += 1;
rawData.copy(buffer, pos);
fs.writeFileSync(outputPath, buffer);
}
getLittleEndian(view, pos, bytes) {
let val = 0;
for (let i = 0; i < bytes; i++) {
val |= view.getUint8(pos + i) << (i * 8);
}
return val;
}
}
// Example usage:
// const handler = new ZstFileHandler('example.zst');
// handler.readAndDecode();
// handler.printProperties();
// handler.writeSimpleFrame('output.zst');
7. C++ Class for .zst File Handling
The following C++ class parses a .zst file, extracts and prints all properties to console, and includes methods for reading (decoding headers) and writing a simple Zstandard frame.
#include <iostream>
#include <fstream>
#include <vector>
#include <map>
#include <cstdint>
#include <cstring>
class ZstFileHandler {
private:
std::string filepath;
std::map<std::string, std::vector<std::map<std::string, std::string>>> properties; // Simplified structure
public:
ZstFileHandler(const std::string& fp) : filepath(fp) {}
void readAndDecode() {
std::ifstream file(filepath, std::ios::binary);
if (!file) throw std::runtime_error("Cannot open file");
file.seekg(0, std::ios::end);
size_t size = file.tellg();
file.seekg(0);
std::vector<uint8_t> data(size);
file.read(reinterpret_cast<char*>(data.data()), size);
size_t pos = 0;
std::vector<std::map<std::string, std::string>> frames;
while (pos < size) {
uint32_t magic;
std::memcpy(&magic, data.data() + pos, 4);
pos += 4;
if (magic == 0xFD2FB528) {
std::map<std::string, std::string> frame;
frame["type"] = "Zstandard";
uint8_t fhd = data[pos++];
uint8_t fcsFlag = (fhd >> 6) & 0x03;
uint8_t singleSeg = (fhd >> 5) & 0x01;
uint8_t checksumFlag = (fhd >> 2) & 0x01;
uint8_t dictFlag = fhd & 0x03;
frame["FCS_Flag"] = std::to_string(fcsFlag);
frame["Single_Segment"] = std::to_string(singleSeg);
frame["Checksum_Flag"] = std::to_string(checksumFlag);
frame["Dict_Flag"] = std::to_string(dictFlag);
uint64_t windowSize = 0;
if (!singleSeg) {
uint8_t wd = data[pos++];
uint8_t exp = (wd >> 3) & 0x1F;
uint8_t mant = wd & 0x07;
windowSize = (1ULL << (10 + exp)) + (mant * (1ULL << (10 + exp)) / 8);
frame["Window_Exponent"] = std::to_string(exp);
frame["Window_Mantissa"] = std::to_string(mant);
frame["Window_Size"] = std::to_string(windowSize);
}
uint64_t dictId = 0;
if (dictFlag) {
std::memcpy(&dictId, data.data() + pos, [0,1,2,4][dictFlag]);
pos += [0,1,2,4][dictFlag];
frame["Dictionary_ID"] = std::to_string(dictId);
}
uint64_t contentSize = 0;
if (fcsFlag) {
size_t fcsSize = [0,1,2,4,8][fcsFlag > 2 ? 3 : fcsFlag];
std::memcpy(&contentSize, data.data() + pos, fcsSize);
if (fcsSize == 2) contentSize += 256;
pos += fcsSize;
frame["Frame_Content_Size"] = std::to_string(contentSize);
}
if (singleSeg) windowSize = contentSize;
frame["Data_Blocks"] = "Blocks: "; // Stub for blocks
bool lastBlock = false;
while (!lastBlock && pos < size) {
uint32_t bh = 0;
std::memcpy(&bh, data.data() + pos, 3);
pos += 3;
lastBlock = bh & 0x01;
uint8_t blockType = (bh >> 1) & 0x03;
uint32_t blockSize = bh >> 3;
frame["Data_Blocks"] += "Last=" + std::to_string(lastBlock) + ", Type=" + std::string({"Raw","RLE","Compressed","Reserved"}[blockType]) + ", Size=" + std::to_string(blockSize) + "; ";
if (blockType == 2) {
frame["Data_Blocks"] += "(Compressed stub); ";
}
pos += blockSize;
}
if (checksumFlag) {
uint32_t checksum;
std::memcpy(&checksum, data.data() + pos, 4);
pos += 4;
frame["Content_Checksum"] = std::to_string(checksum);
}
frames.push_back(frame);
} else if (magic >= 0x184D2A50 && magic <= 0x184D2A5F) {
std::map<std::string, std::string> frame;
frame["type"] = "Skippable";
uint32_t frameSize;
std::memcpy(&frameSize, data.data() + pos, 4);
pos += 4;
frame["Magic_Number"] = std::to_string(magic);
frame["Frame_Size"] = std::to_string(frameSize);
frame["User_Data"] = "Skipped - arbitrary data";
pos += frameSize;
frames.push_back(frame);
} else {
throw std::runtime_error("Invalid frame magic number");
}
}
properties["frames"] = frames;
}
void printProperties() {
std::cout << "Zstandard (.zst) File Properties:" << std::endl;
for (const auto& frame : properties["frames"]) {
std::cout << "\n" << frame.at("type") << " Frame:" << std::endl;
for (const auto& [key, value] : frame) {
if (key != "type") {
std::cout << " " << key << ": " << value << std::endl;
}
}
}
}
void writeSimpleFrame(const std::string& outputPath, const std::vector<uint8_t>& rawData = { 'H','e','l','l','o',',',' ','Z','s','t','a','n','d','a','r','d','!' }) {
// Writes a simple single raw block frame
std::ofstream file(outputPath, std::ios::binary);
uint32_t magic = 0xFD2FB528;
file.write(reinterpret_cast<const char*>(&magic), 4);
uint8_t fhd = 0x20;
file.write(reinterpret_cast<const char*>(&fhd), 1);
uint16_t contentSize = rawData.size() - 256;
file.write(reinterpret_cast<const char*>(&contentSize), 2);
uint32_t bh = (1 << 0) | (0 << 1) | (static_cast<uint32_t>(rawData.size()) << 3);
file.write(reinterpret_cast<const char*>(&bh), 3);
file.write(reinterpret_cast<const char*>(rawData.data()), rawData.size());
}
};
// Example usage:
// int main() {
// ZstFileHandler handler("example.zst");
// handler.readAndDecode();
// handler.printProperties();
// handler.writeSimpleFrame("output.zst");
// return 0;
// }