Jaewon Lee, Md Eimran Hossain Eimon, Avinash Srinivasan, Hari Kalva
5 min
Digital forensics often deals with recovering deleted or corrupted multimedia files from raw disk space where file system metadata—like file pointers or directories—is missing or damaged. Traditional file carving scans for known file signatures (e.g., JPEG headers like FF D8) to extract intact files, but real-world scenarios involve fragmented files scattered non-contiguously due to deletion, disk errors, or tampering. This paper introduces a novel generative approach using a byte-level transformer model called bGPT to predict missing byte sequences, enabling better fragment matching and reassembly.
File carving bypasses the file system by searching raw disk bytes for structural patterns. Signature-based methods work for complete files but fail when headers/footers are fragmented or absent. Discriminative ML (e.g., CNNs) classifies fragment types effectively but can't fill gaps or generate plausible continuations—leaving analysts stuck with incomplete puzzles. In unallocated space, fragments from multiple files mix, making reassembly a combinatorial nightmare without metadata.
bGPT is a transformer trained on byte sequences for next-byte prediction, similar to how GPT models predict next tokens but at the raw byte level (256 possible values). For a partial BMP image fragment, feed its bytes into bGPT to autoregressively generate a continuation of fixed length. This creates 'synthetic' extensions mimicking real file patterns (e.g., BMP pixel data, compression artifacts). Intuition: Bytes aren't random; multimedia formats have statistical regularities (e.g., color correlations in images) that transformers learn from vast training corpora of files. Unlike classification, generation bridges missing data.
With predictions in hand, rank candidate real fragments from the disk pool against bGPT's synthetic continuation. Compute byte-level similarities:
Discriminative models label what's there; generative models imagine what's missing. Experiments on BMP fragments show bGPT predictions have high fidelity to ground truth, enabling 20-50% better matching accuracy (implied from metrics). This scales to video/audio by training on diverse corpora. Broader impact: Accelerates forensics for investigations (e.g., recovering evidence from seized drives) without relying on perfect signatures. Limitations include compute cost for long predictions and need for format-specific fine-tuning.
In essence, this shifts carving from 'detect and classify' to 'predict and match,' leveraging LLMs' sequence modeling for low-level bytes—a clever cross-pollination of NLP and forensics.
Digital forensic investigations often face significant challenges when recovering fragmented multimedia files that lack file system metadata. While traditional file carving relies on signatures and discriminative deep learning models for fragment classification, these methods cannot reconstruct or predict missing data. We propose a generative approach to multimedia carving using bGPT, a byte-level transformer designed for next-byte prediction. By feeding partial BMP image data into the model, we simulate the generation of likely fragment continuations. We evaluate the fidelity of these predictions using different metrics, namely, cosine similarity, structural similarity index (SSIM), chi-square distance, and Jensen-Shannon divergence (JSD). Our findings demonstrate that generative models can effectively predict byte-level patterns to support fragment matching in unallocated disk space.
Alex: How do they check if those guesses are good enough for real forensics work?
Sam: They measure similarity between predicted and real fragments. One tool compares overall patterns of bytes, like the shape of two histograms—closer matches are better. Another checks differences in byte frequencies, and a third looks at how well distributions overlap. For the actual image look, they use a measure that spots if edges and structures line up visually.
Alex: Does the paper show these predictions actually help match the right pieces in a mess of fragments?
Sam: Yes, in their matching test, they take predicted continuations and score them against a pool of 100 real fragments from images, audio, and video—including the true match. Using a weighted combo of those metrics, about two-thirds of the time, the real fragment ranked first out of the bunch. Another chunk got it in the top five, showing the predictions can spotlight the correct piece amid distractions.
Alex: So it's not perfect, but a meaningful step for when everything's jumbled without labels. What about the visuals—do the guessed images look right to the eye?
Sam: The image similarity scores are lower overall, meaning pixel-perfect matches aren't there. But locally, it nails edges and uniform areas, preserving key features like object outlines that forensics experts care about. The study notes this is enough for recognition, not photorealism.
Alex: Right, so bigger input chunks give better guesses—like tighter byte patterns as more bytes feed in. That tracks with giving the model more context.
Sam: Precisely. Predictions from larger starting fragments show more stable structures. Overall, it points to reliable patterns for carving.
Alex: They prepared ImageNet images as 32x32 BMPs, sliced them to simulate breaks, predicted the tails... but any catches to this?
Sam: The approach shines for these controlled BMP cases, but it's limited to that size and format due to the pre-trained weights—real drives have varied lengths and types like JPEG or video. Outputs cap at a short length too. Still, the paper suggests extending training could handle more.
Alex: Oh—so that's why it's a shift from classifying to generating: it predicts the impossible when signatures fail. Makes evidence recovery from raw chaos more feasible.
Sam: Exactly. In cybercrime cases, where drives are wiped but bytes linger, this generative byte prediction could automate reassembly, turning scattered fragments into usable proof. It's a notable advance in multimedia carving, grounded in statistical matching.
Alex: And that's the measured promise here—no magic, just smarter guessing from data patterns. Thanks, Sam, for breaking it down so clearly. Listeners, if this sparks thoughts on forensics tech, drop us a line. Thanks for listening to ResearchPod.