Bun QuickLook thumbnail Cache extraction
Calum Knott

Calum Knott @calumk

Location:
London, UK
Joined:
Mar 4, 2021

Bun QuickLook thumbnail Cache extraction

Publish Date: Jul 5
0 0

I recently stumbled upon this interesting writeup from 2016 (nearly a decade ago) that talks about extracting data from the quicklook cache com.apple.QuickLook.thumbnailcache

QuickLook thumbnails data parser - Article

OSX-QuickLook-Parser - Github

Since time has moved on a bit, I decided to see if the old python code could be ported to Bun.

Im pleased to say it can!

Gone are the Pillow/biplist/xlsxwriter dependancies, and we have an easy to use command prompt version.

Enjoy!

Script rewrite/port for Javascript / Bun.JS #2

Not really an issue, but a comment for future travellers.....

New Version for Bun.JS

I dont like python, so I have a full rewrite/port of this for Bun.JS, it runs and is tested on OSX, using Bun.js

it requires zero dependancies, or installation. as it uses bun native code

Usage

Command Line

bun run extract.js -d "/path/to/com.apple.QuickLook.thumbnailcache" -o "/path/to/output_folder"
Enter fullscreen mode Exit fullscreen mode

Options

  • -d, --thumbcache-dir: Path to com.apple.QuickLook.thumbnailcache folder (required)
  • -o, --output-folder: Path to empty folder to hold report and thumbnails (required)
  • -t, --type: Output format, currently only supports 'tsv' (default: tsv)
  • -h, --help: Show help message

Example

bun run extract.js -d "/Users/username/Library/Caches/com.apple.QuickLook.thumbnailcache" -o "./output"
Enter fullscreen mode Exit fullscreen mode

QuickLook Cache Location

The QuickLook cache is typically located at:

/Users/[username]/Library/Caches/com.apple.QuickLook.thumbnailcache/

This folder should contain:

  • index.sqlite - Database with metadata
  • thumbnails.data - Raw thumbnail data

Output

The script generates:

  • report.csv - Tab-separated values file with metadata
  • thumbnails/ folder - Extracted thumbnail images as PNG files
  • error.log - Any errors encountered during processing

Code

#!/usr/bin/env bun

/**
 * QuickLook Parser - JavaScript/Bun Port
 * Port written by Calum Knott - calum@calumk.com (using Cursor.ai)
 * 
 * Converted from the original Python script by Mari DeGrazia
 * Original: http://az4n6.blogspot.com/
 * 
 * This will parse the Mac QuickLook database which holds metadata for viewed thumbnails in the Mac Finder
 * This includes parsing out the embedded plist file in the version field as well as extracting thumbnails from the thumbnails.data folder
 * 
 * Usage:
 *   bun run extract.js -d "/path/to/com.apple.QuickLook.thumbnailcache" -o "/path/to/output_folder"
 * 
 * Required files in thumbnailcache folder:
 *   - index.sqlite
 *   - thumbnails.data
 */

import { existsSync, mkdirSync, readFileSync, writeFileSync, createWriteStream } from "fs";
import { join, dirname } from "path";
import { Database } from "bun:sqlite";

// Native plist parser using Bun's XML capabilities
function parsePlist(buffer) {
    try {
        const text = buffer.toString('utf8');
        
        // Simple plist parser for binary plists - convert to text format first
        if (text.startsWith('bplist')) {
            // For binary plists, we'll extract key-value pairs manually
            // This is a simplified parser for the common keys we need
            const result = {};
            
            // Look for common patterns in binary plist data
            const dateMatch = buffer.indexOf('date');
            if (dateMatch !== -1) {
                // Extract date value (8 bytes after 'date' key, big-endian double)
                const dateOffset = dateMatch + 4;
                if (dateOffset + 8 <= buffer.length) {
                    const dateView = new DataView(buffer.buffer, buffer.byteOffset + dateOffset, 8);
                    const timestamp = dateView.getFloat64(0, false); // big-endian
                    result.date = timestamp;
                }
            }
            
            // Look for size information
            const sizeMatch = buffer.indexOf('size');
            if (sizeMatch !== -1) {
                // Extract size value
                const sizeOffset = sizeMatch + 4;
                if (sizeOffset + 8 <= buffer.length) {
                    const sizeView = new DataView(buffer.buffer, buffer.byteOffset + sizeOffset, 8);
                    result.size = sizeView.getBigUint64(0, false);
                }
            }
            
            // Look for generator information
            const genMatch = buffer.indexOf('gen');
            if (genMatch !== -1) {
                // Extract generator string
                let genEnd = genMatch + 3;
                while (genEnd < buffer.length && buffer[genEnd] !== 0) genEnd++;
                if (genEnd > genMatch + 3) {
                    result.gen = buffer.subarray(genMatch + 3, genEnd).toString('utf8');
                }
            }
            
            return result;
        }
        
        // For XML plists, use simple XML parsing
        const xmlMatch = text.match(/<plist.*?>(.*?)<\/plist>/s);
        if (xmlMatch) {
            const plistContent = xmlMatch[1];
            const result = {};
            
            // Extract date
            const dateMatch = plistContent.match(/<key>date<\/key>\s*<date>(.*?)<\/date>/);
            if (dateMatch) {
                result.date = new Date(dateMatch[1]).getTime() / 1000;
            }
            
            // Extract size
            const sizeMatch = plistContent.match(/<key>size<\/key>\s*<integer>(.*?)<\/integer>/);
            if (sizeMatch) {
                result.size = parseInt(sizeMatch[1]);
            }
            
            // Extract generator
            const genMatch = plistContent.match(/<key>gen<\/key>\s*<string>(.*?)<\/string>/);
            if (genMatch) {
                result.gen = genMatch[1];
            }
            
            return result;
        }
        
        return {};
    } catch (error) {
        return {};
    }
}

// Native image processing using raw buffer manipulation
async function createPngFromRawRGBA(rawBuffer, width, height, outputPath) {
    try {
        // For simplicity, we'll create a simple PPM file instead of PNG
        // PPM is much simpler to generate and widely supported
        const ppmPath = outputPath.replace('.png', '.ppm');
        
        // PPM header
        const header = `P6\n${width} ${height}\n255\n`;
        const headerBuffer = new TextEncoder().encode(header);
        
        // Convert RGBA to RGB (PPM doesn't support alpha)
        const rgbBuffer = new Uint8Array(width * height * 3);
        for (let i = 0, j = 0; i < rawBuffer.length; i += 4, j += 3) {
            if (i + 3 < rawBuffer.length) {
                rgbBuffer[j] = rawBuffer[i];     // R
                rgbBuffer[j + 1] = rawBuffer[i + 1]; // G
                rgbBuffer[j + 2] = rawBuffer[i + 2]; // B
                // Skip alpha channel (i + 3)
            }
        }
        
        // Combine header and image data
        const totalSize = headerBuffer.length + rgbBuffer.length;
        const ppmBuffer = new Uint8Array(totalSize);
        ppmBuffer.set(headerBuffer, 0);
        ppmBuffer.set(rgbBuffer, headerBuffer.length);
        
        // Write to file
        await Bun.write(ppmPath, ppmBuffer);
        return true;
    } catch (error) {
        console.error(`Error creating image: ${error.message}`);
        return false;
    }
}

// Convert Mac absolute time (seconds from 1/1/2001) to human readable
function convertAbsolute(macAbsoluteTime) {
    try {
        const baseDate = new Date('2001-01-01T00:00:00Z');
        const humanTime = new Date(baseDate.getTime() + (macAbsoluteTime * 1000));
        return humanTime.toISOString();
    } catch (error) {
        return "Error on conversion";
    }
}

// Verify required files exist
function verifyFiles(thumbcacheDir) {
    const indexPath = join(thumbcacheDir, "index.sqlite");
    const thumbnailsPath = join(thumbcacheDir, "thumbnails.data");
    
    if (!existsSync(indexPath)) {
        return `Could not locate the index.sqlite file in the folder ${thumbcacheDir}`;
    }
    
    if (!existsSync(thumbnailsPath)) {
        return `Could not locate the thumbnails.data file in the folder ${thumbcacheDir}`;
    }
    
    return true;
}

// Native argument parsing using Bun.argv
function parseArguments() {
    const args = Bun.argv.slice(2);
    const parsed = {};
    
    for (let i = 0; i < args.length; i++) {
        const arg = args[i];
        
        if (arg === '-h' || arg === '--help') {
            parsed.help = true;
        } else if (arg === '-d' || arg === '--thumbcache-dir') {
            parsed.thumbcacheDir = args[++i];
        } else if (arg === '-o' || arg === '--output-folder') {
            parsed.outputFolder = args[++i];
        } else if (arg === '-t' || arg === '--type') {
            parsed.type = args[++i];
        }
    }
    
    return parsed;
}

// Main database processing function
async function processDatabase(openFolder, saveFolder, outFormat = 'csv') {
    const dbPath = join(openFolder, "index.sqlite");
    const thumbnailsDataPath = join(openFolder, "thumbnails.data");
    
    let thumbnailsFile;
    try {
        thumbnailsFile = readFileSync(thumbnailsDataPath);
    } catch (error) {
        return { error: `Error opening ${thumbnailsDataPath}: ${error.message}` };
    }
    
    let thumbnailsExported = 0;
    
    // Create output directories
    const thumbnailsFolder = join(saveFolder, "thumbnails");
    if (!existsSync(thumbnailsFolder)) {
        mkdirSync(thumbnailsFolder, { recursive: true });
    }
    
    const errorLogPath = join(saveFolder, "error.log");
    const errorLog = createWriteStream(errorLogPath);
    
    let reportFile;
    let reportPath;
    
    if (outFormat === "csv") {
        reportPath = join(saveFolder, "report.csv");
    } else {
        reportPath = join(saveFolder, "report.tsv");
    }
    
    reportFile = createWriteStream(reportPath);
    
    // Write CSV/TSV header with proper escaping for CSV
    const headers = [
        "File Row ID",
        "Folder", 
        "Filename",
        "Hit Count",
        "Last Hit Date",
        "Last Hit Date (UTC)",
        "Has thumbnail",
        "Original File Last Modified Raw",
        "Original File Last Modified(UTC)",
        "Original File Size",
        "Generator",
        "FS ID"
    ];
    
    if (outFormat === "csv") {
        // Escape CSV headers if they contain commas
        const escapedHeaders = headers.map(header => 
            header.includes(',') ? `"${header}"` : header
        );
        reportFile.write(escapedHeaders.join(",") + "\n");
    } else {
        reportFile.write(headers.join("\t") + "\n");
    }
    
    let db;
    try {
        db = new Database(dbPath, { readonly: true });
    } catch (error) {
        return { error: `Error opening database: ${error.message}` };
    }
    
    // Get number of thumbnails
    let totalThumbnails = 0;
    try {
        const thumbnailCountQuery = db.query("SELECT COUNT(*) as count FROM thumbnails");
        const result = thumbnailCountQuery.get();
        totalThumbnails = result.count;
    } catch (error) {
        db.close();
        return { error: `Error executing SQL: ${error.message}` };
    }
    
    // Main query - SQL syntax from http://www.easymetadata.com/2015/01/sqlite-analysing-the-quicklook-database-in-macos/
    const mainQuery = `
        SELECT DISTINCT 
            f_rowid,
            k.folder,
            k.file_name,
            k.version,
            t.hit_count,
            t.last_hit_date,
            t.bitsperpixel,
            t.bitmapdata_location,
            t.bitmapdata_length,
            t.width,
            t.height,
            datetime(t.last_hit_date + strftime('%s', '2001-01-01 00:00:00'), 'unixepoch') AS decoded_last_hit_date,
            k.fs_id
        FROM (
            SELECT rowid as f_rowid, folder, file_name, fs_id, version 
            FROM files
        ) k 
        LEFT JOIN thumbnails t ON t.file_id = k.f_rowid 
        ORDER BY t.hit_count DESC
    `;
    
    let rows;
    try {
        const stmt = db.query(mainQuery);
        rows = stmt.all();
    } catch (error) {
        db.close();
        return { error: `Error executing main query: ${error.message}` };
    }
    
    const totalRows = rows.length;
    
    for (let i = 0; i < rows.length; i++) {
        const row = rows[i];
        const {
            f_rowid: rowid,
            folder,
            file_name,
            version,
            hit_count,
            last_hit_date,
            bitsperpixel,
            bitmapdata_location,
            bitmapdata_length,
            width,
            height,
            decoded_last_hit_date,
            fs_id
        } = row;
        
        let versionLastModifiedRaw = "";
        let versionConvertedDate = "";
        let versionGenerator = "";
        let versionOrgSize = "";
        
        // Parse plist from version field
        if (version) {
            try {
                const plistData = parsePlist(version);
                
                for (const [key, value] of Object.entries(plistData)) {
                    if (key === "date") {
                        const convertedDate = convertAbsolute(value);
                        versionLastModifiedRaw = value.toString();
                        versionConvertedDate = convertedDate;
                    } else {
                        if (key.includes("gen")) {
                            versionGenerator = value.toString();
                        }
                        if (key.includes("size")) {
                            versionOrgSize = value.toString();
                        }
                    }
                }
            } catch (error) {
                errorLog.write(`Error parsing plist for row id ${rowid}: ${error.message}\n`);
            }
        }
        
        // Query for thumbnails for this file
        const thumbnailQuery = `
            SELECT file_id, size, width, height, bitspercomponent, bitsperpixel, 
                   bytesperrow, bitmapdata_location, bitmapdata_length 
            FROM thumbnails 
            WHERE file_id = ?
        `;
        
        let thumbRows;
        try {
            const thumbStmt = db.query(thumbnailQuery);
            thumbRows = thumbStmt.all(rowid);
        } catch (error) {
            errorLog.write(`Error on thumbnails data query for file id ${rowid}: ${error.message}\n`);
            thumbRows = [];
        }
        
        let hasThumbnail = "FALSE";
        
        if (thumbRows.length > 0) {
            hasThumbnail = "TRUE";
            
            for (let thumbIndex = 0; thumbIndex < thumbRows.length; thumbIndex++) {
                const thumb = thumbRows[thumbIndex];
                const countThumb = thumbIndex + 1;
                
                try {
                    const bitspercomponent = thumb.bitspercomponent;
                    const bytesperrow = thumb.bytesperrow;
                    const bitmapDataLocation = thumb.bitmapdata_location;
                    const bitmapDataLength = thumb.bitmapdata_length;
                    
                    // Compute the width from bytes per row
                    const computedWidth = Math.floor(bytesperrow / (bitsperpixel / bitspercomponent));
                    const thumbHeight = thumb.height;
                    
                    // Extract raw bitmap data
                    const rawBitmap = thumbnailsFile.subarray(
                        bitmapDataLocation, 
                        bitmapDataLocation + bitmapDataLength
                    );
                    
                    // Create PNG file
                    const pngFilename = `${rowid}.${file_name}_${countThumb}.png`;
                    const pngPath = join(thumbnailsFolder, pngFilename);
                    
                    if (!existsSync(pngPath)) {
                        try {
                            // Create image from raw RGBA data using native implementation
                            const success = await createPngFromRawRGBA(
                                rawBitmap,
                                computedWidth,
                                thumbHeight,
                                pngPath
                            );
                            
                            if (success) {
                                thumbnailsExported++;
                            }
                        } catch (imageError) {
                            errorLog.write(`Error creating image for row id ${rowid}: ${imageError.message}\n`);
                        }
                    }
                } catch (error) {
                    errorLog.write(`Error with thumbnail for row id ${rowid}: ${error.message}\n`);
                }
            }
        }
        
        // Write to report
        if (reportFile) {
            const reportData = [
                rowid,
                folder || "",
                file_name || "",
                hit_count || "",
                last_hit_date || "",
                decoded_last_hit_date || "",
                hasThumbnail,
                versionLastModifiedRaw,
                versionConvertedDate,
                versionOrgSize,
                versionGenerator,
                fs_id || ""
            ];
            
            if (outFormat === "csv") {
                // Escape CSV fields if they contain commas, quotes, or newlines
                const escapedData = reportData.map(field => {
                    const fieldStr = String(field);
                    if (fieldStr.includes(',') || fieldStr.includes('"') || fieldStr.includes('\n')) {
                        return `"${fieldStr.replace(/"/g, '""')}"`;
                    }
                    return fieldStr;
                });
                reportFile.write(escapedData.join(",") + "\n");
            } else {
                reportFile.write(reportData.join("\t") + "\n");
            }
        }
    }
    
    // Clean up
    db.close();
    if (reportFile) reportFile.end();
    errorLog.end();
    
    return {
        totalRows,
        totalThumbnails,
        thumbnailsExported
    };
}

// Command line interface
async function main() {
    const args = parseArguments();
    
    if (args.help) {
        console.log(`
QuickLook Parser - JavaScript/Bun Port
Usage: bun run extract.js [options]

Options:
  -d, --thumbcache-dir   Path to com.apple.QuickLook.thumbnailcache folder
  -o, --output-folder    Path to empty folder to hold report and thumbnails  
  -t, --type            Output format (csv or tsv) [default: csv]
  -h, --help            Show this help message

Example:
  bun run extract.js -d "/Users/user/Library/Caches/com.apple.QuickLook.thumbnailcache" -o "./output"
        `);
        process.exit(0);
    }
    
    const thumbcacheDir = args.thumbcacheDir;
    const outputFolder = args.outputFolder;
    const outFormat = args.type || 'csv';
    
    if (!thumbcacheDir) {
        console.error("Error: -d THUMBCACHE_DIR argument required");
        process.exit(1);
    }
    
    if (!outputFolder) {
        console.error("Error: -o OUTPUT_FOLDER argument required");
        process.exit(1);
    }
    
    // Verify files exist
    const verification = verifyFiles(thumbcacheDir);
    if (verification !== true) {
        console.error("Error:", verification);
        process.exit(1);
    }
    
    // Create output folder if it doesn't exist
    if (!existsSync(outputFolder)) {
        mkdirSync(outputFolder, { recursive: true });
    }
    
    console.log("Processing QuickLook database...");
    
    try {
        const stats = await processDatabase(thumbcacheDir, outputFolder, outFormat);
        
        if (stats.error) {
            console.error("Error:", stats.error);
            process.exit(1);
        }
        
        console.log("Processing Complete");
        console.log(`Records in table: ${stats.totalRows}`);
        console.log(`Thumbnails available: ${stats.totalThumbnails}`);
        console.log(`Thumbnails extracted: ${stats.thumbnailsExported}`);
        
    } catch (error) {
        console.error("Unexpected error:", error.message);
        process.exit(1);
    }
}

// Run the main function
if (import.meta.main) {
    main().catch(console.error);
}
Enter fullscreen mode Exit fullscreen mode

Comments 0 total

    Add comment