Dev Tools · 3h ago
Microsoft's MarkItDown slashes LLM token waste from PDFs
Uploading PDFs to LLMs wastes tokens on formatting noise, with a single page costing 1,500–3,000 tokens. Microsoft's open-source MarkItDown tool converts PDFs and other files to clean Markdown, reducing token usage by up to 70%. The tool has over 119,000 GitHub stars and also handles Word, Excel, images, and YouTube links.
Meridian48 take
MarkItDown is a practical fix for a costly inefficiency, but its real value lies in how it aligns document structure with LLM training data.
llm-token-optimizationmarkdown-conversion