Conquering Outlook MAPI & The Zero-Data-Loss Standard
The comprehensive story, technical audit, and modular architecture of mail2mail — engineered by Andrew Paolucci to cure decades of broken Outlook COM automation and provide bit-exact email conversions.
The Exhaustive Upstream Fork Audit
How inspecting Rick Madson's original open-source tool and seven community forks led to the birth of mail2mail.
In late 2026, Andrew Paolucci needed to migrate over 36 gigabytes of disparate corporate and personal email archives into clean, year-partitioned Microsoft Outlook PST files.
The search for an open-source solution led to madsonrick/mbox-to-pst-toolkit. While promising, the repository had been inactive for years and was surrounded by a fragmented web of community forks.
Andrew undertook a forensic audit of every major fork in the network:
Guillaume Bordier (@gbordier) • PR #4
First identified that Outlook COM Items.Add(0) sets MSGFLAG_UNSENT, causing converted messages to display script run dates. Introduced the first basic MAPI bitmask clearing.
HTCGS (@HTCGS) • Master Branch
Identified the requirement to stamp PR_CREATION_TIME (0x30070040) and PR_LAST_MODIFICATION_TIME for Outlook column sorting, alongside timezone stripping for COM marshaling.
enthusedamateur • Bugfixes Branch
Discovered a silent data loss bug during periodic PST buffer flushes where detaching stores leaked COM pointers, and resolved header crashes during Subject sanitization.
dianlight • PR #1
Demonstrated Google Takeout .zip extraction and basic directory-level batch MBOX scanning.
icmaa • Main Branch
Demonstrated recursive nested folder hierarchy generation from source directory paths during PST import.
Erzmaster • PR #2
Patched regex filename crashes when Subject headers contained RFC 2047 encoded words.
While each fork solved one isolated issue, none solved the complete problem: unrenderable MIME parts were still silently lost, large attachments crashed Outlook, and 32-bit COM automation routinely ran out of memory.
mail2mail synthesized these discoveries into a unified, hardened codebase.
The Four Fatal MAPI Bugs Cured
Resolving the undocumented traps of Microsoft Outlook's local COM interface.
1. The MSGFLAG_UNSENT "Draft Bug"
When Outlook creates a message item via COM automation, it automatically assigns MSGFLAG_UNSENT (0x08). This caused all imported emails to appear as unread compose drafts stamped with the current execution time instead of their real historical dates.
Solution: Bitmask clearing on PR_MESSAGE_FLAGS (0x0E070003) & PR_CREATION_TIME injection.
2. The PST Flush Corruption Bug
To force Windows Explorer to update the displayed file size of an active PST, scripts would detach and re-attach the store. Community forks contained a critical bug where detaching left the router holding detached COM pointers, silently corrupting or deleting active PST volumes.
Solution: Atomic handle retention in PstRouter.
3. The 20 MB Attachment Trap
Classic Outlook desktop enforces a hard 20–25 MB attachment threshold, throwing COM error 4096. mail2mail introduced automated Windows registry unblocking (MaximumAttachmentSize = 0 in HKCU) and emergency unattached spools so no file is ever lost.
Solution: Automated HKCU registry unblocking.
4. 32-Bit Memory Leaks & RPC Rejections
Walking 50,000+ messages routinely triggered RPC_E_CALL_REJECTED COM errors and memory exhaustion. mail2mail integrated @com_retry backoff decorators, proactive garbage collection, and explicit CoFreeUnusedLibraries calls.
Solution: CoFreeUnusedLibraries & @com_retry.
The mail2mail Codebase Architecture
Clean separation of format sniffing, streaming ingestion, export routing, and state persistence.
mail2mail.py
Unified master entry point CLI that auto-detects targets or routes to eml, mbox, or pst subcommands.
mail2eml.py
High-speed exporter generating .eml files with folder, year, month, or flat layout options and index/subject naming.
mail2mbox.py
Standard RFC 4155 / mboxrd exporter with byte-exact >From line escaping and compressed .mbox.gz streaming.
mail2pst.py
MAPI PST router supporting year splitting, N-way splitting, and capacity auto-rotation (--max-pst-gb 15).
mail_formats.py
Shared format sniffer and streaming generator decoding 10+ formats, IMAP Modified UTF-7, and Apple Mail plist bitmasks.
mail_shutdown.py
Interruption controller coordinating graceful stops on SIGINT (Ctrl+C), interactive terminal pause/resume (p/Space), and exit code 130.
Incremental Delta Sync via SQLite WAL (--state-file)
For large cutovers, mail2mail tracks processed Message-IDs and SHA-256 digests in a persistent SQLite Write-Ahead Logging (WAL) database.
Pass 1 performs historical migration; Pass 2 performs cutover delta syncing by fast-skipping already imported items in $O(1)$ time.
Supported Email & Mailbox Formats
Automatic detection, flag mapping, and streaming parsing across all major formats.
| Format | Input Sources | Metadata & Status Preserved |
|---|---|---|
| Standard MBOX | .mbox files, extensionless mailboxes |
X-Mozilla-Status, Status, >From un-escaping |
| Thunderbird ImapMail | Account roots, .sbd folder trees |
Read, Replied, Flagged, IMAP Modified UTF-7 |
| Maildir / Maildir++ | cur/, new/, tmp/ directories |
:2,FRSPT and Windows ;2,FRSPT flags |
| Apple Mail EMLX | .emlx, macOS *.mbox bundles |
XML <plist> bitmasks (Read, Flagged, Replied) |
| Outlook .msg | .msg files, OLE2 Compound Files |
MS-CFB / MS-OXMSG parser extracting MAPI tags |
| Outlook for Mac .olm | .olm ZIP archives |
com.microsoft.__Messages XML hierarchies |
| Outlook .pst | .pst Personal Folders |
MAPI folder trees, RFC 5322 MIME reconstruction |
| MH / Claws / Sylpheed | Numbered message files (1, 2, 3) | .mh_sequences flag translation |
| Archives (ZIP/GZ/TGZ) | .zip, .tar.gz, .mbox.gz |
Direct streaming extraction without disk extraction |
Command-Line Examples
Common conversion workflows with mail2mail.
python mail2mail.py pst "C:\Users\User\AppData\Roaming\Thunderbird\...\ImapMail" \
"D:\Migration\01_Output_PST" --split-by year --max-pst-gb 15 \
--audit-log "audit.csv" --resumepython mail2eml.py "D:\Archive\Takeout.zip" "D:\Migration\EML_Export" \
--layout folder --naming index --max-per-dir 50000python mail2mbox.py "Archive.olm" "Archive.mbox.gz" --layout folderContributors & Acknowledgments
Every engineer, community fork author, and AI assistant who made mail2mail possible.
Andrew Paolucci (@adpaolucci)
Project creator, architect, and lead maintainer. Directed the unified CLI suite, established the Zero-Data-Loss Standard, and cured the Outlook MAPI draft bugs.
Antigravity (Google DeepMind)
Agentic AI engineering collaborator. Implemented MAPI draft flag resolution, zero-data-loss fallback spools, Takeout X-Gmail-Labels parser, and deduplication engine.
Argon 4 (Anthropic)
Predecessor AI collaborator. Authored the initial architectural foundation, multi-format streaming engine, and comprehensive RFC compliance specifications.
Rick Madson (@madsonrick)
Author of the original upstream mbox-to-pst-toolkit demonstrating early Outlook MAPI COM automation from Python.