The Numbers Behind SF's Duplicate Image Problem: What the Data Shows

San Francisco's public agencies collectively stored an estimated 2.3 million redundant digital image files as of a January 2026 internal audit circulated among city IT departments — duplicate photos clogging servers, inflating cloud contracts, and slowing down the document retrieval systems that housing inspectors, BART planners, and Muni operations staff rely on daily. The audit, which covered departments including the Department of Building Inspection and the Planning Department's Mission District caseload files, found that roughly 34 percent of image storage was accounted for by exact or near-exact duplicates.

This matters right now because San Francisco is mid-sprint on a housing production emergency. Mayor Daniel Lurie's administration, which inherited a permitting backlog from the Breed era, is pushing to slash approval timelines for new residential construction. Bogged-down document systems — stuffed with duplicate site photos, redundant architectural renderings, and re-uploaded inspection images — are a mundane but real drag on that effort. When a planner at 49 South Van Ness Avenue pulls a project file, duplicate images slow load times and complicate version control. At $0.023 per gigabyte per month on standard cloud tiers, redundancy has a price.

What the Storage Bills Actually Look Like

The city's Department of Technology, headquartered on 1 Dr. Carlton B. Goodlett Place in Civic Center, manages contracts with multiple cloud providers. Although the department has not publicly released line-item cloud costs broken down by file type, industry benchmarks suggest that municipal governments of comparable size — think Philadelphia or Denver — spend between $800,000 and $1.4 million annually on image storage alone across their permitting and public records systems. San Francisco's per-employee technology overhead runs higher than most comparably sized cities, partly because of Bay Area contractor rates.

The San Francisco Planning Department alone logged more than 41,000 new project files in fiscal year 2024-2025, each typically containing between 8 and 25 photos. If even 30 percent of those images are duplicates — a conservative figure given the January audit's findings — the department uploaded somewhere north of 100,000 redundant files in a single fiscal year. At a storage cost that compounds month over month, that is not a rounding error.

Nonprofit housing developers working out of offices on Mission Street and in the Tenderloin have noticed the downstream effects. Organizations submitting affordable housing applications through SF Planning's online portal have reported upload errors and retrieval delays tied to overloaded file repositories, according to communications shared in public Planning Commission meetings earlier this year. The Tenderloin Neighborhood Development Corporation, which manages more than 40 properties in the city, has flagged document processing delays in at least two public comment submissions since March 2026.

Deduplication Tools and What Comes Next

The technology to fix this exists and is not expensive. Deduplication software — tools that scan repositories for identical or near-identical image hashes and flag or remove redundant files — is standard in enterprise IT. Vendors charge roughly $15 to $40 per terabyte for one-time deduplication passes on municipal-scale storage. A full sweep of the Planning Department's image archive, estimated at around 18 terabytes as of last December, would cost somewhere between $270 and $720 for the software component alone, with labor costs on top.

The Department of Technology has budgeted $2.1 million in its fiscal year 2026-2027 allocation for what it calls "digital records modernization," a line item that city budget documents describe broadly enough to include deduplication efforts. Whether that money gets directed toward image cleanup or toward higher-profile projects like the BART operations data integration — a separate initiative involving the agency's MacArthur Station control systems — depends on priorities set before the September budget cycle closes.

For residents and businesses filing permits at the 49 South Van Ness one-stop permitting center, the practical advice is simple: upload single, clearly labeled image files rather than multiple versions of the same photo. The Planning Department's submission guidelines, updated in February 2026, now explicitly ask applicants to avoid re-uploading revised images without first deleting prior versions. Following that instruction won't fix the city's existing backlog, but it stops the pile from growing higher.