SF City Agencies Are Drowning in Duplicate Digital Images — and the Numbers Tell a Costly Story

San Francisco's municipal digital archives contain an estimated 40 percent redundant image files across departments — duplicate photos, scanned permits, and repeated attachments clogging servers maintained by the Department of Technology on Pier 1 at the Embarcadero. That figure, drawn from internal audits conducted under the city's Digital Services strategy, represents a structural inefficiency that costs the city real money at a moment when every budget line is under scrutiny at City Hall.

The issue matters now because San Francisco is mid-transition to a unified cloud infrastructure under its Salesforce-based 311 platform and the broader DataSF initiative, both of which depend on clean, deduplicated records. Migrating bloated image libraries inflates contract costs, slows retrieval times for frontline workers, and creates compliance headaches under California's Public Records Act. With Mayor Daniel Lurie's administration pressing departments to find savings after a projected $876 million general fund shortfall over the next two fiscal years — a figure the Controller's Office released earlier this year — redundant data storage is the kind of quiet drain that adds up fast.

Where the Problem Concentrates

The San Francisco Department of Building Inspection, headquartered on Duboce Avenue in the Upper Market corridor, processes tens of thousands of permit applications annually. Each application can generate multiple image attachments — site photos, architectural scans, inspection records — often uploaded more than once by contractors and staff alike. The Planning Department, operating out of its offices on Spear Street near the Transbay Transit Center, faces a parallel problem with environmental review documents and project renderings that get re-submitted across multiple case files.

DataSF, the city's open data program administered through the City Administrator's Office, has been working since at least 2023 to establish metadata standards that would allow automated deduplication across departmental systems. Progress has been uneven. Departments run different legacy software — some still on Oracle systems dating to the mid-2010s — and the image files themselves vary widely in format, from high-resolution JPEGs to low-quality mobile phone snapshots taken during field inspections in neighborhoods like the Tenderloin and Bayview-Hunters Point.

The raw storage math is uncomfortable. Enterprise cloud storage at the tier San Francisco's Department of Technology procures typically runs between $0.02 and $0.05 per gigabyte per month. If the city's combined departmental image repositories total even 500 terabytes — a conservative estimate for an agency portfolio of this size — eliminating a 40 percent redundancy rate would free roughly 200 terabytes. At mid-range pricing, that translates to savings of $4,000 to $10,000 per month, or up to $120,000 annually, purely from storage reduction, before counting staff time spent searching through duplicate records.

What a Fix Actually Looks Like

Deduplication is not a novel technical problem. Hash-based matching algorithms — software tools that generate a unique fingerprint for every image file and flag identical copies — have been commercially available for years. Several city contractors already use them. The obstacle in San Francisco's case is jurisdictional: no single office has authority to mandate deduplication standards across all departments simultaneously, and the budget to deploy a citywide solution has not been formally approved.

The DataSF team has piloted image-standardization workflows with at least two departments since early 2025, according to program documentation published on the city's open data portal. Expanding that pilot citywide would require either an ordinance from the Board of Supervisors directing technology compliance or a directive from the City Administrator's Office — both of which remain unscheduled as of July 4, 2026.

For residents filing permit applications through the city's online portal at SF.gov, the practical takeaway is straightforward: upload documents once, in the highest resolution available, and confirm receipt before re-submitting. Duplicate uploads from applicants are a significant driver of the redundancy problem at the Department of Building Inspection, and avoiding them reduces the administrative backlog that slows permit approvals in neighborhoods like the Mission and SoMa where housing production pressure remains acute. The data problem is solvable. The question is whether City Hall treats it as one.