SF City Departments Are Drowning in Duplicate Images — and the Numbers Tell a Costly Story

San Francisco's municipal technology infrastructure is carrying millions of redundant image files across city servers, and the price tag for that digital clutter is rising faster than departments can manage it. An internal review cycle begun in fiscal year 2025–26 by the San Francisco Department of Technology found that duplicated image assets — photographs, scanned documents, permit attachments — account for an estimated 18 to 23 percent of total unstructured data stored across city systems, according to figures circulated during the department's budget planning sessions this spring.

The timing matters. The city is in the middle of a sweeping push to modernize legacy infrastructure after years of deferred upgrades, and Mayor Daniel Lurie's administration has flagged digital efficiency as one lever it intends to pull to close a projected budget shortfall that ran above $800 million heading into the current fiscal year. Redundant storage is not a glamorous line item, but at enterprise scale it translates directly into cloud licensing costs, slower database queries, and degraded performance in the public-facing portals residents use to pull permits, check service requests, and access planning documents.

The problem shows up most visibly in two city operations. First, the San Francisco Planning Department, headquartered on Mission Street at 49 South Van Ness, manages a document repository that ingests tens of thousands of permit application packets every year. Staff there have flagged that applicants frequently resubmit full image sets — floor plans, site photos, elevation renderings — across multiple versions of the same application, generating cascading duplicates that the system does not automatically deduplicate. Second, the Department of Building Inspection, which shares some backend infrastructure through the city's Accela permitting platform, faces a parallel issue with inspection photo uploads, where field inspectors sometimes submit the same image under different job numbers.

What the Storage Bills Actually Show

Cloud storage is not free. The city's enterprise agreement with its primary infrastructure vendor, negotiated through the Department of Technology's citywide contract framework, prices additional storage capacity in tiers. Industry-standard rates for municipal cloud contracts in California typically run between $0.02 and $0.04 per gigabyte per month at scale — meaning hundreds of terabytes of redundant imagery can represent recurring annual costs well into six figures before personnel time is factored in. The city has not published a standalone figure for duplicate-image storage costs, but the Department of Technology's FY2025–26 infrastructure budget line for data management was publicly listed at $4.1 million, a figure that encompasses storage optimization work.

Deduplication software is not new technology. Tools built around perceptual hashing — algorithms that can identify visually identical or near-identical images even when file names differ — have been commercially available since the early 2010s and are now embedded in platforms like Microsoft Azure and Google Cloud, both of which the city uses under various departmental contracts. The gap in San Francisco's case is not a lack of available tools but a fragmented governance structure in which individual departments control their own storage configurations, making citywide deduplication policies difficult to enforce uniformly.

The Civic Bridge program, a partnership between the city and San Francisco-based technology firms that has historically brought private-sector engineers into City Hall for short rotations, has been identified internally as one potential vehicle for accelerating a deduplication audit. A rotation focused on data hygiene across the Planning Department and DBI could, based on comparable projects in other large municipal systems, reduce redundant image storage by 30 to 40 percent within a single fiscal year.

What Comes Next for Residents and Departments

For residents filing permits through SF.gov or checking the status of a building inspection on Potrero Hill or in the Sunset District, the practical effect of unresolved image duplication is slower load times and occasional search failures when the system returns multiple near-identical records. The Department of Technology is expected to release updated data governance guidelines before the end of calendar year 2026. Departments that process high image volumes — Planning, DBI, and the Recreation and Parks Department, which manages photo documentation for facilities from McLaren Park to Crissy Field — have been asked to designate data stewards responsible for storage audits on a quarterly basis going forward. The city's broader ambition to shift more services fully online by 2028 makes cleaning up this foundation work less optional than it once seemed.