How San Francisco's City Agencies Ended Up Buried in Duplicate Digital Images — And What It's Costing Taxpayers

San Francisco's Department of Technology is sitting on a problem that didn't arrive overnight. Across the city's sprawling network of municipal databases — from the Planning Department's permit portals on South Van Ness Avenue to the Department of Public Works asset management systems — duplicated digital images have accumulated for more than a decade, inflating storage costs, slowing case-processing workflows, and complicating public-records requests that residents and attorneys file daily at City Hall.

The issue matters urgently now because the city is in the middle of a sweeping cloud migration, a multi-year contract with state-certified vendors aimed at consolidating legacy infrastructure that dates, in some cases, to the early 2000s. Redundant image files don't just cost money in cloud storage fees — they degrade the accuracy of machine-learning tools the city has been piloting to speed up housing permit reviews, a pressure point that Planning Director Rich Hillis has publicly acknowledged as a drag on the city's housing production targets.

How the Backlog Built Up

The roots stretch back to at least 2012, when individual departments began digitizing paper records independently, with no citywide file-naming convention and no deduplication standard. The San Francisco Public Library's digitization program, the Department of Building Inspection's permit imaging workflow, and the Municipal Transportation Agency's infrastructure photo archives each developed their own protocols. Files shot on-site in the Tenderloin or the Bayview were uploaded in multiple formats — JPEG, TIFF, PDF-embedded raster — sometimes by field inspectors using different mobile apps, sometimes by contractors billing hourly for scanning work.

The COVID-19 pandemic accelerated the mess. Between March 2020 and December 2021, city departments rushed to digitize paper backlogs so staff working remotely could access records. The Controller's Office later estimated in a fiscal audit summary that city departments collectively added storage capacity at roughly three times the pre-pandemic rate during that period. Deduplication was not a priority when offices were scrambling to keep services running with skeleton crews.

By fiscal year 2023-24, the Department of Technology's budget included a line item exceeding $4 million annually for cloud and data-center storage across municipal agencies — a figure drawn from the city's published budget documents. Technology staff have described the image duplication problem internally as one of several factors driving that number upward, though the precise share attributable to redundant files has not been publicly broken out.

The Cleanup Effort and What Comes Next

In late 2025, the city awarded a data-governance contract to a vendor through the standard competitive-bid process managed by the Office of Contract Administration on Polk Street. The scope includes an audit of image repositories across at least seven departments, automated deduplication using hash-matching tools, and the establishment of a citywide digital-asset taxonomy that departments would be required to follow going forward. The contract runs through June 2027.

The Planning Department's Permit Center, which relocated to 49 South Van Ness Avenue in 2021, is among the first agencies scheduled for the audit. Staff there process thousands of permit applications per year, many of which require attaching site photographs, architectural drawings, and inspection images. Duplicate files in that pipeline have caused delays in automated pre-screening tools the department uses to flag incomplete applications — pushing more files into manual review queues.

For residents, the practical upshot is straightforward: if the deduplication effort succeeds on schedule, permit-processing times for routine projects — accessory dwelling units, seismic retrofits, minor façade work — could improve as the automated tools become more reliable. Housing advocates at organizations including the San Francisco Housing Action Coalition have long argued that permitting speed is one of the few levers city government can pull without state legislation. Cleaning up the underlying data infrastructure is, unglamourously, part of that work.

The Department of Technology has not publicly released interim findings from the vendor audit. A status report is expected before the Board of Supervisors' Government Audit and Oversight Committee before the end of calendar year 2026.