SF City Hall's Duplicate Image Problem: What Officials and Experts Are Saying About the Digital Records Backlog

San Francisco's Department of Technology is facing pointed questions this week about a years-long failure to address duplicate digital images clogging the city's records management infrastructure, a problem that archivists and open-government advocates say is costing taxpayers in wasted cloud storage contracts and slowing public records requests across multiple departments.

The issue has been simmering since at least 2023, when the city's Controller's Office flagged redundant file storage as a line-item concern during a broader audit of the city's data infrastructure spending. Cloud storage contracts held through the Department of Technology run through vendors including Microsoft Azure and Amazon Web Services, and unnecessary duplication of scanned permit documents, police incident reports, and planning files inflates the volume — and the bill — of what the city pays to store them. No official cost figure has been released publicly for the duplicate-image portion specifically.

Why It Matters Now

The timing is pointed. San Francisco is under intense pressure to find budget savings after a deficit that, according to the Controller's Office's Five-Year Financial Plan published in December 2025, is projected to reach roughly $800 million by fiscal year 2028. Every department is being asked to identify redundancies. Digital records hygiene — long treated as a back-office nuisance — is now a line item that budget analysts are actually scrutinizing.

The San Francisco Planning Department, which processes permit applications for projects stretching from the Tenderloin to the Bayview, confirmed earlier this year that it had begun a review of its electronic document management system, Accela, after staff flagged duplicate scan uploads as a recurring workflow problem. The Department of Building Inspection, headquartered on Seventh Street near Market, runs a parallel permitting database that archivists say compounds the redundancy when the two systems exchange files without automated deduplication protocols in place.

Experts in municipal records management — including staff at the California State Archives in Sacramento, which advises local governments on retention schedules — have long recommended hash-based deduplication tools as a low-cost fix. The technology identifies identical files by generating a unique fingerprint for each image, flagging exact copies before they consume additional storage. Several Bay Area municipalities, including Oakland and San José, have implemented versions of these tools inside their Laserfiche document management installations within the past three years.

What Key Figures Are Saying

At City Hall, the conversation has spilled into the Budget and Appropriations Committee, where supervisors pressed Department of Technology staff during a June hearing about whether the city had a formal policy for image deduplication. Department representatives, according to meeting minutes published on the Board of Supervisors website, acknowledged there was no citywide standard and said a working group had been formed to draft one by the end of calendar year 2026.

Open-government advocates at the San Francisco chapter of the League of Women Voters have raised a separate concern: that duplicate records slow the city's responses to California Public Records Act requests, because staff must manually sort through redundant files before releasing documents. Under state law, agencies have 10 days to respond to records requests, with extensions available — but bottlenecks inside document management systems are a practical obstacle that legal deadlines alone cannot fix.

Archivists at the San Francisco History Center, located inside the Main Library on Larkin Street in the Civic Center neighborhood, say the problem is not new. Physical records scanning projects from the early 2010s, many conducted under federal digitization grants, produced duplicate TIFF files that were never reconciled. Those legacy files are now mixed into modern cloud storage buckets alongside current documents.

The Department of Technology's working group is expected to present draft deduplication standards to the City Administrator's Office by December 31, 2026. In the meantime, individual departments have been encouraged to conduct their own file audits using existing IT staff. Budget analysts say any departments that complete audits before the next fiscal year begins in July 2027 could see storage-line savings reflected in their FY2028 budget proposals — a modest but real incentive for bureaucracies not known for moving fast on housekeeping.