How San Francisco's Public Records Offices Ended Up Drowning in Duplicate Images — And What Comes Next

San Francisco's municipal recordkeeping system is carrying a problem that nobody designed but everyone inherited: hundreds of thousands of duplicate digital images stored across incompatible systems in at least a dozen city departments, from the Department of Building Inspection on South Van Ness Avenue to the City Assessor-Recorder's office at City Hall. The city's Department of Technology has begun a formal audit of the redundancy problem, according to public budget documents filed in May 2026, and what it found was not pretty.

The scale matters because the city is not operating in isolation. San Francisco has spent the past three years under pressure to modernize its public-facing digital infrastructure — accelerated by the post-pandemic pivot to online permitting, the Breed administration's push to streamline housing approvals, and the AI-sector boom in SoMa and Mission Bay that brought tens of thousands of new workers into the city and, with them, demands for faster government services. Every bottleneck in the back office eventually shows up at the public counter.

How the Backlog Built Up

The root cause is not carelessness — it is architectural. Over roughly two decades, individual city agencies built or purchased their own document management platforms without a unifying standard. The Planning Department on Mission Street runs one system. The Department of Public Works uses another. The San Francisco Public Library's preservation division, based at the main branch on Larkin Street in the Civic Center neighborhood, maintains a third for digitized historical materials. When a permit, map, or inspection report crossed departmental lines — which happens constantly — clerks often re-scanned or re-uploaded the same image rather than retrieve it from a sister system they couldn't easily access.

A 2024 report by the San Francisco Controller's Office found that city departments collectively spent approximately $14.7 million annually on digital storage infrastructure, a figure that budget analysts noted was growing at roughly 11 percent per year. The Controller's team flagged image duplication as a contributing factor but stopped short of quantifying it precisely. That job fell to the Department of Technology audit now underway.

The problem became acute during the 2021-to-2023 period, when the city's online permitting portal — launched in a rush to handle pandemic-era housing applications — accepted uploaded files without deduplication logic. Contractors and property owners resubmitting corrected drawings sometimes uploaded full image sets rather than individual corrected pages, and the system stored everything. By some internal estimates, as much as 30 percent of storage in the permitting system alone consists of functionally identical files, though that figure has not been independently verified and the Department of Technology has not yet released audit findings publicly.

What the City Is Doing About It

The Department of Technology issued a request for proposals in March 2026 for a deduplication and records consolidation platform, with responses due back in late April. The procurement process is now in evaluation. City vendors already working in the space include firms with contracts through the Mayor's Office of Housing and Community Development, though no award has been announced as of July 4, 2026.

The Assessor-Recorder's office, which manages property deed images and transfer documents going back to the 1850s, has separately partnered with the San Francisco History Center at the Main Library on a pilot project to standardize metadata tagging — a necessary precondition before any deduplication algorithm can reliably determine whether two files are truly identical or merely similar.

For residents and small-business owners who use the city's online portal at SF.gov, the practical consequence of a cleaner system would be faster search results and fewer instances of misfiled records pulling up wrong documents on property lookups. Housing advocates at organizations like the Tenderloin Housing Clinic on Turk Street, who rely on permit histories to challenge code violations, have complained for years about unreliable document retrieval.

The Department of Technology audit results are expected before the Board of Supervisors' fall budget hearings, giving supervisors a data-grounded basis to decide whether to fund a full remediation contract or accept a phased approach that addresses only the highest-volume departments first. Either way, the problem now has a price tag attached to it — and city officials can no longer credibly describe it as a minor housekeeping matter.