SF City Agencies Push to Purge Duplicate Images From Public Records Systems, and Officials Are Taking It Seriously

San Francisco's municipal technology offices are confronting a problem hiding in plain sight: tens of thousands of duplicate images clogging the city's public-facing document and permitting systems, and officials say the cleanup is long overdue. The Department of Technology, headquartered at 1 South Van Ness Avenue, confirmed earlier this year that a citywide audit of digital asset libraries had flagged redundant image files across at least six major departmental databases, creating retrieval bottlenecks and inflating cloud storage costs.

The issue landed on the agenda of the city's Committee on Information Technology, which oversees digital infrastructure for roughly 60 departments, in the spring of 2026. With the city's tech budget under renewed scrutiny after several years of deficit pressure, duplicate data has become a line-item concern rather than a back-office nuisance.

Why This Matters Right Now

San Francisco spends millions annually on cloud data contracts — and storage bloat driven by redundant files is one of the less glamorous drains on that spending. City technologists briefed members of the Budget and Appropriations Committee in March 2026 on the scope of the problem, describing image duplication rates in some legacy permit systems running above 30 percent of total stored files. The SF Planning Department's permit portal, which serves contractors and property owners across neighborhoods from the Richmond District to Bayview-Hunters Point, carries photo attachments going back to digitization efforts that began in the early 2010s. Many of those files were uploaded multiple times by applicants who hit system errors and resubmitted.

SFMTA's asset management database, which tracks everything from signal infrastructure on Market Street to station hardware at Embarcadero, has faced similar problems. Engineers who upload field inspection photos from multiple devices sometimes generate parallel file records for the same image, a known flaw in older asset-management software the agency has been working to replace since 2023.

Across San Francisco's public hospital system, including Zuckerberg San Francisco General Hospital on Potrero Avenue, medical imaging systems use entirely separate deduplication protocols required under federal standards — but administrative and facilities-side photo records operate under no such mandate, leaving room for the same redundancy problems found elsewhere in city government.

What Experts and Officials Are Recommending

Technology advisers brought in by the Department of Technology have pointed to several approaches already in use by peer cities. Perceptual hashing — a technique that identifies visually identical or near-identical images even if their file names or metadata differ — is the method most frequently cited in briefing materials reviewed by The Daily San Francisco. Unlike simple file-comparison tools, perceptual hashing can catch images resized or reformatted before re-upload, which is common in permit workflows.

Urban planning technologists and open-government advocates who follow San Francisco's digital infrastructure say the city's relatively late adoption of unified content management across departments has made this harder to fix quickly. The Controller's Office Citywide Data Warehouse initiative, launched in fiscal year 2024-25, was designed partly to address exactly this kind of siloed data problem, but image deduplication was not initially in scope for the first phase of that project.

Some city commissioners have urged the Department of Technology to prioritize the Planning Department's systems first, given that permit-related slowdowns have a direct impact on housing production — a politically charged issue in a city trying to meet its state-mandated Regional Housing Needs Allocation targets. San Francisco is required to plan for roughly 82,000 new homes by 2031 under the current RHNA cycle, and any friction in the permitting pipeline draws attention at City Hall.

For residents and contractors who interact with city systems, the practical advice from city IT staff, as communicated through the SF Digital Services office, is straightforward: upload documents once, use the system's confirmation screen to verify receipt before resubmitting, and contact the help desk at 415-701-2311 if a file appears to have failed. The city says it expects to roll out automated deduplication tools in the Planning Department's e-permit portal by the end of calendar year 2026, with SFMTA's asset database following in early 2027.