San Francisco Is Quietly Overhauling How City Records Handle Duplicate Images — and It's Ahead of Most Peers

San Francisco's Department of Technology confirmed this spring that a multi-year audit of the city's public-facing digital archives had uncovered thousands of duplicate image files embedded in permit records, environmental impact reports, and community planning documents — many of them misfiled or pulling incorrect metadata going back to at least 2019. The cleanup effort, folded into the city's broader Digital Services modernization push, is now being held up internally as a test case for municipal data hygiene at a moment when AI-assisted search tools are making bad image data much harder to hide.

The timing matters. San Francisco's Planning Department, headquartered on Mission Street, has spent the past two years digitizing a backlog of paper records under pressure from the state's housing production mandates. That sprint created fertile ground for duplicate image replacement problems: the same site photo, scanned twice under different file names, can attach itself to two separate project files, skewing automated compliance checks and creating confusion for developers, neighbors, and appellants alike. Meanwhile, the city's 311 portal — which fields tens of thousands of service requests monthly — has faced its own version of the problem, with street-condition images duplicated across tickets for the same pothole or broken sidewalk on, say, Valencia Street in the Mission.

How SF Compares to London, Singapore, and Amsterdam

Other major cities have hit the same wall. London's equivalent struggle played out inside the Greater London Authority's planning portal, where a 2024 records review flagged duplicate attachments on roughly 12 percent of submitted planning applications, according to a report published by the GLA's Digital Team that year. Amsterdam's municipal archive, the Stadsarchief, completed a de-duplication project in 2023 that removed more than 40,000 redundant image records from its public database — a figure city archivists there described in published documentation as larger than anticipated. Singapore's Urban Redevelopment Authority has taken the most systematic approach of any comparable city, embedding hash-based image fingerprinting directly into its eDeveloper submission portal since 2022, meaning duplicate uploads are flagged and rejected at the point of entry rather than discovered years later during an audit.

San Francisco is not there yet, but it is closer than most American cities. The Department of Technology's current protocol relies on a combination of perceptual hashing software and manual review by a small team based at City Hall. The city has not published a full accounting of how many records were affected, but internal briefings described to The Daily San Francisco suggest the audit covered records dating back seven years. The San Francisco Public Library's San Francisco History Center on Larkin Street, which maintains its own parallel archive of civic images, completed a similar de-duplication exercise in 2024 and reported a smoother process partly because its collections management software — PastPerfect Online — had flagged potential duplicates automatically for years.

What Comes Next for City Records and Residents

The practical stakes are real. A duplicate image attached to the wrong permit file can surface during a Discretionary Review hearing before the Planning Commission and muddy the evidentiary record. For a city still processing an enormous volume of Accessory Dwelling Unit applications under state law AB 2221, errors in image metadata are not merely administrative noise — they can slow approvals and trigger additional rounds of staff review, adding weeks to timelines that developers and homeowners are already fighting.

The Department of Technology has indicated it plans to adopt upstream validation tools, similar in concept to Singapore's approach, by the end of fiscal year 2027. The Civic Bridge program, which pairs city departments with private-sector technologists, is one channel through which vendors are being evaluated. Several Mission District-based civic tech firms have reportedly participated in early briefings, though no contracts have been awarded publicly as of July 4, 2026.

For residents dealing with 311 complaints or tracking a neighborhood project through the Planning Department's online portal, the near-term advice is straightforward: if an image attached to a case looks wrong or out of place, file a correction request directly through the department rather than waiting for the automated audit to catch it. The city's public records team, reachable through SF.gov, has added a dedicated image-dispute intake form as of May 2026.