SF City Agencies Push to Root Out Duplicate Images From Public Records — Here's What Officials and Experts Are Saying

San Francisco's Department of Technology is under renewed pressure this summer to address a persistent problem inside the city's public records infrastructure: thousands of duplicate images clogging document management systems used by agencies from the Planning Department on Van Ness Avenue to the Office of the Assessor-Recorder at City Hall. The issue, long treated as a low-priority IT housekeeping matter, has taken on new urgency as the city prepares a sweeping digital-records modernization push slated to begin in the third quarter of 2026.

Why now? The timing is not accidental. San Francisco's Controller's Office has been auditing city data systems since early 2026 as part of a broader efficiency drive tied to budget pressures — the city is managing a projected general fund shortfall that has forced agencies to justify every line of operational spending. Storage costs, redundant files, and degraded search functionality in legacy databases have become concrete budget arguments, not just abstract technology complaints. Advocates for government transparency say the duplicate-image problem directly hampers public records requests filed under California's Public Records Act, slowing response times for journalists, researchers, and residents.

The San Francisco Public Library's main branch at Larkin and Fulton streets, which manages a significant portion of the city's digitized historical image collections, is one of the institutions most visibly affected. Librarians have flagged that some digitization projects carried out between 2018 and 2022 produced duplicate scans stored across multiple servers, with no unified deduplication protocol in place at the time. The San Francisco Planning Department, which maintains millions of permit-related property images, has acknowledged the issue internally as it moves toward integration with the city's upgraded Accela permitting platform.

What Officials and Experts Are Actually Saying

City technology officials have been careful in their public statements, stopping short of attaching a specific dollar figure to the cleanup effort without a completed audit to back it up. What is on record: the Department of Technology's 2025-2026 annual plan, published on the city's official sfgov.org portal, identifies data deduplication as one of five priority areas for the citywide enterprise content management system. Officials from that department have described the issue in public budget hearings as a solvable but resource-intensive problem requiring both automated tools and manual review workflows.

Outside city government, specialists in municipal records management point to a specific technical gap. San Francisco adopted its current document management infrastructure in phases between 2014 and 2019, meaning different agencies onboarded at different times with inconsistent naming conventions and metadata standards. That patchwork approach is widely seen as the root cause. The nonprofit OpenGov Foundation, which works with municipalities nationally, has argued publicly that cities of San Francisco's scale — managing records for roughly 870,000 residents — need centralized image-hash verification systems to prevent duplication at the point of ingestion, not after the fact.

Tech-sector voices in the city, particularly those connected to the Civic Bridge program run through the San Francisco Mayor's Office of Civic Innovation, have suggested that AI-assisted deduplication tools already in commercial use could be adapted for city systems at relatively low cost. The Civic Bridge program, which pairs private-sector technologists with city agencies on short-term projects, has completed more than 40 engagements since its founding and is seen as a natural vehicle for piloting exactly this kind of solution.

What Comes Next for City Residents and Requesters

For anyone who has filed a California Public Records Act request with a San Francisco agency in the past year and waited weeks for image-heavy document packages, the practical stakes are real. City officials have indicated that the modernization timeline targets a pilot deduplication rollout in at least two agencies — likely Planning and the Assessor-Recorder — before the end of fiscal year 2026-2027, which closes June 30, 2027.

Watchdog groups including the First Amendment Coalition, based in the Bay Area, have urged the city to publish a public-facing dashboard tracking records request response times alongside the modernization effort, so residents can see whether cleaner archives actually translate to faster document delivery. That request has not yet received a formal response from the Mayor's Office. The Fourth of July weekend has city hall running on reduced staff, but the deadline pressure on the modernization project does not pause for holidays.