SF City Agencies Push to Fix Duplicate Image Problem in Public Records — Here's What Officials and Experts Are Saying

San Francisco's digital public infrastructure has a clutter problem. City agencies — including the Planning Department on Mission Street and the San Francisco Municipal Transportation Agency — are sitting on databases riddled with duplicate images: permit photos uploaded twice, outdated street-view captures cross-posted across platforms, and redundant map tiles that inflate storage costs and slow down public-facing portals. Officials and technology experts who work with the city say the problem has quietly compounded for years, and a reckoning is overdue.

The timing matters. The city is in the middle of a broader push to modernize its digital services, part of a citywide technology modernization initiative that the Department of Technology has been advancing since early 2025. That effort has spotlighted inefficiencies that were easy to ignore when budgets were flush — but with San Francisco facing a projected general fund shortfall heading into fiscal year 2027, every dollar spent on redundant cloud storage is a dollar that isn't going toward housing production or Muni service restoration.

What the Agencies Are Saying

At the Planning Department's offices at 49 South Van Ness Avenue, staff have acknowledged internally that the Permit and Project Tracking System accumulates duplicate image attachments each time an applicant resubmits documentation. The problem is structural: the system does not automatically flag identical or near-identical files before they are stored. A single building permit application in the Mission District or the Sunset can generate dozens of image files, some of them exact copies, all of them billed to the city's cloud storage contract.

SFMTA, which maintains visual records for roughly 1,600 lane miles of city streets as part of its Vision Zero documentation program, faces a parallel issue. Field inspectors using mobile upload tools have reported that connectivity problems near Muni Metro tunnels — particularly around the West Portal and Castro stations — cause the upload software to retry and store the same photograph multiple times. Those duplicates accumulate in the agency's asset management system.

Technology consultants who have worked on municipal data projects in the Bay Area describe the phenomenon as common in organizations that adopted cloud storage quickly during the pandemic without putting deduplication protocols in place. One approach gaining traction in city IT circles is hash-based deduplication, a technique that assigns each image file a unique fingerprint and prevents identical files from being stored more than once. Several Bay Area counties moved to similar systems between 2022 and 2024.

Experts Flag Costs and Accountability Gaps

The financial stakes are real. Cloud storage for image-heavy municipal databases can run into hundreds of thousands of dollars annually for a city the size of San Francisco, according to published rate schedules from major providers. When duplicate files account for even 10 to 15 percent of stored volume — a conservative estimate cited in general industry literature on municipal data management — the wasted expenditure adds up across a multi-year contract.

The San Francisco Controller's Office, which publishes annual performance audits of city technology spending, has flagged data governance as an area warranting closer review in recent budget cycles. Advocates at Code for San Francisco, the civic technology volunteer group that meets regularly at the SFGovTV offices and partners with city departments on open-data projects, have pushed for clearer metadata standards that would make duplicate detection easier to automate.

Librarians at the San Francisco Public Library's main branch on Larkin Street, who manage their own digital archive of city photographs dating to the early 20th century, have dealt with the deduplication challenge longer than most. Their experience points toward a practical lesson: the earlier an organization builds deduplication into its upload workflow, rather than attempting a database-wide cleanup retroactively, the lower the long-term cost.

City officials have not announced a formal program or timeline to address the duplicate image problem across all agencies. The Department of Technology is expected to release updated data governance guidelines before the end of calendar year 2026. In the meantime, planning commissioners and SFMTA board members who want to hold agencies accountable have a straightforward tool available: requesting storage audits as part of routine budget oversight hearings, which are scheduled to resume in September at City Hall.