San Francisco Confronts Thousands of Duplicate Digital Assets in City Hall

San Francisco's Department of Technology is facing a reckoning over duplicate digital imagery embedded across city systems — a sprawling, under-reported problem that affects everything from the Planning Department's permit portals to the Recreation and Park Department's online facility catalogs. At stake is an estimated consolidation effort that officials have flagged internally as a priority before the fiscal year closes in December 2026.

The issue matters now because the city is simultaneously trying to modernize three major public-facing platforms: the SF311 service request app, the updated SF.gov portal that launched in late 2024, and the DataSF open data infrastructure. Each system inherited image libraries from legacy software, and none of those libraries were deduplicated before migration. The result is a digital archive riddled with redundant files that slow load times, inflate cloud storage costs, and complicate accessibility compliance under ADA standards.

Where the Backlog Lives

The problem is concentrated in a handful of high-traffic corners of city government. The San Francisco Planning Department, headquartered on Rahaul Street near the Civic Center, maintains project files for thousands of active and completed permits — many of which include scanned site photographs uploaded multiple times by different staff. The SF Public Works bureau, which manages infrastructure from the Embarcadero seawall to the Twin Peaks reservoirs, has its own image catalog tied to inspection records, and duplicate uploads have accumulated there since at least the department's 2019 software transition.

The Tenderloin and SoMa districts are particularly visible in these records, because both neighborhoods have been the focus of intensive street-condition reporting under the city's fentanyl crisis response and homelessness encampment documentation programs. Hundreds of 311 field photos from those neighborhoods exist in duplicate or triplicate inside the city's content management system, according to the structure of the DataSF database as publicly visible on the city's open data portal.

The SF Digital Services team, which operates out of City Hall and has been expanding since 2023, is the unit tasked with resolving the problem. The team has already piloted a deduplication workflow on a subset of the Recreation and Parks image library — roughly 14,000 files tied to facilities like McLaren Park and the Panhandle — using hash-matching software to flag identical or near-identical files before archiving. That pilot, which wrapped up in spring 2026, is the closest thing the city has to a proven template for a wider rollout.

The Decisions That Will Define the Outcome

Three choices will determine whether the city cleans this up or carries the problem into the next budget cycle. First, officials must decide whether to run deduplication centrally through SF Digital Services or push the work down to individual departments. Central processing is faster but requires departments to cede control of assets they often treat as proprietary. Distributed processing preserves departmental autonomy but risks inconsistent standards.

Second, the city must settle on a retention policy. Deleting duplicates sounds simple, but archivists at the San Francisco History Center, housed inside the Main Branch of the San Francisco Public Library on Larkin Street, have raised concerns about bulk deletion of municipal imagery — even routine field photos can have evidentiary or historical value in litigation or public records requests.

Third, and most practically, the Department of Technology needs to decide whether to procure a new enterprise digital asset management platform or integrate deduplication tools into the existing Salesforce and Accela systems the city already licenses. A new platform would cost more upfront — procurement alone typically runs six to eighteen months under the city's contracting rules — but could eliminate the problem systemically rather than through repeated manual audits.

The next formal checkpoint is a technology infrastructure review scheduled before the Board of Supervisors' budget committee this fall. If SF Digital Services brings a consolidated proposal by September, the funding could theoretically be included in supplemental appropriations. If the decision slips to the spring 2027 budget cycle, the duplicate backlog will grow through another full year of permit activity, 311 filings, and street-condition photography — adding cost and complexity to a fix that is already overdue.