San Francisco Is Quietly Leading the Fight Against Duplicate Digital Images in City Records — Here's How It Stacks Up Against London and Singapore

San Francisco's Department of Technology confirmed this spring that the city's centralized digital asset repository had grown to more than 14 million stored images across planning, public works, and emergency-services databases — with internal audits estimating that roughly 30 percent of those files are near-identical duplicates consuming server capacity and slowing permitting workflows at the Planning Department's offices on Rayes Street near City Hall. That duplication problem, long treated as a mundane IT nuisance, has quietly become a policy headache as the city pushes to accelerate housing approvals under its state-mandated Housing Element obligations.

The stakes are real. California's Department of Housing and Community Development has been pressing San Francisco to dramatically cut permitting timelines, and redundant image files attached to permit applications — site photos, architectural drawings, inspection records — have been identified by city technology staff as one of several friction points slowing the review queue at the Permit Center on Edmonds Street in the Civic Center neighborhood. When a single property inspection generates dozens of near-duplicate JPEGs, each requiring manual review, the cumulative drag across thousands of permits adds up fast.

What San Francisco Is Actually Doing

The Department of Technology began piloting a machine-learning deduplication tool in February 2026, working alongside the city's Office of Civic Innovation. The system uses perceptual hashing — a technique that identifies visually similar images even when file names, metadata, or resolution differ — to flag redundant assets before they enter the permanent record. According to materials the department presented to the city's Committee on Information Technology in March, the pilot phase targeted the Planning Department's image archive first, covering roughly 2.1 million files dating back to 2015.

Early results from that pilot, as described in publicly posted committee documents, suggested the tool flagged approximately 620,000 files as candidates for consolidation or deletion. City staff have not yet confirmed how many were ultimately removed, and the program remains in a review phase pending sign-off from the City Attorney's Office on records-retention compliance.

San Francisco is not operating in isolation. London's Government Digital Service rolled out a comparable deduplication protocol across its 33 borough councils beginning in late 2024, with the Greater London Authority reporting in January 2026 that the effort had reduced its planning-portal image storage by 22 percent within 18 months. Singapore's Urban Redevelopment Authority went further, mandating standardized image formats and automated deduplication at the point of upload for all development applications starting January 1, 2025 — a requirement that has no equivalent yet in San Francisco's system.

The Gap With Global Peers

The contrast with Singapore is instructive. Singapore's approach is preventive: images are screened before they enter the archive. San Francisco's current pilot is corrective — cleaning up years of accumulated redundancy after the fact. That distinction matters because it determines how quickly the city can realize efficiency gains in its permitting pipeline.

Tokyo's Bureau of Urban Development took a middle path, implementing deduplication checks at quarterly intervals rather than in real time, a model that San Francisco's Office of Civic Innovation has reportedly studied. Tokyo's bureau has not published outcome data in English, making direct comparison difficult.

For San Francisco residents and developers, the practical consequences are still unfolding. Housing advocates at the Tenderloin Neighborhood Development Corporation and commercial applicants working through the Eastern Neighborhoods Community Advisory Committee have both flagged permitting slowdowns in recent months, though neither organization has publicly linked those delays specifically to image-management issues.

The Department of Technology has indicated it expects to present a full program evaluation to the Committee on Information Technology by October 2026. If the pilot clears the City Attorney's records-retention review, a citywide rollout — covering public works, fire inspection, and emergency-services databases — could follow in early 2027. Whether that timeline holds will depend partly on budget negotiations this fall, when the city faces a projected general-fund shortfall that has already forced cuts across several technology modernization programs. Developers and housing advocates say they will be watching the October presentation closely.