SF City Agencies Push to Fix Duplicate Image Problem Plaguing Digital Records This Week

San Francisco's Department of Technology confirmed this week that a longstanding duplicate image problem across several city databases had worsened significantly, with redundant files now consuming a measurable share of municipal server capacity and slowing permit processing at the Planning Department's office on Goodwin Place in Civic Center. The issue, which tech staff have flagged intermittently since at least 2023, reached a threshold that prompted an internal escalation memo circulated to department heads on July 1.

The timing matters. The city is midway through a broader digital infrastructure overhaul tied to the Mayor's Office of Housing and Community Development's push to accelerate permit approvals — a priority that has become politically loaded given San Francisco's state-mandated housing production targets. Clogged image repositories slow automated document review, which in turn delays the very permitting workflows city leaders have spent 18 months trying to streamline. With the fiscal year just turned and a new technology budget cycle underway, officials are under pressure to show progress before October, when the next phase of the citywide enterprise content management rollout is scheduled to go live.

Where the Problem Shows Up

The duplicate image backlog is most acute in two systems: the Department of Building Inspection's permit document portal, which serves contractors and homeowners filing from neighborhoods like the Mission, the Sunset, and the Richmond, and the city's legacy property records database maintained jointly with the Office of the Assessor-Recorder at City Hall. Staff at DBI's Laney Lane satellite office have reported that some permit application packets contain the same uploaded photo appearing three or four times, a byproduct of a 2024 migration to a new cloud storage vendor that failed to deduplicate files during transfer.

The San Francisco Digital Services team, housed at 200 Paul Ave. in the Bayview, has been running a deduplication script on a subset of records since late June. The program targets image files flagged by hash-comparison tools — software that identifies identical files regardless of filename — and routes them to a manual review queue before deletion. Digital Services staff have not publicly disclosed how many files are involved, but city technology procurement documents reviewed this week show a contract amendment signed June 27 with a data management vendor for work described as large-scale binary file remediation, valued at just under $340,000.

Practical Stakes for Residents and Contractors

For anyone pulling permits or researching property records, the real-world effect has been slower response times on public records requests and occasional error messages when the city's online portal tries to render a document packet. Contractors working in SoMa and the Tenderloin have noted delays in getting sign-off on inspection photos submitted through the PermitSF platform, with some turnaround times stretching past the standard five-business-day window.

The deduplication push also has implications for the city's nascent AI document-review pilot, which the Department of Technology began testing in March at the Planning Department. That pilot uses machine-learning tools to pre-screen uploaded images for completeness and code compliance. Duplicate images confuse the model's classification logic, generating false positives that require human override. Cleaning the underlying data is, in effect, a prerequisite for expanding that AI program to additional departments — a step city technology officials have been publicly committed to completing before year-end.

Digital Services is expected to present a progress report to the city's Committee on Information Technology, known as COIT, at its next scheduled meeting. Contractors and property owners with active permit applications are advised to log into PermitSF and verify that uploaded documents are rendering correctly; re-uploads may be needed in cases where image packets show blank pages or duplicate thumbnails. The Department of Building Inspection's public counter at 49 South Van Ness Ave. can assist with manual file corrections for anyone who has trouble resolving issues through the online system.