SF's Digital Archive Overhaul: The Key Decisions Ahead on Duplicate Image Replacement

San Francisco's public agencies are sitting on a quiet crisis buried in their digital infrastructure. Across city departments — from the Planning Department's permit portals on Van Ness Avenue to the San Francisco Public Library's digital archive on Larkin Street — duplicate and degraded images have accumulated for years inside public-facing databases, creating a tangle of outdated visual records that costs staff time, distorts public information, and in some cases undermines legal and permitting processes.

The issue has moved from a background IT annoyance to a front-burner policy question because of two converging pressures: a citywide push to modernize digital services under San Francisco's Department of Technology, and a parallel surge in AI-assisted cataloguing tools that make large-scale duplicate detection cheaper and faster than at any point in the past decade. The window to act is now — but the decisions ahead are neither obvious nor cheap.

What's Driving the Urgency

The San Francisco Planning Department alone maintains thousands of permit and property records, each of which may carry multiple image attachments — site photos, architectural drawings, inspection stills — uploaded by different staff members at different times, sometimes without standardized naming conventions. When duplicate or superseded images are not removed or replaced, records reviewers and members of the public accessing the City's online portal can pull incorrect or outdated visual information about a given parcel. In a city where a single building permit in the Mission District or SoMa can involve multiple appeals and community hearings, visual accuracy is not a trivial matter.

The San Francisco Public Library's Digital Collections, which houses historical photographs of neighborhoods from the Tenderloin to the Sunset District, has faced similar challenges. Digitization campaigns conducted before roughly 2015 used lower-resolution standards that now look inadequate alongside more recent scans. Duplicate entries — the same photograph scanned twice under different catalog numbers — inflate collection counts and make search results unreliable for researchers.

The Department of Technology's 2025 annual report noted that the city operates more than 200 distinct software applications across its agencies, many of which were procured independently and do not share image management standards. That fragmentation is the root cause of the duplicate problem, and it will be the hardest thing to fix.

The Decisions That Will Define the Next Phase

Three choices loom over any serious duplicate-image replacement effort in San Francisco's public sector. First, agencies must decide whether to run replacement programs in-house or contract out to vendors. The city has used Salesforce and other enterprise platforms for constituent services, and several vendors now pitch AI-powered deduplication modules as add-ons to existing government content management systems. But procurement through the City Administrator's Office takes time, and any contract above $10 million requires Board of Supervisors approval — a process that has historically stretched past 12 months on complex technology bids.

Second, there is the question of standards. The city's Office of Digital Services, which sits under the Department of Technology at 1 South Van Ness Avenue, has been developing a unified digital asset standard since late 2024. Whether individual departments adopt that standard voluntarily or are mandated to comply will determine whether a replacement effort produces a coherent archive or simply substitutes one batch of inconsistent images for another.

Third — and most politically fraught — is funding. San Francisco's current fiscal year budget, passed under significant deficit pressure, left most departmental IT line items flat. Any large-scale image remediation project will likely need to compete for discretionary technology funds or seek state grants through the California Department of Technology's Office of Digital Innovation, which has issued competitive rounds for local government modernization projects in prior years.

For residents trying to understand the practical stakes: if you have ever pulled up a property record on SF's online permit tracker and found a photograph that clearly shows a building that no longer exists in that form, you have encountered this problem firsthand. The fix is achievable, and the tools are available. What the city decides over the next six to twelve months — on vendors, on standards, and on money — will determine whether San Francisco's public digital records finally catch up to the city's expectations of itself.