debshots/db
Christoph Haas c5cd77e9ee Stop concurrent imports from racing on package names
Production imports kept aborting with
ActiveRecord::RecordNotUnique on packages_name_key and - on
environments that never got the constraint - silently created
duplicate packages. Root cause: overlapping update_from_deb_repos
runs (scheduled + manual) both do find-by-name then insert; whichever
wins the race aborts the whole run, whichever loses gets a
duplicate if the constraint was missing.

Three things so the mechanism stops being the problem:

- An advisory lock serializes update_from_deb_repos runs. A second
  run that finds the lock held simply skips with a warning instead
  of racing the first one. The lock is session-scoped, so it is
  released automatically when the process exits.
- A per-package rescue for ActiveRecord::RecordNotUnique. If a
  create still races (e.g. a package being created by a web upload),
  the package is re-fetched and updated instead of aborting the run.
- A real migration for packages_name_key, which previously only
  existed where it had been added by hand. From now on db:migrate
  creates it everywhere, new databases included.

Also verified with a reproduction: a single sequential run of the
deployed importer against a fresh database (both architectures,
~54k packages) creates zero duplicates.
2026-09-04 21:57:35 +02:00
..
migrate Stop concurrent imports from racing on package names 2026-09-04 21:57:35 +02:00
cable_schema.rb Massive revamping for Rails 8 and Zurb 2025-03-16 00:44:43 +01:00
cache_schema.rb Massive revamping for Rails 8 and Zurb 2025-03-16 00:44:43 +01:00
development.sqlite3 Massive revamping for Rails 8 and Zurb 2025-03-16 00:44:43 +01:00
queue_schema.rb Massive revamping for Rails 8 and Zurb 2025-03-16 00:44:43 +01:00
schema.rb Add semantic package search via pgvector embeddings 2026-08-22 22:07:56 +02:00
seeds.rb Massive revamping for Rails 8 and Zurb 2025-03-16 00:44:43 +01:00