Stop concurrent imports from racing on package names
Production imports kept aborting with ActiveRecord::RecordNotUnique on packages_name_key and - on environments that never got the constraint - silently created duplicate packages. Root cause: overlapping update_from_deb_repos runs (scheduled + manual) both do find-by-name then insert; whichever wins the race aborts the whole run, whichever loses gets a duplicate if the constraint was missing. Three things so the mechanism stops being the problem: - An advisory lock serializes update_from_deb_repos runs. A second run that finds the lock held simply skips with a warning instead of racing the first one. The lock is session-scoped, so it is released automatically when the process exits. - A per-package rescue for ActiveRecord::RecordNotUnique. If a create still races (e.g. a package being created by a web upload), the package is re-fetched and updated instead of aborting the run. - A real migration for packages_name_key, which previously only existed where it had been added by hand. From now on db:migrate creates it everywhere, new databases included. Also verified with a reproduction: a single sequential run of the deployed importer against a fresh database (both architectures, ~54k packages) creates zero duplicates.
This commit is contained in:
parent
3f37e2d857
commit
c5cd77e9ee
2 changed files with 78 additions and 14 deletions
|
|
@ -0,0 +1,28 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
# Packages must be unique by name - the same package is listed once per
|
||||
# architecture/component in the Debian archive and the importer looks it up
|
||||
# by name before inserting, so anything that lets two rows for one name appear
|
||||
# (e.g. concurrent import runs) breaks the site. Production had accidentally
|
||||
# relied on a constraint that was only ever added ad-hoc on some databases
|
||||
# and never tracked as a migration; this makes db:migrate the single source
|
||||
# of truth.
|
||||
class AddUniqueNameConstraintToPackages < ActiveRecord::Migration[8.1]
|
||||
def up
|
||||
add_unique_constraint :packages, :name, name: :packages_name_key unless constraint_exists?
|
||||
end
|
||||
|
||||
def down
|
||||
remove_unique_constraint :packages, :name, name: :packages_name_key if constraint_exists?
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def constraint_exists?
|
||||
select_value(<<~SQL.squish)
|
||||
SELECT 1
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'packages'::regclass AND conname = 'packages_name_key' AND contype = 'u'
|
||||
SQL
|
||||
end
|
||||
end
|
||||
Loading…
Add table
Add a link
Reference in a new issue