Production imports kept aborting with
ActiveRecord::RecordNotUnique on packages_name_key and - on
environments that never got the constraint - silently created
duplicate packages. Root cause: overlapping update_from_deb_repos
runs (scheduled + manual) both do find-by-name then insert; whichever
wins the race aborts the whole run, whichever loses gets a
duplicate if the constraint was missing.
Three things so the mechanism stops being the problem:
- An advisory lock serializes update_from_deb_repos runs. A second
run that finds the lock held simply skips with a warning instead
of racing the first one. The lock is session-scoped, so it is
released automatically when the process exits.
- A per-package rescue for ActiveRecord::RecordNotUnique. If a
create still races (e.g. a package being created by a web upload),
the package is re-fetched and updated instead of aborting the run.
- A real migration for packages_name_key, which previously only
existed where it had been added by hand. From now on db:migrate
creates it everywhere, new databases included.
Also verified with a reproduction: a single sequential run of the
deployed importer against a fresh database (both architectures,
~54k packages) creates zero duplicates.
Package text search now combines several strategies: exact name match,
name prefix promotion, compound word splitting ("sqlite browser" finds
"sqlitebrowser") and semantic nearest neighbor search over description
embeddings computed by an external embedding service (all-MiniLM-L6-v2,
384 dims) stored with the pgvector extension. Classic PostgreSQL
full-text search remains as fallback when the vector service is
unreachable. Gibberish queries without lexical overlap with the package
data return empty results instead of random matches.
Embeddings can be backfilled with bin/rails debshots:compute_vectors.
Also drops the unused lograge gem from the Gemfile.
To migrate the application as of April 2020:
• leave models/screenshot.rb with “has_attached_file”
• bundle exec rake db:migrate
• bin/move_paperclip_to_activestorage
• set models/screenshot.rb to “has_one_attached”
