chaas/debshots
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
debshots/db/migrate
History
Exact
Christoph Haas c5cd77e9ee Stop concurrent imports from racing on package names 
Production imports kept aborting with
ActiveRecord::RecordNotUnique on packages_name_key and - on
environments that never got the constraint - silently created
duplicate packages. Root cause: overlapping update_from_deb_repos
runs (scheduled + manual) both do find-by-name then insert; whichever
wins the race aborts the whole run, whichever loses gets a
duplicate if the constraint was missing.

Three things so the mechanism stops being the problem:

- An advisory lock serializes update_from_deb_repos runs. A second
  run that finds the lock held simply skips with a warning instead
  of racing the first one. The lock is session-scoped, so it is
  released automatically when the process exits.
- A per-package rescue for ActiveRecord::RecordNotUnique. If a
  create still races (e.g. a package being created by a web upload),
  the package is re-fetched and updated instead of aborting the run.
- A real migration for packages_name_key, which previously only
  existed where it had been added by hand. From now on db:migrate
  creates it everywhere, new databases included.

Also verified with a reproduction: a single sequential run of the
deployed importer against a fresh database (both architectures,
~54k packages) creates zero duplicates.
2026-09-04 21:57:35 +02:00
..
20130726224319_create_packages.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20130727112955_create_screenshots.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20130808195123_add_search_index_to_packages.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20141206182405_drop_debtags_table.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20141229173351_fix_search_index_in_packages.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150103182901_add_attachment_image_to_screenshots.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150103194230_add_fingerprint_column_to_screenshot.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150422223156_change_package_longdescription_to_text.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150426141106_create_logs.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150503130412_delete_admins_table.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150503133405_create_users.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20150628113425_make_package_name_not_null.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20160302135740_add_devise_to_users.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20160304140037_rename_uploaddatetime.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20160408143714_add_default_to_screenshot_approved.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20160715180453_screenshots_enlarge_description.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20160810063320_add_visits_to_packages.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20160812085722_make_fulltext_search_index_weighted.rb Added explicit Rails version to migrations as needed for Rails 6 2020-08-16 19:22:47 +02:00
20170405144026_rename_user_realname_to_name.rb Renamed realname to name in users table 2017-04-05 16:42:10 +02:00
20170405204143_add_omniauth_to_users.rb Experimental authentication for Launchpad and Google added 2017-04-06 12:56:06 +02:00
20170406155141_change_users_unique_index_to_uid_provider.rb Allow same email address with different SSO providers 2017-04-06 18:32:20 +02:00
20170422181908_change_users_default_provider_local.rb Added welcome and profile actions to the my controller 2017-04-23 13:37:48 +02:00
20170422182228_add_admin_field_to_users.rb Added welcome and profile actions to the my controller 2017-04-23 13:37:48 +02:00
20170422183028_add_screenshot_reference_to_users.rb Added welcome and profile actions to the my controller 2017-04-23 13:37:48 +02:00
20200401000000_create_action_text_tables.action_text.rb Rails update. Fancybox update. Migrations from Paperclip. 2020-10-19 18:49:47 +02:00
20200413194248_create_active_storage_tables.active_storage.rb Migrations and helpers to move from Paperclip to ActiveStorage 2020-04-20 20:24:59 +02:00
20200414002409_add_foreign_key_constraint_to_active_storage_attachments_for_blob_id.active_storage.rb Migrations and helpers to move from Paperclip to ActiveStorage 2020-04-20 20:24:59 +02:00
20200901000000_create_action_mailbox_tables.action_mailbox.rb Rails update. Fancybox update. Migrations from Paperclip. 2020-10-19 18:49:47 +02:00
20201031112024_add_shrine_image_data_to_screenshot.rb Migration to Shrine attachment handling 2020-11-01 15:07:16 +01:00
20210222102040_extend_ip_field.rb Extend uploaderip field to cope with IPv6 addresses 2021-02-22 11:25:13 +01:00
20210222102719_change_ip_field_to_inet.rb Recreate ipaddress field as PostgreSQL's inet type 2021-02-22 11:31:41 +01:00
20210228014424_drop_deprecated_screenshots_fields.rb Authorisation management using CanCanCan added 2021-02-28 21:36:46 +01:00
20210228100636_add_roles_to_users.rb Authorisation management using CanCanCan added 2021-02-28 21:36:46 +01:00
20210228191717_add_pseudo_column_to_user.rb Authorisation management using CanCanCan added 2021-02-28 21:36:46 +01:00
20210303170421_add_user_field_to_log.rb Audit logging improved. References users, screenshots, packages 2021-03-03 23:21:24 +01:00
20210303170557_remove_token_field_from_log.rb Audit logging improved. References users, screenshots, packages 2021-03-03 23:21:24 +01:00
20210303203723_change_type_of_user_field_in_logs.rb Audit logging improved. References users, screenshots, packages 2021-03-03 23:21:24 +01:00
20210303213303_add_screenshot_and_package_field_to_logs.rb Audit logging improved. References users, screenshots, packages 2021-03-03 23:21:24 +01:00
20210309233239_add_hidden_field_to_screenshots.rb Allow hiding and un-hiding of screenshot by moderators 2021-03-10 01:04:37 +01:00
20210321215433_add_upload_counter_to_users.rb Added per-user counter of approved/rejected screenshots 2021-03-21 23:06:51 +01:00
20210528190641_add_service_name_to_active_storage_blobs.active_storage.rb General Rails and Ruby updates 2021-05-28 21:07:45 +02:00
20210528190642_create_active_storage_variant_records.active_storage.rb General Rails and Ruby updates 2021-05-28 21:07:45 +02:00
20231126173151_remove_not_null_on_active_storage_blobs_checksum.active_storage.rb Merge branch 'prod' 2023-12-26 01:21:15 +01:00
20260821203833_install_neighbor_vector.rb Add semantic package search via pgvector embeddings 2026-08-22 22:07:56 +02:00
20260821210000_add_embedding_to_packages.rb Add semantic package search via pgvector embeddings 2026-08-22 22:07:56 +02:00
20260904120000_add_unique_name_constraint_to_packages.rb Stop concurrent imports from racing on package names 2026-09-04 21:57:35 +02:00