Christoph Haas
4948fe50e3
Package text search now combines several strategies: exact name match,
name prefix promotion, compound word splitting ("sqlite browser" finds
"sqlitebrowser") and semantic nearest neighbor search over description
embeddings computed by an external embedding service (all-MiniLM-L6-v2,
384 dims) stored with the pgvector extension. Classic PostgreSQL
full-text search remains as fallback when the vector service is
unreachable. Gibberish queries without lexical overlap with the package
data return empty results instead of random matches.
Embeddings can be backfilled with bin/rails debshots:compute_vectors.
Also drops the unused lograge gem from the Gemfile.
15 lines
585 B
Ruby
15 lines
585 B
Ruby
# frozen_string_literal: true
|
|
|
|
# Adds a vector column to store text embeddings of the package
|
|
# description and long description. The vectors are computed by an
|
|
# external web service and used for nearest neighbor search.
|
|
#
|
|
# The pgvector extension is enabled by the InstallNeighborVector migration.
|
|
class AddEmbeddingToPackages < ActiveRecord::Migration[8.1]
|
|
def change
|
|
add_column :packages, :embedding, :vector, limit: 384
|
|
|
|
# HNSW index for fast approximate nearest neighbor searches
|
|
add_index :packages, :embedding, using: :hnsw, opclass: :vector_cosine_ops
|
|
end
|
|
end
|