Switch to bge-small-en-v1.5 for asymmetric query/passage embeddings
All checks were successful
Build and push container image / build-and-push (push) Successful in 4m10s

Replace all-MiniLM-L6-v2 with bge-small-en-v1.5, which separates
short search queries from package descriptions via the retrieval
instruction. Requests now default to query mode and opt out with
type=passage for documents. Uses the export's pooled, L2-normalized
sentence_embedding output instead of manual mean pooling.
This commit is contained in:
Christoph Haas 2026-09-13 22:10:01 +02:00
parent 1b4d24cada
commit 111af2d66e
3 changed files with 68 additions and 110 deletions

122
README.md
View file

@ -1,100 +1,64 @@
# Vector Service
A minimal Docker container that provides text embedding vectors using the all-MiniLM-L6-v2 model. The service accepts POST requests with text and returns a 384-dimensional embedding vector.
A minimal Go container that provides text embedding vectors using the
[bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) model
(384 dimensions). The service accepts POST requests and returns an embedding
vector, ready to be consumed by pgvector.
## Features
## Model
- **Small image size**: ~373 MB (much smaller than typical Python-based solutions)
- **Fast inference**: Uses ONNX Runtime for efficient model execution
- **API authentication**: Optional API secret protection
- **Pre-converted ONNX**: Uses ready-to-use ONNX model from HuggingFace
bge-small-en-v1.5 uses **asymmetric** embeddings: queries and passages are
embedded differently, which is what lets a short search query align with a
package description. To support this, the request carries a `type`:
## Usage
- **default (omitted)** — treated as a **query**: the text is prefixed with
the bge retrieval instruction
(`Represent this sentence for searching relevant passages: `). This keeps
existing callers that send a bare `{"text": ...}` working unchanged.
- `"passage"` — embedded as-is (used for documents/descriptions).
### Build the container
## API
`POST /vector`
```bash
podman build -t vector-service .
curl -X POST http://localhost:8080/vector \
-H "Content-Type: application/json" \
-d '{"text": "sqlite editor", "type": "query"}'
```
Or with Docker:
```json
{ "vector": [0.023, -0.118, ...] }
```
Endpoints:
- `POST /vector` — embed text.
- `GET /ok` — health check.
- `GET /version` — build revision.
Optional Bearer auth via the `API_SECRET` environment variable.
## Build
```bash
docker build -t vector-service .
```
The build process will:
1. Download the pre-converted ONNX model from `onnx-community/all-MiniLM-L6-v2-ONNX` on HuggingFace
2. Install only the runtime dependencies (ONNX Runtime + NumPy)
3. Create a minimal image (~373 MB)
The build downloads the pre-converted ONNX model
(`onnx-community/bge-small-en-v1.5-ONNX`) and the ONNX Runtime shared library,
then compiles the Go binary.
### Run the service
Without authentication:
```bash
podman run --rm -p 8080:8080 -d vector-service
```
With API secret authentication:
```bash
podman run --rm -p 8080:8080 -e API_SECRET=your-secret-key -d vector-service
```
### Query the service
Send a POST request with JSON body:
## Run
```bash
curl -X POST http://localhost:8080/vector \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-secret-key" \
-d '{"text": "Hello World"}'
docker run --rm -p 8080:8080 -e API_SECRET=your-secret-key vector-service
```
Response:
```json
{
"vector": [0.1868536774709355, 0.8120285351760685, ...]
}
```
## Technical details
The vector has 384 dimensions.
### Without authentication
If you didn't set `API_SECRET`, you can query without the Authorization header:
```bash
curl -X POST http://localhost:8080/vector \
-H "Content-Type: application/json" \
-d '{"text": "Hello World"}'
```
## Files
- `Dockerfile`: Single-stage build with pre-downloaded ONNX model
- `server.py`: HTTP server with ONNX inference
## Technical Details
### Model
- **Model**: all-MiniLM-L6-v2 (80 MB on disk)
- **Source**: Pre-converted ONNX from [onnx-community/all-MiniLM-L6-v2-ONNX](https://huggingface.co/onnx-community/all-MiniLM-L6-v2-ONNX)
- **Dimensions**: 384
- **Format**: ONNX (pre-converted)
### Dependencies
- Runtime: Python 3.11, ONNX Runtime, NumPy
- No build-time dependencies needed (uses pre-converted model)
### Image Size Breakdown
- Model files (ONNX + ONNX data + tokenizer + vocab): ~95 MB
- Python runtime and dependencies: ~278 MB
- Total: ~373 MB
## Notes
- The build will download the pre-converted ONNX model from HuggingFace (~95 MB total)
- Much faster builds since no PyTorch or model conversion is needed
- The ONNX model includes an external data file (`model.onnx_data`) which is normal for larger models
- For production use, consider adding rate limiting and HTTPS
- **Model**: bge-small-en-v1.5 (ONNX), 384 dimensions
- **Pooling/normalization**: the export's `sentence_embedding` output is
mean-pooled and L2-normalized
- **Runtime**: Go + ONNX Runtime (no Python)
- **Files**: `main.go` (server), `Dockerfile`