Switch to bge-small-en-v1.5 for asymmetric query/passage embeddings
All checks were successful
Build and push container image / build-and-push (push) Successful in 4m10s
All checks were successful
Build and push container image / build-and-push (push) Successful in 4m10s
Replace all-MiniLM-L6-v2 with bge-small-en-v1.5, which separates short search queries from package descriptions via the retrieval instruction. Requests now default to query mode and opt out with type=passage for documents. Uses the export's pooled, L2-normalized sentence_embedding output instead of manual mean pooling.
This commit is contained in:
parent
1b4d24cada
commit
111af2d66e
3 changed files with 68 additions and 110 deletions
122
README.md
122
README.md
|
|
@ -1,100 +1,64 @@
|
|||
# Vector Service
|
||||
|
||||
A minimal Docker container that provides text embedding vectors using the all-MiniLM-L6-v2 model. The service accepts POST requests with text and returns a 384-dimensional embedding vector.
|
||||
A minimal Go container that provides text embedding vectors using the
|
||||
[bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) model
|
||||
(384 dimensions). The service accepts POST requests and returns an embedding
|
||||
vector, ready to be consumed by pgvector.
|
||||
|
||||
## Features
|
||||
## Model
|
||||
|
||||
- **Small image size**: ~373 MB (much smaller than typical Python-based solutions)
|
||||
- **Fast inference**: Uses ONNX Runtime for efficient model execution
|
||||
- **API authentication**: Optional API secret protection
|
||||
- **Pre-converted ONNX**: Uses ready-to-use ONNX model from HuggingFace
|
||||
bge-small-en-v1.5 uses **asymmetric** embeddings: queries and passages are
|
||||
embedded differently, which is what lets a short search query align with a
|
||||
package description. To support this, the request carries a `type`:
|
||||
|
||||
## Usage
|
||||
- **default (omitted)** — treated as a **query**: the text is prefixed with
|
||||
the bge retrieval instruction
|
||||
(`Represent this sentence for searching relevant passages: `). This keeps
|
||||
existing callers that send a bare `{"text": ...}` working unchanged.
|
||||
- `"passage"` — embedded as-is (used for documents/descriptions).
|
||||
|
||||
### Build the container
|
||||
## API
|
||||
|
||||
`POST /vector`
|
||||
|
||||
```bash
|
||||
podman build -t vector-service .
|
||||
curl -X POST http://localhost:8080/vector \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"text": "sqlite editor", "type": "query"}'
|
||||
```
|
||||
|
||||
Or with Docker:
|
||||
```json
|
||||
{ "vector": [0.023, -0.118, ...] }
|
||||
```
|
||||
|
||||
Endpoints:
|
||||
|
||||
- `POST /vector` — embed text.
|
||||
- `GET /ok` — health check.
|
||||
- `GET /version` — build revision.
|
||||
|
||||
Optional Bearer auth via the `API_SECRET` environment variable.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
docker build -t vector-service .
|
||||
```
|
||||
|
||||
The build process will:
|
||||
1. Download the pre-converted ONNX model from `onnx-community/all-MiniLM-L6-v2-ONNX` on HuggingFace
|
||||
2. Install only the runtime dependencies (ONNX Runtime + NumPy)
|
||||
3. Create a minimal image (~373 MB)
|
||||
The build downloads the pre-converted ONNX model
|
||||
(`onnx-community/bge-small-en-v1.5-ONNX`) and the ONNX Runtime shared library,
|
||||
then compiles the Go binary.
|
||||
|
||||
### Run the service
|
||||
|
||||
Without authentication:
|
||||
```bash
|
||||
podman run --rm -p 8080:8080 -d vector-service
|
||||
```
|
||||
|
||||
With API secret authentication:
|
||||
```bash
|
||||
podman run --rm -p 8080:8080 -e API_SECRET=your-secret-key -d vector-service
|
||||
```
|
||||
|
||||
### Query the service
|
||||
|
||||
Send a POST request with JSON body:
|
||||
## Run
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/vector \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer your-secret-key" \
|
||||
-d '{"text": "Hello World"}'
|
||||
docker run --rm -p 8080:8080 -e API_SECRET=your-secret-key vector-service
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"vector": [0.1868536774709355, 0.8120285351760685, ...]
|
||||
}
|
||||
```
|
||||
## Technical details
|
||||
|
||||
The vector has 384 dimensions.
|
||||
|
||||
### Without authentication
|
||||
|
||||
If you didn't set `API_SECRET`, you can query without the Authorization header:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/vector \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"text": "Hello World"}'
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
- `Dockerfile`: Single-stage build with pre-downloaded ONNX model
|
||||
- `server.py`: HTTP server with ONNX inference
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Model
|
||||
- **Model**: all-MiniLM-L6-v2 (80 MB on disk)
|
||||
- **Source**: Pre-converted ONNX from [onnx-community/all-MiniLM-L6-v2-ONNX](https://huggingface.co/onnx-community/all-MiniLM-L6-v2-ONNX)
|
||||
- **Dimensions**: 384
|
||||
- **Format**: ONNX (pre-converted)
|
||||
|
||||
### Dependencies
|
||||
- Runtime: Python 3.11, ONNX Runtime, NumPy
|
||||
- No build-time dependencies needed (uses pre-converted model)
|
||||
|
||||
### Image Size Breakdown
|
||||
- Model files (ONNX + ONNX data + tokenizer + vocab): ~95 MB
|
||||
- Python runtime and dependencies: ~278 MB
|
||||
- Total: ~373 MB
|
||||
|
||||
## Notes
|
||||
|
||||
- The build will download the pre-converted ONNX model from HuggingFace (~95 MB total)
|
||||
- Much faster builds since no PyTorch or model conversion is needed
|
||||
- The ONNX model includes an external data file (`model.onnx_data`) which is normal for larger models
|
||||
- For production use, consider adding rate limiting and HTTPS
|
||||
- **Model**: bge-small-en-v1.5 (ONNX), 384 dimensions
|
||||
- **Pooling/normalization**: the export's `sentence_embedding` output is
|
||||
mean-pooled and L2-normalized
|
||||
- **Runtime**: Go + ONNX Runtime (no Python)
|
||||
- **Files**: `main.go` (server), `Dockerfile`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue