LanceDb handles calculating embeddings on the way into the database – for bot…

LanceDb handles calculating embeddings on the way into the database – for both the query term, and the source material.

To take advantage of this using pydantic, you have to implement the following:

• Have a model that inherits from LanceModel
• Define an embedding function, ie:

  from lancedb.embeddings import get_registry
  
  embeddings_function = get_registry().get("openai").create(name="text-embedding-ada-002")
  

• Use that embedding function as the provider of both your source, and embedding field type:

  from lancedb.pydantic import LanceModel, Vector
  
  class Chunk(LanceModel):
      chunk_id:str
      passage: str = embeddings_function.SourceField()
      embedding: Vector(embeddings_function.ndims()) = embeddings_function.VectorField()
  

With this in place, you can pass your embedding model when you create your LanceDb table and the magic will happen automatically.