Email content

Pinecone logo-1

Full-text search is generally available in Pinecone Database.

Semantic search is built for finding data by meaning, and it's very good at it. A natural language question finds the right answer. Something like a part number is a different story, though. It's a literal string, and the catalog around it is full of numbers that differ by a character or two. All of them look about the same.

Vector search has no good, consistent way of ensuring the correct record is returned and ranked first when the query is a literal string. That record likely comes back in the result set, but who knows if it's marked as the correct one, and your agent takes whatever's on top of the list and answers from it whether or not it's the right data.

Full-text search is the solution to this problem. BM25 keyword ranking now runs in the same index as your vector search, so an exact string is matched and scored at the top of the list.

Functionality includes:

BM25 keyword ranking across multiple text fields

Lucene query syntax, including boolean operators, phrase queries, and fuzzy matching for typo tolerance

Text-match filters that constrain a semantic or keyword search to the documents containing a specific term or phrase

Tokenization and stemming in 18 languages

Read the announcement →

Best,

Pinecone Team

pinecone-logo-black

Pinecone Systems, Inc, 127 W. 26th St., Floor 6, New York, New York 10001, United States

Unsubscribe

Manage preferences