Bug in the findAndRerank overload that accepts a class parameter?

[skedwards88]

Bob claims that this is a client bug, though it could just be user error on my part:

❌ This prints empty documents:

import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.collections.Collection;
import com.datastax.astra.client.collections.commands.cursor.CollectionFindAndRerankCursor;
import com.datastax.astra.client.collections.commands.options.CollectionFindAndRerankOptions;
import com.datastax.astra.client.collections.definition.documents.Document;
import com.datastax.astra.client.core.query.Filter;
import com.datastax.astra.client.core.query.Projection;
import com.datastax.astra.client.core.query.Sort;
import com.datastax.astra.client.core.rerank.RerankedResult;
import lombok.Data;

public class Example {

  @Data
  public static class Projected {
    boolean is_checked_out;
    String title;
  }

  public static void main(String[] args) {
    // Get an existing collection
    Collection<Document> collection =
        new DataAPIClient("**APPLICATION_TOKEN**")
            .getDatabase("**API_ENDPOINT**")
            .getCollection("**COLLECTION_NAME**");

    // Find documents
    Filter filter = null;
    CollectionFindAndRerankCursor<Document, Projected> cursor =
        collection.findAndRerank(
            filter,
            new CollectionFindAndRerankOptions()
                .projection(Projection.include("is_checked_out", "title"))
                .sort(Sort.hybrid("A tree in the woods")),
            Projected.class);

    // Iterate over the results
    for (RerankedResult<Projected> result : cursor) {
      System.out.println(result.getDocument());
    }
  }
}

✅ Omitting the new class works:

import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.collections.Collection;
import com.datastax.astra.client.collections.commands.cursor.CollectionFindAndRerankCursor;
import com.datastax.astra.client.collections.commands.options.CollectionFindAndRerankOptions;
import com.datastax.astra.client.collections.definition.documents.Document;
import com.datastax.astra.client.core.query.Filter;
import com.datastax.astra.client.core.query.Projection;
import com.datastax.astra.client.core.query.Sort;
import com.datastax.astra.client.core.rerank.RerankedResult;

public class Example {

  public static void main(String[] args) {
    // Get an existing collection
    Collection<Document> collection =
        new DataAPIClient("**APPLICATION_TOKEN**")
            .getDatabase("**API_ENDPOINT**")
            .getCollection("**COLLECTION_NAME**");

    // Find documents
    Filter filter = null;
    CollectionFindAndRerankCursor<Document, Document> cursor =
        collection.findAndRerank(
            filter,
            new CollectionFindAndRerankOptions()
                .projection(Projection.include("is_checked_out", "title"))
                .sort(Sort.hybrid("A tree in the woods"))
            );

    // Iterate over the results
    for (RerankedResult<Document> result : cursor) {
      System.out.println(result.getDocument());
    }
  }
}

✅ Specifying a projection on the cursor works:

import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.collections.Collection;
import com.datastax.astra.client.collections.commands.cursor.CollectionFindAndRerankCursor;
import com.datastax.astra.client.collections.commands.options.CollectionFindAndRerankOptions;
import com.datastax.astra.client.collections.definition.documents.Document;
import com.datastax.astra.client.core.query.Filter;
import com.datastax.astra.client.core.query.Projection;
import com.datastax.astra.client.core.query.Sort;
import com.datastax.astra.client.core.rerank.RerankedResult;

public class Example {

  public static void main(String[] args) {
    // Get an existing collection
    Collection<Document> collection =
        new DataAPIClient("**APPLICATION_TOKEN**")
            .getDatabase("**API_ENDPOINT**")
            .getCollection("**COLLECTION_NAME**");

    // Find documents
    Filter filter = null;
    CollectionFindAndRerankCursor<Document, Document> cursor =
        collection.findAndRerank(
            filter,
            new CollectionFindAndRerankOptions()
                .sort(Sort.hybrid("A tree in the woods")));

    cursor.project(Projection.include("title", "is_checked_out"));

    // Iterate over the results
    for (RerankedResult<Document> result : cursor) {
      System.out.println(result.getDocument());
    }
  }
}

[clun]

Tested extensively today 4/13 and cannot reproduce. (version 2.2.0)

Beans

Getting a few java objects, I am adding vectorize and lexical and id in Book or i could not trigger insertMany with vectorize and text indexation, (otherwize to insert i would have to instanciate Collection<Document>)

    @Data
    public static class Book {
        boolean is_checked_out;
        String title;
        String author;
        String description;
        @JsonProperty("_id")
        String id;
        @JsonProperty("$vectorize")
        String vectorize;
        @JsonProperty("$lexical")
        String lexical;
    }

    @Data
    public static class Projected {
        boolean is_checked_out;
        String title;
    }

Loading some Data

• Creating a CSV file with a few book values

is_checked_out,title,author,description
false,The Great Gatsby,F. Scott Fitzgerald,A classic American novel set in the Jazz Age exploring themes of wealth and the American Dream
true,To Kill a Mockingbird,Harper Lee,A powerful story of racial injustice and childhood innocence in the American South
false,1984,George Orwell,A dystopian novel depicting a totalitarian society under constant surveillance
true,Pride and Prejudice,Jane Austen,A romantic novel exploring manners and marriage in Georgian England

• and a CSV loader

    public static List<Book> loadBooksFromCsv() {
        List<Book> books = new ArrayList<>();
        try (java.io.BufferedReader reader = new java.io.BufferedReader(
                new java.io.InputStreamReader(
                        SampleFindAndRerankCustomClass.class.getClassLoader()
                                .getResourceAsStream("books.csv")))) {
            String line;
            boolean isHeader = true;
            int count = 0;
            while ((line = reader.readLine()) != null) {
                // Skip header line
                if (isHeader) {
                    isHeader = false;
                    continue;
                }
                // Parse CSV line (simple parsing, assumes no commas in fields)
                String[] fields = line.split(",", 4);
                if (fields.length == 4) {
                    Book book = new Book();
                    book.set_checked_out(Boolean.parseBoolean(fields[0]));
                    book.setTitle(fields[1]);
                    book.setAuthor(fields[2]);
                    book.setDescription(fields[3]);
                    book.setId(String.valueOf(count));
                    book.setVectorize(book.getDescription());
                    book.setLexical(book.getTitle());
                    books.add(book);
                    count++;
                }
            }
            System.out.println("Loaded " + count + " books from CSV");
        } catch (Exception e) {
            System.err.println("Error loading books from CSV: " + e.getMessage());
        }
        return books;
    }

Create Collection

public static Collection<Book> createCollectionFarr(Database db) {
        VectorServiceOptions vectorService = new VectorServiceOptions()
                .provider("nvidia")
                .modelName("NV-Embed-QA");
        VectorOptions vectorOptions = new VectorOptions()
                .dimension(1024)
                .metric(SimilarityMetric.COSINE.getValue())
                .service(vectorService);

        LexicalOptions lexicalOptions = new LexicalOptions()
                .enabled(true)
                .analyzer(new Analyzer(STANDARD));
        RerankServiceOptions rerankService = new RerankServiceOptions()
                .modelName("nvidia/llama-3.2-nv-rerankqa-1b-v2")
                .provider("nvidia");
        CollectionRerankOptions rerankOptions = new CollectionRerankOptions()
                .enabled(true)
                .service(rerankService);

       return db.createCollection(COLLECTION_FIND_RERANK, new CollectionDefinition()
                .vector(vectorOptions)
                .lexical(lexicalOptions)
                .rerank(rerankOptions), Book.class);
    }

Ingest

// Pretty cool it is so succinct
createCollectionFarr(db).insertMany(loadBooksFromCsv());

Search (working)

Note

Note the types in the Generic

CollectionFindAndRerankCursor<Book, Projected>

Filter filter = null;
        CollectionFindAndRerankCursor<Book, Projected> cursor =
                c_farr.findAndRerank(
                        filter,
                        new CollectionFindAndRerankOptions()
                                .projection(Projection.include("is_checked_out", "title"))
                                .sort(Sort.hybrid("A tree in the woods")),
                        Projected.class);

        // Iterate over the results
        for (RerankedResult<Projected> result : cursor) {
            System.out.println(result.getDocument());
        }

Full code
https://github.com/datastax/astra-db-java/blob/main/samples/sample-demo/src/main/java/com/datastax/astra/samples/collections/SampleFindAndRerankCustomClass.java