Persian
Why Persian retrieval is a different problem
Two spellings of the same Persian word look identical to a reader and completely different to a tokenizer. That detail is the gap between a useful assistant and a toy.
Sara Ahmadi1 min read
Retrieval Engineer
Sara Ahmadi
Works on Joghd's retrieval layer and spends most of her time asking why the correct chunk ranked seventh.