Introduction

Hora Search Everywhere!

Hora, a approximate nearest neighbor search algorithm library. We implement all code in Rust🦀 for reliability, high level abstraction and high speed comparable to C++, these make it really suitable for Artificial Intellingence Computing scene.

Hora, 「ほら」 in Japanese, sound like [hōlə], means Wow, You see! or Look at that!. he name is inspired by a famous lyrics 「ほら あなたにとって大事な人ほど すぐそばにいるの」 of a famous Japanese song 小さな恋のうた, it means You See, the most important people to you is always stay near you

Approximate Nearest Neighbor

As shown in the picture below, we have a large data set. Each piece of data is an array<float> (usually a float) representing a store. Now the request is also an array<float>. We now hope that we Find k points with the smallest in the data set.

make it in math language:

ND-dim query vectors

1D-dim query vector

Task:Given and , find top K which has smallest

for futher more information, I really recommend this speech tutorial [CVPR20 Tutorial] Billion-scale Approximate Nearest Neighbor Search present in CVPR2020

Key Features

• Performant ⚡️

  • SIMD-Accelerated (packed_simd (opens new window))
  • Stable algorithm implementation
  • Multiple threads design

• Multiple Languages Support ☄️

  • Python
  • Javascript
  • Java
  • Go (WIP)
  • Ruby (WIP)
  • Swift (WIP)
  • R (WIP)
  • Julia (WIP)
  • also can serve as a service

• Multiple Indexes Support 🚀

  • Hierarchical Navigable Small World Graph Index(HNSWIndex) (detail (opens new window))
  • Satellite System Graph (SSGIndex) (detail (opens new window))
  • Product Quantization Inverted File(PQIVFIndex) (detail (opens new window))
  • Random Projection Tree(RPTIndex) (LSH, WIP)
  • BruteForce (BruteForceIndex) (naive implementation with SIMD)

• Portable 💼

  • Support no_std (WIP, partial)
  • Support Windows, Linux and OS X
  • Support IOS and Android (WIP)
  • No heavy dependency, such as BLAS

• Reliability 🔒

  • Rust compiler secure all code
  • Memory managed by Rust for all language libs such as Python lib
  • Broad testing coverage

• Multiple Distances Support 🧮

  • Dot Product Distance
    • 
  • Euclidean Distance
    • 
  • Manhattan Distance
    • 
  • Cosine Similarity
    • 

• Productive ⭐

  • Well documented
  • Elegant and simple API, easy to learn

Related Projects and Comparison

• Faiss (opens new window), Annoy (opens new window), ScaNN (opens new window):

  • Hora's implementation is strongly inspired by these lib.
  • Faiss focus more on the GPu scenerio, and Hora is lighter than Faiss, such as no heavy dependency.
  • Hora expects to support more language, and everything related to performance shall be implemented by Rust🦀.
  • Annoy only support LSH(Random Projection) algorithm.
  • ScaNN and Faiss are less user-friendly, such as lack of document.
  • Hora is ALL IN RUST 🦀.

• Milvus (opens new window), Vald (opens new window), Jina AI (opens new window)

  • Milvus and Vald also support multiple languages, but serve as a service instead of a lib
  • Milvus is built upon some libs such as Faiss, while Hora is an algorithm lib with all the algo implemented by itself