In this video, we talk about a vector compression technique called Product quantization. We first explain conceptually, what the main ideas are and then show how one can use an existing implementation of it from Faiss (IndexPQ). Finally, we also implement the algorithm from scratch. Last but not least, we run some experiments and compare different methods.
Paper: lear.inrialpes.fr/pubs/2011/J...
Code: github.com/jankrepl/mildlyove...
Faiss: github.com/facebookresearch/f...
Nanopq: github.com/matsui528/nanopq
00:00 Intro
00:45 Overview: Similarity search [slides]
01:50 Overview: Main idea [slides]
02:18 Overview: How exactly codes created [slides]
03:25 Overview: Similarity search with PQ [slides]
04:24 PQ in Faiss [code]
09:40 Custom PQ - Constructor [code]
12:25 Custom PQ - Training [code]
13:49 Custom PQ - Encoding and add method [code]
15:11 Custom PQ - Computing distance [code]
17:36 Custom PQ - Search [code]
18:56 Making sure things work [code]
20:48 Results: CustomPQ vs Faiss? [demo]
22:59 Results: Number of segments tradeoff [demo]
24:17 Outro
If you have any video suggestions or you just wanna chat feel free to join the discord server: / discord
Twitter: / moverfitted
Credits logo animation
Title: Conjungation · Author: Uncle Milk · Source: / unclemilk · License: creativecommons.org/licenses/... · Download (9MB): auboutdufil.com/?id=600
Негізгі бет Ғылым және технология Product quantization in Faiss and from scratch
Пікірлер: 22