4

I'm having a hard time understanding why people use any vector they find as a candidate for a recommender system.

In my mind, a recommender system requires a space where distance represents similarity. Of course, before you can construct such a space, first you need to settle on the type of distance you want to use (euclidean, angular, or anything else). Then you need a model (assuming we are talking about ML) to map your input (it could be an image, text, or anything else) to a point in that space. One major aspect of this model is that it's aware of the type of distance we've defined. If there's no notion of the distance in the model, definitely the output of the model is not going to have the attribute of "distance means similarity".

I'm asking this question because I've seen people use any vector they find to construct a recommender system. Here's an example of using a VAE's latent vectors for recommender systems:

https://developer.nvidia.com/blog/building-recommender-systems-faster-using-jupyter-notebooks-from-ngc/

I've also seen people using fastText word embeddings in the same way. I understand that all these embeddings/latent vectors form clusters in their spaces with some interesting patterns. But I don't think this is enough to assume the "distance represents similarity" requirement for a recommender system.

Please let me know if I'm missing anything here.

Mehran
  • 277
  • 1
  • 2
  • 13

2 Answers2

1

You are correct that recommender systems that map similarity to distance is useful.

Vector representations are useful because most machine learning learning tools are based on linear algebra. Vector representations encode raw data in form that amenable to machine learning.

"Any" vector representation is more useful than no vector representation. For example, one-hot encoding is often more useful than not including the feature. However, distance in one-hot encoding is not related to similarity.

What is even more useful are semantic vectors (e.g., word2vec and related techniques). Semantic vectors map contextual meaning into locations in a learned vector space. This is an example of where distance is a proxy for similarity.

Brian Spiering
  • 23,131
  • 2
  • 29
  • 113
0

You are absolutely right - the "distance" between word embeddings does not automatically imply they are automatically semantically similar! This is a common misconception. This is more a result of when you train towards an objective, certain words can be used in place of one another and as a result, they can have similar representations inside of a neural network (hence similar embeddings and therefore reduced distance between your embeddings). However, this definitely does not mean that words are necessarily similar in meaning despite having similar embeddings!

However, models can be trained such that similar sentences/words have similar embeddings. This was the case of Universal Sentence Encoder where it was trained such that similar words and similar sentences had high cosine similarity relative to others. This method of training falls under a number of names: siamese neural networks/one-shot learning/metric learning. Similarly, what we call "similar" can fall under a number of different definitions - for example - "questions" can be "similar" to "answers" and so their embeddings can be trained to be near each other - same with text and images!

Now in order to build a recommendation engine, you will want to attain an embedding of an item and another item that has been purchased and similarly index them so that their embedding space is a lot closer. Then when you index them, you can immediately fetch the closest neighbors of a particular item and return them as results.

I would recommend trying out some of these applications yourself and experimenting with various models/data! These will help you explore some rather interesting results! Here are a few quickstarts to help you get started:

Text to image search https://docs.relevance.ai/docs/quickstart-text-to-image-search

Question answering: https://docs.relevance.ai/docs/quickstart-question-answering

"Semantic Text" Search: https://docs.relevance.ai/docs/quickstart-text-search

Disclaimer: I work at Relevance AI and helped author/edit the quickstarts linked.

Jacky Wong
  • 11
  • 2