Chapter 8 - MOVSOM - State of the art

MOVSOM is a recommender system for movies that will tell a user how much he will like a certain movie and why he should trust the given recommendation. The system visualizes movie recommendations and encourages users to explore the recommendations. Recommendations consist of similar movies visualized (as a two dimensional Top N list) in the form of a map. The system has two usage modes:

1. Non-profiled based MOVSOM (Basic MOVSOM)

2. Profile based MOVSOM (Personalized MOVSOM)

In the first mode the system visualizes recommendations based on the movie the user searched for, thus requiring no user profile, in the second profile based mode, the user is able to rate and tag movies and during exploration of recommendations also get predictions for movies. In addition the profile based mode introduces a user map, consisting of all MOVSOM users. The user is able to locate himself on the map, as well as see where users that have seen the active item are in relation to himself. As predictions are made, the users making the predictions will also be indicated on the user map.

MOVSOM does not focus on providing users with information about movies, instead the visualized relationships between movies is the focus, however a basic set of identifying information is given about each movie, including title, release year and director. Instead links to the IMDb are always provided, encouraging users that find seemingly interesting movies to visit the IMDb for actual information.

MOVSOM map

MOVSOM uses the SOM algorithm to create a SOM that allows mapping of movies onto a two dimensional map. However the SOM will map multiple movies into the same nodes, i.e. movies end up "stacked on top of each other" in the same node on the map, thus to make it possible to provide visual recommendations and to navigate the map by only looking at the relationship between movies, the map needs to be "stretched" out such that movies in the same node are no longer stacked on top of each other.

The map stretching can be done in several ways, a rough but simple solution is to determine how

many movies a node in the map contains on an average (or at a maximum, but that may lead to a unneccessarily sparse stretchmap) and then transform each node into a (quadratic) grid large enough to contain such a number of movies and simply tile up each node's grid into a larger stretch map [stretch map] . Nodes with more than the average number of movies would need to be treated specially by e.g. listing the overflowing movies separately below the viewed map section (assuming the whole map is not viewed at once). Nodes with less than the averge number of movies would have its data placed out in some suitable fashion, e.g. randomly (to cover more of the node's grid) or focused in the center of the node's grid (to emphasize the data is equal to each other).

However, despite that movies ended up in the same node in the SOM, i.e. had the same BMN, they are not going to be exactly the same. A user may pick up on this and consider the randomly selected relative placement of two movies within a node's grid in the stretch map as an indication of a relationship, e.g. since movie A is below movie B, the user may select to navigate down the map for more movies like B. Unless movies are truly equal in a node the movies within a node's grid in the stretchmap should be ordered in some fashion.

A solution would be to create a smaller SOM using only the movies in each node, essentially making a hierarcical SOM and puzzling the hierarcies together into one level. However such a scheme would fail to account for the relationships between movies in the node and movies in neighoring nodes, thus not truly solving the problem.

A optimal solution to map stretching would be to simply arrange the movies within the node's grid such that movies as far as is possible are placed on the edge adjacent to the grid for the neighboring node they are most similar. Thus movies in a node more similar to the node on the "left" should be placed near the left edge of the node. Such a scheme for map stretching is essential to MOVSOM's ability to provide visual recommendations where the users themselves can determine movie similarity based on distances in the movie map.

MOVSOM interface

The MOVSOM interface consists of two basic parts [movsom interface] , the MOVSOM map and the sidebar.

By searching for movies by their title the section of the map containing the movie will be shown with the searched for movie placed in the middle of the map. Clicking on nearby movies will display information about that movie and recenter the map around the new movie, by this method the map can be navigated.

The sidebar always shows at least the title of the currently selected movie (if any) and provides a link to the IMDb where more attribute information can be obtained for the movie. In the personalized mode the user can always rate and add tags to the movie using the sidebar.

In the basic mode the sidebar [movsom basic mode] displays a U-matrix that shows the global movie map with the active movie indicated. The U-matrix can also be used to jump around in the movie map by simply clicking in it. By showing the U-matrix as a global view of the whole movie map the user can get a clearer sense of direction when searching and moving around in the map.

If the user has a rating profile a U-matrix for the user map is also shown in the sidebar [personalized prediction sidebar] indicating where the active user's position and where users that have rated the active movie are located in the user map. Groups of users are shown as green, yellow or red dots, indicating the groups preference for the active movie (red = don't like it, yellow = neutral, green = likes it). Two predictions are also shown, one based on the active user's average rating on movies similar to the active movie and one based on the active user's neighbors opinion on the active movie. Additionally a goodness recommendation combining and summarizing the shown visual recommendations and predictions into a textual recommendation is provided.

The sidebar also contains tags [personalized tag sidebar] , descriptive keywords users can associate with movies in the moviemap, shown as a ordered tag cloud where similar tags are placed near each other and sized according to have many times the tag has been assigned to the movie by different users. The tags provide additional movie information in place of complete movie attribute data.

MOVSOM architecture

The MOVSOM architecture will be described as consisting of several separate modules . Each module has a specific parameter requirement and output specification, typically each module is connected with one very specific part of the MOVSOM interface, the flow chart [movsom architecture] gives a rough idea of how these modules will be used in the live application. A user is placed out and a series of arrows give an idea of the flow the user will set in motion when interacting with various modules.

SOM module

Using the entire database of user ratings on movies a user SOM and a movie SOM is trained using e.g. the SOM implementation described Chapter 5.

Map stretching module

Stretches the movie map created by the movie SOM such that movies "stacked on top of each other" in the same node are no longer stacked on top of each other, and instead are placed beside each other such that they have both a relationship to the movies in the same node and to the directly neighboring nodes.

Map navigation module

Locates a movie in the stretched out maps and serves up the movie's immediate neighborhood in the stretched out map. Given the movies stretchmap position the movie's position in the movie U-matrix is marked (note that the U-matrix for the movie SOM is still representative of the cluster structure in the stretch map).

Search module

Given a seach query a movie database is searched for a movie matching the search query, the map navigation module is then automatically called to retrieve the corresponding map section.

Recommendation module

Provides recommendations for an active user on an active movie, the recommendations are statistical non-personalized recommendations in the basic mode, however if the user has a profile the recommendations are personalized. The statistical recommendations calculated are based on the populations average rating on the active item. In particular the personalized recommendations include calculating the user's average rating on movies similar to the active movie and calculting the user's neighbors average rating on the active movie.

Average rating prediction The prediction can be made using a recommendation algorithm based on the SOMICF technique which simply calculates the active user's average weighted rating on similar movies, i.e. uses the Weighted Sum prediction algorithm. In this case similar movies can include all movies currently in the map section the user is viewing, plus a few nearby movies not visible in the current map section. This prediction can only be made when the user has rated a minimum number of movies. However, because of size of the movie map, even if the user has rated 20 - 100 movies, it is overall not likely those movies lie in the visible neighborhood of an movie the user wants a prediction for, thus the user will not always receive a average rating prediction, however it will be clear to the user why he isn't receiving any average rating prediction, he hasn't yet rated any movies similar to the active movie.

Neighbor prediction The prediction can be made using a recommendation algorithm based on the SOMUCF technique which uses the Weighted Deviation From Mean prediction algorithm, which calculates the active user's neighbors wighted deviation from their mean rating and adds it to the active users mean rating to get a prediction on the active movie. The neighborhood radius can be set dynamically to accommodate a sufficiently large part of the map. This prediction can only be made when the user has rated a minimum number of movies. Since neighbors of the active user can have seen movies not within the neighborhood of any movie the active user has seen, the neighbors may be able to produce predictions with a much larger coverage than for the average rating predictions.

Goodness recommendation The recommendation indicates the level of safety and serendipity of the movie. The recommendation can be based on the SOMGOODNESSCF technique, but give a textual recommendation instead of a weighted prediction based on the SOMUCF and SOMICF technique.

Rating module

Adds the active user's rating on the active movie to the rating database R . If the number of ratings for the active user or the active movie exceeds a minimum ratings threshold, e.g. 5 or 20, then the active user or the active movie's position in the map will be recalculated.

Tagging module

Adds the active user's tag on the active movie to the tag database T .

Challenges

While the SOM introduces a model step that greatly reduces the constraints of the memory based recommendation part of the MOVSOM recommender system, some noteable challenges are however introduced related to the introduction of new movies and new users.

Let the original set of movies and users used to create the user and movie SOM be denoted by M and U respectively.

Introduction of new movies and new users

If a new movie m ∉ M is introduced to the system, since it has no ratings it is imposible to place it out on the movie map. As soon as a suitable number of users u ∈ U has rated the movie it can be placed out by comparing the movie's rating vector with all the reference vectors to find a closest reference vector, the map stretching module can then give the new movie a location in the stretched out map. (Variant of the SOM algorithm such as hierachical SOM's and growing SOM's can also be used to accommodate for new users and movies.)

But a user u ∈ U ratings on the the new movie will not affect the user's position in the user map. Thus nomatter how many new movies are introduced into the system, the user's position in the user map will never change when rating the new movies.

If a new user u ∉ U is introduced to the system, since the user has not rated any movies it is impossible to place the user in the user map. Hence the user can not initially receive predictions by any neighbors while browsing the movie map. The user can not receive any average rating prediction either as the user has not rated any movies. However, as soon as the user has rated a suitable number of movies m ∈ M the user can be placed out in the user map by comparing the user's rating vector with all the reference vectors to find a closest reference vector, the map stretching module can then give the new user a location in the stretched out user map. The user will then be able to receive predictions by his neighbors. But if the user only rates movies m ∉ M the user will never be possible to place out in the user map, and since the user hasn't rated any movies in M , the user wont be able to receive any predictions of either kind until the SOM's are retrained with the new users and movies.

The challenge with making predictions for new movies and new users can be overcome to a limit by adding new movies and new users to the movie map and user map reference vectors correspondingly. Ratings on new movies m ∉ M and by new users u ∉ U will then affect movie and user positions in both maps just likes movies m ∈ M and users u ∈ U will. By requiring new users to initially rate a random sample of 5 - 20 movies m ∈ M the system will be able to place out the new user, and having done so the system will be able to update the corresponding reference vector and reference vectors of neighboring nodes to resemble the new user rating vector. It is not as straightforward to solve the problem of new movies, but a similar approach where users are asked to rate new movies placed in a "New movies queue" could be used.

Retraining the SOM

As the number of users and movies grows, retraining the SOM to accommodate for new users and movies will become nececssary, however retraining will grow more and more timeconsuming as the dimensions of the user and movie rating vectors will grow. To this end some form of dimensionality reduction techniques must be applied. This however introduces a new challenge when introducing new users and movies as reference vectors are now on reduced dimensionality form. However at such a point the SOM and corresponding map would be made very large, reducing the frequent need for updating reference vectors. However, using variants of the SOM algorithm the dimensionality problem can possibly be solved more elegantly, such as creating lots of smaller maps and combining them into a large map (then retraining only has do be done on the smaller maps).

Copyright © 2005-2024 MOVSOM™ Five By Five