As already mentioned earlier, I want to validate my hybrid proposal for obtaining the dimensions of a conceptual space in a second study, which focuses on the domain of shapes. Today I will start reporting on joint work with Margit Scheibel on obtaining a similarity space for the shape domain based on psychological data. This is the first step of the proposed hybrid procedure and will be followed by training a neural network. But for now, let’s focus on obtaining the similarity spaces.
The Stimuli
In our study, we used 60 line drawings of common objects from 12 different categories. You can see one example image from each of these categories in Figure 1.
Six of these categories contain natural objects (e.g., vegetables, insects) and the six remaining categories contain man-made object (e.g., electrical appliances, tools). Moreover, six categories contain only items which are visually similar to each other (e.g., all tools look alike) while the remaining six categories are visually variable (e.g., the fruit category containing both a banana which is elongated and an apple which is round).
As we are interested in a conceptual space for the shape domain, we would like to know how similar the shapes of these object are to each other. By using line drawings, we do not convey information about color or texture to the participants of our psychological experiments. Moreover, by making sure that all drawings are equally large, we also exclude variability with respect to the size domain. Thus, we can be relatively sure that participants in our study are not distracted by visual information from irrelevant domains.
The Similarity Ratings
As described in an earlier post about multidimensional scaling, we first need to obtain similarity ratings before we can construct a conceptual space. As we are interested in a similarity space for shapes, we therefore asked participants of our study to rate the visual similarity of two images on a 5-point scale. As argued above, our images do not contain information about color, texture, or size, hence we can assume that visual similarity in this case corresponds to shape similarity.
However, participants may be influenced by the overall conceptual similarity of the objects (e.g., a horse and a ladybug both being animals despite not looking alike). In order to ensure that the visual similarity ratings indeed refer to visual similarity and not to general conceptual similarity, we therefore also conducted a second experiment where we explicitly asked for the conceptual similarity for all pairs of images.
We then compared the ratings with respect to visual similarity to the ones with respect to conceptual similarity. In Figure 2, you can see some results of this analysis.
In the left column, we have pairs of images with a low visual similarity but a high conceptual similarity. As one can see, these images tend to belong to the same or a similar category, while having clearly different shapes. In the middle column, we see image pairs with high visual similarity and low conceptual similarity. Not surprisingly, we find here images from different categories that are semantically unrelated, but which have a similar overall shape. Finally, in the right column, we have some examples where visual and conceptual similarity are identical – either because the images belong to the same category and have a similar shape, or because they belong to different categories and do not look alike.
The observations from Figure 2 therefore illustrate that our ratings on visual similarity do indeed refer to shape similarity and not to general conceptual similarity. It therefore makes sense to use them for constructing a similarity space of shapes.
The Psychological Candidate Features
You may remember that a conceptual space is spanned by interpretable dimensions. However, the coordinate axes of similarity spaces obtained with MDS are not necessarily meaningful. We have therefore collected for each image some ratings with respect to the following three shape features which are often discussed in the psychological literature:
- FORM: Is the object elongated or blob-like?
- LINES: Are the lines in the drawing mostly straight or mostly curved?
- ORIENTATION: Is the object horizontally, diagonally, or vertically oriented?
Figure 3 illustrates the ratings with respect to the features FORM and LINES for the individual items.
As you can see, elongated objects (such as axe or banana) have low values with respect to the FORM feature, while blob-like objects (such as dishwasher or lemon) receive high values. On the other hand, the LINES feature distinguishes objects made of straight lines (axe and dishwasher, having low values) from objects containing curved lines (banana and lemon, having high values).
One could say that Figure 3 gives us a two-dimensional conceptual space for shapes, using the features FORM and LINES. However, we don’t know whether the distances in this space actually relate well to the similarity ratings collected in our experiments. Moreover, there might be more to shape similarity than just the three features we consider. We therefore conducted a more elaborate analysis.
Our Analysis
So first of all, we used multidimensional scaling (MDS) on the similarity ratings with respect to visual similarity in order to obtain similarity spaces. More specifically, we considered spaces with 1 to 10 dimensions. In order to evaluate the quality of these spaces, we used three core assumptions of the conceptual spaces framework:
- Distance in the conceptual space is inversely related to similarity. We check whether this is the case by computing the correlation between distances and similarities for the similarity spaces as well as for three baselines.
- Concepts are represented by small, non-overlapping convex regions. We expect that especially the categories based on visual similarity obey this assumption, as they can be defined based on the shape domain.
- The conceptual space is spanned by interpretable dimensions. While the individual coordinate axes of the MDS solutions may not be interpretable, we expect that we can identify directions in the similarity space which correspond to the three psychological shape features.
These three analysis steps will be covered in future blog posts. Once we’re done with analyzing the similarity spaces (and picking promising candidates), we can then start to apply machine learning in the second step of the hybrid procedure.
5 thoughts on “A Similarity Space for Shapes (Part 1)”