For example, the value of a particular share at the stock market at a certain point in time is a single value, but it is likely due to many influences which are mostly unknown. It is completely conceivable for many latent variables to influence few observable ones. Note, however, that it's not necessary for the latent space to be smaller than the observable space. That would be a low dimensional latent space. These are likely to be way less then thousands, maybe only a dozen or two. However, each spectrum can be quite well described by the star's temperature (through the black body radiation law) and the concentration of different elements (for the absorption lines). Modern spectrometers measure the intensity at thousands of wavelengths. A spectrum is a long vector of values, light intensities at many different wavelengths. In such cases it may be useful to perform some kind of dimensionality reduction.Īs a real-world example, consider spectra of light-emitting objects, like stars. In practice, you often have many, maybe even millions of observable variables (think of pixel values in images), but they can be sufficiently well computed from a much smaller set of latent variables. Z = np.sqrt(phi) # 3rd observable: z-coordinate Y = np.exp(phi) * np.cos(25 * phi) # 2nd observable: y-coordinate X = phi * np.sin(25 * phi) # 1st observable: x-coordinate phi = np.linspace(0, 1, 100) # the latent variable However, each point is uniquely determined by a single latent variable, $\varphi$ ( phi in the python code). Let me use this image, adapted from GeeksforGeeks, to visualise the idea:Įach observable data point has four visible features: the $x, y,$ and $z$-coordinates, and the colour. the observable variables can be derived (computed) from the latent ones. Latent variables are variables which are not directly observable, but which are $-$ up to the level of noise $-$ sufficient to describe the data. Latent space is a vector space spanned by the latent variables. If this answer helped, please don't forget to up-vote it :) Low dimensional latent space aims to capture the most important features/aspects required to learn and represent the input data (a good example is a low-dimensional bottleneck layer in VAEs). High dimensional latent space is sensitive to more specific features of the input data and can sometimes lead to overfitting when there isn't sufficient training data. The terms "high dimensional" and "low dimensional" help us define how specific or how general the kinds of features we want our latent space to learn and represent. ![]() In all the above examples, we quantitatively represent the complex observation space with a (relatively simple) multi-dimensional latent space that approximates the real latent space of the observed data. ![]() ![]() ( PyLDAvis provides a good visualization of topic models)Ĥ) VAEs & GANs aim to obtain a latent space/distribution that closely approximates the real latent space/distribution of the observed data. Hence, learning a latent space would help the model make better sense of observed data than from observed data itself, which is a very large space to learn from.ġ) Word Embedding Space - consisting of word vectors where words similar in meaning have vectors that lie close to each other in space (as measured by cosine-similarity or euclidean-distance) and words that are unrelated lie far apart (Tensorflow's Embedding Projector provides a good visualization of word embedding spaces).Ģ) Image Feature Space - CNNs in the final layers encode higher-level features in the input image that allows it to effectively detect, for example, the presence of a cat in the input image under varying lighting conditions, which is a difficult task in the raw pixel space.ģ) Topic Modeling methods such as LDA, PLSA use statistical approaches to obtain a latent set of topics from an observed set of documents and word distribution. The motivation to learn a latent space (set of hidden topics/ internal representations) over the observed data (set of events) is that large differences in observed space/events could be due to small variations in latent space (for the same topic). Just as we, humans, have an understanding of a broad range of topics and the events belonging to those topics, latent space aims to provide a similar understanding to a computer through a quantitative spatial representation/modeling. Latent space refers to an abstract multi-dimensional space containing feature values that we cannot interpret directly, but which encodes a meaningful internal representation of externally observed events.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |