Recent work in image manifold learning has shown the prevalence of unsupervised methods that provide compact representation and perceptually meaningful organization of images in certain types of natural image sets. However, in situations where a discriminant factor needs to be discovered from an image set in which multiple latent variation factors exist, unsupervised methods are often limited. Whereas, supervised manifold learning approaches can be robust against irrelevant factors by leveraging image labels which impose additional constraints on the relationships between images. Nonetheless, ground truth labels are usually too costly to obtain and sometimes not entirely available. In this dissertation, we are interested in learning on image manifolds with weak supervision. The weakly supervised learning methods that we present are capable of mitigating the manual labeling effort required by supervised methods. In particular, we consider three variants of weakly supervised learning on image manifolds: (1) image labels not explaining all latent factors of image variation, (2) image labels which are heavily corrupted, and(3) image labels being partly available. We propose an algorithmic solution for each problem and evaluate the performance of the proposed algorithms quantitatively and qualitatively on a wide range of data sets.