In soft matter consisting of many deformable objects, object shapes often carry important information about local forces and their interactions with the local environment, and can be tightly coupled to the bulk properties and functions. In a concentrated emulsion, for example, the shapes of individual droplets are directly related to the local stress arising from interactions with neighboring drops, which in turn determine their stability and the resulting rheological properties. Shape descriptors used in prior work on single drops and dilute emulsions, where droplet-droplet interactions are largely negligible and the drop shapes are simple, are insufficient to fully capture the broad range of droplet shapes in a concentrated system. This paper describes the application of a machine learning method, specifically a convolutional autoencoder model, that learns to: (1) discover a low-dimensional code (8-dimensional) to describe droplet shapes within a concentrated emulsion, and (2) predict whether the drop will become unstable and undergo break-up. The input consists of images (N = 500 002) of two-dimensional droplet boundaries extracted from movies of a concentrated emulsion flowing through a confined microfluidic channel as a monolayer. The model is able to faithfully reconstruct droplet shapes, as well as to achieve a classification accuracy of 91.7% in the prediction of droplet break-up, compared with ∼60% using conventional scalar descriptors based on droplet elongation. It is observed that 4 out of the 8 dimensions of the code are interpretable, corresponding to drop skewness, elongation, throat size, and surface curvature, respectively. Furthermore, the results show that drop elongation, throat size, and surface curvature are dominant factors in predicting droplet break-up for the flow conditions tested. The method presented is expected to facilitate follow-on work to identify the relationship between drop shapes and the interactions with other drops, and to identify potentially new modes of break-up mechanisms in a concentrated system. Finally, the method developed here should also apply to other soft materials such as foams, gels, and cells and tissues.