Care for low birthweight (LBW) infants can contribute to psychological difficulties and stigma among mothers living with HIV, creating challenges for antiretroviral therapy (ART) adherence and retention in HIV care. We explored how caring for LBW infants affects maternal ART adherence and retention in care. We conducted 30 in-depth interviews with postpartum women living with HIV in Accra, Ghana: 15 with LBW infants and 15 with normal birthweight (NBW) infants. Compared to mothers with NBW infants, mothers with LBW infants described how caring for their newborns led to increased caregiver burden, prolonged hospital stays, and stigma-contributing to incomplete ART adherence and missed clinical appointments. For a few women, care for LBW infants created opportunities for re-engagement in HIV care and motivation to adhere to ART. Results suggest women living with HIV and LBW babies in Ghana face increased challenges that impact their adherence to care and ART.