Artificial Intelligence (AI) and Machine Learning (ML) are shaping marketing activities through digital innovations. Competition is a familiar concept for any digital retailer, and the digital transformation provides hopes for gaining a competitive edge over competitors. Those who do not adopt digital innovations risk getting outcompeted by those who do. This study aims to identify AI mar-keting (AIM) adoptions used for ad optimization with Reinforcement Learning (RL). A scoped literature review is used to find ad optimization adoptions re-search trends with RL in AIM. Scoping this is important both to research and practice as it provides spots for novel adaptations and directions of research of digital ad optimization with RL. The results of the review provide several differ-ent adoptions of ad optimization with RL in AIM. In short, the major category is Ad Relevance Optimization that takes several different forms depending on the purpose of the adoption. The underlying found themes of adoptions are Ad Attractiveness, Edge Ad, Sequential Ad and Ad Criteria Optimization. In conclusion, AIM adoptions with RL is scarce, and recommendations for future research are suggested based on the findings of the review.