Smart solutions that make use of machine learning and data analyses are on the rise. Big Data analysis is attracting more and more developers and researchers, and at least five requirements (Velocity, Volume, Value, Variety, and Veracity) show challenges in deploying such solutions. Across the globe, many Smart City initiatives are using Big Data Analytics as a tool for doing predictive analytics which can be helpful to human well being. This work presents a generic architecture named Machine Learning in Microservices Architecture (MLMA) that provides design patterns to transform a monolithic architecture of machine learning pipelines in microservices with separate roles. We present two case studies deployed to a Smart City initiative, where we discuss how each component of the architecture applied in specific applications that use predictions with machine learning. Among the benefits of this architecture, we argue prediction performance, scalability, code maintenance and reusability makes such transition a natural trend in Big Data and machine learning applications.