Personal digital assistants are designed to assist users in easy information retrieval or execute the tasks they are interested in. The conversational medium implies an additional level of intelligence but typically these systems do not support any reference to the user's past interactions. We propose a domain-agnostic approach that enables the system to address queries referring to the past by using an information retrieval approach to rank various entities for a given query. We also add semantic enrichment to the recall process by augmenting the entities with information from a knowledge graph and leverage that in the retrieval process. We mined user interactions for the Cortana digital assistant to extract queries with location and business entities and show that our technique can achieve an accuracy of 89.8% for such recall queries.