The problem we address in this paper is that the potential impact of Large Language Models (LLMs) on the research practice in information systems is not well understood. The focus has been on how LLMs could support literature review processes. Therefore, this paper aims to advance knowledge on how Large Language Models (LLMs) could support knowledge exploration through literature reviews. The knowledge contribution consists of meta-requirements that inform the design of LLM-based tools assisting knowledge exploration. The meta-requirements are theoretically justified by scrutinizing established IS literature review methodologies, reported challenges of LLMs and design process experiences. Furthermore, we introduce an LLM supported literature review process model that maps the relationships between the meta-requirements and specific phases of the process model. This work contributes to the field by providing a foundation for designing transparent, controllable, and resource-efficient tools for knowledge exploration, and supporting the rigor of knowledge exploration in information systems research.