Generative artificial intelligence (AI), which uses the probability distribution of real-world data and generates new content by learning it, is becoming a powerful tool for use in medicine. In this paper, we will systematically review the progress made in research, major applications, and challenges of the use of generative AI in the medical field. Firstly, we give a brief introduction to the probabilistic background of generative models and then describe four kinds of models, including generative adversarial network, diffusion model, variational autoencoder (VAE), and large language model. Technical features and their potential usage in the medical field will be discussed. Secondly, we systematically investigate their applications in five fields, which include clinical decision-making, synthetic medical imaging, drug discovery, processing of medical literature, and medical education. From our review, it is clear that generative AI has a lot of capabilities when it comes to increasing efficiency and accuracy in health care delivery, drug discovery, and medical education. Nonetheless, there are several challenges associated with this technology, including factual errors, hallucinations, data privacy concerns, algorithmic bias, and difficulties in attributing responsibility. In light of these limitations, it is vital for generative AI technology to play the role of an assistant to medical professionals as opposed to taking their roles. The future will require the creation of reliable technologies like uncertainty quantification, explainable AI, and retrieval-augmented generation (RAG).