Prescription (aka Rx) drugs can be easily overprescribed and lead to drug abuse or opioid overdose. Accordingly, a state-run prescription drug monitoring program (PDMP) in the United States has been developed to reduce overprescribing. However, PDMP has limited capability in detecting patients' potential overprescribing behaviors, impairing its effectiveness in preventing drug abuse and overdose in patients. In this paper, we propose a novel model RxNet, which builds 1) a dynamic heterogeneous graph to model Rx refills that are essentially prescribing and dispensing (P&D) relationships among various patients, 2) an RxLSTM network to explore the dynamic Rx-refill behavior and medical condition variation of patients, and 3) a dosing-adaptive network to extract and recalibrate dosing patterns and obtain the refined patient representations which are finally utilized for overprescribing detection. The extensive experimental results on a one-year state-wide PDMP data demonstrate that RxNet consistently outperforms state-of-the-art methods in predicting patients at high risk of opioid overdose and drug abuse.