In the history of AI, two main paradigms have been proposed to solve Sequential Decision Making (SDM) problems: Automated Planning (AP) and Reinforcement Learning (RL). Among the many proposals to unify both fields, the one known as neurosymbolic AI has recently attracted great attention. It combines the Deep Neural Networks used in modern RL with the symbolic representations typical of AP. The main goal of this PhD is to progress the state of the art in neurosymbolic AI for SDM, developing methods for both solving these problems and learning aspects of their structure.