The goal of the thesis is to develop AI models for speech processing which are aware of the computational resources and of the application requirements and are capable of dynamically adapting in order to meet such limitations. This entails not only the search for a trade-off between resources and inference performance but also the possibility to dynamically exploit additional computational resources, eventually expanding the model. The project will address both training and inference phases, starting from state of the art supervised techniques as model compression, neural architecture search, distillation and continual learning and pushing them towards continuous and unsupervised solutions.