The advent of foundation models has led to impressive advancements in all areas of natural language processing. However, their huge size poses limitations due to the significant computational costs associated with their use or adaptation. When applying them to specific tasks, fundamental questions arise: do we actually need all the architectural complexity of large and – by design – general-purpose foundation models? Can we optimize them to achieve higher efficiency? These questions spark interest in research aimed at reducing models’ size, or deploying efficient decoding strategies, so as to accomplish the same tasks while maintaining or even improving performance. Success in this direction would lead to significant practical and economic benefits (e.g., lower adaptation costs, the possibility of local deployment on small-sized hardware devices), as well as advantages from an environmental impact perspective towards sustainable AI. Focusing on automatic translation, this PhD aims to understand the functioning dynamics of general-purpose massive foundation models and explore possibilities to streamline them for specific tasks. Possible areas of interest range from textual and speech translation (e.g., how to streamline a massively multilingual model to best handle a subset of languages?) to scenarios where the latency is a critical factor, such as in simultaneous/streaming translation (e.g., how to streamline the model to reduce latency?), to automatic subtitling of audiovisual content (e.g., how to streamline the model without losing its ability to generate compact outputs suitable for subtitling?).