Enterprises, governments, and other large organizations are deploying AI in increasingly complex workflows where multiple models work in conjunction with one another to perform multi-step processes beyond the capabilities of traditional AI. The capacity to power these complex processes goes beyond what GPUs are capable of. In this discussion we will examine what is required to build an infrastructure that can hold hundreds of models in memory and switch between them in microseconds to meet the throughput, latency, scale, and multimodality requirements necessary to power the next wave of AI, while simultaneously reducing the footprint, power, and cooling requirements necessary for the modern data center.