Generative AI has ushered in a new era of possibilities, but its long-term potential hinges on solving the fundamental challenge of the staggering costly power demands of AI infrastructure and its impact on ROI. Scaling AI sustainably requires more than just incremental improvements—it demands a rethinking of the entire AI stack, starting with the chips, as well as impact of small language models, advanced cooling and energy solutions. The most immediate and impactful breakthrough opportunity in compute architecture lies in memory access for both performance and power, as GPU and accelerator computation and memory bandwidth account for a large majority of the total energy usage for generative AI inference. This session will explore transformative advancements across the AI stack, with focus on in-memory compute architectures capable of unlocking massive gains in power efficiency and affordability, along with discussion of AI infrastructure innovations that will open doors to greater economic viability of a wider range of generative AI applications in the long run.