Magic's Progress on Ultra-Long Context Models

Magic is a company focused on revolutionizing software development with its innovative ultra-long context models. These models, known as LTM (Long-Term Memory) models, have the ability to process and understand up to 100 million tokens of context during inference. This is a significant leap forward from traditional AI models that rely on short contexts.

• Magic's LTM models can access and analyze your code, documentation, and libraries, even those not publicly available on the internet.
• This technology has the potential to significantly enhance code synthesis, making it easier and more efficient to develop software.

Evaluating the Limits of Context Windows

The current evaluation methods for long context models have limitations. These methods often rely on "semantic hints" that make it easier for models to identify the relevant information. This can lead to misleading results, where traditional RNNs and SSMs appear to perform well despite their inherent limitations.

• Magic has developed a new evaluation method called HashHop, which eliminates these semantic hints. This method forces models to store and retrieve information from the maximum possible information content, providing a more accurate measure of their ability to handle large contexts.
• HashHop involves prompting models with hash pairs and asking them to complete chains of hashes. This requires models to attend and jump across multiple points of the entire context in latent space.

Magic's LTM-2-mini Model and its Capabilities

Magic has trained its first 100 million token context model, LTM-2-mini. This model represents a significant milestone in the development of ultra-long context models. Its sequence-dimension algorithm is significantly more efficient than the attention mechanism used in large language models like Llama 3.1.

• LTM-2-mini has demonstrated promising results, particularly when trained on hashes with chain of thought. It can successfully build complex circuits and perform multiple hops in a single step.
• Magic has also trained a prototype model on text-to-diff data, showcasing its capability for code synthesis. While the model's abilities are still under development, it has produced some impressive results, such as creating a calculator using a custom in-context GUI framework.
• Magic is currently training a larger LTM-2 model on its new supercomputer, with the aim of further enhancing its capabilities.

Magic's Partnership with Google Cloud and NVIDIA

Magic has partnered with Google Cloud and NVIDIA to build its next-generation AI supercomputers. These supercomputers, Magic-G4 and Magic-G5, will leverage the power of NVIDIA H100 Tensor Core GPUs and NVIDIA GB200 NVL72 GPUs, respectively, to accelerate the training and deployment of Magic's models.

• The partnership with Google Cloud will provide Magic with access to a rich ecosystem of cloud services and the ability to scale its operations quickly.
• The NVIDIA GB200 NVL72 GPUs will significantly improve the efficiency of training and inference for Magic's models.

Magic's Recent Funding and Future Plans

Magic has raised a total of $465 million, including a recent investment of $320 million from leading investors such as Eric Schmidt, Jane Street, Sequoia, Atlassian, and others. This funding will support Magic's continued development and expansion.

• Magic's goal is to make AI more accessible and powerful, enabling developers to build complex applications quickly and efficiently. They believe that inference-time compute is the next frontier in AI.
• To achieve this goal, Magic is actively hiring engineers and researchers to accelerate its development of ultra-long context models and related technologies.
• Magic plans to scale its operations to tens of thousands of GB200s over time, further advancing its capabilities in the field of AI. They are also hiring supercomputing and systems engineers to work on this ambitious project.
• Magic is committed to developing AI responsibly and safely, and they are hiring a Head of Security to lead their efforts in cybersecurity and regulatory compliance.

Magic's Role in the Future of Software Development

Magic is at the forefront of AI innovation, pushing the boundaries of what's possible with ultra-long context models. Its focus on software development, combined with its partnership with Google Cloud and NVIDIA, positions it as a leading player in the future of AI.

• Magic's advancements in ultra-long context models have the potential to transform software development, making it easier, more efficient, and more accessible for developers. They aim to empower developers to create complex applications with speed and agility, unlocking new possibilities in software development.
• With its ambitious plans for scaling its supercomputing infrastructure and its commitment to responsible AI development, Magic is well-positioned to shape the future of AI and software development.