[AINews] Vision Everywhere: Apple AIMv2 and Jina CLIP v2 • ButtondownTwitterTwitter

Advancements in 'multimodal' (really just vision) embeddings are foundational, with releases from Apple and Jina in the past 48 hours. Apple's AIMv2 introduces joint visual and textual objectives, scaling up very well. Jina's CLIP v2 offers multilingual support, high image resolution, and efficient Matryoshka embeddings compression. Twitter recaps highlight cutting-edge AI models like Tülu 3, Apple's AIMv2, and Jina CLIP v2, as well as AI agents enhancements, AI ethics discussions, bug fixing, collaborations, and memes.

Theme 1: DeepSeek Emerges as Leading Chinese Open Source AI Company

• Chad DeepSeek created a model that matches or exceeds OpenAI's performance while using only 18,000 GPUs compared to OpenAI's 100,000 GPUs. The model shows significant advancements in model training efficiency and resource utilization. Community support focuses on Chinese open-source AI companies like Qwen, DeepSeek, and Yi achieving comparable results with fewer resources. Discussions cover model performance in mathematical reasoning and coding tasks, while highlighting limitations in creative reasoning and nuanced responses. Debate arises regarding political censorship in AI models and the impact of GPU export restrictions.

Theme 2: Innovative Model Architectures: Marco-o1 and OpenScholar

• Marco-o1 from MarcoPolo Alibaba integrates Chain of Thought, Monte Carlo Tree Search, and reasoning action to excel at writing and reasoning tasks across multiple domains. OpenScholar, developed by Allen Institute for AI and University of Washington, outperforms GPT-4o in scientific research with a self-feedback inference loop for output refinement. The model is available as an open-source model on Hugging Face, enhancing accessibility to researchers.

Theme 3: System Prompts and Tokenizer Optimization Insights

• Leaked system prompts from Vercel's V0 reveal the use of MDX components, specialized code blocks, and structured thinking for UI component generation. The system is noted to likely use Claude/Sonnet over GPT-4, with details on tokenizer issues affecting model performance in RPMax v1.3 models. The unconventional RPMax training approach aims to prevent specific character tropes in story generation.

Theme 4: INTELLECT-1: Distributed Training Innovation

• INTELLECT-1 completes training using distributed GPU resources worldwide, sparking community interest and comparisons to protein folding projects. Technical observations include a perplexity and loss bump coinciding with learning rate reduction. Discussions point to the model's open-source status setting it apart from existing models like Olmo and K2-65B, with an emphasis on INTELLECT-1's distributed compute contribution representing a unique approach.

The section discusses recent advancements in AI technology, including the Wave Network's use of complex token representation for high accuracy, debates over learnable positional embeddings like Mamba, and RNNs' capabilities in extrapolating out-of-distribution on algorithmic tasks. It also delves into insights on Muon techniques involving momentum and orthogonalization, emphasizing the importance of sufficient batch sizes. Furthermore, the update in Unsloth AI introduces vision finetuning for models like Llama 3.2 Vision, significantly boosting VRAM efficiency by 30-70% and supporting Pixtral finetuning in a free 16GB Colab environment. The update also merges models into 16bit for streamlined inference and provides long context support for vision models, enhancing overall usability.

• Users have shown interest in video fine-tuning services and servers, particularly referencing the Cogvideo model.
• While Cogvideo is well-known for video generation, users are exploring alternative fine-tuning options.
• Discussions also focus on downloading Stable Diffusion, its relevant use cases, and troubleshooting issues with the Ollama package.

This section highlights recent discussions in various AI-related channels on Discord. Topics include archiving previous reading groups, seeking help for quantum modeling, planning a pre-NeurIPS meetup, discussing the benefits of vector environments for reinforcement learning, and exploring the development of AI agents using open-source tools. Additional conversations delve into the performance of Test-Time Training on the Abstraction and Reasoning Corpus, innovative token representation in the Wave network, effectiveness of learnable positional embeddings, RNN extrapolation capabilities, and insights on Muon and orthogonalization. The discussions provide valuable insights into cutting-edge AI research and practical applications.

Excitement Over AI Art Turing Test:

A recent AI Art Turing Test has prompted discussions, with a member sharing their hope to test it with an expert in art restoration.

• The results from participants indicate mixed experiences, especially in distinguishing between AI and human-generated artworks.

Anthropic Receives $4 Billion from AWS:

Anthropic has secured an additional $4 billion investment from Amazon, solidifying AWS as its primary cloud and training partner.

• This partnership aims to enhance AI model training through collaboration on AWS Trainium hardware.

Launch of LTX Video Model:

Lightricks introduced the open-source LTX Video model, capable of generating 5-second videos in just 4 seconds on high-performance hardware.

• The model supports easy access through APIs, leading to discussions about balancing local processing versus cloud spending.

AI Vibrancy Tool from Stanford:

A member shared their enthusiasm for the Stanford AI Vibrancy Rankings Tool, which ranks countries on AI development metrics.

• The tool allows users to customize the weight of various indicators to reflect their own perspectives on AI vibrancy.

OpenAI's Data Deletion Incident:

Debate arose over OpenAI's recent accidental deletion of training findings, raising questions about competence in managing crucial data.

• While both OpenAI and NYT lawyers acknowledged it was a mistake, concerns about the handling and recovery of data persist.

This section delves into discussions within the OpenAI community regarding various AI-related topics and advancements. It covers areas such as voice cloning experiences, AI accents understanding, dystopian views of tech governance, ChatGPT integration ideas, Copilot image generation speculation, GPT struggles with vocabulary control, alternatives to Dall-E, debate over model ownership and monopolies, and availability of free image generation options.

The Marco-o1 paper delves into the exploration of reasoning models, specifically large reasoning models (LRM), and their application in handling open-ended resolutions in less structured environments. Recognized authors in AI research like Xin Dong, Yonggan Fu, and Jan Kautz have contributed significantly to advancing AI capabilities and introducing new methodologies.

OpenAI's Data Deletion Misstep:

Lawyers for The New York Times and Daily News are suing OpenAI for accidentally deleting data relevant to their copyright lawsuit. This incident occurred after over 150 hours of searching. Concerns have been raised about the potential impact on the case.

Prime Intellect Announces INTELLECT-1:

Prime Intellect completes the first decentralized training of a 10B model, mentioning ongoing post-training with @arcee_ai and an imminent full open-source release. Collaboration for building open-source AGI is encouraged.

Anthropic Secures Major AWS Partnership:

Anthropic expands collaboration with AWS, backed by a $4 billion investment from Amazon. This funding aims to establish AWS as their primary cloud and training partner to enhance AI technology development.

Skepticism Surrounding New AI Models:

There are concerns about the 10B model's effectiveness compared to existing models like LLaMA. Some believe the new model must excel to compete, but cynicism and skepticism in the AI landscape persist.

Navigating AI Development Cynicism:

Discussion on cynicism in AI development, highlighting skepticism towards new models such as Olmo. Despite challenges, advocates encourage pushing through critical feedback for continued development.

This section provides updates on various links shared in discussions and job openings posted in different channels. It includes discussions on tools like Character AI, torch, Vercel's v0, NPU acceleration solutions, Cohere API front-end, Tinygrad, and Mojo. Members share insights, inquire about new features, and discuss recent developments in AI research and model evaluations.

This section covers conversations on a variety of AI-related topics within different channels on Discord. It includes discussions on challenges with installing O1 on Linux, exploring free APIs, using DSPy for VLMs in projects, achievements in decentralized training with INTELLECT-1, excitement for fine-tuning in Axolotl, and interests in Neural Turing Machines and Differentiable Neural Computers.