GPT Store, text-to-3d in under 10 seconds, DeepSeekMoE 16B, jailbreaking advanced LLMs, LLaVA-ϕ, Microsoft's open-source agent framework, and more
Greetings and welcome to this week's AI Brews for a concise roundup of the week's major developments in AI.
In today’s issue (Issue #47 ):
AI Pulse: Weekly News & Insights at a Glance
AI Toolbox: Product Picks of the Week
AI Skillset: Learn & Build
🗞️🗞️ AI Pulse: Weekly News & Insights at a Glance
🔥 News
OpenAI launched the GPT Store for finding GPTs. In Q1, a GPT builder revenue program will be launched. As a first step, US builders will be paid based on user engagement with their GPTs. A new ChatGPT Team' plan was also announced. [Details].
DeepSeek released DeepSeekMoE 16B, a Mixture-of-Experts (MoE) language model with 16.4B parameters. It is trained from scratch on 2T tokens, and exhibits comparable performance with DeepSeek 7B and LLaMA2 7B, with only about 40% of computations [Details].
Microsoft Research introduced TaskWeaver – a code-first open-source agent framework which can convert natural language user requests into executable code, with additional support for rich data structures, dynamic plugin selection, and domain-adapted planning process [Details |GitHub].
Open Interpreter, the open-source alternative to ChatGPT's Code Interpreter, that lets LLMs run code (Python, Javascript, Shell, and more) locally gets a major update. This includes an OS Mode that lets you instruct Open Interpreter to use the Computer API to control your computer graphically [Details].
AI startup Rabbit released r1, an AI-powered gadget that can use your apps for you. Rabbit OS is based on a “Large Action Model”. r1 also has a dedicated training mode, which you can use to teach the device how to do something. Rabbit has sold out two batches of 10,000 r1 over two days [Details].
Researchers introduced LLaVA-ϕ (LLaVA-Phi), a compact vision-language assistant that combines the powerful opensourced multi-modal model, LLaVA-1.5 , with the best-performing open-sourced small language model, Phi2. This highlights the potential of smaller language models to achieve sophisticated levels of understanding and interaction, while maintaining greater resource efficiency [Details].
Luma AI announced Genie 1.0, a text-to-3d model capable of creating any 3d object in under 10 seconds. Available on web and in Luma’s iOS app [Link]
Researchers achieved a 92% success rate in jailbreaking advanced LLMs, such as Llama 2-7b Chat, GPT-3.5, and GPT-4, without any specified optimization. Introduced a taxonomy with 40 persuasion techniques from decades of social science research and tuned LLM to try all of them to generate persuasive adversarial prompts (PAPs) & attack other LLMs [Details].
Microsoft Phi-2 licence has been updated to MIT [Link].
PolyAI introduced Pheme, a neural, Transformer-based TTS framework that aims to maintain high-quality speech generation both in multi-speaker and single-speaker scenarios [Details| Hugging Face Demo].
Runway opens registration for the second edition of GEN:48, an online short film competition where teams of filmmakers have 48 hours to ideate and execute a 1-4 minute film [Details].
Meta AI present MAGNET (Masked Audio Generation using Non-autoregressive Transformers) for text-to-music and text-to-audio generation. The proposed method is able to generate relatively long sequences (30 seconds long), using a single model and has a significantly faster inference time while reaching comparable results to the autoregressive alternative [Details].
ByteDance introduced MagicVideo-V2, a multi-stage Text-to-video framework that integrates Text-to-Image , Image-to-Video, Video-to-Video and Video Frame Interpolation modules into an end-to-end video generation pipeline, demonstrating superior performance over leading Text-to-Video systems such as Runway, Pika 1.0, Morph, Moon Valley and Stable Video Diffusion model via user evaluation at large scale [Details].
Mistral AI released paper of Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model, on Arxiv [Link].
Amazon revealed new generative AI-powered Alexa experiences from AI chatbot platform Character.AI, AI music company Splash and Voice AI game developer Volley [Details].
Researchers from Singapore University of Technology and Design released TinyLlama, an open-source 1.1B language model pretrained on around 1 trillion tokens, with exactly the same architecture and tokenizer as Llama 2 [Paper | GitHub].
Getty Images released Generative AI By iStock, powered by NVIDIA Picasso, providing designers and businesses with a text-to-image generation tool to create ready-to-license visuals, with legal protection and usage rights for generated images included [Details].
Volkswagen plans to install OpenAI’s ChatGPT into its vehicles starting in the second quarter of 2024 [Details].
Microsoft and Department of Energy’s Pacific Northwest National Laboratory (PNNL) used AI to to screen over 32 million candidates to discover and synthesize a new material that has potential for resource-efficient batteries [Details].
Assembly AI announced significant speed improvements along with price reduction to their API’s inference latency with the majority of audio files now completing in well under 45 seconds regardless of audio duration [Details].
OpenAI has started rolling out an experiment personalization ability for ChatGPT, empowering it to carry what it learns between chats, in order to provide more relevant responses [Details].
🔦 Weekly Spotlight
Toms’ Guide: CES 2024 — these are the 11 best AI gadgets so far [Link].
In the largest survey of its kind, 2,778 researchers who had published in top-tier artificial intelligence venues gave predictions on the pace of AI progress and the nature and impacts of advanced AI systems [Paper].
Google Research video: Can AI decode the music you're listening to? [Link].
🔍 🛠️ AI Toolbox: Product Picks of the Week
Rosebud AI: A platform for AI game creation, AI Text-to-Skybox generation, AI generated consistent game assets and AI avatar animation.
Offline Chat: Private AI: an iOS app for AI ChatBot that runs entirely on your device without the Internet. The AI is a fine tuned large language model based on Mistral 7B 0.1, quantized to 3-bit.
GPT Auth: authentication & analytics built specifically for GPTs on the GPT Store.
📕 📚 AI Skillset: Learn & Build
A simple guide to local LLM fine-tuning on a Mac with Apple’s MLX framework [Link].
Making an intelligent Robot NPC using Hugging Face and Unity [Link].
Fine-Tune Mixtral 8x7B (Mistral's Mixture of Experts MoE) Model - Walkthrough Guide [Link].
Use LlamaIndex to Build an AI Shopping Assistant with RAG and Agents [Link]
Thanks for reading and have a nice weekend! 🎉 Mariam.