Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Before any mathematical development, we must first understand the foundation of learning and how it is closely linked to the concept of errorThe Hypothetical CookImagine that, on any given day, you decide to replicate a delicacy you ate at a renowned restaurant. You remember the taste of this delicacy perfectly. Based on this, you search for the recipe online and attempt to reproduce it at home.Let’s denote the taste of the delicacy you ate at the restaurant as T, which will represent the expected taste, your target. Based on the recipe you found online, you hope to achieve this goal, i.e., the taste T.To reproduce this recipe, you follow all the indicated steps, use all the ingredients, th…

Enabling the algorithm to work its magicPhoto by Mourizal Zativa on UnsplashYou must have heard the saying “garbage in, garbage out.” This saying is indeed applicable when training machine learning models. If we train machine learning models using irrelevant data, even the best machine learning algorithms won’t help much. Conversely, using well-engineered meaningful features can achieve superior performance even with a simple machine learning algorithm. So, then, how can we create these meaningful features that will maximize our model’s performance? The answer is feature engineering. Working on feature engineering is especially important when working with traditional machine learning algori…

An explanation of the backpropagation through time algorithmContinue reading on Towards Data Science »

Be mindful of the measure you choosePhoto by Isaac Smith on UnsplashTesting and benchmarking machine learning models by comparing their predictions on a test set, even after deployment, is of fundamental importance. To do this, one needs to think of a measure or score that takes a prediction and a test point and assigns a value measuring how successful the prediction is with respect to the test point. However, one should think carefully about which scoring measure is appropriate. In particular, when choosing a method to evaluate a prediction we should adhere to the idea of proper scoring rules. I only give a loose definition of this idea here, but basically, we want a score that is minimize…

Seeking a system that works for current and future codesContinue reading on Towards Data Science »

I Built a Reusable Dashboard for Read the Docs Traffic Analytics Using Vizro-AI(In less than 50 lines of code)The resulting dashboard from typical traffic dataIn this article, I’ll explain how I built a dashboard to visualize the traffic data for some documentation I maintain as a technical writer. I have few design skills and limited Python experience, so needed a simple, low-code approach to show the impact and usage of the documentation I maintain. This turned out to be an open-source solution: Vizro as a template for a low-code dashboard, and Vizro-AI to build the individual charts with generative AI.TL;DR?If you want to jump right in, you can find the Jupyter Notebook code for the dash…

Learn How to Visualize Time Series Data from Satellite Imagery on a Map Using Folium and Plotly Libraries (Python)Continue reading on Towards Data Science »

Asking for feedback is a secretly powerful tool in data work. Let’s talk about why, and how to do it wellPhoto by Priscilla Du Preez 🇨🇦 on UnsplashA recent conversation with a fellow data practitioner sparked an idea that I want to share today. What is your process for conducting data analysis or modeling, and what do you consider important but perhaps unsung parts of getting the job done well? I realized as we were talking that getting feedback from other people as I go through the work is an extremely important part of my process, but it’s not actually something that is explicitly instructed to junior practitioners in my experience. I thought it would be useful to explain how I do this an…

This blog post describes practices for “what good looks like” in AIML UX, suggests examples, and maps a path forward for product leadersMuch time, blood, sweat, tears, and ink have been spilled in recent years to focus on Artificial Intelligence and Machine Learning (AIML) models, their size and performance, their rapid evolution, their training costs, their security, their latency, and the various model hosting choices in the cloud, locally, and at the edge. One overlooked area has been the Final Mile product User Experience (UX), and how best to incorporate AIML into products.This blog post describes several practices for “what good looks like” in AIML UX, suggests reference examples, and…

How to get insights about model robustness/reliability, training data size adequacy, and moreContinue reading on Towards Data Science »

Advice and tips that helped me get my first promotion as a data scientistContinue reading on Towards Data Science »

Exploring LLMs for ICD Coding — Part 1Building automated clinical coding systems with LLMsClinical coding isn’t common parlance, but it significantly impacts everyone who interacts with the healthcare system in most countries. Clinical coding involves translating and mapping medical information from patient health records, such as diagnoses and procedures, into standardized numeric or alphanumeric codes. These codes are crucial for billing, healthcare analytics, and ensuring that patients receive appropriate care.A representative workflow of automated ICD coding (Image by Author)Clinical coding is typically performed by human coders with medical expertise. These coders navigate complex and …

Learn How to Enhance Your Data Analysis for Advanced Computational Tasks, from Innovative Optimization Strategies to Foundational Machine…Continue reading on Towards Data Science »

Looked at from another angle, there is much more noise than signal in financial data.

Another financial market application of LLMs might be synthetic data creation [4,8].

Then precious real market data could be employed to fine-tune the predictions and determine precisely the optimal speed to trade.

Financial market practitioners are often interested in extreme events, the times when trading strategies are more likely to experience significant gains or losses.

CitationFor attribution in academic contexts or books, please cite this work asRichard Dewey and Ciamac Moallemi, "Financial Market Applications of LLMs," The Gradient, 2024

All of these terms (“AI”, “gender”, and “bias”) can be somewhat overused and ambiguous.

A Short History of Studying Gender Bias in AIHere, I cover a very small sample of papers I’ve found influential studying gender bias in AI.

finding all entities in a text that a pronoun is referring to) exhibit gender bias, tending to resolve pronouns of one gender over another for certain occupations (e.g.

This article mainly focused on gender bias — and particularly, on binary gender.

AcknowledgementsThis post was originally posted on Art Fish IntelligenceCitationFor attribution in academic contexts or books, please cite this work asYennie Jun, "Gender Bias in AI," The Gradient, 2024@article{Jun2024bia…

As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics.

Here we’ll discuss:The advantages (and disadvantages) of Mamba (🐍) vs Transformers (🤖),Analogies and intuitions for thinking about Mamba, andWhat Mamba means for Interpretability, AI Safety and Applications.

The Mamba BlockLike a Transformer made up of stacked transformer blocks, Mamba is made up of stacked Mamba blocks as above.

The Mamba authors write, “the efficiency vs. effectiveness tradeoff of sequence models is characterised by how well they compress their state”.

Thanks to Gonçalo for reading an early draft, Jaden for the nnsight library …

We've just seen 3 prominent families of LLM usage in self-driving cars: Perception, Planning, and Generation.

The first wave of papers mentioning LLMs in Self-Driving Cars is from mid-2023, so let's give it some time.

Next StepsIf you want to get started on LLMs for self-driving cars, there are several things you can do:⚠️ Before this, the most important : If you want to keep learning about self-driving cars.

Author BioJérémy Cohen is a self-driving car engineer and founder of Think Autonomous, a platform to help engineers learn about cutting-edge technologies such as self-driving cars and advanced Computer Vision.

CitationFor attribution in academic contexts or books, please cite this work…

Beyond the requirements of semantic similarity, there are no constraints on what embedding must be assigned for a given text input.

What if someone hacks into the database and gains access to all your text embedding vectors – would this be bad?

From text to embeddings...back to textThe problem of recovering text from embeddings is exactly the scenario we tackle in our paper Text Embeddings Reveal As Much as Text (EMNLP 2023).

Scaling and future workThe fact that text embeddings can be perfectly inverted raises many follow-up questions.

CitationFor attribution in academic contexts or books, please cite this work asJack Morris, "Do text embeddings perfectly encode text?

If your model latches on to these during training, it will appear to work well, but may not work on new data.

This happens when the model training pipeline has access to information it shouldn’t have access to, particularly information that confers an advantage to the model.

This means that knowledge of the test data is implicitly entering the model training pipeline, even if it is not explicitly used to train the model.

Well, this is a common thing to do, but if you’re developing a model iteratively and using the same test set to evaluate the model after each iteration, then you’re basically using that test set to guide the development of the model.

CitationFor attribution in academic cont…

Evolution of single-cell sequencing over timeHaving explored the panorama of single-cell sequencing, let us now delve into the role of deep learning in the context of single-cell sequencing.

Deep Learning on single-cell sequencingDeep learning is increasingly employed in single-cell analysis due to its capacity to handle the complexity of single-cell sequencing data.

The deep learning approach, however, autonomously captures relevant characteristics from single-cell sequencing data, addressing the heterogeneity between single-cell sequencing experiments, as well as the associated noise and sparsity in such data.

As we explore the reasons behind using deep learning in single-cell sequencing …

In order to obtain a license or permit from FERC, hydroelectric dam operators must submit detailed plans and studies demonstrating that their facility meets regulations.

Typically, a hydroelectric dam requires lots of space to store water on one side of it, which means they tend to be located away from population centers.

Enter Computer VisionSome organizations are exploring the use of computer vision and machine learning to significantly automate fish counting.

The annotated images are then used to train a machine learning model.

CitationFor attribution of this in academic contexts or books, please cite this work as:Kevin McCraney, "Salmon in the Loop", The Gradient, 2023.

In recent work with computer networking and machine learning collaborators from ETH Zürich, we studied the applicability of neural algorithmic reasoning in computer networking [27].

Our proposal, the neural algorithmic reasoning blueprint [32], aims to bridge this divide by neuralising the target algorithm.

Neural Algorithmic Reasoning with Causal Regularisation.

Neural Algorithmic Reasoning.

CitationFor attribution in academic contexts or books, please cite this work asPetar Veličković, "Neural Algorithmic Reasoning", The Gradient, 2023.

This community has developed an extensive vocabulary around theories of AI safety and alignment, many first introduced as detailed blog posts in forums like LessWrong and AI Alignment Forum.

One such idea that is useful for contextualizing technical alignment work — and is perhaps the more formal version of what Bostrom was referring to — is the concept of intent alignment.

Anthropic’s product marketing pages are plastered with notes and phrases about their alignment work —“HHH” is also Claude's biggest selling point.

The site uses the phrasing “AI Safety” instead of “AI Alignment” in the title, but the article itself proceeds to use “safety” and “alignment” interchangeably without differen…

This brief introduction is aimed at those working within the AI community who are interested in AI consciousness, but may not know much about the philosophical and scientific work behind consciousness generally or the topic of AI consciousness in particular.

(Image by author)AI and ConsciousnessTwo Problems for AI ConsciousnessLet’s return to the topic of AI consciousness.

The problem of AI consciousness may seem less difficult than the hard problem: the problem of AI consciousness only asks if silicon could support consciousness, but it does not ask for an explanation of why silicon can or cannot, like the hard problem does.

The second test proposed by Schneider and Edwin Turner [27], call…

We’ve identified three key areas where such programs can level up: dataset curation, a pattern language for usability, and filtering.

The format for a friction hinge pattern might look like this:Pattern Name Friction Hinge Pattern Description The Friction Hinge pattern addresses the need for adjustable friction in hinges so as to provide tuneable resistance, but without compromising smooth movement.

Alas, once text-to-CAD models get open-sourced or leaked, many of these queries will be satisfied without compunction.

____In conclusion, the emergence of AI-powered text-to-CAD generation presents both risks and opportunities, the ratio of which is still very much undecided.

CitationFor attribu…

Created Using IdeogramThe AI world anxiously waits to see what Apple is going to do in space!

Unlike other tech incumbents such as Microsoft, Google, and Meta, Apple has been relatively quiet when it comes to contributions in the AI space.

One of the most interesting trends in autonomous agents is based on computer vision models that can infer actions from screens.

Companies like Adept.ai have been pushing screen understanding as the right way to build autonomous agents.

Recently, Apple decided to dabble into this space by publishing a paper outlining Ferret-UI, a multimodal LLM optimized for mobile screen understanding.

Created Using IdeogramIn this Issue:An overview of task decomposition in autonomous agents.

An introduction to the Bazed framework to build autonomous agents in TypeScript💡 ML Concept of the Day: Task-Decomposition Planning in Autonomous AgentsTask-decomposition is, arguably, the most obvious form of planning in autonomous agents.

As the name suggests, task-decomposition focuses on fragmenting a task into smaller sub-tasks and formulate specific plans for those.

The key to robust task-decomposition techniques is to ensure there is a strong correlation between the created subtasks and the original tasks.

Otherwise, it can lead to hallucinations.

You can subscribed to The Sequence below:📝 Editorial: DeepMind’s AI-First Science Quest Continues with AlphaFold 3AI for science is one of my favorite forms of AI 😊.

A few days ago, DeepMind published details about AlphaFold 3, which expands its prediction capabilities beyond just proteins to a broad spectrum of biomolecules.

Starting with a list of molecules, AlphaFold 3 is able to generate a 3D structure that clearly visualizes its joint 3D structure, revealing its intricacies and interactions.

AlphaFold 3 continues Google DeepMind’s incredible scientific achievements in areas such as mathematics, physics, biology, and several others.

Model SpecOpenAI published Model Spec, a set of guidel…

Created Using IdeogramTransformer architectures have been the dominant paradigm in LLMs leading to exceptional advancements in research and development.

The question of whether transformers will be the final architecture to reach AGI versus the real possibility of new architecture paradigm has been a passionate topic of debate in the AI community.

Recently, researchers from Princeton University and Carnegie Mellon proposed the Mamba architecture based on state space models(SSMs) which has become the most viable alternative to transformers.

Instead of thinking about SSMs vs. transformers, could we try to combine the two?

Jamba combines transformers and SSMs in a single architecture that coul…

Created Using IdeogramIn this Issue:We start diving into planning methods in autonomous agents!

Provide an introduction to the XLANG framework for building autonomous agents.

💡 ML Concept of the Day: Understanding Planning in Autonomous AgentsPlanning is, arguably, the most important capability of autonomous agents.

Obviously, there are many ways to implement planning in autonomous agents.

In their paper, the researchers identified five forms of planning in autonomous agents:

Hey there,Hallucination detection alone isn’t enough; enterprise AI teams need to proactively intercept hallucinations before they reach end users.

And so, for the very first time, we’re excited to showcase Galileo Protect – an advanced GenAI firewall that intercepts hallucinations, prompt attacks, security threats, and more in real-time!

Register NowRegister for our upcoming webinar to see Galileo Protect live in action and learn:How Galileo Protect works, including our research-backed metricsThe key features of an enterprise GenAI firewallWhat makes Galileo Protect uniqueCan't attend live?

All registrants will receive an on-demand recording.

Want to read more about Protect?

You can subscribed to The Sequence below:📝 Editorial: Maybe Two Big Research Breakthroughs or Maybe NothingResearch breakthroughs always command a lot of attention in the generative AI space, as they can drive a new wave of innovation for foundation models.

Last week was particularly interesting because we saw two papers that, on the surface, seem to be quite a big deal for generative AI models, but it's quite hard to determine if they are ready for prime-time.

Or maybe not 😊"🔥 Announcing Galileo Protect: Real-Time Hallucination FirewallCan you stop hallucinations in real-time?

We’re excited to support Galileo Protect – an advanced GenAI firewall that intercepts hallucinations, prompt attac…

Created Using IdeogramPretraining Large Language Models (LLMs) on massive text datasets has become the norm.

When these LLMs are applied to specific tasks, it’s often necessary to integrate additional information, such as the latest news or specialized knowledge, into the already trained model.

The main strategies considered are in-context learning through RAG and supervised fine-tuning.

On the other hand, supervised fine-tuning aims to identify broader patterns in the documents, which could lead to better performance in tasks and alignment with user needs.

However, this method may not always take advantage of documents during the testing phase or may overlook errors in document retrieval.

Created Using DALL-EIn this Issue:An overview of function calling in LLMs and its role in autonomous agnets.

An introduction to the Phidata framework for building autonomous agents.

💡 ML Concept of the Day: LLM Function Calling and Autonomous AgentsIn the first few installments of this series about autonomous agents, we have been exploring the ability of agents to integrate with third party tools or APIs.

Function calling refers to the ability of LLMs to invoke functions from external APIs.

In the context of autonomous agents, function calling plays a role by allowing agents to retrieve information or perform actions on external systems.

Again, not that small, but small enough ;) Apple open-sourced OpenELM, a family of LLMs optimized for mobile scenarios.

After all, nobody likes a know-it-all ;)"🔎 ML ResearchPhi-3Microsoft Research published the technical report of Phi-3, their famous small language model that excel at match and computer science task.

The method is an interesting approach to interpretability to prove generative AI models to undestand their behavior —> Read more.

LayerSkipMeta AI Research published a paper introducing LayerSkip, a method for accelerated inference in LLMs.

🤖 Cool AI Tech ReleasesOpenELMApple open sourced OpenELM, a family of small LLMs optimized to run on devices —> Read more.

Created Using IdeogramThe open-source generative AI landscape is experiencing tremendous momentum.

Innovation comes not only from startups like HuggingFace, Mistral, or AI21 but also from large AI labs such as Meta.

Databricks has been one of the tech incumbents exploring different angles in open source generative AI, mainly after the acquisition of MosaicML.

A few days ago, Databricks open sourced DBRX, a massive general-purpose LLM that show incredible performance across different benchmarks.

Databricks released both the baseline model DBRX Base as well as the intstruction fine-tuned one DBRX Instruct.

Created Using IdeogramIn this Issue:An overview of large action models(LAM) in autonomous agents.

An introduction to the MetaGPT framework for building autonomous agents.

💡 ML Concept of the Day: Large Action Models and Autonomous AgentsThe ability to execute actions in a given environment is one of the hallmarks of autonomous agents.

One of the key topics of debate in autonomous agent circles is how much of the action execution relies on external components versus being built into the model itself.

One of the most interesting approaches among the proponents of the latter camp is what is known as large action models (LAMs).

You can subscribed to The Sequence below:📝 Editorial: Some Cool Details About Llama 3I had an editorial prepared for this week’s newsletter, but then Meta AI released Llama 3!

I prefer to use the term "open models," given that these releases are not completely open source, but that’s just my preference.

The release of Llama 3 builds on incredible momentum within the open model ecosystem and brings its own innovations.

The momentum in the generative AI open models space definitely continues, even if it forced me to rewrite the entire editorial.

🤖 Cool AI Tech ReleasesLlama 3Meta AI introduced the highly anticipated Llama 3 model —> Read more.

Created Using IdeogramVideo games have long served as some of the best environments for training AI agents.

However, most of the AI breakthroughs in 3D game environments have been constrained to one or a small number of games.

The goal of the project was to develop instructable agents that can interact with any 3D environment just like a human by following simple language instructions.

Language is the most powerful and yet simple abstraction for communicating instructions about the world or, in this case, a 3D virtual world.

The magic of SIMA is its ability to translate those abstract instructions into mouse and keyboard actions used to navigate an environment.

Created Using IdeogramIn this Issue:Tool learning in autonomous agents.

💡 ML Concept of the Day: Tool Learning in Autonomous AgentsOne of the key differentiators between agents and models is the capability of the former to take actions in a given environment.

Part of that action execution typically involves interactions with different systems or tools.

From this perspective, tool learning has become one of the most important building blocks of autonomous agents.

When it comes to tool learning in autonomous agents, we should identify two main groups of interactions:

In various domains, Diffusion Models (DMs) have emerged as groundbreaking tools, offering an unparalleled blend of realism and diversity…Continue reading on SyncedReview »

In recent years, there has been remarkable advancement in Large Language Models (LLMs) capable of generating and manipulating code. A variety of models exhibiting impressive coding capabilities have emerged. Nevertheless, significant gaps persist within the realm of LLMs tailored for code, particularly concerning enterprise software development.In a new paper Granite Code Models: A Family of Open Foundation Models for Code Intelligence, an IBM research team introduces the Granite Code model family. Specifically optimized for enterprise software development workflows, these models excel across a spectrum of coding tasks, rendering them versatile and well-suited for diverse coding challenges.…

Achieving excellence across diverse medical applications presents significant hurdles for artificial intelligence (AI), demanding advanced reasoning abilities, access to the latest medical knowledge, and comprehension of intricate multimodal data. Gemini models, Google’s cutting-edge AI, stand out for their robust general capabilities in multimodal and long-context reasoning, presenting promising avenues in the realm of medicine.In a new paper Capabilities of Gemini Models in Medicine, a research team from Google Research, Google DeepMind, Google Cloud and Verily introduce Med-Gemini, a family of highly proficient multimodal models is tailored for medical tasks, boasting the capacity to sea…

Multi-layer perceptrons (MLPs) stand as the bedrock of contemporary deep learning architectures, serving as indispensable components in various machine learning applications. Leveraging the expressive power conferred by the universal approximation theorem, MLPs excel in approximating nonlinear functions, embodying a default choice for many tasks.However, despite their widespread adoption, MLPs harbor notable limitations. They often exhaust a significant portion of non-embedding parameters and frequently lack interpretability without supplementary post-analysis techniques.In a new paper KAN: Kolmogorov-Arnold Networks, a research team from Massachusetts Institute of Technology, California In…

Harnessing Hundreds of GPU Power: NVIDIA’s NeMo-Aligner Unleashes Potential for Large Model AlignmentEnsuring that Large Language Models (LLMs) align with human values and preferences is crucial for their utility and safety. Yet, devising effective tools for this alignment presents significant challenges, particularly with the largest and most sophisticated LLMs, which often boast tens or hundreds of billions of parameters.In a new paper NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment, a team of researchers from Nvidia introduces NeMo-Aligner, a toolkit designed for large-scale LLM model alignment that can efficiently harness the power of hundreds of GPUs for training.Aligning …

In recent advancements, the fusion of video foundation models and large language models has emerged as a promising avenue for constructing robust video understanding systems, transcending the constraints of predefined vision tasks. However, while these methods exhibit commendable performance on shorter videos, they encounter significant hurdles when confronted with longer video sequences. The escalating computational complexity and memory demands inherent in sustaining long-term temporal connections pose formidable challenges.In a new paper MovieChat+: Question-aware Sparse Memory for Long Video Question Answering, a pioneering research group introduces MovieChat, a novel framework tailored…

Large language models (LLMs) endowed with long-context capabilities, such as GPT-4 and Gemini, are increasingly finding versatile applications in various domains like chatbots, vision generation, and financial analysis. However, their efficacy is hampered by the inefficient utilization of computational resources and a substantial memory footprint, particularly when tasked with generating long sequences.Addressing these challenges, in a new paper TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding, a research team from Carnegie Mellon University and Meta AI introduces TriForce — a hierarchical speculative decoding system tailored for scalable lo…

In recent years, there has been a surge in the development of large language models (LLMs) tailored for code-related tasks. These LLMs have shown remarkable proficiency in aiding developers with tasks such as writing, editing, explaining, and reviewing code. However, they often stumble when faced with more intricate software engineering challenges that demand a deeper understanding of a program’s runtime behavior.Addressing this gap, in a new paper NExT: Teaching Large Language Models to Reason about Code Execution, a Google DeepMind research team proposes Naturalized Execution Tuning (NExT), a method aims to equip LLMs with the ability to scrutinize program execution traces and deduce runt…

AlphaFold2 (AF2), crafted by DeepMind, stands as a beacon in the realm of artificial intelligence (AI), boasting the remarkable ability to predict the three-dimensional (3D) structures of proteins from amino acid sequences with unprecedented atomic-level precision. While lauded as a revolutionary advancement in protein folding, its training regimen has long been hampered by its laborious nature, failing to reap significant benefits from the scaling up of computational resources.In a new paper ScaleFold: Reducing AlphaFold Initial Training Time to 10 Hours, a team of researchers from NVIDIA presents ScaleFold, a novel and scalable training methodology tailored for the AlphaFold model. Notabl…

In the expansive realm of artificial intelligence and natural language processing, Small Language Models (SLMs) are making significant strides. Unlike their larger counterparts with hefty parameter counts and demanding computational needs, SLMs are sleeker versions crafted for optimal performance even in resource-constrained settings.In a new paper RecurrentGemma: Moving Past Transformers for Efficient Open Language Models, a Google DeepMind research team introduce RecurrentGemma, an open language model built on Google’s innovative Griffin architecture. This model reduces memory usage and facilitates efficient inference on lengthy sequences, thereby unlocking new possibilities for highly ef…

Efficient on-device neural networks offer rapid, real-time, and interactive experiences while safeguarding private data from public internet exposure. Yet, the computational limitations of mobile devices present a formidable challenge in maintaining a delicate balance between accuracy and efficiency.Addressing this challenge head-on, a recent paper titled “MobileNetV4 — Universal Models for the Mobile Ecosystem,” penned by a Google research team, unveils the latest iteration of MobileNets: MobileNetV4 (MNv4). This cutting-edge model boasts an impressive 87% ImageNet-1K accuracy, coupled with an astonishingly low Pixel 8 EdgeTPU runtime of merely 3.8ms.At the heart of this breakthrough lies …

Transformer-based language models have emerged as powerful tools across various tasks, underlining their significance in critical contexts. Understanding the inner workings of these models is paramount for ensuring their safety, reliability, and trustworthiness, given their widespread adoption.In a new paper LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models, a research team from Meta, University College London and Universitat Politècnica de Catalunya introduces the LM Transparency Tool (LM-TT), an open-source interactive toolkit designed for dissecting Transformer-based language models.Existing analysis tools often focus on isolated aspects of decision-making …

The Large Language Model (LLM) has showcased remarkable efficacy across various real-world applications, including intelligent assistants, text summarization, translation, and multi-modality tasks on mobile devices. Nonetheless, the current methodologies for on-device deployment of LLMs are hampered by sluggish inference speeds, leading to subpar user experiences.In a new paper Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUs, researchers from OPPO AI Center have introduced a solution. They present four optimization techniques and introduce a novel mobile inference engine dubbed Transformer-Lite. This engine outperforms CPU-based FastLLM and GPU-bas…

Revolutionizing Video Understanding: Real-Time Captioning for Any Length with Google’s Streaming ModelThe exponential growth of online video platforms has led to a surge in video content, thereby heightening the need for advanced video comprehension. However, existing computer vision models tailored for video understanding often fall short, typically analyzing only a limited number of frames, typically spanning mere seconds, and categorizing these brief segments into predefined concepts.To address the abovementioned challenge, in a new paper Streaming Dense Video Captioning, a Google research team proposes a streaming dense video captioning model, which revolutionizes dense video captioning…

AURORA-M: A Global Symphony of Innovation as 33 Prestigious Institutions Unify for Open-Source Multilingual MasteryLarge Language Models (LLMs) have revolutionized various applications, including machine translation, text summarization, dialogue systems, and code generation. Yet, the hefty computational requirements for pretraining these models pose significant barriers to broader accessibility and development.To address these challenges, recent open-source initiatives like BLOOM, StarCoder, and StarCoder-2 have emerged, aiming to democratize access to pretrained LLMs. However, these models encounter limitations such as restricted multilingual capabilities, computational intensity, and the …

Или кликбейт — и это в Nature?

Статья ниже — подробный разбор достаточно сложного топика, и в некоторых моментах нужно будет сосредоточиться и вдумчиво читать.

Они же пишут код в помощь разработчикам, да и в целом помогают решать разного рода проблемы.

Однако может так получиться, что сет долгое время не выпадает — и не потому, что игроки проворонили, а потому, что его действительно просто нет на столе.

Надеемся, что после прочтения этой статьи стало ясно, что ошибки нейросетей — это не баг, это фича.

Также присоединяйтесь к моему телеграм каналу AI[ex]Time , где я пишу про машинное обучение, NLP, LLM и в том числе про агентов.

LLaVaВ качестве LLM для генерации текстового ответа используется LLaMA, которая декодирует эмбеддинги (то же, что и векторы) картинок и входного текста в ответ.

На вход модель получает картинку и запрос от пользователя (Normal prompt) и на выходе должна дать ответ (Response).

Модель таким образом от решения задачи напрямую переходит к рассуждению по шагам, что в некотором смысле и является декомпозицией задачи.

И это улучшает качество работы модели — в том числе и ризонинга.

Мы помним, что в начале 2023-го продуктом уже пользовалось больше 100 млн человек в месяц.

У самых любознательных читателей может возникнуть вопрос: а как это вообще работает?

Насколько нам известно, это первый раз, когда модель такого масштаба обучается на синтетических данных, а не на произведенных человеком.

Использование такой модели дешевле в 3 раза на текст из промпта, и в 2 раза на генерируемые токены (их обычно меньше).

Но вот как с ними обращаться, когда использовать и как комбинировать — это уже решает AI по контексту диалога.

Меня зовут Валентин Малых, я — руководитель направления NLP-исследований в MTS AI, вот уже 6 лет я читаю курс по NLP.

Поскольку я все время отвечаю одно и то же, появилась идея сделать пост про мой список книг, заодно описав их.

При этом в книге больше информации про информационный поиск (information retrieval) и меньше про NLP, но в наше время эти две области уже (или все еще) очень близки.

Правда, его тоже уже не достать, хотя PDF версия ищется без проблем.

Foundations of Statistical Natural Language ProcessingНасколько мне известно, эта книга не переводилась на русский язык.

Меня зовут Валентин Малых, я — руководитель направления NLP-исследований в MTS AI, вот уже 6 лет я читаю курс по NLP.

Поскольку я все время отвечаю одно и то же, появилась идея сделать пост про мой список книг, заодно описав их.

При этом в книге больше информации про информационный поиск (information retrieval) и меньше про NLP, но в наше время эти две области уже (или все еще) очень близки.

Правда, его тоже уже не достать, хотя PDF версия ищется без проблем.

Foundations of Statistical Natural Language ProcessingНасколько мне известно, эта книга не переводилась на русский язык.

Платформа для экспериментов в RecSysНа примере Киона я показала сложности экспериментов с рекомендательными системами.

В работе над RecSys в реальном сервисе мы строим гипотезы, выкатываем модели в АБ тесты, строим новые гипотезы.

Нашей целью была инфраструктура и пайплайны для максимально быстрых и удобных экспериментов с любыми алгоритмами.

Трекинг экспериментов: метрики и визуальный анализТрекинг экспериментов мы проводили одновременно в Mlflow и в отчётах в репозитории.

Схема на картинке:Инфраструктура платформы для экспериментов и приложения с рекомендациямиЖёлтые стрелки отражают запись результатов экспериментов: логирование метрик с кросс-валидации, сохранение обученных моделей в Min…

В первой части статьи я описывала, как мы с напарником решили выкатить модель из соревнования в онлайн рекомендации, и что из этого вышло.

Давайте построим фичу для бустинга, которая будет решать главную проблему нашей текущей модели - не релевантные айтемы в хороших подборках.

Также у нас были замечательные фичи, проверенные ещё на модели в соревновании.

В задаче классификации мы подаём модели кандидатов, бывших в интеракциях в качестве позитивных таргетов, и не бывших в качестве негативных, и учим его отличать их друг от друга.

Самый стабильный подход к кросс-валидации в рекомендательных системах - это схема скользящего окна, которая больше всего соответствует работе модели в реальном мир…

Смотрим, что мы имеем в Кионе:Распределение популярности топ 100 айтемов в датасете KionЧто там в топе просмотров?

Популярные айтемы заполняют собой полку рекомендаций и на этот запрос, и на многие другие.

Фильм занимает 6 место в топе просмотров в датасете и очень не вовремя попадает в рекомендации у самых разных алгоритмов.

Проверим рекомендации от модели с максимумом Recall без популярного:Рекомендации модели bm25 с максимумом Recall без популярного.

Recall и MAP понизились примерно в 2 раза от модели в соревновании.

Почти сразу после публикации поста про систему поиска новостей о трудовых конфликтах в СНГ я познакомился с коллективом проекта «Диалектик».

Коллектив волнуют процессы, происходящие как в экономическом базисе, так и в его надстройке – в обществе, культуре, политике.

Этот метод познания помогал всем классикам марксизма анализировать сложные ситуации в экономике и обществе, этим он ценен и для коллектива «Диалектика».

Поэтому внутри коллектива ведется разработка ИТ-инструментов для внутреннего (только для редакторов) и для внешнего (для всех пользователей) использования.

Этот публичный агрегатор находится на этапе тестирования и в скором времени будет представлен общественности.

The image diffusion model, in its simplest form, generates an image from the prompt. The prompt can be a text prompt or an image as long as a suitable encoder is available to convert it into a tensor that the model can use as a condition to guide the generation process. Text prompts are probably […]

The post More Prompting Techniques for Stable Diffusion appeared first on MachineLearningMastery.com.

We have just learned about ControlNet. Now, let’s explore the most effective way to control your character based on human pose. OpenPose is a great tool that can detect body keypoint locations in images and video. By integrating OpenPose with Stable Diffusion, we can guide the AI in generating images that match specific poses. In […]

The post Using OpenPose with Stable Diffusion appeared first on MachineLearningMastery.com.

ControlNet is a neural network that can improve image generation in Stable Diffusion by adding extra conditions. This allows users to have more control over the images generated. Instead of trying out different prompts, the ControlNet models enable users to generate consistent images with just one prompt. In this post, you will learn how to […]

The post Using ControlNet with Stable Diffusion appeared first on MachineLearningMastery.com.

Inpainting and outpainting have long been popular and well-studied image processing domains. Traditional approaches to these problems often relied on complex algorithms and deep learning techniques yet still gave inconsistent outputs. However, recent advancements in the form of Stable diffusion have reshaped these domains. Stable diffusion now offers enhanced efficacy in inpainting and outpainting while […]

The post Inpainting and Outpainting with Stable Diffusion appeared first on MachineLearningMastery.com.

Stable Diffusion’s latest models are very good at generating hyper-realistic images, but they can struggle with accurately generating human faces. We can experiment with prompts, but to get seamless, photorealistic results for faces, we may need to try new methodologies and models. In this post, we will explore various techniques and models for generating highly […]

The post Generate Realistic Faces in Stable Diffusion appeared first on MachineLearningMastery.com.

The deep learning model of Stable Diffusion is huge. The weight file is multiple GB large. Retraining the model means to update a lot of weights and that is a lot of work. Sometimes we must modify the Stable Diffusion model, for example, to define a new interpretation of prompts or make the model to […]

The post Using LoRA in Stable Diffusion appeared first on MachineLearningMastery.com.

Generating pictures using Stable Diffusion in all cases would involve to submit a prompt to the pipeline. This is only one of the parameters, but the most important one. An incomplete or poorly constructed prompt would make the resulting image not as you would expect. In this post, you will learn some key techniques to […]

The post Prompting Techniques for Stable Diffusion appeared first on MachineLearningMastery.com.

Launching the Stable Diffusion Web UI can be done in one command. After that, you can control the image generation pipeline from a browser. The pipeline has a lot of moving parts and all are important in one way or another. To effectively command Stable Diffusion to generate images, you should recognize the widgets from […]

The post How to Create Images Using Stable Diffusion Web UI appeared first on MachineLearningMastery.com.

The introduction of GPT-3, particularly its chatbot form, i.e. the ChatGPT, has proven to be a monumental moment in the AI landscape, marking the onset of the generative AI (GenAI) revolution. Although prior models existed in the image generation space, it’s the GenAI wave that caught everyone’s attention. Stable Diffusion is a member of the […]

The post A Technical Introduction to Stable Diffusion appeared first on MachineLearningMastery.com.

The advance of generative machine learning models makes computers capable of creative work. In the scope of drawing pictures, there are a few notable models that allow you to convert a textual description into an array of pixels. The most powerful models today are part of the family of diffusion models. In this post, you […]

The post Brief Introduction to Diffusion Models for Image Generation appeared first on MachineLearningMastery.com.

The path to uncovering meaningful insights often starts with a single step: looking at the data before asking questions. This journey through the Ames Housing dataset is more than an exploration; it’s a narrative about the hidden stories within numbers, waiting to be told. Through a “Data First Approach,” we invite you to dive deep […]

The post Unfolding Data Stories: From First Glance to In-Depth Analysis appeared first on MachineLearningMastery.com.

Data Science embodies a delicate balance between the art of visual storytelling, the precision of statistical analysis, and the foundational bedrock of data preparation, transformation, and analysis. The intersection of these domains is where true data alchemy happens – transforming and interpreting data to tell compelling stories that drive decision-making and knowledge discovery. Just as […]

The post The Da Vinci Code of Data: Mastering The Data Science Mind Map appeared first on MachineLearningMastery.com.

The real estate industry is a vast network of stakeholders including agents, homeowners, investors, developers, municipal planners, and tech innovators, each bringing unique perspectives and objectives to the table. Within this intricate ecosystem, data emerges as the critical element that binds these diverse interests together, facilitating collaboration and innovation. PropTech, or Property Technology, illustrates this […]

The post Finding Value with Data: The Cohesive Force Behind Luxury Real Estate Decisions appeared first on MachineLearningMastery.com.

Data transformations enable data scientists to refine, normalize, and standardize raw data into a format ripe for analysis. These transformations are not merely procedural steps; they are essential in mitigating biases, handling skewed distributions, and enhancing the robustness of statistical models. This post will primarily focus on how to address skewed data. By focusing on […]

The post Skewness Be Gone: Transformative Tricks for Data Scientists appeared first on MachineLearningMastery.com.

Most people will have more fun if they solve puzzles than if they don’t, but you don’t have to solve puzzles quickly to have fun.

I’m still going to explain the solving strategies I’ve learned, but puzzle solving is really an activity where you learn by doing.

Puzzle solving often involves relating two parts of the puzzle together.

Search everythingHonestly, a lot of puzzle solving is about taking random parts of the puzzle and throwing them into a search engine.

Bringing This TogetherTo showcase these strategies together, here is a puzzle I remember speedrunning especially quickly: The Three Little Pigs from Hunt 20 2.1 Puzzle Hunt.

This means we can reduce all the crew battle outcomes down to a single \(n \times n\) matrix, which I’ll call the crew battle matrix.

So I Wrote Some Python Code to Compute \(f\) for Arbitrary Crew BattlesLet’s consider the RPS crew battle again.

Here’s the matchup matrix:and here’s what my code outputs for the crew battle matrix, assuming optimal play.

But also, this suggests that crew battle strategy really isn’t that important in the first place!

If your crew loses, you don’t get to blame bad crew battle strategy.

This has spoilers for MIT Mystery Hunt 2024.

I hunted with teammate again this year, because there is nothing quite like writing a Mystery Hunt to forge friends through fire.

I needed to spend some to avoid hitting the vacation cap, and what better time than Mystery Hunt?

If you were forced to pick how a Mystery Hunt runs long, I think most people would pick the “too many puzzles” side of Mystery Hunt 2024 over the “too difficult puzzles” side of Mystery Hunt 2023.

So did Spoilr, the codebase for Mystery Hunt 2022, and tph-site from Mystery Hunt 2023.

In August 2020, I wrote a post about my AI timelines.

Diffusion based image augmentation has been shown to improve robot learning, and Anthropic has based a lot of its branding on constitutional AI and “RL from AI feedback”.

I don’t like AI Twitter for reasons I’ve explained here, but I especially do not AI twitter post-ChatGPT.

In AI, models can never do everything people claim they can, but what the models can do is ever-growing and never slides backward.

The bull is to say that we can figure out how to scale models, and scaled up models will solve all the other hard problems.

You know how much time I have spent studying how to squeeze Neopoints water out of the Neopets stone?

I’d say you can expect to get about 25k NP a day, depending on how many NP rewards you get.

Trudy’s Surprise also gives items for 7 day streaks, but these items are usually junk and not worth anything.

When you win against CPU opponents, you earn a small amount of Neopoints and an item drop.

Or your needs to…see someone do a lot of research into things that don’t matter?

Everfree Northwest is a My Little Pony convention in the Seattle area.

Except, I heard that with the ending of BronyCon, Everfree Northwest is the largest pony convention left.

My Little Pony: Friendship is Magic is Generation 4, or G4, and is the one that kicked off the brony fandom.

People tend to assume that “pony concert” means “remixes of pony songs”, or at least “overt references to My Little Pony”.

The first was a signed copy of Ponies: The Galloping, the Magic: The Gathering x My Little Pony crossover.

I started working on that post right after Mystery Hunt 2023 finished, and hardcore focused on getting it out ASAP.

The end result was that I was exerting “write Mystery Hunt” levels of effort (10-20 hrs/week) for 1.5 years straight.

markdown 1929 2023 - 04 - 21 - mh - 2023. markdown 114 2023 - 05 - 09 - bootes - 2023. markdown 153 2023 - 07 - 19 - ml - hurry .

markdownPosts in LimboPost about Dominion Online:Odds of writing this year: 5%Odds of writing eventually: 25%My priorities are moving away from Dominion.

Post about AI timelines:Odds of writing this year: 90%Odds of writing eventually: 99%I’m not planning a big update to the last timelines post.

Generative AI is a membership based concept, so it is defined by the set of approaches and applications that get put under that name.

I think this is one part of the intrigue of generative AI for science.

So maybe generative AI applications are part of this drive for a post-theory mode of science.

This generative science, it is hoped, can add vigour into the sciences, especially in cross-disciplinary ways and provide broad-based benefit.

Generative AI for Science presents opportunities for new ways of doing theory and renewed vigour across our sciences.

Keynote at the Stanford Human-centered AI 2023 Fall Conference on New Horizons in Generative AI: Science, Creativity, and Society.

Our conference today on new horizons in generative AI, invites us to think of the frontier of research and innovation.

These stories will expose some features of the sociotechnical foundations of generative AI that is my underlying message and call to action.

What he doesn’t know yet, is that this book will become a cornerstone of the field and industry of weather forecasting.

There is a specific and firm model for responsibility that is built on taking a sociotechnical approach to generative AI and our work.

Abstract: This talk talk has a single objective: to advocate for machine learning infused with social purpose.

In this way, social purpose transforms our field of machine learning: into something that is both technical and social.

So social purpose will make machine learning something that is both technical and social.

A more global field and industry can shift machine learning to be more general: magnifying social purpose.

Your continued volunteering, funding, support, and openness to these groups shows yet another way of infusing machine learning with social purpose.

At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well.

To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages.

We address uncertainty in estimates $\widehat \beta$ of model coefficients with subsampling, our usual strategy for variance estimation in online experiments[12].

If our experiments were smaller then we might need to model our metrics with Poisson, Binomial or other GLMs instead.

We sometimes use $L^1$ regularization when we fit models to our live experiment data to select the most important non-linear c…

Both of these assumptions seem reasonable and general and we can quickly see where Gaussian distribution should start popping out.

Since Gaussian distribution is fully defined by just two numbers - mean and variance, we can, by collecting enough data rather precisely estimate these values.

Below I simulate 400x200=80000 independent pixels, each taking a random value between zero and one [code available here].

These distributions we often refer to as "fat tails" [for putting much more "weight" in the tail of the distribution, as in outside of the "central" part] are really weird and problematic.

Let's get back to why we see Gaussian distribution anywhere at all?

So it should not be surprising that hubris in science, displayed by some scientists strikes me equally as arrogant and irritating as the claims of religious leaders.

With the day of judgment established and imminent we are now ready to spread the gospel of modern day religion and profit.

AGI (artificial general intelligence) isn't even a well defined thing.

We can hardly even imagine such a thing, which is great for religious purposes, because everyone can have their own "AGI" to be scared of.

That we should never question the technocrats, because technology is the only and natural way of making our lives better.

NeurIPS 2023, arguably this year’s biggest AI conference takes place in two weeks from Dec 10–16 in New Orleans.

Here are the main trends I observed:Most NLP work at NeurIPS is related to large language models (LLMs).

Papers studied different aspects of in-context learning such as whether it persists during training and using a Bayesian perspective.

Lexinvariant Language Models (Huang et al.).

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning (Wang et al.).

Sparked by progress in Large Language Models (LLMs), there’s a lot of chatter recently about AGI, its timelines, and what it might look like.

In particular, when I am asked about what I think AGI will look like, I personally like to point to self-driving.

AGISo these are the broad strokes of what I think AGI will look like.

And it looks a lot more like self-driving, the part of our economy that is currently speed-running the development of a major, society-altering automation.

I personally very much look forward to what all the equivalents of that might be with AGI.

The Influence Maximization (IM) problem is a well-known NP-hard combinatorial problem over graphs whose goal is to find the set of nodes in a network that spreads influence at most.

Among the various methods for solving the IM problem, evolutionary algorithms (EAs) have been shown to be particularly effective.

Hypergraphs are a valuable tool for modeling complex interaction networks in various domains; however, they require rethinking of several graph-based problems, including IM.

In this work, we propose a multi-objective EA for the IM problem over hypergraphs that leverages smart initialization and hypergraph-aware mutation.

While the existing methods rely on greedy or heuristic methods, …

Black-box optimization problems often require simultaneously optimizing different types of variables, such as continuous, integer, and categorical variables.

Unlike integer variables, categorical variables do not necessarily have a meaningful order, and the discretization approach of continuous variables does not work well.

Although several Bayesian optimization methods can deal with mixed-category black-box optimization (MC-BBO), they suffer from a lack of scalability to high-dimensional problems and internal computational cost.

This paper proposes CatCMA, a stochastic optimization method for MC-BBO problems, which employs the joint probability distribution of multivariate Gaussian and cat…

In this paper, we introduce the Polish Massive Text Embedding Benchmark (PL-MTEB), a comprehensive benchmark for text embeddings in Polish.

The PL-MTEB consists of 28 diverse NLP tasks from 5 task types.

We adapted the tasks based on previously used datasets by the Polish NLP community.

In addition, we created a new PLSC (Polish Library of Science Corpus) dataset consisting of titles and abstracts of scientific publications in Polish, which was used as the basis for two novel clustering tasks.

We evaluated 15 publicly available models for text embedding, including Polish and multilingual ones, and collected detailed results for individual tasks and aggregated results for each task type and …

We present HyperSum, an extractive summarization framework that captures both the efficiency of traditional lexical summarization and the accuracy of contemporary neural approaches.

HyperSum exploits the pseudo-orthogonality that emerges when randomly initializing vectors at extremely high dimensions ("blessing of dimensionality") to construct representative and efficient sentence embeddings.

Simply clustering the obtained embeddings and extracting their medoids yields competitive summaries.

HyperSum often outperforms state-of-the-art summarizers -- in terms of both summary accuracy and faithfulness -- while being 10 to 100 times faster.

We open-source HyperSum as a strong baseline for unsu…

In learnable scalable coding, previous work increased the utilization of side-information for input reconstruction by also rewarding input reconstruction when learning this shared representation.

We evaluate the impact of this idea in the context of input reconstruction more rigorously and extended it to other computer vision tasks.

We perform experiments using representations trained for object detection on COCO 2017 and depth estimation on the Cityscapes dataset, and use them to assist in image reconstruction and semantic segmentation tasks.

Moreover, using the proposed representations, the performance of the base tasks are also improved.

Results suggest that the proposed method induces s…

Many of these problems are both textual and multimodal.

Because of the hallucination issue, the generated soft negative rationales with high textual quality but illogical semantics do not always help improve answer accuracy.

This study proposes a rationale generation method using soft negative sampling (SNSE-CoT) to mitigate hallucinations in multimodal CoT.

Five methods were applied to generate soft negative samples that shared highly similar text but had different semantics from the original.

Bidirectional margin loss (BML) was applied to introduce them into the traditional contrastive learning framework that involves only positive and negative samples.

With the rapid development of face recognition (FR) systems, the privacy of face images on social media is facing severe challenges due to the abuse of unauthorized FR systems.

Some studies utilize adversarial attack techniques to defend against malicious FR systems by generating adversarial examples.

However, the generated adversarial examples, i.e., the protected face images, tend to suffer from subpar visual quality and low transferability.

In this paper, we propose a novel face protection approach, dubbed DiffAM, which leverages the powerful generative ability of diffusion models to generate high-quality protected face images with adversarial makeup transferred from reference images.

As…

LLM watermarking, which embeds imperceptible yet algorithmically detectable signals in model outputs to identify LLM-generated text, has become crucial in mitigating the potential misuse of large language models.

To address these issues, we introduce MarkLLM, an open-source toolkit for LLM watermarking.

MarkLLM offers a unified and extensible framework for implementing LLM watermarking algorithms, while providing user-friendly interfaces to ensure ease of access.

For evaluation, MarkLLM offers a comprehensive suite of 12 tools spanning three perspectives, along with two types of automated evaluation pipelines.

Through MarkLLM, we aim to support researchers while improving the comprehension …

This paper introduces Grounding DINO 1.5, a suite of advanced open-set object detection models developed by IDEA Research, which aims to advance the "Edge" of open-set object detection.

The suite encompasses two models: Grounding DINO 1.5 Pro, a high-performance model designed for stronger generalization capability across a wide range of scenarios, and Grounding DINO 1.5 Edge, an efficient model optimized for faster speed demanded in many applications requiring edge deployment.

The Grounding DINO 1.5 Edge model, while designed for efficiency with reduced feature scales, maintains robust detection capabilities by being trained on the same comprehensive dataset.

Empirical results demonstrate …

The rapid evolution of large language models (LLMs) has ushered in the need for comprehensive assessments of their performance across various dimensions.

In this paper, we propose LFED, a Literary Fiction Evaluation Dataset, which aims to evaluate the capability of LLMs on the long fiction comprehension and reasoning.

We collect 95 literary fictions that are either originally written in Chinese or translated into Chinese, covering a wide range of topics across several centuries.

Additionally, we conduct an in-depth analysis to ascertain how specific attributes of literary fictions (e.g., novel types, character numbers, the year of publication) impact LLM performance in evaluations.

Through …

This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image.

We observe that current metrics are size-sensitive, where larger objects are focused, and smaller ones tend to be ignored.

In pursuit of this, we propose a generic approach that evaluates each salient object separately and then combines the results, effectively alleviating the imbalance.

We further develop an optimization framework tailored to this goal, achieving considerable improvements in detecting objects of different sizes.

Theoretically, we provide evidence supporting the validity of our new metrics and present …

Algorithmic bias is a major issue in machine learning models in educational contexts.

However, it has not yet been studied thoroughly in Asian learning contexts, and only limited work has considered algorithmic bias based on regional (sub-national) background.

As a step towards addressing this gap, this paper examines the population of 5,986 students at a large university in the Philippines, investigating algorithmic bias based on students' regional background.

The university used the Canvas learning management system (LMS) in its online courses across a broad range of domains.

Subsequently, we examined the data for bias based on students' region.

Remote sensing image-text retrieval constitutes a foundational aspect of remote sensing interpretation tasks, facilitating the alignment of vision and language representations.

This paper introduces a prior instruction representation (PIR) learning paradigm that draws on prior knowledge to instruct adaptive learning of vision and text representations.

However, with massive additional data for pre-training the vision-language foundation model, remote sensing image-text retrieval is further developed into an open-domain retrieval task.

Continuing with the above, we propose PIR-CLIP, a domain-specific CLIP-based framework for remote sensing image-text retrieval, to address semantic noise in re…

The Learning-to-match (LTM) framework proves to be an effective inverse optimal transport approach for learning the underlying ground metric between two sources of data, facilitating subsequent matching.

However, the conventional LTM framework faces scalability challenges, necessitating the use of the entire dataset each time the parameters of the ground metric are updated.

In adapting LTM to the deep learning context, we introduce the mini-batch Learning-to-match (m-LTM) framework for audio-text retrieval problems.

Moreover, to cope with misaligned training data in practice, we propose a variant using partial optimal transport to mitigate the harm of misaligned data pairs in training data.…

Recent advances in generative models trained on large-scale datasets have made it possible to synthesize high-quality samples across various domains.

In this paper, we propose an essential yet under-explored task called generative identity unlearning, which steers the model not to generate an image of a specific identity.

In the generative identity unlearning, we target the following objectives: (i) preventing the generation of images with a certain identity, and (ii) preserving the overall quality of the generative model.

To satisfy these goals, we propose a novel framework, Generative Unlearning for Any Identity (GUIDE), which prevents the reconstruction of a specific identity by unlearni…

Recent deep music generation studies have put much emphasis on long-term generation with structures.

However, we are yet to see high-quality, well-structured whole-song generation.

In this paper, we make the first attempt to model a full music piece under the realization of compositional hierarchy.

The high-level languages reveal whole-song form, phrase, and cadence, whereas the low-level languages focus on notes, chords, and their local patterns.

A cascaded diffusion model is trained to model the hierarchical language, where each level is conditioned on its upper levels.

Robust optimisation is a well-established framework for optimising functions in the presence of uncertainty.

In this work, we study the multi-objective extension of this problem from a computational standpoint.

We identify that the majority of all robust multi-objective algorithms rely on two key operations: robustification and scalarisation.

Whilst scalarisation refers to the procedure that is used to encode the relative importance of each objective.

As part of our analysis, we showcase how many existing risk concepts can be easily integrated into the specification and solution of a robust multi-objective optimisation problem.

However, such methods usually obtain sparse tokens by hand-crafted or parallel-unfriendly designs, posing a challenge to reach a better balance between efficiency and performance.

Different from them, this paper proposes to use learnable meta tokens to formulate sparse tokens, which effectively learn key information meanwhile improving the inference speed.

Technically, the meta tokens are first initialized from image tokens via cross-attention.

Then, we propose Dual Cross-Attention (DCA) to promote information exchange between image tokens and meta tokens, where they serve as query and key (value) tokens alternatively in a dual-branch structure, significantly reducing the computational comp…

In this work, we introduce Libra, a prototype model with a decoupled vision system on a large language model (LLM).

The decoupled vision system decouples inner-modal modeling and cross-modal interaction, yielding unique visual information modeling and effective cross-modal comprehension.

Libra is trained through discrete auto-regressive modeling on both vision and language inputs.

Specifically, we incorporate a routed visual expert with a cross-modal bridge module into a pretrained LLM to route the vision and language flows during attention computing to enable different attention patterns in inner-modal modeling and cross-modal interaction scenarios.

Experimental results demonstrate that th…

Event Stream Super-Resolution (ESR) aims to address the challenge of insufficient spatial resolution in event streams, which holds great significance for the application of event cameras in complex scenarios.

Previous works for ESR often process positive and negative events in a mixed paradigm.

This paradigm limits their ability to effectively model the unique characteristics of each event and mutually refine each other by considering their correlations.

In this paper, we propose a bilateral event mining and complementary network (BMCNet) to fully leverage the potential of each event and capture the shared information to complement each other simultaneously.

To facilitate the exchange of in…

Infrared (IR) image super-resolution faces challenges from homogeneous background pixel distributions and sparse target regions, requiring models that effectively handle long-range dependencies and capture detailed local-global information.

Recent advancements in Mamba-based (Selective Structured State Space Model) models, employing state space models, have shown significant potential in visual tasks, suggesting their applicability for IR enhancement.

In this work, we introduce IRSRMamba: Infrared Image Super-Resolution via Mamba-based Wavelet Transform Feature Modulation Model, a novel Mamba-based model designed specifically for IR image super-resolution.

Additionally, a new wavelet transf…

Calibration in recommender systems is an important performance criterion that ensures consistency between the distribution of user preference categories and that of recommendations generated by the system.

Standard methods for mitigating miscalibration typically assume that user preference profiles are static, and they measure calibration relative to the full history of user's interactions, including possibly outdated and stale preference categories.

We conjecture that this approach can lead to recommendations that, while appearing calibrated, in fact, distort users' true preferences.

In this paper, we conduct a preliminary investigation of recommendation calibration at a more granular leve…

Recent advancements in language models (LMs) have enabled the introduction of a new type of actor in that ecosystem: LM-powered assistants capable of code generation, optimization, and maintenance.

Our study investigates the efficacy of small language models (SLMs) for generating high-quality docstrings by assessing accuracy, conciseness, and clarity, benchmarking performance quantitatively through mathematical formulas and qualitatively through human evaluation using Likert scale.

Further, we introduce DocuMint, as a large-scale supervised fine-tuning dataset with 100,000 samples.

Fine-tuning the CodeGemma 2B model using the DocuMint dataset led to significant improvements in performance a…

Large pretrained transformers are increasingly being developed as generalised foundation models which can underpin powerful task-specific artificial intelligence models.

Histopathology foundation models show promise across many tasks, but analyses have been limited by arbitrary hyperparameters that were not tuned to the specific task/dataset.

We report the most rigorous single-task validation conducted to date of a histopathology foundation model, and the first performed in ovarian cancer subtyping.

Normalisations and augmentations aided the generalisability of ResNet-based models, but these still did not match the performance of UNI, which gave the best external performance in any ovarian …

In the new paradigm of semantic communication (SC), the focus is on delivering meanings behind bits by extracting semantic information from raw data.

Recent advances in data-to-text models facilitate language-oriented SC, particularly for text-transformed image communication via image-to-text (I2T) encoding and text-to-image (T2I) decoding.

However, although semantically aligned, the text is too coarse to precisely capture sophisticated visual features such as spatial locations, color, and texture, incurring a significant perceptual difference between intended and reconstructed images.

To address this limitation, in this paper, we propose a novel language-oriented SC framework that communic…

Scientific document summarization has been a challenging task due to the long structure of the input text.

The long input hinders the simultaneous effective modeling of both global high-order relations between sentences and local intra-sentence relations which is the most critical step in extractive summarization.

In this paper, we propose HAESum, a novel approach utilizing graph neural networks to locally and globally model documents based on their hierarchical discourse structure.

First, intra-sentence relations are learned using a local heterogeneous graph.

We validate our approach on two benchmark datasets, and the experimental results demonstrate the effectiveness of HAESum and the imp…

Hierarchical leaf vein segmentation is a crucial but under-explored task in agricultural sciences, where analysis of the hierarchical structure of plant leaf venation can contribute to plant breeding.

While current segmentation techniques rely on data-driven models, there is no publicly available dataset specifically designed for hierarchical leaf vein segmentation.

To address this gap, we introduce the HierArchical Leaf Vein Segmentation (HALVS) dataset, the first public hierarchical leaf vein segmentation dataset.

It also includes human-annotated ground truth for three orders of leaf veins, with a total labeling effort of 83.8 person-days.

Empirical studies are performed on HALVS, reveali…

Authorship obfuscation techniques hold the promise of helping people protect their privacy in online communications by automatically rewriting text to hide the identity of the original author.

In this work, we introduce an automatic text privatization framework that fine-tunes a large language model via reinforcement learning to produce rewrites that balance soundness, sense, and privacy.

We evaluate it extensively on a large-scale test set of English Reddit posts by 68k authors composed of short-medium length texts.

We study how the performance changes among evaluative conditions including authorial profile length and authorship detection strategy.

Our method maintains high text quality ac…

In this paper, we dive into the reliability concerns of Integrated Gradients (IG), a prevalent feature attribution method for black-box deep learning models.

We particularly address two predominant challenges associated with IG: the generation of noisy feature visualizations for vision models and the vulnerability to adversarial attributional attacks.

Our approach involves an adaptation of path-based feature attribution, aligning the path of attribution more closely to the intrinsic geometry of the data manifold.

Our experiments utilise deep generative models applied to several real-world image datasets.

They demonstrate that IG along the geodesics conforms to the curved geometry of the Rie…

Yet, the escalating demands on AI have highlighted the limitations of AI's current offerings, catalyzing a movement towards Artificial General Intelligence (AGI).

AGI, distinguished by its ability to execute diverse real-world tasks with efficiency and effectiveness comparable to human intelligence, reflects a paramount milestone in AI evolution.

As the realization of AGI requires more advanced capabilities and adherence to stringent constraints, we further discuss necessary AGI alignment technologies to harmonize these factors.

Notably, we emphasize the importance of approaching AGI responsibly by first defining the key levels of AGI progression, followed by the evaluation framework that s…

Over the past year, we’ve made incredible progress in enhancing the quality of our generative media technologies.

We’ve been working closely with the creative community to explore how generative AI can best support the creative process, and to make sure our AI tools are as useful as possible at each stage.

Today, we’re introducing Veo, our latest and most advanced video generation model, and Imagen 3, our highest quality text-to-image model yet.

We’re also sharing some of our recent collaborations with filmmaker Donald Glover and his creative studio, Gilga, and new demo recordings being released by artists Wyclef Jean, Marc Rebillet and songwriter Justin Tranter, made with help from our Mus…

Company Watermarking AI-generated text and video with SynthID ShareCopy link ×Announcing our novel watermarking method for AI-generated text and video, and how we’re bringing SynthID to key Google products Generative AI tools — and the large language model technologies behind them — have captured the public imagination.

Today, we’re expanding SynthID’s capabilities to watermarking AI-generated text in the Gemini app and web experience, and video in Veo, our most capable generative video model.

SynthID text watermarking is less effective on responses to factual prompts because there are fewer opportunities to adjust the token distribution without affecting the factual accuracy.

How video wat…

In December, we launched our first natively multimodal model Gemini 1.0 in three sizes: Ultra, Pro and Nano.

Just a few months later we released 1.5 Pro, with enhanced performance and a breakthrough long context window of 1 million tokens.

This inspired us to keep innovating, so today, we’re introducing Gemini 1.5 Flash: a model that’s lighter-weight than 1.5 Pro, and designed to be fast and efficient to serve at scale.

Both 1.5 Pro and 1.5 Flash are available in public preview with a 1 million token context window in Google AI Studio and Vertex AI.

And now, 1.5 Pro is also available with a 2 million token context window via waitlist to developers using the API and to Google Cloud customers.

They’re made up of proteins, DNA and other molecules, but no single piece works on its own.

In a paper published in Nature, we introduce AlphaFold 3, a revolutionary model that can predict the structure and interactions of all life’s molecules with unprecedented accuracy.

To build on AlphaFold 3’s potential for drug design, Isomorphic Labs is already collaborating with pharmaceutical companies to apply it to real-world drug design challenges and, ultimately, develop new life-changing treatments for patients.

Our new model builds on the foundations of AlphaFold 2, which in 2020 made a fundamental breakthrough in protein structure prediction.

This leap could unlock more transformative science…

Research Google DeepMind at ICLR 2024 ShareCopy link ×Developing next-gen AI agents, exploring new modalities, and pioneering foundational learning Next week, AI researchers from around the globe will converge at the 12th International Conference on Learning Representations (ICLR), set to take place May 7-11 in Vienna, Austria.

Teams from across Google DeepMind will present more than 70 papers this year.

For instance, LLM-based AI agents capable of taking effective actions could transform digital assistants into more helpful and intuitive AI tools.

Until recently, large AI models mostly focused on text and images, laying the groundwork for large-scale pattern recognition and data interpreta…

Responsibility & Safety The ethics of advanced AI assistants ShareCopy link ×Exploring the promise and risks of a future with more capable AI Imagine a future where we interact regularly with a range of advanced artificial intelligence (AI) assistants — and where millions of assistants interact with each other on our behalf.

General-purpose foundation models are paving the way for increasingly advanced AI assistants.

Advanced AI assistants could have a profound impact on users and society, and be integrated into most aspects of people’s lives.

Able to fluidly communicate using natural language, the written output and voices of advanced AI assistants may become hard to distinguish from those…

Research TacticAI: an AI assistant for football tactics ShareCopy link ×As part of our multi-year collaboration with Liverpool FC, we develop a full AI system that can advise coaches on corner kicks 'Corner taken quickly… Origi!'

Our first paper, Game Plan, looked at why AI should be used in assisting football tactics, highlighting examples such as analyzing penalty kicks.

Predicting corner kick outcomes with geometric deep learning A corner kick is awarded when the ball passes over the byline, after touching a player of the defending team.

With TacticAI, we have developed a capable AI assistant for football tactics and achieved a milestone in developing useful assistants in sports AI.

We s…

In a new technical report, we introduce SIMA, short for Scalable Instructable Multiworld Agent, a generalist AI agent for 3D virtual settings.

SIMA: a versatile AI agent SIMA is an AI agent that can perceive and understand a variety of environments, then take actions to achieve an instructed goal.

What’s more, an agent trained in all but one game performed nearly as well on that unseen game as an agent trained specifically on it, on average.

We compare this performance with three types of generalist SIMA agent, each trained across multiple environments.

Advancing AI agent research SIMA’s results show the potential to develop a new wave of generalist, language-driven AI agents.

Today, we’re excited to introduce a new generation of open models from Google to assist developers and researchers in building AI responsibly.

Gemma open modelsGemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models.

This enables Gemma 2B and 7B to achieve best-in-class performance for their sizes compared to other open models.

And Gemma models are capable of running directly on a developer laptop or desktop computer.

Notably, Gemma surpasses significantly larger models on key benchmarks while adhering to our rigorous standards for safe and responsible outputs.

A note from Google and Alphabet CEO Sundar Pichai:Last week, we rolled out our most capable model, Gemini 1.0 Ultra, and took a significant step forward in making Google products more helpful, starting with Gemini Advanced.

Today, developers and Cloud customers can begin building with 1.0 Ultra too — with our Gemini API in AI Studio and in Vertex AI.

Our teams continue pushing the frontiers of our latest models with safety at the core.

In fact, we’re ready to introduce the next generation: Gemini 1.5.

It shows dramatic improvements across a number of dimensions and 1.5 Pro achieves comparable quality to 1.0 Ultra, while using less compute.

We’re excited by the progress, for example with our Search Generative Experience, or SGE, which you can try in Search Labs.

Introducing Gemini AdvancedBard has been the best way for people to directly experience our most capable models.

To reflect the advanced tech at its core, Bard will now simply be called Gemini.

The version with Ultra will be called Gemini Advanced, a new experience far more capable at reasoning, following instructions, coding, and creative collaboration.

You can start using Gemini Advanced by subscribing to the new Google One AI Premium plan, which offers the best of Google’s AI features in a single place.

Research AlphaGeometry: An Olympiad-level AI system for geometry ShareCopy link ×Our AI system surpasses the state-of-the-art approach for geometry problems, advancing AI reasoning in mathematicsReflecting the Olympic spirit of ancient Greece, the International Mathematical Olympiad is a modern-day arena for the world's brightest high-school mathematicians.

In a paper published today in Nature, we introduce AlphaGeometry, an AI system that solves complex geometry problems at a level approaching a human Olympiad gold-medalist - a breakthrough in AI performance.

In a benchmarking test of 30 Olympiad geometry problems, AlphaGeometry solved 25 within the standard Olympiad time limit.

Solving Ol…

Today we’re announcing a suite of advances in robotics research that bring us a step closer to this future.

AutoRT, SARA-RT, and RT-Trajectory build on our historic Robotics Transformers work to help robots make decisions faster, and better understand and navigate their environments.

So the AutoRT system comprises layers of practical safety measures from classical robotics.

SARA-RT: Making Robotics Transformers leaner and faster Our new system, Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT), converts Robotics Transformer (RT) models into more efficient versions.

We will continue to tackle challenges in robotics today and to adapt to the new capabilities and technologies …

Research Images altered to trick machine vision can influence humans too ShareCopy link ×New research shows that even subtle changes to digital images, designed to confuse computer vision systems, can also affect human perception Computers and humans see the world in different ways.

Our discovery highlights a similarity between human and machine vision, but also demonstrates the need for further research to understand the influence adversarial images have on people, as well as AI systems.

An adversarial image is one that has been subtly altered by a procedure that causes an AI model to confidently misclassify the image contents.

Indeed, security concerns have led researchers to investigate …

Retrieval augmented generation (RAG)Retrieval augmented generation, or RAG, helps ensure model outputs are grounded on your data.

Instead of relying on the model’s training knowledge, AI apps architected for RAG can search your data for information relevant to a query, then pass that information into the prompt.

Supervised fine-tuning (SFT)If you want to give a model specific instructions for a well-defined task, you might consider SFT, also often referred to as Parameter Efficient Fine Tuning (PEFT).

To perform supervised fine tuning, you need to have the model learn from input-output pairs that you provide the model.

RAG also has the benefit that you can control who has access to which gr…

This solution focuses on how game development teams can harness the capabilities of Model Garden on Vertex AI, which incorporates partner integrations such as Hugging Face and Civitai.

Read on to learn how to accomplish the first step of game asset creation – generating concept art – on Google Cloud using Vertex AI and Model Garden with Stable Diffusion.

We'll cover how to access and download popular LoRA (Low-Rank Adaptation) adapters from Hugging Face or Civitai, and serve them alongside the stabilityai/stable-diffusion-xl-base-1.0 model (from Model Garden) on Vertex AI for online prediction.

APIs enabled: Enable both the Vertex AI and Compute Engine APIs.

You're ready to run your Jupyter…

One challenge is getting access to the AI infrastructure they need.

Today, we are thrilled to announce our seamless integration of Ray, a powerful distributed Python framework, with Google Cloud's Vertex AI is generally available.

Vertex AI's comprehensive security framework can help ensure that your Ray applications comply with strict enterprise security requirements.

To fine-tune Gemma using Ray on Vertex AI, first you need a Ray cluster on Vertex AI, which Ray on Vertex AI lets you create in just a few minutes, using either the console or the Vertex AI SDK for Python.

You can monitor the cluster either by leveraging the integration with Google Cloud Logging or using the Ray Dashboard.

Trillium TPUs achieve an impressive 4.7X increase in peak compute performance per chip compared to TPU v5e.

We doubled the High Bandwidth Memory (HBM) capacity and bandwidth, and also doubled the Interchip Interconnect (ICI) bandwidth over TPU v5e.

Trillium TPUs make it possible to train the next wave of foundation models faster and serve those models with reduced latency and lower cost.

Critically, our sixth-generation TPUs are also our most sustainable: Trillium TPUs are over 67% more energy-efficient than TPU v5e.

Next-generation HBM enables higher memory bandwidth, improved power efficiency, and a flexible channel architecture to increase memory throughput.

PaliGemma, available in Vertex AI Model Garden, is the first vision-language model in the Gemma family of open models, and is well-suited for tasks like image captioning and visual question-answering.

And to empower developers to more flexibly and quickly build AI agents, we’ve made Firebase Genkit and LlamaIndex available on Vertex AI.

Facilitated through the Vertex AI plugin, Firebase developers can now take advantage of Google models like Gemini and Imagen 2, as well as text embeddings.

Now Vertex AI customers can leverage Google’s models and AI-optimized infrastructure alongside LlamaIndex’s simple, flexible, open-source data framework, to connect custom data sources to generative model…

Translation API introduces a specialized generative AI model, fine-tuned for translationsThe latest addition to Google Cloud’s Translation AI portfolio lets Translation API customers choose between our traditional machine translation model (a.k.a.

At Google Cloud Next ‘24 in April, AI-enabled translation platform Smartling co-presented on responsive translation using generative AI.

The findings included Google Adaptive Translation outperforming Google Translate with an up to 23% increase in quality.

“Adding Google Adaptive Translation to our portfolio of engines is a pivotal point in the Smartling translation and AI strategy due to its easily customizable, dynamic and dramatic improvement i…

Like our other recommendations, you can view them through our Recommendation Hub UI, API, and BigQuery export.

Deprecation and breaking change recommendations are offered at no charge and are available for all users today.

IAM for BigQuery recommendationsWe’ve expanded the popular IAM Recommender to include IAM recommendations on BigQuery datasets.

These recommendations help you enforce the principle of least privilege by ensuring that principals have only the permissions that they actually need.

The new recent change recommendations automatically flags recent risky changes to cloud resources that are identified as important based on their usage and other signals.

But combining these different data types often requires advanced analytics to interpret them meaningfully.

However, integrating unstructured data within an existing analytics framework of structured data, for example user behavior data in database tables, is not without its hurdles.

Maintaining data integrity across layers: As insights are extracted from unstructured data, preserving the original source of truth and ensuring consistency across intermediate (interstitial) layers is crucial for reliable, iterative analysis.

This approach uses BigQuery's built-in generative AI functions, coupled with remote User Defined Functions (UDFs) that interface with Vertex AI APIs.

This architecture sho…

The intricate hierarchical data structures in data warehouses and lakes sourced from diverse origins can make data modeling a protracted and error-prone process.

To quickly adapt and create data models that meet evolving business requirements without having to rework them excessively, you need data models that are flexible, modular and adaptable enough to accommodate many requirements.

Multimodal large language models (LLMs) can analyze examples of data in the data lake, including text descriptions, code, and even images of existing databases.

In this blog, we walk you through how to use multimodal LLMs in BigQuery to create a database schema.

STEP1 : Create an entity relationship diagramTh…

AI Infrastructure for RAGPrior to the rise of Generative AI, a typical application architecture might involve a database, a set of microservices, and a frontend.

Even the most basic RAG applications introduce new requirements for serving LLMs, processing, and retrieving unstructured data.

Many customers choose to access AI infrastructure like TPUs and GPUs via a fully managed platform, such as Vertex AI.

This is why we have developed a quickstart solution and reference architecture for RAG applications built on top of GKE, Cloud SQL, and open-source frameworks Ray, LangChain and Hugging Face.

Benefits of RAG on GKE and Cloud SQLGKE and Cloud SQL accelerate your journey to production in a va…

If reading and constructing a batch takes longer than GPU computation, then the GPU is effectively stalled and underutilized, leading to longer training times.

In Dataflux, we employ a Cloud Storage feature called Compose Objects that can dynamically combine many smaller objects into a larger object.

Another optimization that Dataflux Datasets employs is high-throughput parallel-listing, speeding up the initial metadata needed for the dataset.

Dataflux Dataset uses these client libraries under the hood.

Dataflux is available nowGive the Dataflux Dataset for PyTorch (or the Dataflux Python client library if writing your own ML training dataset code) a try and let us know how it boosts your w…

Early signals tell us that generative AI is that missing catalyst, and Google Cloud is a unique partner for your journey.

Why generative AI catalyzes your teamA business strategy with generative AI at the center benefits customers and employees.

It’s not just about generative AI; it’s about what it takes to be good at generative AI.

There’s more than one way to proceed with your generative AI strategy, but at Google Cloud, we see three crucial building blocks for your success.

Register for our Building Apps in an AI Era webinar to learn more about how Google Cloud can help you innovate faster, deliver unparalleled customer experiences, and secure a lasting competitive advantage.

“It recognises the value of our award-winning journalism and will give us early insights into how content is surfaced through AI.

“Apart from the benefits to the FT, there are broader implications for the industry.

It’s right, of course, that AI platforms pay publishers for the use of their material.

“We value the opportunity to be inside the development loop as people discover content in new ways.

As with any transformative technology, there is potential for significant advancements and major challenges, but what’s never possible is turning back time.

OpenAI, alongside industry leaders including Amazon, Anthropic, Civitai, Google, Meta, Metaphysic, Microsoft, Mistral AI, and Stability AI, has committed to implementing robust child safety measures in the development, deployment, and maintenance of generative AI technologies as articulated in the Safety by Design principles.

By adopting comprehensive Safety by Design principles, OpenAI and our peers are ensuring that child safety is prioritized at every stage in the development of AI.

Responsibly source our training datasets, detect and remove child sexual abuse material (CSAM) and child sexual exploitation material (CSEM) from training data, and report any confirmed CSAM to the relevant a…

Customers with a sustained level of tokens per minute (TPM) usage on GPT-4 or GPT-4 Turbo can request access to provisioned throughput to get discounts ranging from 10–50% based on the size of the commitment.

Reduced costs on asynchronous workloads: Customers can use our new Batch API to run non-urgent workloads asynchronously.

Batch API requests are priced at 50% off shared prices, offer much higher rate limits, and return results within 24 hours.

We plan to keep adding new features focused on enterprise-grade security, administrative controls, and cost management.

For more information on these launches, visit our API documentation or get in touch with our team to discuss custom solution…

Our new local presence also gets us closer to leading businesses like Daikin, Rakuten, and TOYOTA Connected who are using ChatGPT Enterprise to automate complex business processes, assist in data analysis, and optimize internal reporting.

ChatGPT also helps accelerate the efforts of local governments, such as Yokosuka City, which is leveraging the technology to improve the efficiency of public services in Japan.

Over the past year, the city has gradually provided ChatGPT access to almost all city employees, and 80% have reported increases in productivity.

Now Yokosuka City has formed a network with 21 local governments—including the Tokyo Metropolitan Government and the City of Kobe—to …

Assisted Fine-TuningAt DevDay last November, we announced a Custom Model program designed to train and optimize models for a specific domain, in partnership with a dedicated group of OpenAI researchers.

Since then, we've met with dozens of customers to assess their custom model needs and evolved our program to further maximize performance.

Today, we are formally announcing our assisted fine-tuning offering as part of the Custom Model program.

Fully custom-trained models imbue new knowledge from a specific domain by modifying key steps of the model training process using novel mid-training and post-training techniques.

Our team modified every step of the model training process, from domain-s…

We’ve also introduced additional content safeguards for this experience, such as blocking prompts and generations in a wider range of categories.

There are many benefits to creating an account including the ability to save and review your chat history, share chats, and unlock additional features like voice conversations and custom instructions.

For anyone that has been curious about AI’s potential but didn’t want to go through the steps to set-up an account, start using ChatGPT today.

We recognize that generating speech that resembles people's voices has serious risks, which are especially top of mind in an election year.

We are engaging with U.S. and international partners from across government, media, entertainment, education, civil society and beyond to ensure we are incorporating their feedback as we build.ÂThe partners testing Voice Engine today have agreed to our usage policies, which prohibit the impersonation of another individual or organization without consent or legal right.

In addition, our terms with these partners require explicit and informed consent from the original speaker and we don’t allow developers to build ways for individual users to create the…

Starting his career at DreamWorks Animation, Don Allen III is a multidisciplinary creator, speaker and consultant who collaborates with major tech and entertainment companies on mixed reality, virtual reality and AI applications.

“For a long time I've been making augmented reality hybrid creatures that I think would be fun combinations in my head.

Now I have a much easier way of prototyping the ideas before I fully build out the 3-D characters to place in spatial computers.” Don cites Sora’s “weirdness” as its greatest strength: “It’s not bound by traditional laws of physics or conventions of thought.” He says that working with Sora shifted his focus from “technical hurdle…

Echoing this sentiment, Louis Dreyfus, CEO of Le Monde, stated, "At the moment we are celebrating the 80th anniversary of Le Monde, this partnership with OpenAI allows us to expand our reach and uphold our commitment to providing accurate, verified, balanced news stories at scale.

Collaborating with OpenAI ensures that our authoritative content can be accessed and appreciated by a broader, more diverse audience. ÂEvery shift in the media landscape has presented Le Monde with new opportunities.

From the transition to digital platforms to embracing the era of free media, Le Monde has consistently seized these moments to underscore its commitment to independence, expertise, and journalistic i…

Additionally, Sam Altman, CEO, will rejoin the OpenAI Board of Directors.ÂSue, Nicole and Fidji have experience in leading global organizations and navigating complex regulatory environments, including backgrounds in technology, nonprofit and board governance.

They will work closely with current board members Adam D’Angelo, Larry Summers and Bret Taylor as well as Sam and OpenAI’s senior management.ÂBret Taylor, Chair of the OpenAI board, stated, “I am excited to welcome Sue, Nicole, and Fidji to the OpenAI Board of Directors.

She also served as President of Sony Entertainment, Inc., and simultaneously served as President of Sony Corporation of America.

She also serves as a member of …

The Special Committee of the OpenAI Board today announced the completion of the review by WilmerHale.

The firm conducted dozens of interviews with members of OpenAI’s prior Board, OpenAI executives, advisors to the prior Board, and other pertinent witnesses; reviewed more than 30,000 documents; and evaluated various corporate actions.

“We have unanimously concluded that Sam and Greg are the right leaders for OpenAI,” stated Bret Taylor, Chair of the OpenAI Board.

The Special Committee acknowledged the important work done by WilmerHale in conducting this extensive review and thanked OpenAI current and former Board members, advisors and employees for their cooperation.

The Special Commi…

Date: January 31, 2018 at 11:54:30 PM PSTSubject: Re: Top AI institutions todayWorking at the cutting edge of AI is unfortunately expensive.

For example,In addition to DeepMind, Google also has Google Brain, Research, and Cloud.

If historical trends are any indication, progress in AI is primarily driven by systems - compute, data, infrastructure.

Not only that, but any algorithmic advances published in a paper somewhere can be almost immediately re-implemented and incorporated.

The “second stage” would be a full self driving solution based on large-scale neural network training, which OpenAI expertise could significantly help accelerate.

This technical report focuses on (1) our method for turning visual data of all types into a unified representation that enables large-scale training of generative models, and (2) qualitative evaluation of Sora’s capabilities and limitations.

Model and implementation details are not included in this report.

Much prior work has studied generative modeling of video data using a variety of methods, including recurrent networks,[^1][^2][^3] generative adversarial networks,[^4][^5][^6][^7] autoregressive transformers,[^8][^9] and diffusion models.

[^10][^11][^12] These works often focus on a narrow category of visual data, on shorter videos, or on videos of a fixed size.

Sora is a generalist mo…

Based on collaboration and information sharing with Microsoft, we disrupted five state-affiliated malicious actors: two China-affiliated threat actors known as Charcoal Typhoon and Salmon Typhoon; the Iran-affiliated threat actor known as Crimson Sandstorm; the North Korea-affiliated actor known as Emerald Sleet; and the Russia-affiliated actor known as Forest Blizzard.

The identified OpenAI accounts associated with these actors were terminated.

Salmon Typhoon used our services to translate technical papers, retrieve publicly available information on multiple intelligence agencies and regional threat actors, assist with coding, and research common ways processes could be hidden on a system.…

We’re testing memory with ChatGPT.

Remembering things you discuss across all chats saves you from having to repeat information and makes future conversations more helpful.

You're in control of ChatGPT's memory.

You can explicitly tell it to remember something, ask it what it remembers, and tell it to forget conversationally or through settings.

We are rolling out to a small portion of ChatGPT free and Plus users this week to learn how useful it is.

O’NEILL: Yes, yes.

O’NEILL: Yeah, yeah.

Wasn’t there …O’NEILL: Yes, yes, yes.

O’NEILL: Yeah, yeah, yes.

O’NEILL: Yeah, yeah.

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

NEW RESEARCHInjecting New Knowledge into Large Language Models via Supervised Fine-TuningLarge language models (LLMs) have shown remarkable performance in generating text similar to that created by people, proving to be a valuable asset across various applications.

Recently, several Indic (Indian language) LLMs have been created to help build more locally and culturally relevant LLMs.

The researchers intend for MS MARCO Web Search to lay the groundwork for future advancements in AI and systems research.…

The ways people engage with technology, through its design and functionality, determine its utility and acceptance in everyday use, setting the stage for widespread adoption.

The ACM CHI Conference on Human Factors in Computing Systems is a premier forum that brings together researchers and experts in the field, and Microsoft is honored to support CHI 2024 as a returning sponsor.

This research aims to redefine how people work, collaborate, and play using technology, with a focus on design innovation to create more personalized, engaging, and effective interactions.

Spotlight: Microsoft research newsletter Microsoft Research Newsletter Stay connected to the research community at Microsoft.

L…

Over the past decade, robotics has revolutionized numerous industries that rely on storage systems, such as manufacturing and warehousing.

Our paper, published at ICRA 2024, introduces RASCAL: A Scalable, High-redundancy Robot for Automated Storage and Retrieval Systems, which addresses these concerns.

RASCAL is an untethered robot that improves the efficiency of vertical storage systems by operating across evenly spaced, parallel shelves and horizontal rails.

RASCAL was inspired by the challenges of managing archival storage media in datacenters, and it’s the key component of Project Silica’s storage and retrieval system.

Advancing robotics and automationAs digital economies grow, the need…

In the quest for groundbreaking materials crucial to nanoelectronics, energy storage, and healthcare, a critical challenge looms: predicting a material’s properties before it is even created.

These factors drastically affect atomic interactions within materials, making accurate property prediction and behavior simulation exceedingly demanding.

Here at Microsoft Research, we developed MatterSim, a deep-learning model for accurate and efficient materials simulation and property prediction over a broad range of elements, temperatures, and pressures to enable the in silico materials design.

MatterSim can model materials properties and behaviors under realistic temperature and pressure condition…

However, developing and fine-tuning autoscaling algorithms, which govern this process, present significant challenges.

In our paper, “VASIM: Vertical Autoscaling Simulator Toolkit,” presented at ICDE 2024, we introduce a tool designed to address the complexities involved in assessing autoscaling algorithms.

The Simulation Run feature enables the modification of algorithm parameters, imported via a configuration file, and the execution of simulations based on the selected trace.

Because VASIM is similar to standard autoscaling architecture (as in the Kubernetes Vertical Pod Autoscaler (opens in new tab) [VPA]) it allows us to test autoscaling algorithms for pods, applications, and virtual ma…

Consequently, the KV cache can become prohibitively large as the complexity of the tasks increases, sometimes requiring up to 320 GB for a single operation.

Observations of the KV cacheThe development of FastGen is underpinned by our observations of how the KV cache functions.

Also, some LLM modules primarily attend only to special tokens, such as punctuation, for which it is possible to create a KV cache that retains only those tokens.

Finally, some LLM modules broadly need all tokens, and for these we can employ the standard KV cache and store all words.

FastGen accounts for the diversity of KV cache structuresBecause different KV caches have different structures, they need to be handled …

LoftQ’s strength lies in its ability to combine quantization and adaptive initialization during fine-tuning.

How LoftQ worksLoftQ builds on the principles of LoRA (opens in new tab) and QLoRA (opens in new tab).

LoRA is a method that greatly reduces the number of parameters needed for training, decreasing the memory requirements for fine-tuning.

That is, LoftQ seeks to identify a combination of a quantized matrix and a low rank matrix such that their sum closely approximates the original pretrained weight.

This table compares LoftQ and QLoRA during the fine-tuning of two Llama-2 models on the Wikitext-2 and GSM8K datasets.

GRETCHEN HUIZINGA: Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on world-class research in brief.

And the second stage, we actually had humans, annotators, just certify that the selected data is actually of high quality.

Well, my favorite part of a research paper is where it says, “and what we found was … ,” so talk a little bit about your results.

Michel, if there was one thing you wanted our listeners to take away from this research, kind of golden nugget, what would it be?

I think that’s what makes MathVista stand out compared to other datasets.

NEW RESEARCHCan Large Language Models Transform Natural Language Intent into Formal Method Postconditions?

However, there is no guarantee that a program’s implementation aligns with its natural language documentation.

However, this information is often underutilized, due to the inherent ambiguity of natural language which makes natural language intent challenging to check programmatically.

The “emergent abilities” of large language models (LLMs) have the potential to facilitate the translation of natural language intent to programmatically checkable assertions.

In a new paper: Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?

Modern computer systems and applications, with unprecedented scale, complexity, and security needs, require careful co-design and co-evolution of hardware and software.

We are pleased to share that eight papers from Microsoft researchers and their collaborators have been accepted to the conference, spanning a broad spectrum of topics.

Regarding infrastructure, topics include memory safety with CHERI, I/O prefetching in modern storage, and smart oversubscription of burstable virtual machines.

Burstable virtual machines (BVMs) are a type of virtual machine in the cloud that allows temporary increases in resource allocation.

We are always pushing the boundaries of computer systems to improve t…

What would it take to build an interactive AI system that could assist you with any task in the physical world, just as a real-time expert would?

To begin exploring the core competencies that such a system would require, we developed and released the Situated Interactive Guidance, Monitoring, and Assistance (SIGMA) system, an open-source research platform and testbed prototype (opens in new tab) for studying mixed-reality task assistance.

SIGMA provides a basis for researchers to explore, understand, and develop the capabilities required to enable in-stream task assistance in the physical world.

Physical and social intelligenceFor AI systems to fluidly collaborate with people in the physica…

Well, I’ve heard other people on your teams use words like surprise, sometimes even shock …HOSN: Yeah, yeah, there are a lot of “wow” factors.

HUIZINGA: Yeah, yeah.

AI research is moving at such speeds that I feel like we need to get accustomed to a timing of now.

HOSN: That’s right.

Well, as we close, Rafah, I want to ask a question anchored on the big idea behind AI Frontiers.

New generations of language models like GPT-4 and Mixtral 8x7B advance the capability to process long input texts.

The first structure, the task description, remains static and independent of the input as a result of conventional prompt optimization techniques.

Despite previous efforts in prompt optimization, the evolution towards more complex prompt structures has rendered many older strategies ineffective in this new context.

SAMMO: A prompt optimization approachDownload SAMMOTo address these challenges, we developed the Structure-Aware Multi-objective Metaprompt Optimization (SAMMO) framework.

We compared it against Automatic Prompt Optimization (APO) and GrIPS, applying open-source mode…

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

NEW RESEARCHAppropriate reliance on Generative AI: Research synthesisAppropriate reliance on AI happens when people accept correct AI outputs and reject incorrect ones.

Spotlight: Event Series Microsoft Research Forum Join us for a continuous exchange of ideas about research in the era of general AI.

They also develop AFRICOMET: COMET evaluation metrics for African languages by leveraging DA data from well-resourced languages and an African-centric multilingual encoder (AfroXLMR) to create state-of-the-…

Phase transitions in water are so common most of us probably don’t even think about them, but phase transitions in novel materials or complex physical systems are an important area of study.

Importantly, because their approach leverages generative models, it does not require huge, labeled training datasets used in other machine-learning techniques.

“This is a really nice way of incorporating something you know about your physical system deep inside your machine-learning scheme.

And because the researchers directly approximate the probability distributions underlying measurements from the physical system, the classifier has system knowledge.

And because it can work automatically without the …

"Imagine a new way to help language models understand and generate text, like a game.

It's a pretty exciting demonstration of how bringing game-theoretic strategies into the mix can tackle some big challenges in making language models more reliable and consistent."

In practice, implementing the consensus game approach to language model querying, especially for question-answering tasks, does involve significant computational challenges.

The potential for such a method to significantly improve the base model's performance is high, which could result in more reliable and factual outputs from ChatGPT and similar language models that people use daily.

"The proposal by the MIT researchers is an i…

They had no way to bring mobile software development — about to become part of everyday life — into the classroom.

Like Scratch, App Inventor is a block-based language, allowing programmers to visually snap together pre-made “blocks” of code rather than need to learn specialized programming syntax.

Friedman describes it as novel for the time, particularly for mobile development, to make it as easy as possible to build simple mobile apps.

App Inventor’s long-term sustainability now rests with the App Inventor Foundation, created in 2022 to grow its resources and further drive its adoption.

“The opportunity for App Inventor and MIT is to democratize those new possibilities for young people — …

The team also built a simulator to test control algorithms for deformable soft robots on a series of challenging, shape-changing tasks.

While reconfigurable soft robots are still in their infancy, such a technique could someday enable general-purpose robots that can adapt their shapes to accomplish diverse tasks.

“When people think about soft robots, they tend to think about robots that are elastic, but return to their original shape.

The researchers built a simulator to test control algorithms for deformable soft robots on a series of challenging, shape-changing tasks.

In this way, the control algorithm follows a coarse-to-fine methodology.

He studies the effect of certain bacteria that have been observed encouraging the spread of ovarian cancer and possibly reducing the effectiveness of chemotherapy and immunotherapy.

This opens an avenue to develop therapies to see if we can start to undo some of these changes.”Kumar’s research combines microbiology, bioengineering, artificial intelligence, big data, and materials science.

Using microbiome sequencing and AI, he aims to define microbiome changes that may correlate with poor patient outcomes.

Kumar started inching toward work in the health sciences just months into earning his bachelor's degree at IIT Bombay.

“It has been amazing to work with Ashutosh on this ovarian cancer mi…

MIT President Sally Kornbluth and OpenAI CEO Sam Altman covered all that and more in a wide-ranging discussion on MIT’s campus May 2.

“I think we’ve made surprisingly good progress around how to align a system around a set of values,” Altman said.

Kornbluth also brought up privacy concerns associated with the vast amounts of data needed to train today’s large language models.

I don’t know what the answers will be.”For both privacy and energy consumption concerns surrounding AI, Altman said he believes progress in future versions of AI models will help.

Prosperity, abundance, a better life next year, a better life for our children.

As a researcher with the MIT-IBM Watson AI Lab and the Computer Science and Artificial Intelligence Laboratory, Ragan-Kelley specializes in high-performance, domain-specific programming languages and machine learning that enable 2D and 3D graphics, visual effects, and computational photography.

One such project with the MIT-IBM Watson AI Lab leverages Exo, a language developed in Ragan-Kelley’s group.

Ragan-Kelley and his group are developing methods that straddle each technique, balancing trade-offs to achieve effective and resource-efficient programming.

In addition, his team can add their bespoke schedulers on top, which can help target specialized hardware like machine-learning accelera…

The PIs on these projects, who have common interests but different strengths, create more powerful teams by working together.

Transmitting shared values, attitudes, and beliefs to improve cybersecurity across supply chainsThe MIT and HPI cybersecurity researchers say that most ransomware attacks aren’t reported.

The HPI team includes Professor Gerard de Melo; HPI School of Entrepreneurship Director Frank Pawlitschek; and doctoral student Michael Mansfeld.

Furthering the goals of the HPI-MIT Joint Research ProgramThese three diverse projects all advance the mission of the HPI-MIT collaboration.

MIT MAD aims to use design to transform learning, catalyze innovation, and empower society by insp…

From cutting-edge robotics, design, and bioengineering to sustainable energy solutions, ocean engineering, nanotechnology, and innovative materials science, MechE students and their advisors are doing incredibly innovative work.

He is currently working on methods for the scalable fabrication of nano-architected materials and predicting their mechanical properties.

The ability to fine-tune the mechanical properties of specific materials brings versatility and adaptability, making these materials suitable for a wide range of applications across multiple industries.

While the research applications are quite diverse, Dhulipala is passionate about making space habitable for humanity, a crucial s…

Luckily, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers have found a treasure trove of abstractions within natural language.

Still, their work represents a step forward for how language models can facilitate increasingly elaborate coding activities.

The method trains on potential tasks and their natural language descriptions, then a language model proposes action abstractions from this dataset.

Just like LILO and Ada, LGA has a novel focus on how natural language leads us to those better abstractions.

These recent papers demonstrate a compelling way forward by placing large language models in an interactive loop with symbolic search, compression, and plannin…

Within the corpus of weather radar data, tornadoes are extremely rare events.

They were particularly eager to apply deep learning, a form of machine learning that excels at processing visual data.

Fusing multiple types of data could improve the accuracy of machine learning models.

Taking steps toward operationsOn top of detecting tornadoes, Kurdzo hopes that models might help unravel the science of why they form.

“I think the forecaster community is still, understandably, skeptical of machine learning.

When introducing new teaching and learning technologies, panelists stressed the importance of iteration and teaching students how to develop critical thinking skills while leveraging technologies like generative AI.

Incorporating generative AI into learning experiencesMIT faculty and instructors aren’t just willing to experiment with generative AI — some believe it’s a necessary tool to prepare students to be competitive in the workforce.

But, she saw an opportunity for teaching experimentation with generative AI.

When incorporating generative AI into assignments, the key is to be clear about learning goals and open to sharing examples of how generative AI could be used in ways that align w…

Slater Professor in Aeronautics and Astronautics, has been named the new head of the Department of Aeronautics and Astronautics (AeroAstro), effective May 1.

She currently directs the Interactive Robotics Group in MIT’s Computer Science and Artificial Intelligence Lab (CSAIL), and MIT’s Industrial Performance Center.

Shah and her team at the Interactive Robotics Group conduct research that aims to imagine the future of work by designing collaborative robot teammates that enhance human capability.

Shah was also named a Bisplinghoff Faculty Fellow, was named to MIT Technology Review’s TR35 List, and received an NSF Faculty Early Career Development Award.

Shah succeeds Professor Steven Barrett…

To do this, they set out to map the spatio-temporal brain dynamics involved in recognizing a visual image.

What they found was that a more distributed network of brain regions than previously thought are actively involved in the encoding and retention processes that underpin memorability.

“We've identified a brain signature of visual memorability that emerges around 300 milliseconds after seeing an image, involving areas across the ventral occipital cortex and temporal cortex, which processes information like color perception and object recognition.

They created a “representational matrix,” which is like a detailed chart, showing how similar neural responses are in various brain regions.

Wh…

Their chip can keep a user’s health records, financial information, or other sensitive data private while still enabling huge AI models to run efficiently on devices.

A digital IMC chip performs computations inside a device’s memory, where pieces of a machine-learning model are stored after being moved over from a central server.

In a side-channel attack, a hacker monitors the chip’s power consumption and uses statistical techniques to reverse-engineer data as the chip computes.

First, they employed a security measure where data in the IMC are split into random pieces.

Safety testingTo test their chip, the researchers took on the role of hackers and tried to steal secret information using s…

Modeling Extremely Large Images with xTAs computer vision researchers, we believe that every pixel can tell a story.

However, there seems to be a writer’s block settling into the field when it comes to dealing with large images.

Today, we make one of two sub-optimal choices when handling large images: down-sampling or cropping.

Why bother handling large images anyways?

That’s basically what we do with large images with $x$T.

As computer vision researchers, we believe that every pixel can tell a story. However, there seems to be a writer’s block settling into the field when it comes to dealing with large images. Large images are no longer rare—the cameras we carry in our pockets and those orbiting our planet snap pictures so big and detailed that they stretch our current best models and hardware to their breaking points when handling them. Generally, we face a quadratic increase in memory usage as a function of image size.

Today, we make one of two sub-optimal choices when handling large images: down-sampling or cropping. These two methods incur significant losses in the amount of information and context present…

2024 BAIR Graduate DirectoryEvery year, the Berkeley Artificial Intelligence Research (BAIR) Lab graduates some of the most talented and innovative minds in artificial intelligence and machine learning.

Our Ph.D. graduates have each expanded the frontiers of AI research and are now ready to embark on new adventures in academia, industry, and beyond.

These fantastic individuals bring with them a wealth of knowledge, fresh ideas, and a drive to continue contributing to the advancement of AI.

Join us in celebrating the achievements of BAIR’s latest PhD graduates.

Thank you to our friends at the Stanford AI Lab for this idea!

Every year, the Berkeley Artificial Intelligence Research (BAIR) Lab graduates some of the most talented and innovative minds in artificial intelligence and machine learning. Our Ph.D. graduates have each expanded the frontiers of AI research and are now ready to embark on new adventures in academia, industry, and beyond.

These fantastic individuals bring with them a wealth of knowledge, fresh ideas, and a drive to continue contributing to the advancement of AI. Their work at BAIR, ranging from deep learning, robotics, and natural language processing to computer vision, security, and much more, has contributed significantly to their fields and has had transformative impacts on society.

This…

AI caught everyone’s attention in 2023 with Large Language Models (LLMs) that can be instructed to perform general tasks, such as translation or coding, just by prompting. This naturally led to an intense focus on models as the primary ingredient in AI application development, with everyone wondering what capabilities new LLMs will bring.

As more developers begin to build using LLMs, however, we believe that this focus is rapidly changing: state-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models.

For example, Google’s AlphaCode 2 set state-of-the-art results in programming through a carefully engineered system that uses L…

In this post, we analyze the trend toward compound AI systems and what it means for AI developers.

We argue that compound AI systems will likely be the best way to maximize AI results in the future, and might be one of the most impactful trends in AI in 2024.

We define a Compound AI System as a system that tackles AI tasks using multiple interacting components, including multiple calls to models, retrievers, or external tools.

Developing Compound AI SystemsWhile compound AI systems can offer clear benefits, the art of designing, optimizing, and operating them is still emerging.

However, new compound AI systems contain non-differentiable components like search engines or code interpreters, a…

The structure of Ghostbuster, our new state-of-the-art method for detecting AI-generated text. Large language models like ChatGPT write impressively well—so well, in fact, that they’ve become a problem. Students have begun using these models to ghostwrite assignments, leading some schools to ban ChatGPT. In addition, these models are also prone to producing text with factual errors, so wary readers may want to know if generative AI tools have been used to ghostwrite news articles or other sources before trusting them.

What can teachers and consumers do? Existing tools to detect AI-generated text sometimes do poorly on data that differs from what they were trained on. In addition, if these m…

Ghostbuster: Detecting Text Ghostwritten by Large Language ModelsThe structure of Ghostbuster, our new state-of-the-art method for detecting AI-generated text.

Large language models like ChatGPT write impressively well—so well, in fact, that they’ve become a problem.

Existing tools to detect AI-generated text sometimes do poorly on data that differs from what they were trained on.

Our recent paper introduces Ghostbuster, a state-of-the-art method for detecting AI-generated text.

Many current AI-generated text detection systems are brittle to classifying different types of text (e.g., different writing styles, or different text generation models or prompts).

Asymmetric Certified Robustness via Feature-Convex Neural Networks TLDR: We propose the asymmetric certified robustness problem, which requires certified robustness for only one class and reflects real-world adversarial scenarios. This focused setting allows us to introduce feature-convex classifiers, which produce closed-form and deterministic certified radii on the order of milliseconds. Figure 1. Illustration of feature-convex classifiers and their certification for sensitive-class inputs. This architecture composes a Lipschitz-continuous feature map $\varphi$ with a learned convex function $g$. Since $g$ is convex, it is globally underapproximated by its tangent plane at $\varphi(x)$, y…

Asymmetric Certified Robustness via Feature-Convex Neural NetworksAsymmetric Certified Robustness via Feature-Convex Neural NetworksTLDR: We propose the asymmetric certified robustness problem, which requires certified robustness for only one class and reflects real-world adversarial scenarios.

We argue that these issues can be addressed by refining the certified robustness problem to be more aligned with practical adversarial settings.

The Asymmetric Certified Robustness ProblemCurrent certifiably robust classifiers produce certificates for inputs belonging to any class.

Feature-convex classifiersWe propose feature-convex neural networks to address the asymmetric robustness problem.

Conclu…

Goal Representations for Instruction Following A longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans. Natural language has the potential to be an easy-to-use interface for humans to specify arbitrary tasks, but it is difficult to train robots to follow language instructions. Approaches like language-conditioned behavioral cloning (LCBC) train policies to directly imitate expert actions conditioned on language, but require humans to annotate all training trajectories and generalize poorly across scenes and behaviors. Meanwhile, recent goal-conditioned approaches perform much better at general manipulation tasks, but do not e…

Goal Representations for Instruction FollowingGoal Representations for Instruction FollowingA longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans.

Goal Representations for Instruction FollowingThe GRIF model consists of a language encoder, a goal encoder, and a policy network.

Our approach, Goal Representations for Instruction Following (GRIF), jointly trains a language- and a goal- conditioned policy with aligned task representations.

In particular, we exploit this structure by requiring that language- and goal- representations be similar for the same semantic task.

We train dual image and text encoders by doing contrastiv…

Rethinking the Role of PPO in RLHF TL;DR: In RLHF, there’s tension between the reward learning phase, which uses human preference in the form of comparisons, and the RL fine-tuning phase, which optimizes a single, non-comparative reward. What if we performed RL in a comparative way? Figure 1: This diagram illustrates the difference between reinforcement learning from absolute feedback and relative feedback. By incorporating a new component - pairwise policy gradient, we can unify the reward modeling stage and RL stage, enabling direct updates based on pairwise responses. Large Language Models (LLMs) have powered increasingly capable virtual assistants, such as GPT-4, Claude-2, Bard and Bing…

Rethinking the Role of PPO in RLHFRethinking the Role of PPO in RLHFTL;DR: In RLHF, there’s tension between the reward learning phase, which uses human preference in the form of comparisons, and the RL fine-tuning phase, which optimizes a single, non-comparative reward.

By incorporating a new component - pairwise policy gradient, we can unify the reward modeling stage and RL stage, enabling direct updates based on pairwise responses.

Proximal Policy Optimization (PPO), the dominant RL optimizer in this process, has been reported to exhibit instability and implementation complications.

Derivation of P3OOur idea stems from the vanilla policy gradient (VPG).

Under this framework, we develop …

Goal Representations for Instruction FollowingGoal Representations for Instruction FollowingA longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans.

Goal Representations for Instruction FollowingThe GRIF model consists of a language encoder, a goal encoder, and a policy network.

Our approach, Goal Representations for Instruction Following (GRIF), jointly trains a language- and a goal- conditioned policy with aligned task representations.

In particular, we exploit this structure by requiring that language- and goal- representations be similar for the same semantic task.

We train dual image and text encoders by doing contrastiv…

In this post, we share how LotteON improved their recommendation service using Amazon SageMaker and machine learning operations (MLOps).

The main AWS services used are SageMaker, Amazon EMR, AWS CodeBuild, Amazon Simple Storage Service (Amazon S3), Amazon EventBridge, AWS Lambda, and Amazon API Gateway.

The SageMaker pipeline predefined in CodeBuild runs, and sequentially runs steps such as preprocessing including provisioning, model training, and model registration.

Recommendation model using NCFNCF is an algorithm based on a paper presented at the International World Wide Web Conference in 2017.

Let’s walk through the steps from model training to deployment, using some code examples.

We cover the technical implementation using the Anthropic Claude large language model (LLM) on Amazon Bedrock and AWS Lambda deployed with the AWS Serverless Application Model (AWS SAM).

With the ability to scale up to 200,000-token context windows, Anthropic Claude v2.1 on Amazon Bedrock is our preferred choice for this post.

The 200,000 tokens supported by Anthropic Claude v2.1 on Amazon Bedrock would be equivalent to roughly 150,000 words or over 500 pages of documents.

Enable model access through Amazon BedrockYou can add access to a model from the Amazon Bedrock console.

With the capabilities of Amazon Bedrock and the AWS SAM, you can increase educators’ productivity and foster student…

ONNX Runtime is the runtime engine used for model inference and training with ONNX.

In this post, we show how to run ONNX Runtime inference on AWS Graviton3-based EC2 instances and how to configure them to use optimized GEMM kernels.

As shown in the following diagrams, the optimized GEMM kernels are integrated into the ONNX Runtime CPU EP as MLAS kernels.

Enable the optimizationsThe optimizations are part of the ONNX Runtime 1.17.0 release, and are available starting with onnxruntime-1.17.0 python wheels and conda-1.17.0 packages.

Then we compared the improvements from the optimized MMLA kernels from ONNX Runtime 1.17.1 for the same model inference.

Amazon Ads applied scientists use SageMaker Studio as the web-based interface to work with SageMaker (step ii).

The product image is fetched from an image repository, which is a part of an existing solution predating this creative feature.

Then, the deployed text-to-image model is used for image generation using the prompt and the processed image (step 5).

Here, Amazon SageMaker is at the center of the solution, starting from JumpStart to final SageMaker deployment.

If you plan to build your generative AI application on Amazon SageMaker, the fastest way is with SageMaker JumpStart.

In this post, we provide an overview of the state-of-the-art embedding models by Voyage AI and show a RAG implementation with Voyage AI’s text embedding model on Amazon SageMaker Jumpstart, Anthropic’s Claude 3 model on Amazon Bedrock, and Amazon OpenSearch Service.

Voyage AI’s embedding models are the preferred embedding models for Anthropic.

In addition to general-purpose embedding models, Voyage AI offers domain-specific embedding models that are tuned to a particular domain.

As part of SageMaker JumpStart, you can deploy Voyage AI embedding models with a few clicks and start running your RAG stack on AWS.

Get started or level up your existing RAG stack on AWS today with Voyage AI embedd…

You can learn how to improve your LLMs with RLHF on Amazon SageMaker, see Improving your LLMs with RLHF on Amazon SageMaker.

Amazon SageMaker Sample and used Amazon SageMaker documentation as the knowledge base.

ConclusionThis post presented solutions for incorporating both offline and online human workflows into generative AI applications on AWS.

Together, these human-in-the-loop techniques, offline RLHF workflows, and online real-time workflows enable you to develop responsible and robust generative AI applications.

The provided solutions integrate multiple AWS services, like Amazon Bedrock, SageMaker, SageMaker Ground Truth, Lambda, Amazon S3, and Step Functions.

Amazon Titan Text Premier, the latest addition to the Amazon Titan family of large language models (LLMs), is now generally available in Amazon Bedrock.

Exclusive to Amazon Bedrock, Amazon Titan Text models support a wide range of text-related tasks, including summarization, text generation, classification, question-answering, and information extraction.

Document Explorer sample applicationThe Document Explorer sample generative AI application can help you quickly understand how to build end-to-end generative AI applications on AWS.

This app deploys an Amazon Bedrock agent that can consult an Amazon Bedrock knowledge base backed by Amazon OpenSearch Serverless as a vector store.

To learn mor…

Amazon Bedrock also comes with a broad set of capabilities required to build generative AI applications with security, privacy, and responsible AI.

We also explore the utility of the RAG prompt engineering technique as it applies to the task of summarization.

This model is used for the clinical summarization tasks where we evaluate the few-shot and zero-shot prompting techniques.

Because we used only the radiology report text data, we downloaded just one compressed report file (mimic-cxr-reports.zip) from the MIMIC-CXR website.

Findings: {} Assistant:""" Few-shot prompting examples_string = '' for ex in examples: examples_string += f"""H:{ex['findings']} A:{ex['impression']}""" prompt_few_s…

In today’s technological landscape, artificial intelligence (AI) and machine learning (ML) are becoming increasingly accessible, enabling builders of all skill levels to harness their power.

As more companies adopt AI solutions, there’s a growing need to upskill both technical and non-technical teams in responsibly expanding AI usage.

Through the AWS DeepRacer League, we have educated over 550,000 developers, crowned five AWS DeepRacer champions, recognized over 100 monthly virtual circuit winners, and rewarded over 10,000 participants worldwide with Amazon gift cards, cash prizes, and paid trips to AWS re:Invent to compete for the annual AWS DeepRacer Championship Cup.

Deep Racer is proof …

Amazon SageMaker Canvas and Amazon SageMaker JumpStart democratize this process, offering no-code solutions and pre-trained models that enable businesses to fine-tune LLMs without deep technical expertise, helping organizations move faster with fewer technical resources.

For businesses invested in the Amazon SageMaker ecosystem, using SageMaker Canvas with SageMaker JumpStart models provides continuity in operations and granular control over deployment options through SageMaker’s wide range of instance types and configurations.

For information on using SageMaker Canvas with Amazon Bedrock models, see Fine-tune and deploy language models with Amazon SageMaker Canvas and Amazon Bedrock.

Alter…

In this post, we show you how LotteON implemented dynamic A/B testing for their personalized recommendation system.

The dynamic A/B testing system monitors user reactions, such as product clicks, in real-time from the recommended item lists provided.

For a hands-on workshop on dynamic A/B testing with MAB and Thompson sampling algorithms, see Dynamic A/B Testing on Amazon Personalize & SageMaker Workshop.

Dynamic A/B test architectureThe following figure shows the architecture for the dynamic A/B test that LotteON implemented.

Dynamic A/B Test exercises can also be found in AWS Workshop – Dynamic A/B Testing on Amazon Personalize & SageMaker.

Verisk FAST started building a RAG pipeline for their AI companion and have iteratively enhanced this solution.

Data governance – With a wide variety of users accessing the platform and with different users having access to different data, data governance and isolation was paramount.

Although PII isn’t typically necessary for interactions with the AI companion, Verisk employed Amazon Comprehend to detect any potential PII within queries.

Business ImpactVerisk initially rolled out the AI companion to one beta customer to demonstrate real-world performance and impact.

ConclusionVerisk’s journey in developing an AI companion for their FAST platform showcases the immense potential of generative…

Training and enablementTo help educate employees on AI/ML concepts, tools, and techniques, the AI/ML CoE can develop training programs, workshops, certification programs, and hackathons.

To provide ethical integrity, an AI/ML CoE helps integrate robust guidelines and safeguards across the AI/ML lifecycle in collaboration with stakeholders.

Lifecycle managementWithin the AI/ML CoE, the emphasis on scalability, availability, reliability, performance, and resilience is fundamental to the success and adaptability of AI/ML initiatives.

Ava Kong is a Generative AI Strategist at the AWS Generative AI Innovation Center, specializing in the financial services sector.

Rifat Jafreen is a Generative AI…

Whether looking for new adventures, epic storylines or games to play with a friend, GeForce NOW members are covered.

Start off with the much-anticipated sequel to the Men of War franchise or cozy up with some adorable pals in Palworld, both part of five games GeForce NOW is bringing to the cloud this week.

No Guts, No GloryGet transported to the battlefields of World War II with historical accuracy and attention to detail in Men of War II, the newest entry in the real-time strategy series from Fulqrum Publishing.

With advanced enemy AI and diverse gameplay modes, Men of War II promises an immersive experience for both history enthusiasts and casual gamers.

Cloud PalsStep into a world teemin…

Senior executives from NVIDIA, Siemens and Teradyne Robotics gathered this week in Odense, Denmark, to mark the launch of Teradyne’s new headquarters and discuss the massive advances coming to the robotics industry.

The grand opening showcased the latest AI robotic applications and featured a panel discussion on the future of advanced robotics.

Speakers included Ujjwal Kumar, group president at Teradyne Robotics; Rainer Brehm, CEO of Siemens Factory Automation; and Deepu Talla, vice president of robotics and edge computing at NVIDIA.

The alliance between NVIDIA and Teradyne Robotics, which includes an AI-based intra-logistics solution alongside Siemens, showcases the strength of collaborati…

A collaboration between NVIDIA and academic researchers is prepping robots for surgery.

The physics-based framework was built using NVIDIA Isaac Sim, a robotics simulation platform for designing, training and testing AI-based robots.

A Stitch in AI Saves NineORBIT-Surgical is based on Isaac Orbit, a modular framework for robot learning built on Isaac Sim.

Orbit includes support for various libraries for reinforcement learning and imitation learning, where AI agents are trained to mimic ground-truth expert examples.

By developing a surgical simulator that takes advantage of GPU acceleration and parallelization, the team is able to boost robot learning speed by an order of magnitude compared …

Basecamp Research is on a mission to capture the vastness of life on Earth at an unprecedented scale.

Basecamp Research is a member of the NVIDIA Inception program for cutting-edge startups.

Time Stamps1:31: What is Basecamp Research?

5:15: What is the collected biodiversity data used for?

198Scientists at Matice Biosciences are using AI to study the regeneration of tissues in animals known as super-regenerators, such as salamanders and planarians.

Mozilla Firefox, the popular open-source browser, is the latest partner to incorporate NVIDIA RTX Video, a technology that uses AI to improve video quality on Windows PCs and workstations.

RTX Video Super Resolution upscales low-resolution video for cleaner, crisper imagery.

“Mozilla is integrating RTX Video into Firefox to improve video quality for our users with compatible RTX GPUs.”Firefox joins other Chromium-based browsers, including Google Chrome and Microsoft Edge, in supporting RTX Video.

RTX Video Super Resolution is also supported in popular video players like VLC.

Enabling RTX Video is easy:Update to the latest GeForce RTX Game Ready Driver, NVIDIA Studio or NVIDIA RTX Enterprise…

This guide explores how RAPIDS helps unlock GPU acceleration on Databricks to transform data processing and analytics with familiar APIs and plugins.

With this new feature, single-node users can now easily switch between RAPIDS cuDF (cuDF) and pandas for large data manipulation tasks.

Multi-node Databricks: Accelerating Rapids with SparkIn Databricks, a Spark cluster handles large-scale data processing with Apache Spark, distributing workloads across multiple nodes to achieve parallelism.

Dask on Databricks with RAPIDS quick start exampleTo get started, first configure an init script with the following contents to install Dask, Dask on Databricks, RAPIDS libraries, and all other dependencie…

At Google I/O’24, Laurence Moroney, head of AI Advocacy at Google, announced that RAPIDS cuDF is now integrated into Google Colab.

Developers can now instantly accelerate pandas code up to 50x on Google Colab GPU instances, and continue using pandas as data grows—without sacrificing performance.

RAPIDS cuDF brings the power of accelerated computing to pandas, so you can continue using pandas as data grows—without compromising performance.

Get startedReady to try RAPIDS cuDF on Google Colab?

To learn more about using RAPIDS cuDF on Google Colab, explore these example notebooks:For a complete overview of RAPIDS cuDF, check out the GTC session, Accelerating Pandas with Zero Code Change Using R…

Gemma + NIMUsing TensorRT-LLM, NVIDIA worked with Google to optimize three new models it introduced at the event: Gemma 2, PaliGemma and RecurrentGemma.

PaliGemma is an open vision language model (VLM) inspired by PaLI-3.

is an open vision language model (VLM) inspired by PaLI-3.

RecurrentGemma is an open language model based on Google’s novel Griffin architecture, which requires lesser memory and achieves faster inference on long sequences.

Beyond optimization, Gemma will be offered with NVIDIA NIM inference microservices, part of the NVIDIA AI Enterprise software platform, which simplifies the deployment of AI models at scale.

The integration of AI has become pivotal in shaping the future of driving experiences.

The platform, unveiled in December, showcases the future of in-car interaction, with an automotive- and mobility-specific assistant that provides an integrated in-cabin experience.

Cerence is striving to empower vehicles with the cognitive capabilities necessary to seamlessly assist drivers in navigating their daily routines.

“Generative computing is going to change your in-car experience,” said Iqbal.

With generative AI at its core, driving will evolve into a personalized, connected and, ultimately, safer experience for all.

Japan to invest more than $740 million in AI infrastructure partners with NVIDIA and local firms.

Following an announcement by Japan’s Ministry of Economy, Trade and Industry, NVIDIA will play a central role in developing the nation’s generative AI infrastructure as Japan seeks to capitalize on the technology’s economic potential and further develop its workforce.

With this move, Japan becomes the latest nation to embrace the concept of sovereign AI, aiming to fortify its local startups, enterprises and research efforts with advanced AI technologies.

Japan’s technology powerhouses are already moving fast to embrace AI.

Last week, SoftBank Corp. announced that it will invest ¥150 billion, ap…

BioHive-2 packs 504 NVIDIA H100 Tensor Core GPUs linked on an NVIDIA Quantum-2 InfiniBand network to deliver 2 exaflops of AI performance.

That work is vital; Recursion’s scientists run more than 2 million such experiments a week.

But going forward, they’ll use AI models on BioHive-2 to direct their platform to the most promising biology areas to run their experiments.

Over time, it also amassed a more than 50-petabyte database of biological, chemical and patient data, helping it build powerful AI models that are accelerating drug discovery.

Learn about NVIDIA’s AI platform for healthcare and life sciences and subscribe to NVIDIA healthcare news.

Latest accelerators and networking improve performance for advanced simulations, AI, quantum computing, data analytics and more.

Quantum computing.

Multiplying Scientific Computing Simulations With BlackwellScientific computing and physics-based simulation often rely on what’s known as double-precision formats, or FP64 (floating point), to solve problems.

Driving Extreme Performance for Scientific Computing with NVIDIA NetworkingThe NVIDIA Quantum-X800 InfiniBand networking platform offers the highest throughput for scientific computing infrastructure.

It includes NVIDIA Quantum Q3400 and Q3200 switches and the NVIDIA ConnectX-8 SuperNIC, together hitting twice the bandwidth of the prior ge…

Scientists and researchers are applying generative AI to HPC jobs in code generation, weather forecasting, genetics and materials science using NVIDIA technologies.

Generative AI is taking root at national and corporate labs, accelerating high-performance computing for business and science.

Microsoft Proposes Novel MaterialsMicrosoft Research showed how generative AI can accelerate work in materials science.

Companies such as Carbon3D are already finding opportunities, applying generative AI to materials science in commercial 3D printing operations.

It’s just the beginning of what researchers will be able to do for HPC and science with generative AI.

Data centers need an upgraded dashboard to guide their journey to greater energy efficiency, one that shows progress running real-world applications.

The formula for energy efficiency is simple: work done divided by energy used.

Many standards exist for data center efficiency.

It Takes a VillageThanks to metrics like PUE and rankings like the Green500, data centers and supercomputing centers have made enormous progress in energy efficiency.

Metrics of energy consumed doing useful work on today’s top applications can take supercomputing and data centers to a new level of energy efficiency.

To help personalize content, tailor and measure ads and provide a safer experience, we use cookies.

By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies.

Learn more, including about available controls: Cookie PolicyAccept

The next generation of Meta’s large-scale infrastructure is being built with AI in mind, including supporting new generative AI (GenAI) products and services, recommendation systems, and advanced AI research.

It’s an investment we expect will grow in the years ahead as the compute requirements to support AI models increase alongside the models’ sophistication.

Last year, we unveiled the Meta Training and Inference Accelerator (MTIA) v1, our first-generation AI inference accelerator that we designed in-house with Meta’s AI workloads in mind – specifically our deep learning recommendation models that are improving a variety of experiences across our products.

MTIA is a long-term venture to pr…

Bandwidth estimation (BWE) and congestion control play an important role in delivering high-quality real-time communication (RTC) across Meta’s family of apps.

Network characterizationAn ML model-based approach leverages time series data to improve the bandwidth estimation by using offline parameter tuning for characterized network types.

The first component is offline ML model learning using ML to categorize the network type (random packet loss versus bursty loss).

The non-time series data or dense data will pass through a dense layer (i.e., a fully connected layer).

Use case: Random packet loss classificationLet’s consider the use case of categorizing packet loss as either random or conge…

In this post, we present the design behind Logarithm, and show how it powers AI training debugging use cases.

AI training debugging with LogarithmBefore looking at Logarithm’s internals, we present support for training systems and model issue debugging, one of the prominent use cases of Logarithm at Meta.

ML model training workflows tend to have a wide range of failure modes, spanning data inputs, model code and hyperparameters, and systems components (e.g., PyTorch, data readers, checkpointing, framework code, and hardware).

Logarithm ingests both systems logs from the training stack, and model telemetry from training jobs that the stack executes.

Filter–by-callsite enables hiding known lo…

While we’ve had a long history of building AI infrastructure, we first shared details on our AI Research SuperCluster (RSC), featuring 16,000 NVIDIA A100 GPUs, in 2022.

Under the hoodOur newer AI clusters build upon the successes and lessons learned from RSC.

Our out-of-box performance for large clusters was initially poor and inconsistent, compared to optimized small cluster performance.

Commitment to open AI innovationMeta maintains its commitment to open innovation in AI software and hardware.

The future of Meta’s AI infrastructureThese two AI training cluster designs are a part of our larger roadmap for the future of AI.

These improvements helped us find and remove many unnecessary dependencies, making build graph analysis and overall build times much better.

In response to this challenge, we implemented a new solution for the packaging and distribution of Python executables – the Content Addressable Filesystem (CAF).

LazyCAF and enforcing uniform revisions: Areas for further ML iteration improvementsThe improvements we implemented have proven highly effective, drastically reducing the overhead and significantly elevating the efficiency of our ML engineers.

Faster build times and more efficient packaging and distribution of executables have reduced overhead by double-digit percentages.

We plan to enable all…

At Meta, the quest for faster model training has yielded an exciting milestone: the adoption of Lazy Imports and the Python Cinder runtime.

The challenges of adopting Lazy ImportsWhile Lazy Imports’ approach significantly improved ML development, it was not all a bed of roses.

With Lazy Imports, Meta’s ML developers are now equipped to work more efficiently, experiment more rapidly, and achieve results faster.

Here’s a glimpse into our future endeavors:Streamlining developer onboardingThe learning curve associated with Lazy Imports can be a challenge for newcomers.

Building a robust community that helps supporting paradigms and patterns that play well with Lazy Imports is one of our future …

What’s going on with generative AI (GenAI) at Meta?

In this episode of the Meta Tech Podcast, Meta engineer Pascal Hartig (@passy) speaks with Devi Parikh, an AI research director at Meta.

They cover a wide range of topics, including the history and future of GenAI and the most interesting research papers that have come out recently.

And, of course, they discuss some of Meta’s latest GenAI innovations, including:Audiobox, a foundational model for generating sound and soundscapes using natural language prompts.

And if you’re interested in AI career opportunities at Meta visit the Meta Careers page.

In this post, we will provide an overview of the end-to-end debugging workflows supported by HawkEye, components of the system, and the product surface for Meta product and monetization teams to debug AI model and feature issues.

HawkEye includes infrastructure for continuously collecting data on serving and training models, data generation, and analysis components for mining root causes.

However, significant differences indicate problems with either the training data or loss divergence (e.g., loss or gradient explosion) in the bad snapshot.

Such issues can happen for several hard-to-diagnose reasons, ranging from the complex data pipelines behind training data, to data corruptions.

HawkEye…

The 2023 edition of Networking at Scale focused on how Meta’s engineers and researchers have been designing and operating the network infrastructure over the last several years for Meta’s AI workloads, including our numerous ranking and recommendation workloads and the immense Generative AI models.

But the sheer scale and complexity of GenAi models means new challenges for Meta’s network infrastructure.

Meta’s Network Journey to Enable AIHany Morsy, Network EngineerSusana Contrera, Network EngineerOver the years, Meta’s AI infrastructure has transitioned from CPU-based to GPU-based training due to growing AI workloads.

Traffic Engineering for AI Training NetworksShuqiang Zhang, Software Eng…

Fueling the success of these products are world-class infrastructure teams, including Meta’s custom AI silicon team, led by Olivia Wu, a leader in the silicon industry for 30 years.

In 2018, I saw a social media post from Yann LeCun, our Chief AI Scientist, that Meta was looking for someone to help build AI silicon in-house.

I knew of just a few other companies designing their own custom AI silicon, but they were mainly focused only on silicon and not the software ecosystem and products.

What’s next for the AI silicon design team?

We’re continuing to gather feedback and input from our AI software teams to shape the features of our future AI silicon.

Meta presents Chakra execution traces , an open graph-based representation of AI/ML workload execution, laying the foundation for benchmarking and network performance optimization.

The limitations of traditional AI benchmarking methodologyTraditionally, benchmarking AI systems has largely relied on running full ML workloads.

How Meta leverages Chakra execution tracesAt Meta, we collect execution traces from our production servers every day.

Future plansEnhancing the benchmarking capability of Chakra execution tracesWhile the execution trace replayer enables replay of execution traces, it brings forth challenges.

Using AI to generate representative execution tracesChakra execution traces are…

We’re introducing Arcadia, Meta’s unified system that simulates the compute, memory, and network performance of AI training clusters.

Arcadia gives Meta’s researchers and engineers valuable insights into the performance of AI models and workloads in an AI cluster – enabling data-driven decision making in the design of AI clusters.

That’s where Arcadia, Meta’s end-to-end AI system performance simulator, comes in.

By providing insights into the impact of these factors on system performance, Arcadia facilitates data-driven decision-making processes and fosters the evolution of models and hardware.

This comprehensive set of metrics empowers stakeholders to analyze the impact of different factor…

Today, we are releasing Code Llama, a large language model (LLM) that can use text prompts to generate code.

Additionally, we have further fine-tuned two additional variations of Code Llama: Code Llama - Python and Code Llama - Instruct.

Code Llama - Python is a language-specialized variation of Code Llama, further fine-tuned on 100B tokens of Python code.

Code Llama - Instruct is an instruction fine-tuned and aligned variation of Code Llama.

We recommend using Code Llama - Instruct variants whenever using Code Llama for code generation since Code Llama - Instruct has been fine-tuned to generate helpful and safe answers in natural language.

[...]

Read More...

The post Meta Connect 2023: September 27 – 28 appeared first on Engineering at Meta.

Using Optuna and a hands-on example, you will learn about the ideas behind Bayesian hyperparameter optimization, how it works, and how to perform Bayesian optimization for any of your machine-learning models.

Advantages of Bayesian optimization over other hyperparameter optimization methodsWe’ve seen that Bayesian optimization is superior to simpler hyperparameter optimization approaches because it takes into account past information.

Global optimization: Bayesian optimization is well-suited for global optimization tasks where the goal is to find the global optimum rather than just a local one.

Optimizing hyperparameter search using Bayesian optimization and OptunaOptuna is an open-source h…

We can further control the output of LLMs through parameters such as “temperature” or a “frequency penalty,” which influence an LLM’s output on a token-by-token basis.

How the Greedy Decoding, Beam Search, and Sampling post-processing techniques determine the next token to output.

How advanced techniques like frequency penalties, logit bias, and structured output give you even more control over an LLM’s output.

Before we dive into post-processing techniques for customizing LLM outputs, it’s crucial to understand how an LLM generates its output in the first place.

Adjusting the temperature parameter modifies the softmax function, influencing the diversity and predictability of a large langua…

In this article, we’ll survey the most commonly used deep learning optimization algorithms, including Gradient Descent, Stochastic Gradient Descent, and the Adam optimizer.

Understanding different optimization algorithms and their strengths and weaknesses is crucial for any data scientist training deep learning models.

Optimization in deep learning Have a look at other articles on our blog exploring aspects of optimization in deep learning: Deep Learning Model Optimization Methods: Deep learning models exhibit excellent performance but require high computational resources.

:Mini-batch Gradient DescentMini-batch Gradient Descent strikes a balance between the thorough, calculated approach of …

On top of that, neptune.ai integrates with any MLOps stack, and it just works.

Now, with less boilerplate code, you can log and visualize information from your ZenML pipeline steps (e.g., models, parameters, metrics).

You’re looking for a more visually interactive way of navigating the results produced from your ZenML pipeline runs (e.g., models, metrics, datasets).

In this example, we log a simple ZenML pipeline to Neptune using the Experiment Tracker stack component.

The example assumes that you have ZenML installed together with the Neptune integration.

Scatter plotsIf you have two metrics or parameters that you wish to compare or see how they relate to each other throughout the runs, you can now create a scatter plot.

See an example of a scatter plot in the Neptune app.

Distraction-free modeYou can now view run dashboards and compare views in distraction-free mode.

(neptune 1.5.0)We added a new environment variable NEPTUNE_DATA_DIRECTORY, which lets you specify where the .neptune folder should be created instead of the current working directory.

(neptune 1.6.0)Web applicationWe fixed an issue where a field named “type” could not be displayed as a runs table column.

We are excited to announce that Neptune and Modelbit have partnered to release an integration to enable better ML model deployment and experiment tracking.

Data scientists and machine learning engineers can use the integration to train and deploy machine learning models in Modelbit while logging and visualizing training progress in Neptune.

We’ll log the model’s hyperparameters and accuracy to Neptune and then deploy the model to a REST endpoint.

First, import “modelbit” and “neptune” and authenticate your notebook with Modelbit:import modelbit, neptune mb = modelbit.login()If your “NEPTUNE_API_TOKEN” isn’t already in your notebook’s environment, add it:import os os.environ["NEPTUNE_API_TOK…

(neptune 1.9.0): When fetching runs with , you have 4 new parameters: , , , and .

(neptune 1.9.0) All messages that Neptune prints to the console are prefixed with [neptune].

(neptune 1.9.0)that Neptune prints to the console are prefixed with [neptune].

(neptune 1.9.0) We introduced querying functionality to table fetching methods.

(neptune 1.10.0)IntegrationsWe introduced the log_model_summary parameter to NeptuneCallback() ( neptune- tensorflow-keras 2.2.1)parameter to ( neptune- 2.2.1) We added support for logging project requirements with the dependencies parameter.

This page lists all functions, parameters, and constants exposed by the Neptune Python API.

(kedro-neptune 0.3.0)Web applicationWhen modifying a search query in the runs table, you can now edit the operator and value without changing the other parts.

You can now click on a username in the Owner column (in the runs table) to instantly create a filter for runs created by that account.

column (in the runs table) to instantly create a filter for runs created by that account.

(neptune 1.8.6)Web applicationWe fixed an issue where a field named “type” could not be displayed as a runs table column.

Strategies for improving GPU usage include mixed-precision training, optimizing data transfer and processing, and appropriately dividing workloads between CPU and GPU.

The GPU memory usage metric reflects the amount of memory allocated to the GPU relative to its total memory capacity.

Optimizing data preprocessingIn addition to I/O and data transfer to the GPU, data preprocessing can become a bottleneck for GPU utilization.

This indicates that by improving GPU utilization, training could be sped up, and GPU resources could be used more efficiently.

However, across the many projects I’ve worked on, the following guidelines for optimizing GPU usage have often proven helpful:Always monitor GPU…

The role of zero-shot and few-shot learning in LLMsThe goal of zero-shot and few-shot learning is to get a machine-learning model to perform a new task it was not trained for.

“Few-shot learning” and “zero-shot learning” are well-known concepts in machine learning that were studied long before LLMs appeared on the scene.

In the context of LLMs, these terms are sometimes used interchangeably with “few-shot prompting” and “zero-shot prompting.” However, they are not the same.

Where zero-shot prompting failsLet’s turn to two use cases where zero-shot prompting is insufficient.

Where few-shot prompting failsFinally, let’s look at a situation where few-shot prompting won’t get us far.

Large Language Models (LLMs) like Meta AI’s LLaMA models, MISTRAL AI’s open models, and OpenAI’s GPT series have improved language-based AI.

Monitoring Monitor model performance for data drift and model degradation, often using automated monitoring tools.

Automate prompt selection: Implement systems that automatically choose the best prompt for a given task using historical data on prompt performance and the specifics of the current request.

Monitoring and observability are about tracking LLMs’ performance, health, and operational metrics in production to ensure they perform optimally and reliably.

Use platforms that offer comprehensive observability into LLM performance, including function…

The canonical MLflow setup for teams: The MLflow client embedded in the training code communicates with the MLflow tracking server, which handles access to cloud storage (artifact store) and a database (metadata store).

| Modified based on: sourceDeploying the MLflow tracking serverMLflow’s tracking server is relatively lightweight.

With this in mind, there are three main options for deploying the MLflow tracking server on AWS:Deploying the MLflow tracking server on an Amazon EC2 instance.

Recommended The Best MLflow alternatives Read alsoSetting up an artifact storeThe artifact store is the third relevant cost item in an MLflow deployment.

Further, everyone who has access to your MLflow tr…

Knowledge distillation transfers insights from a complex “teacher” model to a simpler “student” model, maintaining performance with less computational demand.

Distillation loss: The student model is trained not just to replicate the output of the teacher model but to match the output distributions produced by the teacher model.

Model architecture compatibility: The effectiveness of knowledge distillation depends on how well the student model can learn from the teacher model, which is greatly influenced by their architectural compatibility.

Data is fed to both a complex ‘Teacher Model’ and a simpler ‘Student Model.’ The Teacher Model, which consists of multiple layers (from Layer 1 to Layer …

Continual learning scenariosDepending on the data stream characteristics, problems within the continual learning scenario can be divided into three, each with a common solution.

Task incremental continual learningTask Incremental (TI) continual learning is classic multi-task learning but in an incremental way.

Continual learning methodsOver the second decade of the 2000s, there has been a rapid improvement in recent advances in continual learning methods.

Recommended How to Build Machine Learning Systems with a Feature Store See alsoHow to choose the right continual learning method for your projectAcross the three groups of continual learning approaches, many techniques exist.

Continual lea…

Paper: https://arxiv.org/abs/2403.07691 Abstract:

While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence. In this paper, we study the crucial role of SFT within the context of preference alignment, emphasizing that a minor penalty for the disfavored generation style is sufficient for preference-aligned SFT. Building on this foundation, we introduce a straightforward and innovative reference model-free monolithic odds ratio preference optimization algorithm, ORPO, eliminating the necessity for an additional preference alignment phase. We demonstrate, both empiri…

OUTLINE:

0:00 - Intro

0:19 - Our next-generation Meta Training and Inference Accelerator

01:39 - ALOHA Unleashed

03:10 - Apple Inks $50M Deal with Shutterstock for AI Training Data

04:28 - OpenAI Researchers, Including Ally of Sutskever, Fired for Alleged Leaking

05:01 - Adobe's Ethical Firefly AI was Trained on Midjourney Images

05:52 - Trudeau announces $2.4billion for AI-related investments

06:48 - RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

07:15 - CodeGemma - an official Google release for code LLMs

07:24 - Mistral AI: Cheaper, Better, Faster, Stronger

08:08 - Vezora/Mistral-22B-v0.1

09:00 - WizardLM-2, next generation state-of-the-art-LLM

09:31 - Idefic…

Paper: https://arxiv.org/abs/2404.09173 Abstract:

While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs. We propose Feedback Attention Memory (FAM), a novel Transformer architecture that leverages a feedback loop to enable the network to attend to its own latent representations. This design fosters the emergence of working memory within the Transformer, allowing it to process indefinitely long sequences. TransformerFAM requires no additional weights, enabling seamless integration with pre-trained models. Our experiments show that TransformerFAM significantly improves Transformer performance on long-…

OUTLINE:

0:00 - Intro

0:21 - Debunking Devin: "First AI Software Engineer" Upwork lie exposed!

07:24 - NeurIPS 2024 will have a track for papers from high schoolers.

13:29 - Opus can operate as a Turing machine.

13:47 - An AI-Powered, Self-Running Propaganda Machine for $105

14:27 - TechScape: How cheap, outsourced labour in Africa is shaping AI English

16:25 - Is ChatGPT Transforming Academics' Writing Style? References:

https://news.ycombinator.com/item?id=40008109&s=09

https://www.youtube.com/watch?v=tNmgmwEtoWE

https://www.youtube.com/watch?v=xE2fxcETP5E

https://twitter.com/itsandrewgao/status/1779369373737668669?t=omW3DvRNmZyce8oo0Ehf1g&s=09

https://twitter.com/0interestrates/status/17…

Google researchers achieve supposedly infinite context attention via compressive memory. Paper: https://arxiv.org/abs/2404.07143 Abstract:

This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M …

Meta's Llama 3 is out. New model, new license, new opportunities. References:

https://llama.meta.com/llama3/

https://ai.meta.com/blog/meta-llama-3/

https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

https://llama.meta.com/trust-and-safety/

https://ai.meta.com/research/publications/cyberseceval-2-a-wide-ranging-cybersecurity-evaluation-suite-for-large-language-models/

https://github.com/meta-llama/llama-recipes/tree/main/recipes/responsible_ai

https://llama.meta.com/llama3/license/

https://about.fb.com/news/2024/04/meta-ai-assistant-built-with-llama-3/?utm_source=twitter&utm_medium=organic_social&utm_content=thread&utm_campaign=imagineflash

https://twitter.com/minchoi/status/178277…

Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2

Litecoin (LTC): LQW2TRyKYetVC8WjFkhpP…

Some updates from industry in the Machine Learning world Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507…

A flurry of new models continues to appear. Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC…

Flow matching is a more general method than diffusion and serves as the basis for models like Stable Diffusion 3. Paper: https://arxiv.org/abs/2210.02747 Abstract:

We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, …

Paper: https://arxiv.org/abs/2402.14083 Abstract:

While Transformers have enabled tremendous progress in various application settings, such architectures still lag behind traditional symbolic planners for solving complex decision making tasks. In this work, we demonstrate how to train Transformers to solve complex planning tasks and present Searchformer, a Transformer model that optimally solves previously unseen Sokoban puzzles 93.7% of the time, while using up to 26.8% fewer search steps than standard A∗ search. Searchformer is an encoder-decoder Transformer model trained to predict the search dynamics of A∗. This model is then fine-tuned via expert iterations to perform fewer search step…

OUTLINE:

0:00 - Intro

0:15 - XAI releases Grok-1

2:00 - Nvidia GTC

4:45 - Comment of the Week

5:35 - Brute-forcing OpenAI model names

7:30 - Inflection AI gets eaten by Microsoft

9:25 - EU AI Act moving forward

11:45 - Advances in Robotics

14:00 - India retracts controversial advisory

14:30 - OpenSora

15:20 - Improved Gemma fine-tuning

16:20 - Decoding encrypted LLM traffic

17:45 - Varia References:

https://x.ai/blog/grok-os

https://github.com/xai-org/grok-1

https://finance.yahoo.com/news/nvidia-debuts-next-generation-blackwell-ai-chip-at-gtc-2024-205825161.html?guccounter=1&guce_referrer=aHR0cHM6Ly9uZXdzLmdvb2dsZS5jb20v&guce_referrer_sig=AQAAAHYRVePPrDnH3HxPV8smDzUiia_ztWttteAmHKxy-x_Z75lq…

Your weekly dose of ML News OUTLINE:

0:00 - Intro

0:15 - Devin: AI software engineer

5:50 - Mira Murati on Sora training data

6:50 - Inflection accused of copying Claude

9:00 - Tools & papers

16:30 - GPT-4.5-turbo mystery

17:30 - US government report: total extinction by AI

19:20 - Various other news References:

https://www.cognition-labs.com/introducing-devin

https://twitter.com/cognition_labs/status/1767548763134964000?t=ZECIn-uqbguwHtY8X_Gvtw&s=09

https://news.google.com/stories/CAAqNggKIjBDQklTSGpvSmMzUnZjbmt0TXpZd1NoRUtEd2lWMUwyU0N4RnVWM3pSRWhWX01pZ0FQAQ?hl=en-US&gl=US&ceid=US%3Aen

https://www.bloomberg.com/news/articles/2024-03-12/cognition-ai-is-a-peter-thiel-backed-coding-assistant?…

#mlnews #ainews #openai OUTLINE:

0:00 - Intro

0:20 - Elon sues OpenAI

14:00 - Mistral Large

16:40 - ML Espionage

18:30 - More Gemini Drama

24:00 - Copilot generates spicy images

26:55 - Gemma bugs

28:45 - Varia References: https://gist.github.com/yk/0c065cdc8e414738abfaae4f8e417e00 Thumbnail pictures: Wikipedia Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and …

Hey everyone! Thank you so much for watching this overview of Llama 3 looking at the release notes and seeing a demo of how to integrate it with DSPy through Ollama and how to use DSPy's MIPRO to find the optimal prompt when using this new large language model for RAG! We are hosting an event in San Francisco on May 1st with Arize AI and Cohere, featuring a talk from Omar Khattab, the lead author of DSPy! Hope to see you there! https://lu.ma/dspy Introducing Meta Llama 3: https://ai.meta.com/blog/meta-llama-3/ Ollama Llama 3: https://ollama.com/library/llama3 Weaviate Recipes: https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Llama3.ipynb Chapters

0:00 Llama3!!

1:28 Relea…

Hey everyone! Thank you so much for watching this overview of Command R+ showing you how you can use the new model in DSPy and a quick RAG demo, as well as walking through the details of the release post! Congratulations to the Cohere team! Super exciting times to be working with LLM systems! Introducing Command R+: A Scalable LLM Built for Business - https://txt.cohere.com/command-r-plus-microsoft-azure/ Link to demo notebook - https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Command-R-Plus.ipynb Chapters

0:00 Welcome! Command R+!

1:12 Demo with Cohere, DSPy, and Weaviate

6:06 Command R+ Announcement Post

9:24 LLM Evals

Unfortunately, Large Language Models will not consistently follow the instructions that you give them. This is a massive problem when you are building AI systems that require a particular type of output from the previous step to feed into the next one! For example, imagine you are building a blog post writing system that first takes a question and retrieved context to output a list of topics. These topics have to be formatted in a particular way, such as a comma-separated list or a JSON of Topic objects, such that the system can continue writing the blog post! I am SUPER excited to share the 4th video in my DSPy series, diving into 3 solutions to structuring outputs in DSPy programs: (1) **…

Hey everyone! Thank you so much for watching the 3rd edition of the DSPy series, Adding Depth to DSPy Programs!! You can find the examples and links to community resources / news on https://github.com/weaviate/recipes! Chapters

0:00 Intro

0:50 Chapters Overview

5:06 Weaviate Recipes

5:24 DSPy News and Community Notes

13:51 Adding Depth to RAG Programs

18:40 Multi-Model DSPy Programs

20:18 DSPy Optimizers

25:30 Deep Dive Optimizers

27:55 Into the Optimizer Code!

37:48 Demo #1: Adding Depth to RAG

1:05:25 Demo #2: Questions to Blogs

1:07:48 Thank you so much for watching!

Hey everyone! Thank you so much for watching this tutorial on getting started with RAG programming in DSPy! This video will take you through 4 major aspects of building DSPy programs (1) Installation, settings, and Datasets with dspy.Example, (2) LLM Metrics, (3) The DSPy programming model, and (4) Optimization!! The notebook used in the video can be found here: https://github.com/weaviate/recipes/blob/main/integrations/dspy/1.Getting-Started-with-RAG-in-DSPy.ipynb All future videos, as well as additional utils like data import scripts, will be in this folder: https://github.com/weaviate/recipes/tree/main/integrations/dspy Please leave a star, it helps a lot! DSPy on GitHub: https://github.…

Hey everyone! Thank you so much for watching this explanation of DSPy! DSPy is a super exciting new framework for developing LLM programs! Pioneered by frameworks such as LangChain and LlamaIndex, we can build much more powerful systems by chaining together LLM calls! This means that the output of one call to an LLM is the input to the next, and so on. We can think of chains as programs, with each LLM call analogous to a function that takes text as input and produces text as output. DSPy offers a new programming model, inspired by PyTorch, that gives you a massive amount of control over these LLM programs. Further the Signature abstraction wraps prompts and structured input / outputs to cle…

This comes from a full video breaking down how LLMs work. The link is on the bottom of the screen (in the shorts feed at least), or here for reference: https://youtu.be/wjZofJX0v4M

This comes from a full video dissecting how LLMs work. In the shorts player, you can click the link at the bottom of the screen, or for reference: https://youtu.be/wjZofJX0v4M

Demystifying attention, the key mechanism inside transformers and LLMs.

Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support

Special thanks to these supporters: https://www.3blue1brown.com/lessons/attention#thanks

An equally valuable form of support is to simply share the videos. Demystifying self-attention, multiple heads, and cross-attention.

Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support The first pass for the translated subtitles here is machine-generated, and therefore notably imperfect. To contribute edits or fixes, visit https://translate.3blue1brown.com/ ------------------ Here are …

An introduction to transformers and their prerequisites

Early view of the next chapter for patrons: https://3b1b.co/early-attention Other recommended resources on the topic. Richard Turner's introduction is one of the best starting places:

https://arxiv.org/pdf/2304.10557.pdf Coding a GPT with Andrej Karpathy

https://youtu.be/kCc8FmEb1nY Introduction to self-attention by John Hewitt

https://web.stanford.edu/class/cs224n/readings/cs224n-self-attention-transformers-2023_draft.pdf History of language models by Brit Cruise:

https://youtu.be/OFS90-FX6pg ------------------ Timestamps 0:00 - Predict, sample, repeat

3:03 - Inside a transformer

6:36 - Chapter layout

7:20 - The premise of Deep Learni…

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/aXRTczANuIs

A link to the full video answering this is at the bottom of the screen. Or, for reference: https://youtu.be/LqbZpur38nw

A link to the full video answering this is at the bottom of the screen. Or, for reference: https://youtu.be/bOXCLR3Wric

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/pQa_tWZmlGs The full video this comes from proves why slicing a cone gives the same shape as the two-thumbtacks-and-string construction, which is beautiful. Editing from long-form to short by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/HZGCoVF3YvM Editing from long-form to short by Dawid Kołodziej

A link to the full video about Bayesian thinking is at the bottom of the screen.

Or, for reference: https://youtu.be/lG4VkPoG3ko Long-to-short editing by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/OkmNXy7er84 Editing from the original video into this short by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/Cz4Q4QOuoo8 That video answers various viewer questions about the index of refraction. Editing from long-form to short by Dawid Kołodziej

A link to the full video is at the bottom of the screen. Or, for reference: https://youtu.be/ltLUadnCyi0

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/KTzGBJPuJwM That video unpacks the mechanism behind how light slows down in passing through a medium, and why the slow-down rate would depend on color. Editing from long-form to short by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab Editing from long-form to short by Dawid Kołodziej

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai Official link: https://openai.com/index/spring-update/

Try it out: https://chatgpt.com/

Singing: https://openai.com/index/hello-gpt-4o/

(look for the "Two GPT-4os interacting and singing.") 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le,…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "DrEureka: Language Model Guided Sim-To-Real Transfer" is available here:

https://eureka-research.github.io/dr-eureka/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Micha…

📝 Check out AlphaFold 3 here:

https://dpmd.ai/yt-tmp-alphafold3 📝 Or try it out through AlphaFold server for free:

https://alphafoldserver.com/ This video was made in partnership with Google DeepMind. My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard S…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 More on Unreal Engine 5.4 is available here:

https://www.unrealengine.com/en-US/blog/unreal-engine-5-4-is-now-available 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers Meta's Llama3 is available here (in select countries, Europe likely later):

https://llama.meta.com/llama3/

https://ai.meta.com/blog/meta-llama-3/ Try it out (everyone):

https://huggingface.co/chat/ Try Gemini 1.5 Pro (in select countries, Europe likely later):

https://aistudio.google.com/ Note - this is not 1.5 Pro yet as of the making of this video: https://gemini.google.com/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We wo…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "Learning agile soccer skills for a bipedal robot with deep reinforcement learning" is available here:

https://sites.google.com/view/op3-soccer

https://www.science.org/doi/10.1126/scirobotics.adi8022 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, Jo…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers ChatGPT:

https://chat.openai.com/ Chatbot arena leaderboard:

https://chat.lmsys.org/?leaderboard Video studying Devin: https://www.youtube.com/watch?v=tNmgmwEtoWE Try this instead! https://github.com/princeton-nlp/SWE-agent 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gord…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis " is available here:

https://research.nvidia.com/labs/nxp/lp3d/ Fully Connected conference: https://wandb.ai/site/resources/events/fully-connected MKBHD, iJustine ,Brian Tong persona source: https://www.youtube.com/watch?v=dtp6b76pMak 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous …

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "Grandmaster-Level Chess Without Search" is available here:

https://arxiv.org/abs/2402.04494 +1:

https://www.anthropic.com/news/decomposing-language-models-into-understandable-components

https://transformer-circuits.pub/2023/monosemantic-features/index.html 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Ba…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "ViCMA: Visual Control of Multibody Animations" is available here:

https://research.nvidia.com/labs/prl/vicma/ 📝 The paper "Inverse-Foley Animation: Synchronizing rigid-body motions to sound" is available here:

https://www.cs.cornell.edu/projects/Sound/ifa/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Ba…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context" is available here:

https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf 📝 The paper "Gemma: Open Models Based on Gemini Research and Technology" is available here:

https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf Try Gemma:

https://huggingface.co/chat Sources:

https://twitter.com/skirano/status/1760468624706351383

https://twitter.com/mckaywrigley/status/1761113846520131816

https://simonwillison.net/2024/Feb/21/gemini-pro-video/

https://twitter.com/ha…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers Get Blender: https://www.blender.org/ Demo files: https://www.blender.org/download/demo-files/

Blend Swap: https://www.blendswap.com/ Andrew Price's donut tutorial:

https://www.youtube.com/watch?v=B0J27sf9N1Y&list=PLjEaoINr3zgEPv5y--4MKpciLaoQYZB1Z&index=2 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Be…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 My Master thesis on fluids, with source code: https://users.cg.tuwien.ac.at/zsolnai/gfx/fluid_control_msc_thesis/

📝 Paper/poster on fluid control, with source code: https://users.cg.tuwien.ac.at/zsolnai/gfx/real_time_fluid_control_eg/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Bri…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers The conference:

https://wandb.ai/site/resources/events/fully-connected First impressions from creatives: https://openai.com/blog/sora-first-impressions 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Brizzee, Gaston Ingaramo, Gordon Child, Jace O'Brien, John Le, Kyle Davis, Lukas Biewald…

Подписывайтесь - https://dzen.ru/spbai !

Участники: - Петр Ермаков, ML Brand Director, Яндекс

- Иван Барабанов, Разработчик, ВКонтакте, deep vk

- Валентин Хрульков, ведущий исследователь, Yandex Research Денис Димитров, Исполнительный директор по исследованию данных Sber AI, научный консультант AIRI Вместе со специалистами в области диффузионных картиночных моделей порассуждаем, куда развивается область. Поговорим про текущие положение дел и актуальные технические барьеры области.

Спикер: Валентин Хрульков, ведущий исследователь, Yandex Research Рассмотрим все этапы обучения foundational диффузионных моделей, начиная от подготовки датасета до регулярных замеров качества в процессе обучения. Обсудим scaling law эксперименты и их предварительные результаты. Так же обсудим различные аспекты применения этих моделей на практике: генерации рекламных баннеров, персонализация, сервис-социальная сеть Шедеврум.

Спикер: Андрей Кузнецов, Исполнительный директор по исследованию данных, Sber AI Рассмотрим теорию и доступную информацию о диффузионном процессе. Подробно покажу, чем отличается архитектура Kandinsky 2.1 от 2.2. Обсудим метрики для оценки качества генераций, поговорим про ключевые результаты релизов. Вместе посмотрим, в каких сценариях для бизнеса и практических приложениях можно применять нашу модель.

Добро пожаловать на вечерний митап для ML-инженеров от Яндекса. В этот раз поговорим про модные нынче «генеративки», а именно про диффузионные картиночные модели! программа:

00:00:00 – таймер обратного отсчёта перед эфиром

00:02:12 – открытие конференции

00:04:52 – Kandinsky 2.X

Андрей Кузнецов, исполнительный директор по исследованию данных, Sber AI.

00:46:55 – перерыв

01:03:40 – Практические аспекты обучения масштабных диффузионных моделей Валентин Хрульков, ведущий исследователь, Yandex Research. 01:49:34 – Дискуссия «Ближайшее будущее диффузионных моделей» Присоединяйтесь к нашем сообществу в телеграм, чтобы быть в курсе всех событий и мероприятий Яндекса https://t.me/yadatadojo. Вопрос…

Спикеры: Дмитрий Колодезев, Алексей Пустынников, Алексей Натекин Страница соревнования: https://ods.ai/tracks/data-fusion-2024-competitions

Дедлайн по соревнованию 5 апреля 2024 года, присоединяйтесь!

Спикеры: Алексей Натекин, Дмитрий Колодезев Страница соревнования: https://ods.ai/tracks/data-fusion-2024-competitions

Все презентации можно скачать на странице митапа https://ods.ai/tracks/data-fusion-2024-competitions/meetup Дедлайн по соревнованию 5 апреля 2024 года, присоединяйтесь!

С радостью приглашаем вас на уникальное событие, посвященное силе и вкладу женщин в мире данных - митап "Woman In Data Science"! Это не просто встреча, это праздник ума, таланта и вдохновения, организованный ODS SPB при поддержке компании Samokat.tech. Полная программа доступна на ODS:

https://ods.ai/events/wids-meetup-2024 Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Курс Open ML Course: Деревья и их ансамбли:

https://ods.ai/tracks/trees-autumn23

Сезон курсов: https://ods.ai/events/course_season_autumn_23 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс Open ML Course: Деревья и их ансамбли:

https://ods.ai/tracks/trees-autumn23

Сезон курсов: https://ods.ai/events/course_season_autumn_23 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс Deep Reinforcement Learning 2023: https://ods.ai/tracks/drlcourse23

Сезон курсов:https://ods.ai/events/course_season_autumn_23 В четвертой лекции:

- Рассматривается случай MDP с неизвестными функциями награды и перехода между состояниями

- Рассмотрели подход Monte-Carlo и Temporal-Difference для нахождения Q-функции в этом случае

- Обсудили epsilon-жадные политики

- Вывили алгоритмы Monte-Carlo, SARSA и Q-learning Автор курса: Антон Плаксин, исследователь в группе Yandex.Research и доцент Уральского федерального университета. Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам:…

Курс Deep Reinforcement Learning 2023: https://ods.ai/tracks/drlcourse23

Сезон курсов:https://ods.ai/events/course_season_autumn_23 В третьей лекции:

- Поговорили про принцип динамического программирования

- Рассмотрели понятия v- и q-функций, а также понятия оптимальной политики.

- Выписали уравнения Белламана и научились их решать методами Policy Iteration и Value Iteration. Автор курса: Антон Плаксин, исследователь в группе Yandex.Research и доцент Уральского федерального университета. Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат …

Курс Deep Reinforcement Learning 2023: https://ods.ai/tracks/drlcourse23

Сезон курсов:https://ods.ai/events/course_season_autumn_23 На втором практическом занятии: - Разбираемся с PyTorch

- Пишем линейную регрессию

- Решаем задачу регрессии с помощь нейронных сетей

- Реализуем Deep Cross-Entropy метод и решаем CartPole Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс Open ML Course: Линейные модели 2023: https://ods.ai/tracks/linear-models-autumn23

Сезон курсов: https://ods.ai/events/course_season_autumn_23 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

На третьем практическом занятии: - Разбираемся с со средой Frozen Lake

- Пишем Policy Iteration Автор курса: Антон Плаксин, исследователь в группе Yandex.Research и доцент Уральского федерального университета. Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс Open ML Course: Линейные модели 2023: https://ods.ai/tracks/linear-models-autumn23

Сезон курсов: https://ods.ai/events/course_season_autumn_23 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Страница курса: https://ods.ai/tracks/my_first_data_project

Все доп.материалы в блоке на странице курса: https://ods.ai/tracks/my_first_data_project/blocks/98c41cb4-aaff-4c5e-8ddf-252be36ed722 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс Open ML Course: Деревья и их ансамбли:

https://ods.ai/tracks/trees-autumn23

Сезон курсов: https://ods.ai/events/course_season_autumn_23 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс ML System Design: https://ods.ai/tracks/ml-system-design-23

Сезон курсов: https://ods.ai/events/course_season_autumn_23 Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Курс Deep Reinforcement Learning 2023: https://ods.ai/tracks/drlcourse23

Сезон курсов:https://ods.ai/events/course_season_autumn_23 Во второй лекции:

- рассмотрены понятия нейрона, функции активации, нейронных сетей;

- кратко изложен нейросетевой подход к решению задач регрессии и классификации;

- приведена Теорема Цибенко об аппроксимации нейронными сетями непрерывных функций;

- рассказана модификация алгоритма Кросс-Энтропии с использованием нейронных сетей для решения задач обучения с подкреплением с бесконечными пространствами состояний и действий. Автор курса: Антон Плаксин, исследователь в группе Yandex.Research и доцент Уральского федерального университета. Наши соц.сети:

Telegram: h…

Twitch: https://www.twitch.tv/primerjustin

Discord: https://discord.gg/NbruaNW

Store: https://store.dftba.com/collections/primer

Patreon: https://www.patreon.com/primerlearning

Twitter: @primerlearning 0:00 - Introduction

0:19 - Simulation rules

3:23 - First simulations

5:21 - Game theory analysis

8:45 - Alternate reward matrices

15:58 - Requirements for an evolutionarily stable strategy

16:69 - Discussion questions Made possible by support from these wonderful Patrons:

abledbody

Alba Caparros-Roissard

Andrew Lang

Anthony Eufemio

Brian Cloutier

Captain Chinchilla

Christy Serbus

Daniel Rolandsgard Kjellevold

Erik Broeders

Flavio Kindler

Gabriele Siino

Garrett

Guguke

James Manning

Jeff

Jeremy…

Sean Carroll is a theoretical physicist, author, and host of Mindscape podcast.

Please support this podcast by checking out our sponsors:– HiddenLayer: https://hiddenlayer.com/lex– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off– Notion: https://notion.com/lex– Shopify: https://shopify.com/lex to get $1 per month trial– NetSuite: http://netsuite.com/lex to get free product tourTranscript: https://lexfridman.com/sean-carroll-3-transcriptEPISODE LINKS:Sean’s Website: https://preposterousuniverse.comMindscape Podcast: https://www.preposterousuniverse.com/podcast/Sean’s YouTube: https://youtube.com/@seancarrollSean’s Patreon: https://www.patreon.com/seanmcarrollSean’s Twitte…

Neil Adams is a judo world champion, 2-time Olympic silver medalist, 5-time European champion, and often referred to as the Voice of Judo.

Please support this podcast by checking out our sponsors:– ZipRecruiter: https://ziprecruiter.com/lex– Eight Sleep: https://eightsleep.com/lex to get special savings– MasterClass: https://masterclass.com/lexpod to get 15% off– LMNT: https://drinkLMNT.com/lex to get free sample pack– NetSuite: http://netsuite.com/lex to get free product tourEPISODE LINKS:Neil’s Instagram: https://instagram.com/naefightingNeil’s YouTube: https://youtube.com/NAEffectiveFightingNeil’s TikTok: https://tiktok.com/@neiladamsmbeNeil’s Facebook: https://facebook.com/NeilAdamsJudo…

Edward Gibson is a psycholinguistics professor at MIT and heads the MIT Language Lab.

Please support this podcast by checking out our sponsors:– Yahoo Finance: https://yahoofinance.com– Listening: https://listening.com/lex and use code LEX to get one month free– Policygenius: https://policygenius.com/lex– Shopify: https://shopify.com/lex to get $1 per month trial– Eight Sleep: https://eightsleep.com/lex to get special savingsTranscript: https://lexfridman.com/edward-gibson-transcriptEPISODE LINKS:Edward’s X: https://x.com/LanguageMITTedLab: https://tedlab.mit.edu/Edward’s Google Scholar: https://scholar.google.com/citations?user=4FsWE64AAAAJTedLab’s YouTube: https://youtube.com/@Tedlab-MITP…

Andrew Callaghan is the host of Channel 5 on YouTube, where he does street interviews with fascinating humans at the edges of society, the so-called vagrants, vagabonds, runaways, outlaws, from QAnon adherents to Phish heads to O Block residents and much more.

Please support this podcast by checking out our sponsors:– ShipStation: https://shipstation.com/lex and use code LEX to get 60-day free trial– BetterHelp: https://betterhelp.com/lex to get 10% off– LMNT: https://drinkLMNT.com/lex to get free sample pack– MasterClass: https://masterclass.com/lexpod to get 15% off– AG1: https://drinkag1.com/lex to get 1 month supply of fish oilTranscript: https://lexfridman.com/andrew-callaghan-transcri…

Bassem Youssef is an Egyptian-American comedian & satirist, referred to as the Jon Stewart of the Arab World.

Please support this podcast by checking out our sponsors:– AG1: https://drinkag1.com/lex to get 1 month supply of fish oil– Shopify: https://shopify.com/lex to get $1 per month trial– Eight Sleep: https://eightsleep.com/lex to get special savings– LMNT: https://drinkLMNT.com/lex to get free sample packEPISODE LINKS:Bassem’s X: https://x.com/ByoussefBassem’s Instagram: https://instagram.com/bassemBassem’s Facebook: https://facebook.com/bassemyousseftvBassem’s Website: https://bassemyoussef.xyzPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co…

Tulsi Gabbard is a politician, veteran, and author of For Love of Country.

Please support this podcast by checking out our sponsors:– Riverside: https://creators.riverside.fm/LEX and use code LEX to get 30% off– ExpressVPN: https://expressvpn.com/lexpod to get 3 months free– NetSuite: http://netsuite.com/lex to get free product tour– Notion: https://notion.com/lexEPISODE LINKS:For Love of Country (book): https://amzn.to/3VLlofMTulsi’s X: https://x.com/tulsigabbardTulsi’s YouTube: https://youtube.com/@TulsiGabbardTulsi’s Podcast: https://youtube.com/@TheTulsiGabbardShowTulsi’s Instagram: https://instagram.com/tulsigabbardTulsi’s Facebook: https://facebook.com/TulsiGabbardTulsi’s Website: htt…

Mark Cuban is a businessman, investor, star of TV series Shark Tank, long-time principal owner of Dallas Mavericks, and founder of Cost Plus Drugs.

Please support this podcast by checking out our sponsors:– Listening: https://listening.com/lex and use code LEX to get one month free– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off– Notion: https://notion.com/lex– Eight Sleep: https://eightsleep.com/lex to get special savings– Shopify: https://shopify.com/lex to get $1 per month trialEPISODE LINKS:Mark’s X: https://twitter.com/mcubanMark’s Instagram: https://instagram.com/mcubanCost Plus Drugs: https://costplusdrugs.comShark Tank: https://abc.com/shows/shark-tankDallas Mav…

Dana White is the CEO and president of the UFC.

Please support this podcast by checking out our sponsors:– LMNT: https://drinkLMNT.com/lex to get free sample pack– Notion: https://notion.com/lex– AG1: https://drinkag1.com/lex to get 1 month supply of fish oil– InsideTracker: https://insidetracker.com/lex to get 20% offTranscript: https://lexfridman.com/dana-white-transcriptEPISODE LINKS:Dana’s X: https://x.com/danawhiteDana’s Instagram: https://instagram.com/danawhiteDana’s Facebook: https://facebook.com/danawhiteUFC’s YouTube: https://youtube.com/@UFCUFC’s Website: https://ufc.com/PODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co/2lwqZIrSpotify: h…

Annie Jacobsen is an investigative journalist and author of “Nuclear War: A Scenario” and many other books on war, weapons, government secrecy, and national security.

Please support this podcast by checking out our sponsors:– HiddenLayer: https://hiddenlayer.com/lex– BetterHelp: https://betterhelp.com/lex to get 10% off– Policygenius: https://policygenius.com/lex– NetSuite: http://netsuite.com/lex to get free product tourEPISODE LINKS:Nuclear War: A Scenario (book): https://amzn.to/3THZHfrAnnie’s Twitter: https://twitter.com/anniejacobsenAnnie’s Website: https://anniejacobsen.com/Annie’s Books: https://amzn.to/3TGWyMJAnnie’s Books (audio): https://adbl.co/49ZnI7cPODCAST INFO:Podcast website…

Sam Altman is the CEO of OpenAI, the company behind GPT-4, ChatGPT, Sora, and many other state-of-the-art AI technologies.

Please support this podcast by checking out our sponsors:– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off– Shopify: https://shopify.com/lex to get $1 per month trial– BetterHelp: https://betterhelp.com/lex to get 10% off– ExpressVPN: https://expressvpn.com/lexpod to get 3 months freeTranscript: https://lexfridman.com/sam-altman-2-transcriptEPISODE LINKS:Sam’s X: https://x.com/samaSam’s Blog: https://blog.samaltman.com/OpenAI’s X: https://x.com/OpenAIOpenAI’s Website: https://openai.comChatGPT Website: https://chat.openai.com/Sora Website: https://op…

Norman Finkelstein and Benny Morris are historians.

Mouin Rabbani is a Middle East analyst.

Steven Bonnell (aka Destiny) is a political livestreamer.

On some podcast players you should be able to click the timestamp to jump to that time.

(00:00) – Introduction(12:11) – 1948(1:10:43) – Partition(2:15:16) – October 7(3:09:27) – Gaza(3:36:02) – Peace(4:40:47) – Hope for the future

Kimbal Musk is a chef, entrepreneur, and author of The Kitchen Cookbook: Cooking for Your Community.

Please support this podcast by checking out our sponsors:– Eight Sleep: https://eightsleep.com/lex to get special savings– ExpressVPN: https://expressvpn.com/lexpod to get 3 months free– NetSuite: http://netsuite.com/lex to get free product tour– BetterHelp: https://betterhelp.com/lex to get 10% offTranscript: https://lexfridman.com/kimbal-musk-transcriptEPISODE LINKS:Kimbal’s X: https://x.com/kimbalKimbal’s Instagram: https://instagram.com/kimbalmusk/Kimbal’s Facebook: https://facebook.com/kimbalmuskofficial/The Kitchen Cookbook: https://amzn.to/4ccaCoEThe Kitchen (restaurants): https://www…

Yann LeCun is the Chief AI Scientist at Meta, professor at NYU, Turing Award winner, and one of the most influential researchers in the history of AI.

Please support this podcast by checking out our sponsors:– HiddenLayer: https://hiddenlayer.com/lex– LMNT: https://drinkLMNT.com/lex to get free sample pack– Shopify: https://shopify.com/lex to get $1 per month trial– AG1: https://drinkag1.com/lex to get 1 month supply of fish oilEPISODE LINKS:Yann’s Twitter: https://twitter.com/ylecunYann’s Facebook: https://facebook.com/yann.lecunMeta AI: https://ai.meta.com/PODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co/2lwqZIrSpotify: https://spoti.fi/2nEwCF8R…

Serhii Plokhy is a Ukrainian historian at Harvard University, director of the Ukrainian Research Institute, and an author of many books on history of Eastern Europe, including his latest book The Russo-Ukrainian War: The Return of History.

Please support this podcast by checking out our sponsors:– Eight Sleep: https://eightsleep.com/lex to get special savings– Shopify: https://shopify.com/lex to get $1 per month trial– NetSuite: http://netsuite.com/lex to get free product tour– AG1: https://drinkag1.com/lex to get 1 month supply of fish oilEPISODE LINKS:Serhii’s X: https://x.com/splokhySerhii’s Website: https://history.fas.harvard.edu/people/serhii-plokhiiHarvard Ukrainian Research Institut…

We are sorry, the page you requested cannot be found.

The page you are looking for could not be found or is no longer available.

Well, I’ve heard other people on your teams use words like surprise, sometimes even shock …HOSN: Yeah, yeah, there are a lot of “wow” factors.

HUIZINGA: Yeah, yeah.

AI research is moving at such speeds that I feel like we need to get accustomed to a timing of now.

HOSN: That’s right.

Well, as we close, Rafah, I want to ask a question anchored on the big idea behind AI Frontiers.

GRETCHEN HUIZINGA: Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on world-class research in brief.

CHAKRABORTY: So satellite connectivity is nothing new and has been there for long.

So we are talking about the satellites that are at least 10 to 20 times cheaper and smaller than state-of-the-art satellites.

So the device sends some packet to the satellite; satellite sends some packet to the device—it’s all about packet exchange.

So our vision is clear: to bring 24-7 connectivity for devices anywhere on Earth with just a click of power button.

Behind every emerging technology is a great idea propelling it forward. In the new Microsoft Research Podcast series, Ideas, members of the research community at Microsoft discuss the beliefs that animate their research, the experiences and thinkers that inform it, and the positive human impact it targets. In this episode, host Gretchen Huizinga talks with Principal Researcher Kalika Bali. Inspired by an early vision of “talking computers” and a subsequent career in linguistics, Bali has spent the last two decades bringing the two together. Aided by recent advances in large language models and motivated by her belief that everyone should have access to AI in their own language, Bali and her…

And so I just want to start here: for you, Ida, what is general intelligence?

Different people at different times provide different criteria for what would be the artificial general intelligence notion.

One is artificial general intelligence and the other is humanlike intelligence or human-level intelligence.

Artificial general intelligence and humanlike, human-level intelligence—how do these two concepts relate to you?

LLORENS: So it sounds like a very extensive set of experiments across many different tasks and with many different leading AI models, and you’ve uncovered a lack of robustness across some of these different tasks.

GRETCHEN HUIZINGA: Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on world-class research in brief.

These two examples are also the differences from other deep learning OFDFT works.

This is the generalization challenge and is one of the major challenges of deep learning method for molecular science applications.

This somehow shows the benefits of leveraging the OFDFT framework for using a deep learning method to solve molecular tasks.

You can also read it on arXiv, or you can check out the March 2024 issue of Nature Computational Science.

And so we realized that working with generative AI really parallels these different aspects of what a manager does, right.

So this requires having self-awareness of the applicability of generative AI to your workflows and maintaining an appropriate level of confidence in completing tasks manually or relying on generative AI.

For example, whether it’s worth it for you to actually learn how to work with generative AI more effectively.

But I think, given how generative AI has rolled out in the world today, I mean, a lot of the focus has been on productivity and workflows.

If you want to read the full paper on metacognition and generative AI, you can find a link at aka.ms/abstracts, or you can …

NICOLE FORSGREN: Yeah, it’s, you know, it’s, kind of, this ridiculous story.

I was there for, you know, two, three years, and I’m doing really, really well.

GEHRKE: This is just “I had a feeling.”FORSGREN: In my gut, I’m like, I’m doing really well.

After that, things were going really well, but we were also growing and scaling really, really rapidly.

Because I realized there were pieces about research that I really, really loved.

In the Microsoft Research Podcast series What’s Your Story, Johannes Gehrke explores the who behind the technical and scientific advancements helping to reshape the world.

A systems expert whose 10 years with Microsoft spans research and product, Gehrke talks to members of the company’s research community about what motivates their work and how they got where they are today.

Partner Software Architect Ivan Tashev’s expertise in audio signal processing has contributed to the design and study of audio components for Microsoft products such as Kinect, Teams, and HoloLens.

In this episode, Tashev discusses how a first-place finish in the Mathematical Olympiad fueled a lifelong passion for shoot…

And up until now, there’s been a lot of work on pruning model parameters for a variety of reasons.

But generally, these papers show that as parameters are removed from the model, performance just does not degrade.

You can, overall, keep performance roughly the same even with a fairly drastic reduction of model parameters.

HUIZINGA: So, Jordan, I often think of an abstract as a sort of appetizer for a research paper.

I think for one, as a practical matter, there’s this question of just what’s the best way to find the best LASER intervention?

LLORENS: Your new book lays out foundations in statistics and probability theory for modern machine learning.

LLORENS: Another concept that is key in machine learning is generalization.

So that’s, that’s … we show that in the book, in fact.

BISHOP: [LAUGHS] Well, that’s, that’s really interesting.

So I personally, actually, find this one of the most exciting frontiers not only of the natural sciences but also of machine learning.

Members of the research community at Microsoft work continuously to advance their respective fields.

Abstracts brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.

In this episode, Senior Principal Research Manager Tao Qin and Senior Researcher Lijun Wu discuss “FABind: Fast and Accurate Protein-Ligand Binding.” The paper, accepted at the 2023 Conference on Neural Information Processing Systems (NeurIPS), introduces a new method for predicting the binding structures of proteins and ligands during drug development.

The method demonstrates improved speed and accuracy over current methods.

Learn moreFABind: Fast and Ac…

Members of the research community at Microsoft work continuously to advance their respective fields.

Abstracts brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.

In this episode, Principal Researcher Alessandro Sordoni joins host Gretchen Huizinga to discuss “Joint Prompt Optimization of Stacked LLMs using Variational Inference.” In the paper, which was accepted at the 2023 Conference on Neural Information Processing Systems (NeurIPS), Sordoni and his coauthors introduce Deep Language Networks, or DLNs, an architecture that treats large language models as layers within a network and natural language prompts as eac…

Members of the research community at Microsoft work continuously to advance their respective fields.

Abstracts brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.

In this episode, Xing Xie, a Senior Principal Research Manager of Microsoft Research Asia, joins host Dr. Gretchen Huizinga to discuss “Evaluating General-Purpose AI with Psychometrics.” As AI capabilities move from task specific to more general purpose, the paper explores psychometrics, a subfield of psychology, as an alternative to traditional methods for evaluating model performance and for supporting consistent and reliable systems.

Read the paper: Ev…

Curious about the safety of LLMs?

🤔 Join us for an insightful new episode featuring Suchin Gururangan, Young Investigator at Allen Institute for Artificial Intelligence and Data Science Engineer at Appuri.

🚀 Don't miss…

This podcast episode features Dr. Mohamed Elhoseiny, a true luminary in the realm of computer vision with over a decade of groundbreaking research.

As an Assistant Professor at KAUST, Dr. Elhoseiny's work delves into the…

Our first guest with this new format is Kyle Lo, the most senior lead scientist in the Semantic Scholar team at Allen Institute for AI (AI2), who kindly agreed to share his perspective on #Science of #Science (#scisci) o…

Ben is interested in evolutionary biology and uses his experience as a software developer to build a software program called SLiM.

Ben discussed what SLiM does — running genetic simulations.

Ben also discussed the technical and biological background expected of a user to create models on SLiM.

He discussed the inputs (Eidos code) the model needs to create simulations.

Learn more about SLiMSLiM webpageSLiM Workshops

Behavioral GeneticsOur guest today is Jessica Hekman, the President of Functional Dog Collaborative and a teacher of behavioral biology at Virginia Tech.

She joins us to discuss her work on behavioral genetics, particularly in dogs.

Jessica gave background information about what the Functional Dog Collaborative does.

Jessica discussed how dog breeders can breed dogs with reduced risks of undesirable traits or diseases.

She also discussed how she and her coauthors got data to understand breed behaviors that are scientific or based on our perception.

She is interested in studying and understanding the neural mechanism of the honeybee waggle dance.

Barbara and Anna shared some breakthroughs in the field of animal communication.

Anna discussed how the honeybee uses the waggle dance to communicate.

Our guests explained how they captured the waggle dance of honeybees in their hives.

She also discussed how researchers use the neural socket to understand the workings of insects' brains.

Talmo PereiraDr. Talmo Pereira is a Principal Investigator at the Salk Institute for Biological Studies in San Diego, CA where he leads a research group as a Salk Fellow.

His lab (talmolab.org) focuses on the development of deep learning-based computational methods for recognition and modeling of complex biological systems, with applications ranging from neuroscience to cancer and plant biology.

His recent work has demonstrated how advances in deep learning and computer vision can enable quantitative phenotyping of complex behaviors through the development and application of approaches for markerless motion capture (sleap.ai).

This work has been published in Nature Methods and featured in T…

Modeling Group BehaviorOur guest in this episode is Sebastien Motsch, an assistant professor at Arizona State University, working in the School of Mathematical and Statistical Science.

Sebastien discussed two approaches to modeling the group behavior of animals, for instance, a flock of birds.

He discussed how Boltzmann's questions and kinetic theory help them understand birds' interactions with respect to velocity changes.

Papers discussedA new model for self-organized dynamics and its flocking behaviorHeterophilious dynamics enhances consensusResourceBoltzmann equation$$f(t + \Delta t, \mathbf{x} + \Delta \mathbf{x}, \mathbf{v} + \Delta \mathbf{v}) = f(t, \mathbf{x}, \mathbf{v}) + \left( …

Ryan discussed how the behavior of rattlesnakes is studied in the natural world, particularly with an increase in temperature.

He discussed how they collect data about the rattlesnake hunt using loggers and how they determine the loggers do not affect the animals' behavior.

Ryan discussed how he built a machine learning model to predict the behavior of the animals.

Ryan discussed what he discovered about the eating habits of rattlesnakes.

Rounding up, Ryan shared some future plans for the project.

What you Know About Intelligence is WrongWe are joined by Hank Schlinger, a professor of psychology at California State University, Los Angeles.

His research revolves around theoretical issues in psychology and behavioral analysis.

He also discussed how intelligence is measured in a given context.

Hank mentioned why the current measure of intelligence is fundamentally flawed.

Hank discussed how psychologists can perform behavioral experiments to understand consciousness.

What you Know About Intelligence is WrongWe are joined by Hank Schlinger, a professor of psychology at California State University, Los Angeles.

His research revolves around theoretical issues in psychology and behavioral analysis.

He also discussed how intelligence is measured in a given context.

Hank mentioned why the current measure of intelligence is fundamentally flawed.

Hank discussed how psychologists can perform behavioral experiments to understand consciousness.

Louis and the interim director at the Whitney R. Harris World Ecology Center.

Aimee discussed how animals perceive information and what they use it for.

She also discussed the costs required for learning and factors that affect animal learning.

She also discussed the different kinds of evolutionary experiments that can be performed.

Aimee discussed some models that researchers use during evolutionary experiments.

Octopus CognitionWe are joined by Tamar Gutnick, a visiting professor at the University of Naples Federico II, Napoli, Italy.

She studies the octopus nervous system and their behavior, focusing on cognition and learning behaviors.

Tamar gave a background to the kind of research she does — lab research.

She discussed the octopus nervous system and why they are unique compared to other animals.

She discussed how they measure the behavior of octopuses using a video recording and a logger to track brain activity.

Claire discussed how bumblebees make foraging decisions and how they communicate when foraging.

She discussed how they set up experiments in the lab to address questions about bumblebees foraging.

Claire discussed factors that drive an animal's foraging decisions.

She also touched on some irrational foraging behaviors she observed in her study.

She discussed the effect of climate change on foraging bees' learning behavior.

On today’s show, we are joined by our co-host, Becky Hansis-O’Neil. Becky is a Ph.D. student at the University of Missouri, St Louis, where she studies bumblebees and tarantulas to understand their learning and cognitive work. She joins us to discuss the paper: Perception in Chess. The paper aimed to understand how chess players perceive the positions of chess pieces on a chess board. She discussed the findings paper. She spoke about situations where grandmasters had better recall of chess positions than beginners and situations where they did not. Becky and Kyle discussed the use of chess engines for cheating. They also discussed how chess players use chunking. Becky discussed some approac…

OpenwormOn this episode, we are joined by Stephen Larson, the CEO of MetaCell and an affiliate of the OpenWorm foundation.

Stephen discussed what the Openworm project is about.

They hope to use a digital C. elegans nematode (C. elegans for short) to study the basics of life.

Stephen discussed why C. elegans is an ideal organism for studying life in the lab.

He also mentioned how students can get involved in the Openworm project.

What the Antlion KnowsOur guest is Becky Hansis-O’Neil, a Ph.D. student at the University of Missouri, St Louis.

Becky discussed how they designed an experiment using a T-maze to understand antlions' behavior.

Becky discussed some interesting findings from the experiment.

Becky gave her thoughts on the findings of the paper and the methodologies used.

Paper in focusOperant conditioning in antlion larvae and its impairment following exposure to elevated temperaturesFollow our guestXWebsiteThis season’s cover art features original photographs by Becky Hansis-O’Neil

Aligning LLMs: How can we teach pre-trained LLMs to hold a conversation and learn new information from each other?

This was where Sinan Ozdemir began his investigation into aligning LLMs.

In this episode, he talks to Jon…

Recent advances in GenAI, how to tackle the climate crisis with advanced technology, and addressing the knowledge gap in understanding AI: Jon Krohn speaks to Flypower co-founder and CEO Navdeep Martin about the advances…

Hear Jon Krohn’s favorite five clips from his April interviews.

Chief Scientist at Posit PBC Hadley Wickham on the subtle differences between Python and R. Professor of Business Analytics Barrett Thomas walks through the…

Sol Rashidi, a distinguished data executive who has served in C-suite roles at Fortune 100 companies, joins Jon Krohn to delve into successful enterprise AI strategies and the reasons behind the high turnover among Chief…

Want to become a data scientist?

Jon and Adam discuss the key steps to becoming a data scientist, with a focus on developing portfolio projects.

Hear about the 10 project ideas Adam recommends in his book to help you sta…

Tidyverse, ggplot2, and the secret to a tech company’s longevity: Hadley Wickham talks to Jon Krohn about Posit’s rebrand, Tidyverse and why it needs to be in every data scientist’s toolkit, and why getting your hands di…

Mixtral 8x22B is the focus on this week's Five-Minute Friday.

Jon Krohn examines how this model from French AI startup Mistral leverages its mixture-of-experts architecture to redefine efficiency and specialization in AI…

Generative AI is reshaping our world, and Bernard Marr, world-renowned futurist and best-selling author, joins Jon Krohn to guide us through this transformation.

In this episode, Bernard shares his insights on how AI is …

What are the risks of AI progressing beyond a point of no return?

What do we stand to gain?

On this Five-Minute Friday, Jon Krohn talks ‘books’ as he outlines two nonfiction works on AI and futurism by Oxford philosopher…

Tech entrepreneurship, artificial superintelligence, and the future of education: Aleksa Gordić speaks to Jon Krohn about his strategies for self-directed learning, the traits that help people succeed in moving from big …

Covariant's RFM-1: Jon Krohn explores the future of AI-driven robotics with RFM-1, a groundbreaking robot arm designed by Covariant and discussed by A.I.

roboticist Pieter Abbeel.

Explore how this innovation aims to merg…

Dr. Barrett Thomas, an award-winning Research Professor at the University of Iowa, explores the intricacies of Markov decision processes and their connection to Deep Reinforcement Learning.

Discover how these concepts ar…

Pytorch benefits, how to get funding for your AI startup, and managing scientific silos: In our new series for SuperDataScience, “In Case You Missed It”, host Jon Krohn engages in some “reinforcement learning through hum…

Kirill Eremenko joins Jon Krohn for another exclusive, in-depth teaser for a new course just released on the SuperDataScience platform, “Machine Learning Level 2”.

Kirill walks listeners through why decision trees and ra…

Explore the science of confidence with Lucy Antrobus, as she unveils neuroscience-backed strategies to build and boost confidence through practice, positive energy, and the power of laughter.

An essential listen for fost…

In this episode of “Rust in the Cosmos” we delve into the challenges of building embedded applications for space.

Did you know that once you ship your app to space… you can’t get it back?

Build robotics applications in minutes, not months.

Amethix works to create and maximize the impact of the world’s leading corporations and startups, so they can create a better future for everyone they serve.

CommunitiesAeroRust, Intrepid, BytenookAeroRust Discord invite: https://discord.com/invite/6jJyx5nEUqAeroRust website: AeroRust.orgIntrepid AI Discord https://discord.gg/cSSzche6Cthttps://discord.gg/cSSzche6Ct Intrepid AI website: https://intrepid.aiReferences

In this episode of “Rust in the Cosmos” we delve into the challenge of testing software for… ehm … spaceHow can Rust help?

Build robotics applications in minutes, not months.

Amethix works to create and maximize the impact of the world’s leading corporations and startups, so they can create a better future for everyone they serve.

We provide solutions in AI/ML, Fintech, Defense, Robotics and Predictive maintenance.

CommunitiesAeroRust, Intrepid, BytenookAeroRust Discord invite: https://discord.com/invite/6jJyx5nEUqAeroRust website: AeroRust.orgIntrepid AI Discord https://discord.gg/cSSzche6Cthttps://discord.gg/cSSzche6Ct Intrepid AI website: https://intrepid.aiReferences

In this inaugural episode of “Rust in the Cosmos,” we delve into the intricacies of communication in space and some of the challenges in space application development.

In this episode we delve into the dynamic realm of game development and the transformative role of artificial intelligence (AI).

Join Frag, Jim and Mike as they explore the current landscape of game development processes, from initial creative ideation to the integration of AI-driven solutions.

With Mike’s expertise as a software executive and avid game developer, we uncover the potential of AI to revolutionize game design, streamline development cycles, and enhance player experiences.

SponsorsIntrepid AI is an AI assisted all-in-one platform for robotics teams.

Build robotics applications in minutes, not months.

In this episode, join me and the Kaggle Grand Master, Konrad Banachewicz, for a hilarious journey into the zany world of data science trends.

From algorithm acrobatics to AI, creativity, Hollywood movies, and music, we just can’t get enough.

It’s the typical episode with a dose of nerdy comedy you didn’t know you needed.

Buckle up, it’s a data disco, and we’re breaking down the binary!

SponsorsIntrepid AI is an AI assisted all-in-one platform for robotics teams.

In this episode of Data Science at Home, we explore the game-changing impact of low-code solutions in robotics development.

Discover how these tools bridge the coding gap, simplify integration, and enable trial-and-error development.

We’ll also uncover challenges with traditional coding methods using ROS.

Join us for a concise yet insightful discussion on the future of robotics!

Join us in a dynamic conversation with Yori Lavi, Field CTO at SQream, as we unravel the data analytics landscape.

From debunking the data lakehouse hype to SQream’s GPU-based magic, discover how extreme data challenges are met with agility.

Yori shares success stories, insights into SQream’s petabyte-scale capabilities, and a roadmap to breaking down organizational bottlenecks in data science.

Dive into the future of data analytics with SQream’s commitment to innovation, leaving legacy formats behind and leading the charge in large-scale, cost-effective data projects.

Tune in for a dose of GPU-powered revolution!

In this episode from a month ago, join me as we unravel the controversial CEO firing at OpenAI in December 2023.

I share my insights on the events, decode the intricacies, and explore what lies ahead for this influential organization.

Don’t miss this concise yet insightful take on the intersection of leadership and artificial intelligence innovation.

SponsorLearn what the new year holds for ransomware as a service, Active Directory, artificial intelligence and more when you download the 2024 Arctic Wolf Labs Predictions Report today at arcticwolf.com/datascience

!!WARNING!!

Due to some technical issues the volume is not always constant during the show.

I sincerely apologise for any inconvenienceFrancescoIn this episode, I speak with Richie Cotton, Data Evangelist at DataCamp, as he delves into the dynamic intersection of AI and education.

Richie, a seasoned expert in data science and the host of the podcast, brings together a wealth of knowledge and experience to explore the evolving landscape of AI careers, the skills essential for generative AI technologies, and the symbiosis of domain expertise and technical skills in the industry.

Dive into the world of Data Science at Home with our latest episode, where we explore the dynamic relationship between Artificial Intelligence and the redemption of open source software.

In this thought-provoking discussion, I share my insights on why now, more than ever, is the opportune moment for open source to leave an indelible mark on the field of AI.

Join me as I unpack my opinions and set expectations for the near future, discussing the pivotal role open source is set to play in shaping the landscape of data science and artificial intelligence.

Don’t miss out—tune in to gain a deeper understanding of this revolutionary intersection!

This episode is available as YouTube stream at htt…

In this captivating podcast episode, join renowned financial expert Chris Skinner as he delves into the fascinating realm of the future of money.

From cryptocurrencies to government currencies, the metaverse to artificial intelligence (AI), Skinner explores the intricate interplay between technology and humanity.

Gain valuable insights as he defines the future of money, examines the potential impact of cryptocurrencies on traditional government currencies, and addresses the advantages and disadvantages of digital currencies.

Brace yourself for an enlightening discussion on the integration of AI in the financial sector and its potential impact on humanity.

Once subscribed, you get full acces…

In this thought-provoking episode, we sit down with the renowned AI expert, Filip Piekniewski, Phd, who fearlessly challenges the prevailing narratives surrounding artificial general intelligence (AGI) and the singularity.

With a no-nonsense approach and a deep understanding of the field, Filip dismantles the hype and exposes some of the misconceptions about AI, LLMs and AGI.

If you’re seeking a refreshingly pragmatic perspective on the future of AI, this episode is an absolute must-listen.

Filip Piekniewski BioFilip Piekniewski is a distinguished computer vision researcher and engineer, specializing in visual object tracking and perception.

He is known for his realistic perspective on AI, …

Brace yourselves, dear friends!

In this episode, we delve into the earth-shattering revelation that OpenAI might have stumbled upon AGI (lol) and we’re all just seconds away from being replaced by highly sophisticated toasters (lol lol).

Spoiler alert: OpenAI’s CEO is just playing 7D chess with the entire human race.

So, sit back, relax, and enjoy this totally not ominous exploration into the ‘totally not happening’ future of AI!

Dive into the cool world of AI chips with us!

🚀 We’re breaking down how these special computer chips for AI have evolved and what makes them different.

Think of them like the superheroes of the tech world!

Don’t miss out!

🎙️🔍 #AIChips #TechTalk #SimpleScience

Hey there, engineering enthusiasts!

Ever wondered how engineers deal with the wild, unpredictable twists and turns in their projects?

In this episode, we’re spilling the beans on uncertainty and why it’s the secret sauce in every engineering recipe, not just the fancy stuff like deep learning and neural networks!

Join us for a ride through the world of uncertainty quantification.

Tune in and let’s demystify the unpredictable together!