ParaRNN: Large-Scale Nonlinear RNNs, Trainable in Parallel

International Conference on Learning Representations (ICLR) 2026

International Conference on Learning Representations (ICLR) 2026 Apple is presenting new research at the annual International Conference on Learning Representations (ICLR), which takes place in person in Rio de Janeiro, Brazil, from April 23 to 27. We are proud to again sponsor the conference, which brings together the scientific and industrial research communities focused on deep learning. Below is an overview of Apple’s participation at ICLR 2026: Jump to a section: Schedule Stop by the Apple booth #204 during exhibition hours: 9:30 AM - 5:30 PM (Thursday, April 23 - Saturday, April 25). All times referenced in schedule are in BRT (local time). Schedule Thursday, April 23 - Pretraining with Hierarchical Memories: Separating Long-Tail and Common Knowledge - 10:30 AM - 1:00 PM, Poster Session 1, Pavilion 3, #0309 - Hadi Pour Ansari, C Thomas, David Grangier, Michael Kirchhof, Oncel…

MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining

MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining AuthorsBingbing Wen**, Sirajul Salekin, Feiyang Kang†, Lucy Lu Wang‡, Bill Howe‡, Javier Movellan, Manjot Bilkhu MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining AuthorsBingbing Wen**, Sirajul Salekin, Feiyang Kang†, Lucy Lu Wang‡, Bill Howe‡, Javier Movellan, Manjot Bilkhu This paper was accepted at the Workshop on Navigating and Addressing Data Problems for Foundation Models (NADPFM) at ICLR 2026. Principled domain reweighting can substantially improve sample efficiency and downstream generalization; however, data-mixture optimization for multimodal pretraining remains underexplored. Current multimodal training recipes tune mixtures from only a single perspective such as data format or task type. We introduce MixAtlas, a principled framework for compute-efficient multimodal mixture optimization via systematic domain decomposition and smaller proxy models. MixAtlas factorizes the training data along two interpretable axes - image concepts and task supervision -…

STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows

STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows AuthorsJiatao Gu†, Ying Shen‡**, Tianrong Chen, Laurent Dinh, Yuyang Wang, Miguel Ángel Bautista, David Berthelot, Josh Susskind, Shuangfei Zhai STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows AuthorsJiatao Gu†, Ying Shen‡**, Tianrong Chen, Laurent Dinh, Yuyang Wang, Miguel Ángel Bautista, David Berthelot, Josh Susskind, Shuangfei Zhai Normalizing flows (NFs) are end-to-end likelihood-based generative models for continuous data, and have recently regained attention with encouraging progress on image generation. Yet in the video generation domain, where spatiotemporal complexity and computational cost are substantially higher, state-of-the-art systems almost exclusively rely on diffusion-based models. In this work, we revisit this design space by presenting STARFlow-V, a normalizing flow-based video generator with substantial benefits such as end-to-end learning, robust causal prediction, and native likelihood estimation. Building upon the recently proposed STARFlow, STARFlow-V operates in the spatiotemporal…

Adaptive Thinking: Large Language Models Know When to Think in Latent Space

Adaptive Thinking: Large Language Models Know When to Think in Latent Space AuthorsPingzhi Li†‡, Bairu Hou, Yun Zhu†, Yihao Feng, Ke Ye†, Tao Lei, Zhifeng Chen, Tianlong Chen‡, Xianzhi Du Adaptive Thinking: Large Language Models Know When to Think in Latent Space AuthorsPingzhi Li†‡, Bairu Hou, Yun Zhu†, Yihao Feng, Ke Ye†, Tao Lei, Zhifeng Chen, Tianlong Chen‡, Xianzhi Du Recent advances in large language models (LLMs) test-time computing have introduced the capability to perform intermediate chain-of-thought (CoT) reasoning (thinking) before generating answers. While increasing the thinking budget yields smooth performance improvements at inference time, the relationship between LLM capability, query complexity, and optimal budget allocation remains poorly understood for achieving compute-optimal inference. To address this challenge, we utilize self-consistency, the agreement among multiple reasoning paths, as a proxy for thinking necessity. We first identify that lower self-consistency indicates when…

Local Mechanisms of Compositional Generalization in Conditional Diffusion

Local Mechanisms of Compositional Generalization in Conditional Diffusion AuthorsArwen Bradley Local Mechanisms of Compositional Generalization in Conditional Diffusion AuthorsArwen Bradley Conditional diffusion models appear capable of compositional generalization, i.e., generating convincing samples for out-of-distribution combinations of conditioners, but the mechanisms underlying this ability remain unclear. To make this concrete, we study length generalization, the ability to generate images with more objects than seen during training. In a controlled CLEVR setting (Johnson et al.,2017), we find that length generalization is achievable in some cases but not others, suggesting that models only sometimes learn the underlying compositional structure. We then investigate locality as a structural mechanism for compositional generalization. Prior works proposed score locality as a mechanism for creativity in unconditional diffusion models (Kamb & Ganguli, 2024; Niedoba et al., 2024), but did not address flexible conditioning or compositional generalization. In this…

StereoFoley: Object-Aware Stereo Audio Generation from Video

StereoFoley: Object-Aware Stereo Audio Generation from Video AuthorsTornike Karchkhadze†**, Kuan-Lin Chen, Mojtaba Heydari, Robert Henzel, Alessandro Toso, Mehrez Souden, Joshua Atkins StereoFoley: Object-Aware Stereo Audio Generation from Video AuthorsTornike Karchkhadze†**, Kuan-Lin Chen, Mojtaba Heydari, Robert Henzel, Alessandro Toso, Mehrez Souden, Joshua Atkins We present StereoFoley, a video-to-audio generation framework that produces semantically aligned, temporally synchronized, and spatially accurate stereo sound at 48 kHz. While recent generative video-to-audio models achieve strong semantic and temporal fidelity, they largely remain limited to mono or fail to deliver object-aware stereo imaging, constrained by the lack of professionally mixed, spatially accurate video-to-audio datasets. First, we develop and train a base model that generates stereo audio from video, achieving state-of-the-art in both semantic accuracy and synchronization. Next, to overcome dataset limitations, we introduce a synthetic data generation pipeline that combines video analysis, object tracking, and audio…

Apple Machine Learning Research at ICLR 2026

Apple is advancing AI and ML with fundamental research, much of which is shared through publications and engagement at conferences in order to accelerate progress in this important field and support the broader community. This week, the Fourteenth International Conference on Learning Representations (ICLR) will be held in Rio de Janeiro, Brazil, and Apple is proud to again participate in this important event for the research community and to support it with sponsorship. At the main conference and associated workshops, Apple researchers will present new research across a variety of topics, including work unlocking large-scale training for Recurrent Neural Networks, a technique for improving State Space Models, a new approach to unifying image understanding and generation, a method for generating 3D scenes from a single photo, and a new approach to protein folding. During exhibition hours, attendees will be able…

Bootstrapping Sign Language Annotations with Sign Language Models

Bootstrapping Sign Language Annotations with Sign Language Models AuthorsColin Lea, Vasileios Baltatzis, Connor Gillis, Raja Kushalnagar†**, Lorna Quandt†**, Leah Findlater Bootstrapping Sign Language Annotations with Sign Language Models AuthorsColin Lea, Vasileios Baltatzis, Connor Gillis, Raja Kushalnagar†**, Lorna Quandt†**, Leah Findlater AI-driven sign language interpretation is limited by a lack of high-quality annotated data. New datasets including ASL STEM Wiki and FLEURS-ASL contain professional interpreters and 100s of hours of data but remain only partially annotated and thus underutilized, in part due to the prohibitive costs of annotating at this scale. In this work, we develop a pseudo-annotation pipeline that takes signed video and English as input and outputs a ranked set of likely annotations, including time intervals, for glosses, fingerspelled words, and sign classifiers. Our pipeline uses sparse predictions from our fingerspelling recognizer and isolated sign recognizer (ISR), along with…

LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning

LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning AuthorsHaoqiang Kang†, Yizhe Zhang, Nikki Lijing Kuang†, Nicklas Majamaki†, Navdeep Jaitly, Yi-An Ma†, Lianhui Qin† LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning AuthorsHaoqiang Kang†, Yizhe Zhang, Nikki Lijing Kuang†, Nicklas Majamaki†, Navdeep Jaitly, Yi-An Ma†, Lianhui Qin† Large Language Models (LLMs) demonstrate their reasoning ability through chain-of-thought (CoT) generation. However, LLM’s autoregressive decoding may limit the ability to revisit and refine earlier tokens in a holistic manner, which can also lead to inefficient exploration for diverse solutions. In this paper, we propose LaDiR (Latent Diffusion Reasoner), a novel reasoning framework that unifies the expressiveness of continuous latent representation with the iterative refinement capabilities of latent diffusion models for an existing LLM. We first construct a structured latent reasoning space using a Variational Autoencoder (VAE) that encodes text reasoning steps into blocks…

Can Large Language Models Understand Context?

Can Large Language Models Understand Context? AuthorsYilun Zhu†**, Joel Ruben Antony Moniz, Shruti Bhargava, Jiarui Lu, Dhivya Piraviperumal, Site Li, Yuan Zhang, Hong Yu, Bo-Hsiang Tseng Can Large Language Models Understand Context? AuthorsYilun Zhu†**, Joel Ruben Antony Moniz, Shruti Bhargava, Jiarui Lu, Dhivya Piraviperumal, Site Li, Yuan Zhang, Hong Yu, Bo-Hsiang Tseng Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent. However, though the evaluation of LLMs encompasses various domains within the realm of Natural Language Processing, limited attention has been paid to probing their linguistic capability of understanding contextual features. This paper introduces a context understanding benchmark by adapting existing datasets to suit the evaluation of generative models. This benchmark comprises of four distinct tasks and nine datasets, all featuring prompts designed to…

DSO: Direct Steering Optimization for Bias Mitigation

DSO: Direct Steering Optimization for Bias Mitigation AuthorsLucas Monteiro Paes‡, Nivedha Sivakumar‡, Oliver Wang†‡**, Masha Fedzechkina, Barry-John Theobald, Luca Zappella, Nicholas Apostoloff DSO: Direct Steering Optimization for Bias Mitigation AuthorsLucas Monteiro Paes‡, Nivedha Sivakumar‡, Oliver Wang†‡**, Masha Fedzechkina, Barry-John Theobald, Luca Zappella, Nicholas Apostoloff Generative models are often deployed to make decisions on behalf of users, such as vision-language models (VLMs) identifying which person in a room is a doctor to help visually impaired individuals. Yet, VLM decisions are influenced by the perceived demographic attributes of people in the input, which can lead to biased outcomes like failing to identify women as doctors. Moreover, when reducing bias leads to performance loss, users may have varying needs for balancing bias mitigation with overall model capabilities, highlighting the demand for methods that enable controllable bias reduction during inference. Activation steering is a…

Learning Long-Term Motion Embeddings for Efficient Kinematics Generation

Learning Long-Term Motion Embeddings for Efficient Kinematics Generation AuthorsNick Stracke†‡, Kolja Bauer†‡, Stefan Andreas Baumann†‡, Miguel Ángel Bautista, Josh Susskind, Björn Ommer†‡ Learning Long-Term Motion Embeddings for Efficient Kinematics Generation AuthorsNick Stracke†‡, Kolja Bauer†‡, Stefan Andreas Baumann†‡, Miguel Ángel Bautista, Josh Susskind, Björn Ommer†‡ Understanding and predicting motion is a fundamental component of visual intelligence. Although modern video models exhibit strong comprehension of scene dynamics, exploring multiple possible futures through full video synthesis remains prohibitively inefficient. We model scene dynamics orders of magnitude more efficiently by directly operating on a long-term motion embedding that is learned from large-scale trajectories obtained from tracker models. This enables efficient generation of long, realistic motions that fulfill goals specified via text prompts or spatial pokes. To achieve this, we first learn a highly compressed motion embedding with a temporal compression factor of 64×. In…

Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts

Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts AuthorsJiayuan Ye, Vitaly Feldman, Kunal Talwar Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts AuthorsJiayuan Ye, Vitaly Feldman, Kunal Talwar This paper was accepted at the Workshop on Navigating and Addressing Data Problems for Foundation Models at ICLR 2026. Large language models (LLMs) can struggle to memorize factual knowledge in their parameters, often leading to hallucinations and poor performance on knowledge-intensive tasks. In this paper, we formalize fact memorization from an information-theoretic perspective and study how training data distributions affect fact accuracy. We show that fact accuracy is suboptimal (below the capacity limit) whenever the amount of information contained in the training data facts exceeds model capacity. This is further exacerbated when the fact frequency distribution is skewed (e.g. a power law). We propose…

International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2026

International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2026 Apple is presenting new research at the annual International Conference on Acoustics, Speech and Signal Processing (ICASSP), which takes place in person in Barcelona, Spain, from May 4 to 8. We are proud to again sponsor the conference, which brings together the scientific and industrial research communities focused on signal processing and its applications. Below is an overview of Apple’s participation at ICASSP 2026. Jump to a section: Schedule Stop by the Apple booth #P2 during exhibition hours at the Centre de Convencions Internacional de Barcelona (CCIB) in Barcelona, Spain. All times listed in CEST (local time): - Monday, May 4: 19:00 - 21:30 - Tuesday, May 5 to Friday, May 8: 09:00 - 17:00 Schedule Wednesday, May 6 - Leveraging Audio-Visual Data to Reduce the Multilingual Gap in Self-Supervised…

ACM Human-Computer Interaction Conference (CHI) 2026

ACM Human-Computer Interaction Conference (CHI) 2026 Apple is presenting new research at the annual ACM (Association of Computing Machinery) CHI Conference on Human Factors in Computing Systems, which takes place in person in Barcelona, Spain, from April 13 to 17. We are proud to again sponsor the conference, which brings together the scientific and industrial research communities focused on human-computer interaction. Below is an overview of Apple’s participation at CHI 2026. Below is the schedule of Apple-sponsored presentations, demos, and events at CHI 2026. Jump to a section: Schedule Stop by the Apple booth during exhibition hours at the CHI 2026 venue in Barcelona, Spain. All times listed in CEST (local time): - Monday, April 13: 10:30 - 16:30; CHI Reception 18:00 - 20:00 - Tuesday, April 14: 10:00 - 18:00 - Wednesday, April 15: 10:00 - 18:00 - Thursday,…

Efficient Privacy Loss Accounting for Subsampling and Random Allocation

Efficient Privacy Loss Accounting for Subsampling and Random Allocation AuthorsVitaly Feldman, Moshe Shenfeld† Efficient Privacy Loss Accounting for Subsampling and Random Allocation AuthorsVitaly Feldman, Moshe Shenfeld† We consider the privacy amplification properties of a sampling scheme in which a user’s data is used in k steps chosen randomly and uniformly from a sequence (or set) of t steps. This sampling scheme has been recently applied in the context of differentially private optimization (Chua et al., 2024a; Choquette-Choo et al., 2025) and communication-efficient high-dimensional private aggregation (Asi et al., 2025), where it was shown to have utility advantages over the standard Poisson sampling. Theoretical analyses of this sampling scheme (Feldman & Shenfeld, 2025; Dong et al., 2025) lead to bounds that are close to those of Poisson sampling, yet still have two significant shortcomings. First, in many practical settings, the resulting…

What Do Your Logits Know? (The Answer May Surprise You!)

What Do Your Logits Know? (The Answer May Surprise You!) AuthorsMasha Fedzechkina, Eleonora Gualdoni, Rita Ramos, Sinead Williamson What Do Your Logits Know? (The Answer May Surprise You!) AuthorsMasha Fedzechkina, Eleonora Gualdoni, Rita Ramos, Sinead Williamson Recent work has shown that probing model internals can reveal a wealth of information not apparent from the model generations. This poses the risk of unintentional or malicious information leakage, where model users are able to learn information that the model owner assumed was inaccessible. Using vision-language models as a testbed, we present the first systematic comparison of information retained at different “representational levels” as it is compressed from the rich information encoded in the residual stream through two natural bottlenecks: low-dimensional projections of the residual stream obtained using tuned lens, and the final top- logits most likely to impact model’s answer. We show…