Papers

Most single-cell foundation models are adapted from language models, representing each cell as a sequence of gene tokens. This discards the relationships among genes and often the magnitude of their expression. We present scVision, a vision foundation model that instead renders each cell as a continuous image. Using optimal transport, it places genes at fixed positions on a single shared, pan-tissue layout so that co-expressed genes become spatial neighbours, turning a transcriptome into an image in which gene programs appear as local texture. We pretrain a vision transformer by masked image modelling on 72 million human cells and use the frozen encoder with no fine-tuning. In zero-shot evaluations on six independent, held-out studies, scVision is the most accurate cell-type annotator and recovers gene programs without supervision, ahead of existing foundation models and classical baselines; on multi-study integration it matches the strongest token-based model while conserving the most biological structure, without ever seeing a batch label. Permuting the gene layout with the network fixed sharply lowers accuracy, more than removing the vision transformer itself, showing that biologically meaningful position, not the network, carries the signal. By preserving expression magnitude and gene relationships, scVision reframes single-cell representation learning as a vision problem, connecting it to the mature methods of computer vision.

While large language models (LLMs) can support clinical documentation needs, standalone tools struggle with "workflow friction" from manual data entry. We developed ChatEHR, a system that enables the use of LLMs with the entire patient timeline spanning several years. ChatEHR enables automations - which are static combinations of prompts and data that perform a fixed task - and interactive use in the electronic health record (EHR) via a user interface (UI). The resulting ability to sift through patient medical records for diverse use-cases such as pre-visit chart review, screening for transfer eligibility, monitoring for surgical site infections, and chart abstraction, redefines LLM use as an institutional capability. This system, accessible after user-training, enables continuous monitoring and evaluation of LLM use. In 1.5 years, we built 7 automations and 1075 users have trained to become routine users of the UI, engaging in 23,000 sessions in the first 3 months of launch. For automations, being model-agnostic and accessing multiple types of data was essential for matching specific clinical or administrative tasks with the most appropriate LLM. Benchmark-based evaluations proved insufficient for monitoring and evaluation of the UI, requiring new methods to monitor performance. Generation of summaries was the most frequent task in the UI, with an estimated 0.73 hallucinations and 1.60 inaccuracies per generation. The resulting mix of cost savings, time savings, and revenue growth required a value assessment framework to prioritize work as well as quantify the impact of using LLMs. Initial estimates are $6M savings in the first year of use, without quantifying the benefit of the better care offered. Such a "build-from-within" strategy provides an opportunity for health systems to maintain agency via a vendor-agnostic, internally governed LLM platform.

We present an approach of using AI to model and simulate biology and life. Why is it important? Because at the core of medicine, pharmacy, public health, longevity, agriculture and food security, environmental protection, and clean energy, it is biology at work. Biology in the physical world is too complex to manipulate and always expensive and risky to tamper with. In this perspective, we layout an engineering viable approach to address this challenge by constructing an AI-Driven Digital Organism (AIDO), a system of integrated multiscale foundation models, in a modular, connectable, and holistic fashion to reflect biological scales, connectedness, and complexities. An AIDO opens up a safe, affordable and high-throughput alternative platform for predicting, simulating and programming biology at all levels from molecules to cells to individuals. We envision that an AIDO is poised to trigger a new wave of better-guided wet-lab experimentation and better-informed first-principle reasoning, which can eventually help us better decode and improve life.

quantum-classical supercomputing: quantum chemistry of protein-ligand complexes

Robotics draws inspiration from biology, particularly animal locomotion based on muscle-driven contractions. While traditional engineering assembles components sequentially, locomotive animals are built via self-organized developmental programs. Stem cells, under the right conditions, can mimic these processes in vitro, offering a pathway to develop muscle-propelled biobots in a self-organized building process. Here, we demonstrate that existent cardiogenic gastruloid protocols can produce motile aggregates from mouse embryonic stem cells, although with very limited efficiency. We then identify a novel protocol that yields contractile aggregates with higher frequency and larger contractile areas. In this novel protocol, mesendoderm induction using TGF-beta ligands is followed by cardiogenic induction with FGFs and VEGF. Synthetic organizers further control contraction localization. Aggregates developed via this protocol show enhanced motility, marking a step forward towards building motile cardiobots from self-organized biological material. This strategy opens new possibilities for designing autonomous biobots and studying the evolution of muscle-powered movement of multicellular organisms and cardiovascular development.

The ability to study human cardiac development in health and disease is highly limited by the capacity to model the complexity of the human heart in vitro. Developing more efficient organ-like platforms that can model complex in vivo phenotypes, such as organoids and organs-on-a-chip, will enhance the ability to study human heart development and disease. This paper describes a protocol to generate highly complex human heart organoids (hHOs) by self-organization using human pluripotent stem cells and stepwise developmental pathway activation using small molecule inhibitors. Embryoid bodies (EBs) are generated in a 96-well plate with round-bottom, ultra-low attachment wells, facilitating suspension culture of individualized constructs. The EBs undergo differentiation into hHOs by a three-step Wnt signaling modulation strategy, which involves an initial Wnt pathway activation to induce cardiac mesoderm fate, a second step of Wnt inhibition to create definitive cardiac lineages, and a third Wnt activation step to induce proepicardial organ tissues. These steps, carried out in a 96-well format, are highly efficient, reproducible, and produce large amounts of organoids per run. Analysis by immunofluorescence imaging from day 3 to day 11 of differentiation reveals first and second heart field specifications and at day 15, highly complex tissues inside hHOs, including myocardial tissue with regions of atrial and ventricular cardiomyocytes, as well as internal chambers lined with endocardial tissue. The organoids also exhibit an intricate vascular network throughout the structure and an external lining of epicardial tissue. From a functional standpoint, hHOs beat robustly and present normal calcium activity as determined by Fluo-4 live imaging. Overall, this protocol constitutes a solid platform for in vitro studies in human organ-like cardiac tissues.

AI security red-teaming competitions – in which participants compete to develop new attacks against AI models and defenses – provide a unique way to assess how secure today’s AI systems are in the face of adversarial pressure. CAISI recently partnered with Gray Swan, the UK AI Security Institute (UK AISI), and several frontier AI labs to publish a new research paper based on data from a large-scale public AI agent red-teaming competition, revealing several insights into the robustness of current leading AI models.

Accurately predicting cellular responses to genetic perturbations is essential for understanding disease mechanisms and designing effective therapies. Yet, exhaustively exploring the space of possible perturbations (for example, multigene perturbations or across tissues and cell types) is prohibitively expensive, motivating methods that can generalize to unseen conditions. We present TxPert, a latent-transfer-based deep learning method that uses multiple knowledge graphs of gene (product)–gene (product) relationships to predict transcriptomic perturbation effects. Different knowledge graphs encode complementary information and we show that a combination of graphs derived from biological databases and high-throughput perturbation screens yields the best performance. For predictions of single unseen perturbations, TxPert approaches the performance of split-half experimental reproducibility. For double unseen perturbations and single perturbations in a different cell line, its predictions increase Person Δ for unseen single perturbations by 8–25% over existing methods.

Neural crest cells (NCCs) are a multipotent embryonic cell population of ectodermal origin that extensively migrate during early development and contribute to the formation of multiple tissues. Cardiac NCCs play a critical role in heart development by orchestrating outflow tract septation, valve formation, aortic arch artery patterning, parasympathetic innervation, and maturation of the cardiac conduction system. Abnormal migration, proliferation, or differentiation of cardiac NCCs can lead to severe congenital cardiovascular malformations. However, the complexity and timing of early embryonic heart development pose significant challenges to studying the molecular mechanisms underlying NCC-related cardiac pathologies. Here, we present a sophisticated functional model of human heart assembloids derived from induced pluripotent stem cells, which, for the first time, recapitulates cardiac NCC integration into the human embryonic heart in vitro. NCCs successfully integrated at developmentally relevant stages into heart organoids, and followed developmental trajectories known to occur in the human heart. They demonstrated extensive migration, differentiated into cholinergic neurons capable of generating nerve impulses, and formed mature glial cells. Additionally, they contributed to the mesenchymal populations of the developing outflow tract. Through transcriptomic analysis, we revealed that NCCs acquire molecular features of their cardiac derivatives as heart assembloids develop. NCC-derived parasympathetic neurons formed functional connections with cardiomyocytes, promoting the maturation of the cardiac conduction system. Leveraging this model’s cellular complexity and functional maturity, we uncovered that early exposure of NCCs to antidepressants harms the development of NCC derivatives in the context of the developing heart. The commonly prescribed antidepressant Paroxetine disrupted the expression of a critical early neuronal transcription factor, resulting in impa

Recently, as the demand for cleaning robots has steadily increased, therefore household electricity consumption is also increasing. To solve this electricity consumption issue, the problem of efficient path planning for cleaning robot has become important and many studies have been conducted. However, most of them are about moving along a simple path segment, not about the whole path to clean all places. As the emerging deep learning technique, reinforcement learning (RL) has been adopted for cleaning robot. However, the models for RL operate only in a specific cleaning environment, not the various cleaning environment. The problem is that the models have to retrain whenever the cleaning environment changes. To solve this problem, the proximal policy optimization (PPO) algorithm is combined with an efficient path planning that operates in various cleaning environments, using transfer learning (TL), detection nearest cleaned tile, reward shaping, and making elite set methods. The proposed method is validated with an ablation study and comparison with conventional methods such as random and zigzag. The experimental results demonstrate that the proposed method achieves improved training performance and increased convergence speed over the original PPO. And it also demonstrates that this proposed method is better performance than conventional methods (random, zigzag).

This article surveys reinforcement learning approaches in social robotics. Reinforcement learning is a framework for decision-making problems in which an agent interacts through trial-and-error with its environment to discover an optimal behavior. Since interaction is a key component in both reinforcement learning and social robotics, it can be a well-suited approach for real-world interactions with physically embodied social robots. The scope of the paper is focused particularly on studies that include social physical robots and real-world human-robot interactions with users. We present a thorough analysis of reinforcement learning approaches in social robotics. In addition to a survey, we categorize existent reinforcement learning approaches based on the used method and the design of the reward mechanisms. Moreover, since communication capability is a prominent feature of social robots, we discuss and group the papers based on the communication medium used for reward formulation. Considering the importance of designing the reward function, we also provide a categorization of the papers based on the nature of the reward. This categorization includes three major themes: interactive reinforcement learning, intrinsically motivated methods, and task performance-driven methods. The benefits and challenges of reinforcement learning in social robotics, evaluation methods of the papers regarding whether or not they use subjective and algorithmic measures, a discussion in the view of real-world reinforcement learning challenges and proposed solutions, the points that remain to be explored, including the approaches that have thus far received less attention is also given in the paper. Thus, this paper aims to become a starting point for researchers interested in using and applying reinforcement learning methods in this particular research field.

Intelligent manufacturing is becoming increasingly important due to the growing demand for maximizing productivity and flexibility while minimizing waste and lead times. This work investigates automated secondary robotic food packaging solutions that transfer food products from the conveyor belt into containers. A major problem in these solutions is varying product supply which can cause drastic productivity drops. Conventional rule-based approaches, used to address this issue, are often inadequate, leading to violation of the industry's requirements. Reinforcement learning, on the other hand, has the potential of solving this problem by learning responsive and predictive policy, based on experience. However, it is challenging to utilize it in highly complex control schemes. In this paper, we propose a reinforcement learning framework, designed to optimize the conveyor belt speed while minimizing interference with the rest of the control system. When tested on real-world data, the framework exceeds the performance requirements (99.8% packed products) and maintains quality (100% filled boxes). Compared to the existing solution, our proposed framework improves productivity, has smoother control, and reduces computation time.

One of the biggest hurdles robotics faces is the facet of sophisticated and hard-to-engineer behaviors. Reinforcement learning offers a set of tools, and a framework to address this problem. In parallel, the misgivings of robotics offer a solid testing ground and evaluation metric for advancements in reinforcement learning. The two disciplines go hand-in-hand, much like the fields of Mathematics and Physics. By means of this survey paper, we aim to invigorate links between the research communities of the two disciplines by focusing on the work done in reinforcement learning for locomotive and control aspects of robotics. Additionally, we aim to highlight not only the notable successes but also the key challenges of the application of Reinforcement Learning in Robotics. This paper aims to serve as a reference guide for researchers in reinforcement learning applied to the field of robotics. The literature survey is at a fairly introductory level, aimed at aspiring researchers. Appropriately, we have covered the most essential concepts required for research in the field of reinforcement learning, with robotics in mind. Through a thorough analysis of this problem, we are able to manifest how reinforcement learning could be applied profitably, and also focus on open-ended questions, as well as the potential for future research.

Large Vision-Language Models (LVLMs) have recently shown great promise in advancing robotics by combining embodied reasoning with robot control. A common approach involves training on embodied reasoning tasks related to robot control using Supervised Fine-Tuning (SFT). However, SFT datasets are often heuristically constructed and not explicitly optimized for improving robot control. Furthermore, SFT often leads to issues such as catastrophic forgetting and reduced generalization performance. To address these limitations, we introduce Robot-R1, a novel framework that leverages reinforcement learning to enhance embodied reasoning specifically for robot control. Robot-R1 learns to predict the next keypoint state required for task completion, conditioned on the current scene image and environment metadata derived from expert demonstrations. Inspired by the DeepSeek-R1 learning approach, Robot-R1 samples reasoning-based responses and reinforces those that lead to more accurate predictions. To rigorously evaluate Robot-R1, we also introduce a new benchmark that demands the diverse embodied reasoning capabilities for the task. Our experiments show that models trained with Robot-R1 outperform SFT methods on embodied reasoning tasks. Despite having only 7B parameters, Robot-R1 even surpasses GPT-4o on reasoning tasks related to low-level action control, such as spatial and movement reasoning.

As surgical robots become more common, automating away some of the burden of complex direct human operation becomes ever more feasible. Model-free reinforcement learning (RL) is a promising direction toward generalizable automated surgical performance, but progress has been slowed by the lack of efficient and realistic learning environments. In this paper, we describe adding reinforcement learning support to the da Vinci Skill Simulator, a training simulation used around the world to allow surgeons to learn and rehearse technical skills. We successfully teach an RL-based agent to perform sub-tasks in the simulator environment, using either image or state data. As far as we know, this is the first time an RL-based agent is taught from visual data in a surgical robotics environment. Additionally, we tackle the sample inefficiency of RL using a simple-to-implement system which we term hybrid-batch learning (HBL), effectively adding a second, long-term replay buffer to the Q-learning process. Additionally, this allows us to bootstrap learning from images from the data collected using the easier task of learning from state. We show that HBL decreases our learning times significantly.

Conventional feedback control methods can solve various types of robot control problems very efficiently by capturing the structure with explicit models, such as rigid body equations of motion. However, many control problems in modern manufacturing deal with contacts and friction, which are difficult to capture with first-order physical modeling. Hence, applying control design methodologies to these kinds of problems often results in brittle and inaccurate controllers, which have to be manually tuned for deployment. Reinforcement learning (RL) methods have been demonstrated to be capable of learning continuous robot controllers from interactions with the environment, even for problems that include friction and contacts. In this paper, we study how we can solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL. The final control policy is a superposition of both control signals. We demonstrate our approach by training an agent to successfully perform a real-world block assembly task involving contacts and unstable objects.