Sensory landscape

Aircraft noise changes how Pedregal frogs call and thins out their chorus.

Densely annotated underwater image datasets of Red Sea coral reefs from EPFL and partners for training segmentation models; V2 adds panoptic labels, 95 classes and fish instances, and is the training data behind DeepReefMap.

Edited volume from Farsi, Daneshkhah, Hosseinian-Far and Jahankhani, 2019, on using digital twins to monitor, diagnose and forecast city infrastructure, with attention to security.

Open-access edited volume from Shi, Goodchild, Batty, Kwan and Zhang, 2021, on the data, methods and infrastructure used to sense, model and run cities.

Open object detector from the ConservationDronesAI group, built on RF-DETR, that finds people, mammals and vehicles in drone imagery for wildlife surveys and counts.

Open-source streaming 3D reconstruction model from the Robbyant team at Ant Group; turns a video or image sequence into camera poses, depth and point clouds in one feed-forward pass.

Hosted mapping and visual positioning service from MultiSet AI that turns one drone flight over a site into a localisation map which drones, phones, headsets and robots then query for their position.

Catalogue from Niantic Spatial of 100 real-world environments captured as simulation-ready USDZ scenes, for training and testing robot policies in Isaac Sim and other OpenUSD simulators.

NVIDIA's family of world foundation models and tools for physical AI; generates and reasons over video of the physical world for robot training, simulation and synthetic sensor data.

NVIDIA's open-source robotics simulator built on Omniverse and OpenUSD; used to train, test and validate robots in physically based virtual environments and to generate synthetic training data.

World model from Odyssey that generates an interactive 3D environment from a prompt and predicts how it changes as a person or agent moves and acts in it.

Universal Scene Description, the open 3D scene format and software platform created by Pixar and now stewarded by the Alliance for OpenUSD; the scene layer under NVIDIA Omniverse and many simulation pipelines.

Open-source family of wildlife detection models and tools from Microsoft's AI for Good Lab; MegaDetector finds animals, people and vehicles in camera-trap images, and PyTorch-Wildlife is the framework that runs it and its siblings for aerial and acoustic data.

Real-time underwater SLAM from the University of Michigan field robotics group that fuses learned stereo depth with IMU, pressure and Doppler velocity log measurements; developed and tested on shipwreck surveys.

Feed-forward 3D vision model from Meta AI and Oxford's Visual Geometry Group that predicts camera poses, depth and point clouds from video; used as a reconstruction backend in DeepReefMap for coral reef mapping.

How do sound, light, smell and barriers affect people and other animals?

Mehmet Kerem Turkcan's city-scale digital twin of New York City's five boroughs, compiled from public municipal records, with real-time rendering, traffic and pedestrian simulation and a CARLA-style Python API.