TDW also simulates noise attenuation and reverberation in accordance with the geometry of the space and the objects in it. To synthesize true-to-life audio, TDW uses generative models of impact sounds that are triggered by collisions or other object interactions within the simulation. The team has also created furnished virtual floor plans that researchers can fill with agents and avatars. Dynamic lighting models accurately simulate scene illumination, causing shadows and dimming that correspond to the appropriate time of day and sun angle. These models respond accurately to lighting changes, and their material composition and orientation in the scene dictate their physical behaviors in the space. Researchers construct and populate a scene by pulling from an extensive 3D model library of objects, like furniture pieces, animals, and vehicles. The simulation consists of two components: The build, which renders images, synthesizes audio, and runs physics simulations and the controller, which is a Python-based interface where the user sends commands to the build. To achieve this, the researchers built TDW on a video game platform called Unit圓D Engine and committed to incorporating both visual and auditory data rendering without any animation. "So, we thought that this sort of environment, where you can have objects that will interact with each other and then render realistic sensory data from them, would be a valuable way to start to study that." "We were all interested in the idea of building a virtual world for the purpose of training AI systems that we could actually use as models of the brain," says McDermott, who studies human and machine hearing. TDW brought these together in one platform. The work began as a collaboration between a group of MIT professors along with Stanford and IBM researchers, tethered by individual research interests into hearing, vision, cognition, and perceptual intelligence. McDermott, Gan, and their colleagues are presenting this research at the conference on Neural Information Processing Systems (NeurIPS) in December. There's the possibility of creating these very rich sensory scenarios, where you still have total control and complete knowledge of what is happening in the environment." In addition, "many of us are excited about the doors that these sorts of virtual worlds open for doing experiments on humans to understand human perception and cognition.
Torque 3d virtual world trial#
Robotic systems, which rely on trial and error, can be taught in an environment where they cannot cause physical harm. "We are trying to build a general-purpose simulation platform that mimics the interactive richness of the real world for a variety of AI applications," says study lead author Chuang Gan, MIT-IBM Watson AI Lab research scientist.Īnother advantage of TDW, McDermott notes, is that it provides a controlled setting for understanding the learning process and facilitating the improvement of AI robots. And using virtual reality (VR), human attention and play behavior within the space can provide real-world data, for example. Different types of robotic agents and avatars can also be spawned within the controlled simulation to perform, say, task planning and execution. TDW is unique in that it is designed to be flexible and generalizable, generating synthetic photo-realistic scenes and audio rendering in real time, which can be compiled into audio-visual datasets, modified through interactions within the scene, and adapted for human and neural network learning and prediction tests. Object orientations, physical characteristics, and velocities are calculated and executed for fluids, soft bodies, and rigid objects as interactions occur, producing accurate collisions and impact sounds. A new study from researchers at MIT, the MIT-IBM Watson AI Lab, Harvard University, and Stanford University is enabling a rich virtual world, very much like stepping into "The Matrix." Their platform, called ThreeDWorld (TDW), simulates high-fidelity audio and visual environments, both indoor and outdoor, and allows users, objects, and mobile agents to interact like they would in real life and according to the laws of physics.
0 kommentar(er)
