Learning and inferring underlying motion patterns of captured 2D scenes and then re-creating dynamic evolution consistent with the real-world natural phenomena have high appeal for graphics and animation. To bridge the technical gap between virtual and real environments, we focus on the inverse modeling and reconstruction of visually consistent and property-verifiable oceans, taking advantage of deep learning and differentiable physics to learn geometry and constitute waves in a self-supervised manner. First, we infer hierarchical geometry using two networks, which are optimized via the differentiable renderer. We extract wave components from the sequence of inferred geometry through a network equipped with a differentiable ocean model. Then, ocean dynamics can be evolved using the reconstructed wave components. Through extensive experiments, we verify that our new method yields satisfactory results for both geometry reconstruction and wave estimation. Moreover, the new framework has the inverse modeling potential to facilitate a host of graphics applications, such as the rapid production of physically accurate scene animation and editing guided by real ocean scenes.

Laparoscopic cholecystectomy is used to treat cholecystitis and cholelithiasis. Because the high risk of the surgery prevents novice doctors from practicing it on real patients, VR-based surgical simulation has been developed to simulate surgical procedures to train surgeons without patients, cadavers, or animals. In this study, we propose a real-time system designed to provide plausible visual and tactile simulation of the main surgical procedures. To achieve this, the physical properties of organs are modeled by particles, and cluster-based shape matching is used to simulate soft deformation. The haptic interaction between tools and soft tissue is modeled as a collision between a capsule and particles. Constraint-based haptic rendering is used to generate feedback force and the non-penetrating position of the virtual tool. The proposed system can simulate the major steps of laparoscopic cholecystectomy, such as the anatomy of Calot’s triangle, clipping of the cystic duct and biliary artery, disjunction of the cystic duct and biliary artery, and separation of the gallbladder bed. The experimental results show that haptic rendering can be performed at a high frequency (> 900 Hz), whereas mesh skinning and graphics rendering can be performed at 60 frames per second (fps).

For multi-person 2D pose estimation, current deep learning based methods have exhibited impressive performance, but the trade-offs among efficiency, robustness, and accuracy in the existing approaches remain unavoidable. In principle, bottom-up methods are superior to top-down methods in efficiency, but they perform worse in accuracy. To make full use of their respective advantages, in this paper we design a novel bidirectional optimization coupled lightweight network (BOCLN) architecture for efficient, robust, and general-purpose multi-person 2D (2-dimensional) pose estimation from natural images. With the BOCLN framework, the bottom-up network focuses on global features, while the top-down network places emphasis on detailed features. The entire framework shares global features along the bottom-up data stream, while the top-down data stream aims to accelerate the accurate pose estimation. In particular, to exploit the priors of human joints’ relationship, we propose a probability limb heat map to represent the spatial context of the joints and guide the overall pose skeleton prediction, so that each person’s pose estimation in cluttered scenes (involving crowd) could be as accurate and robust as possible. Therefore, benefiting from the novel BOCLN architecture, the time-consuming refinement procedure could be much simplified to an efficient lightweight network. Extensive experiments and evaluations on public benchmarks have confirmed that our new method is more efficient and robust, yet still attain competitive accuracy performance compared with the state-of-the-art methods. Our BOCLN shows even greater promise in online applications.