Skip to content

NVIDIA AR SDK - API headers and sample applications

License

Notifications You must be signed in to change notification settings

jonahhawk/MAXINE-AR-SDK

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

48 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

README

NVIDIA MAXINE AR SDK: API Source Code and Sample Applications

NVIDIA MAXINE AR SDK enables real-time modeling and tracking of human faces from video. The SDK is powered by NVIDIA graphics processing units (GPUs) with Tensor Cores, and as a result, the algorithm throughput is greatly accelerated, and latency is reduced.

The SDK has the following features:

  • Face detection and tracking, which detects, localizes, and tracks human faces in images or videos by using bounding boxes.
  • Facial landmark detection and tracking, which predicts and tracks the pixel locations of human facial landmark points and head poses in images or videos. It can predict 68 and 126 landmark points. The 68 detected facial landmarks follow the Multi-PIE 68 point mark-ups information in facial point annotations. The 126 landmark points detector can predict more points on the cheeks, the eyes, and on laugh lines.
  • Face 3D mesh and tracking, which reconstructs and tracks a 3D human face and its head pose from the provided facial landmarks.
  • 3D Body Pose and tracking, which predicts and tracks the 3D human pose from images or videos. It predicts 34 keypoints of body pose in 2D and 3D.

Face detection and tracking Facial landmark detection and tracking - 68 pts

Facial landmark detection and tracking - 126 pts Face 3D mesh and tracking

Body 3D Pose and tracking

The SDK provides two sample applications that demonstrate the features listed above in real time by using a webcam or offline videos.

  • FaceTrack App which demonstrates the face tracking, landmark tracking and 3D mesh tracking features.
  • BodyTrack App which demonstrates the 3D Body Pose tracking feature.

NVIDIA MAXINE AR SDK is distributed in the following parts:

Please refer to SDK System guide for configuring and integrating the SDK, compiling and running the sample applications. Please visit the NVIDIA MAXINE AR SDK webpage for more information about the SDK.

System requirements

The SDK is supported on NVIDIA GPUs that are based on the NVIDIA® Turing™ or Ampere™ architecture and have Tensor Cores.

  • Windows OS supported: 64-bit Windows 10 or later
  • Microsoft Visual Studio: 2017 (MSVC15.0) or later
  • CMake: v3.12 or later
  • NVIDIA Graphics Driver for Windows: 465.89 or later
  • NVIDIA CUDA Toolkit: 11.3.1
  • NVIDIA TensorRT: 8.0.1.6

NVIDIA MAXINE Branding Guidelines

If you integrate an NVIDIA MAXINE SDK within your product, please follow the required branding guidelines that are available here

Compiling the sample app

Steps

The open source repository includes the source code to build the sample application, and a proxy file nvARProxy.cpp to enable compilation without explicitly linking against the SDK DLL.

Note: To download the models and runtime dependencies required by the features, you need to run the SDK Installer.

  1. In the root folder of the downloaded source code, start the CMake GUI and specify the source folder and a build folder for the binary files.
  • For the source folder, ensure that the path ends in OSS.
  • For the build folder, ensure that the path ends in OSS/build.
  1. Use CMake to configure and generate the Visual Studio solution file.
  • Click Configure.
  • When prompted to confirm that CMake can create the build folder, click OK.
  • Select Visual Studio for the generator and x64 for the platform.
  • To complete configuring the Visual Studio solution file, click Finish.
  • To generate the Visual Studio Solution file, click Generate.
  • Verify that the build folder contains the NvAR_SDK.sln file.
  1. Use Visual Studio to generate the FaceTrack.exe or BodyTrack.exe file from the NvAR_SDK.sln file.
  • In CMake, to open Visual Studio, click Open Project.
  • In Visual Studio, select Build > Build Solution.

Documentation

Please refer to the online documentation guides -

PDF versions of these guides are also available at the following locations -

About

NVIDIA AR SDK - API headers and sample applications

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C 72.6%
  • C++ 26.3%
  • CMake 1.1%