-
Notifications
You must be signed in to change notification settings - Fork 31
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #17 from creativeIKEP/develop
v1.3.0
- Loading branch information
Showing
7 changed files
with
68 additions
and
91 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
115 changes: 47 additions & 68 deletions
115
Packages/BlazePoseBarracuda/Documentation~/BlazePoseBarracuda.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,90 +1,69 @@ | ||
| # BlazePoseBarracuda Usage Documentation | ||
| ### Dependencies | ||
| ## Dependencies | ||
| BlazePoseBarracuda uses the following sub packages: | ||
| - [PoseDetectionBarracuda](https://github.com/creativeIKEP/PoseDetectionBarracuda) | ||
| - [PoseLandmarkBarracuda](https://github.com/creativeIKEP/PoseLandmarkBarracuda) | ||
|
|
||
| ### Install | ||
| ## Install | ||
| BlazePoseBarracuda can be installed from npm or GitHub URL. | ||
|
|
||
| ### Install from npm (Recommend) | ||
| BlazePoseBarracuda can be installed by adding following sections to the manifest file (`Packages/manifest.json`). | ||
|
|
||
| To the `scopedRegistries` section: | ||
| ``` | ||
| { | ||
| "name": "creativeikep", | ||
| "url": "https://registry.npmjs.com", | ||
| "scopes": [ "jp.ikep" ] | ||
| } | ||
| ``` | ||
| To the `dependencies` section: | ||
| ``` | ||
| "jp.ikep.mediapipe.blazepose": "1.3.0" | ||
| ``` | ||
| Finally, the manifest file looks like below: | ||
| ``` | ||
| { | ||
| "scopedRegistries": [ | ||
| { | ||
| "name": "creativeikep", | ||
| "url": "https://registry.npmjs.com", | ||
| "scopes": [ "jp.ikep" ] | ||
| } | ||
| ], | ||
| "dependencies": { | ||
| "jp.ikep.mediapipe.blazepose": "1.3.0", | ||
| ... | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Install from GitHub URL | ||
| BlazePoseBarracuda can be installed by adding below URLs from the Unity Package Manager's window | ||
| ``` | ||
| https://github.com/creativeIKEP/PoseDetectionBarracuda.git?path=Packages/PoseDetectionBarracuda#v1.0.0 | ||
| https://github.com/creativeIKEP/PoseDetectionBarracuda.git?path=Packages/PoseDetectionBarracuda#v1.0.1 | ||
| ``` | ||
| ``` | ||
| https://github.com/creativeIKEP/PoseLandmarkBarracuda.git?path=Packages/PoseLandmarkBarracuda#v1.0.1 | ||
| https://github.com/creativeIKEP/PoseLandmarkBarracuda.git?path=Packages/PoseLandmarkBarracuda#v1.1.1 | ||
| ``` | ||
| ``` | ||
| https://github.com/creativeIKEP/BlazePoseBarracuda.git?path=Packages/BlazePoseBarracuda#v1.0.0 | ||
| https://github.com/creativeIKEP/BlazePoseBarracuda.git?path=Packages/BlazePoseBarracuda#v1.3.0 | ||
| ``` | ||
| or, appending lines to your manifest file(`Packages/manifest.json`) `dependencies` block. | ||
| Example is below. | ||
| ``` | ||
| { | ||
| "dependencies": { | ||
| "jp.ikep.mediapipe.posedetection": "https://github.com/creativeIKEP/PoseDetectionBarracuda.git?path=Packages/PoseDetectionBarracuda#v1.0.0", | ||
| "jp.ikep.mediapipe.poselandmark": "https://github.com/creativeIKEP/PoseLandmarkBarracuda.git?path=Packages/PoseLandmarkBarracuda#v1.0.1", | ||
| "jp.ikep.mediapipe.blazepose": "https://github.com/creativeIKEP/BlazePoseBarracuda.git?path=Packages/BlazePoseBarracuda#v1.0.0", | ||
| "jp.ikep.mediapipe.posedetection": "https://github.com/creativeIKEP/PoseDetectionBarracuda.git?path=Packages/PoseDetectionBarracuda#v1.0.1", | ||
| "jp.ikep.mediapipe.poselandmark": "https://github.com/creativeIKEP/PoseLandmarkBarracuda.git?path=Packages/PoseLandmarkBarracuda#v1.1.1", | ||
| "jp.ikep.mediapipe.blazepose": "https://github.com/creativeIKEP/BlazePoseBarracuda.git?path=Packages/BlazePoseBarracuda#v1.3.0", | ||
| ... | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Usage Demo | ||
| Below code is the demo that estimate human pose from a image and get pose landmark. | ||
| Check ["/Assets/Script/PoseVisuallizer.cs"](https://github.com/creativeIKEP/BlazePoseBarracuda/blob/main/Assets/Script/PoseVisuallizer.cs) and ["/Assets/Scenes/SampleScene.unity"](https://github.com/creativeIKEP/BlazePoseBarracuda/blob/main/Assets/Scenes/SampleScene.unity) for BlazePoseBarracuda usage demo details. | ||
| ```cs | ||
| using UnityEngine; | ||
| using Mediapipe.BlazePose; | ||
|
|
||
| public class <YourClassName>: MonoBehaviour | ||
| { | ||
| // Set "Packages/BlazePoseBarracuda/ResourceSet/BlazePose.asset" on the Unity Editor. | ||
| [SerializeField] BlazePoseResource blazePoseResource; | ||
| [SerializeField] bool isUpperBodyOnly; | ||
|
|
||
| BlazePoseDetecter detecter; | ||
|
|
||
| void Start(){ | ||
| detecter = new BlazePoseDetecter(blazePoseResource, isUpperBodyOnly); | ||
| } | ||
|
|
||
| void Update(){ | ||
| Texture input = ...; // Your input image texture | ||
| // Predict pose by neural network model. | ||
| // Switchable anytime between upper body and full body with 2nd argment. | ||
| detecter.ProcessImage(input, isUpperBodyOnly); | ||
|
|
||
| /* | ||
| `detecter.outputBuffer` is pose landmark result and ComputeBuffer of float4 array type. | ||
| 0~(24 or 32) index datas are pose landmark. | ||
| Check below Mediapipe document about relation between index and landmark position. | ||
| https://google.github.io/mediapipe/solutions/pose#pose_landmarks | ||
| Each data factors are | ||
| x: x cordinate value of pose landmark ([0, 1]). | ||
| y: y cordinate value of pose landmark ([0, 1]). | ||
| z: Landmark depth with the depth at the midpoint of hips being the origin. | ||
| The smaller the value the closer the landmark is to the camera. ([0, 1]). | ||
| This value is full body mode only. | ||
| **The use of this value is not recommended beacuse in development.** | ||
| w: The score of whether the landmark position is visible ([0, 1]). | ||
| (25 or 33) index data is the score whether human pose is visible ([0, 1]). | ||
| This data is (score, 0, 0, 0). | ||
| */ | ||
| ComputeBuffer result = detecter.outputBuffer; | ||
|
|
||
| // `detecter.vertexCount` is count of pose landmark vertices. | ||
| // `detecter.vertexCount` returns 25 or 33 by upper or full body mode. | ||
| int count = detecter.vertexCount; | ||
|
|
||
| // Your custom processing from here, i.e. rendering. | ||
| // For example, below is CPU log debug. | ||
| var data = new Vector4[count]; | ||
| result.GetData(data); | ||
| Debug.Log("---"); | ||
| foreach(var d in data){ | ||
| Debug.Log(d); | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
| ## Usage Demo | ||
| This repository has the demo for inferencing pose and visualizing landmarks. | ||
| Check ["/Assets/Script/PoseVisuallizer.cs"](https://github.com/creativeIKEP/BlazePoseBarracuda/blob/main/Assets/Script/PoseVisuallizer.cs) and ["/Assets/Scenes/2DSampleScene.unity"](https://github.com/creativeIKEP/BlazePoseBarracuda/blob/main/Assets/Scenes/2DSampleScene.unity) for BlazePoseBarracuda usage demo details in the 2D pose estimation. | ||
| Check ["/Assets/Script/PoseVisuallizer3D.cs"](https://github.com/creativeIKEP/BlazePoseBarracuda/blob/main/Assets/Script/PoseVisuallizer3D.cs) and ["/Assets/Scenes/3DSampleScene.unity"](https://github.com/creativeIKEP/BlazePoseBarracuda/blob/main/Assets/Scenes/3DSampleScene.unity) for BlazePoseBarracuda usage demo details in the 3D pose estimation. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters