+
## Introduction
-
+{: .no_toc }
We will guide you through the steps to create an Android app that can train a simple image classification model using on-device training techniques. This tutorial showcases the `transfer learning` technique where knowledge gained from training a model on one task is leveraged to improve the performance of a model on a different but related task. Instead of starting the learning process from scratch, transfer learning allows us to transfer the knowledge or features learned by a pre-trained model to a new task.
For this tutorial, we will leverage the `MobileNetV2` model which has been trained on large-scale image datasets such as ImageNet (which has 1,000 classes). We will use this model for classifying custom data into one of four classes. The initial layers of MobileNetV2 serve as a feature extractor, capturing generic visual features applicable to various tasks, and only the final classifier layer will be trained for the task at hand.
@@ -24,26 +24,10 @@ In this tutorial, we will use data to learn to:
- Classify animals into one of four categories using a pre-packed animals dataset.
- Classify celebrities into one of four categories using a custom celebrities dataset.
-## Contents
-
-- [Introduction](#introduction)
-- [Prerequisites](#prerequisites)
-- [Offline Phase - Building the training artifacts](#offline-phase---building-the-training-artifacts)
- - [Export the model to ONNX](#op1)
- - [Define the trainable and non trainable parameters](#op2)
- - [Generate the training artifacts](#op3)
-- [Training Phase - Android application development](#training-phase---android-application-development)
- - [Setting up the project in Android Studio](#tp1)
- - [Adding the ONNX Runtime dependency](#tp2)
- - [Packaging the Prebuilt Training Artifacts and Dataset](#tp3)
- - [Interfacing with ONNX Runtime - C++ Code](#tp4)
- - [Image Preprocessing](#tp5)
- - [Application Frontend](#tp6)
-- [Training Phase - Running the application on a device](#training-phase---running-the-application-on-a-device)
- - [Running the application on a device](#tp7)
- - [Training with a pre-loaded dataset - Animals](#tp8)
- - [Training with a custom dataset - Celebrities](#tp9)
-- [Conclusion](#conclusion)
+
+## Table of Contents
+* TOC placeholder
+{:toc}
## Prerequisites
@@ -791,7 +775,7 @@ To follow this tutorial, you should have a basic understanding of Android app de
b. Launching the application on the device should look like this:
- 
+
2. Training with a pre-loaded dataset - Animals
@@ -805,7 +789,7 @@ To follow this tutorial, you should have a basic understanding of Android app de
e. Use any animal image from your library for inferencing now.
- 
+
As can be seen from the image above, the model correctly predicted `Cow`.
@@ -825,7 +809,7 @@ To follow this tutorial, you should have a basic understanding of Android app de
g. That's it!. Hopefully the application classified the image correctly.
-
+
## Conclusion
diff --git a/docs/tutorials/on-device-training/ios-app.md b/docs/tutorials/on-device-training/ios-app.md
index fff1347923ef0..e61bab68596ff 100644
--- a/docs/tutorials/on-device-training/ios-app.md
+++ b/docs/tutorials/on-device-training/ios-app.md
@@ -7,7 +7,7 @@ nav_order: 2
---
# Building an iOS Application
-
+{: .no_toc }
In this tutorial, we will explore how to build an iOS application that incorporates ONNX Runtime's On-Device Training solution. On-device training refers to the process of training a machine learning model directly on an edge device without relying on cloud services or external servers.
In this tutorial, we will build a simple speaker identification app that learns to identify a speaker's voice. We will take a look at how to train a model on-device, export the trained model, and use the trained model to perform inference.
@@ -18,6 +18,7 @@ Here is what the application will look like:
## Introduction
+{: .no_toc }
We will guide you through the process of building an iOS application that can train a simple audio classification model using on-device training techniques. The tutorial showcases the `transfer learning` technique where knowledge gained from training a model on one task is leveraged to improve the performance of a model on a different but related task. Instead of starting the learning process from scratch, transfer learning allows us to transfer the knowledge or features learned by a pre-trained model to a new task.
In this tutorial, we will leverage the [`wav2vec`](https://huggingface.co/superb/wav2vec2-base-superb-sid) model which has been trained on large-scale celebrity speech data such as `VoxCeleb1`. We will use the pre-trained model to extract features from the audio data and train a binary classifier to identify the speaker. The initial layers of the model serve as a feature extractor, capturing the important features of the audio data. Only the last layer of the model is trained to perform the classification task.
@@ -29,23 +30,9 @@ In the tutorial, we will:
- Use the exported model to perform inference
-## Contents
-- [Building an iOS Application](#building-an-ios-application)
- - [Introduction](#introduction)
- - [Contents](#contents)
- - [Prerequisites](#prerequisites)
- - [Generating the training artifacts](#generating-the-training-artifacts)
- - [Building the iOS application](#building-the-ios-application)
- - [Xcode Setup](#xcode-setup)
- - [Application Overview](#application-overview)
- - [Training the model](#training-the-model)
- - [Inference with the trained model](#inference-with-the-trained-model)
- - [Recording Audio](#recording-audio)
- - [Train View](#train-view)
- - [Infer View](#infer-view)
- - [ContentView](#contentview)
- - [Running the iOS application](#running-the-ios-application)
- - [Conclusion](#conclusion)
+## Table of Contents
+* TOC placeholder
+{:toc}
## Prerequisites
@@ -947,27 +934,27 @@ Now, we are ready to run the application. You can run the application on the sim
a. Now, when you run the application, you should see the following screen:
-
+
b. Next, click on the `Train` button to navigate to the `TrainView`. The `TrainView` will prompt you to record your voice. You will need to record your voice `kNumRecordings` times.
-
+
c. Once all the recordings are complete, the application will train the model on the given data. You will see the progress bar indicating the progress of the training.
-
+
d. Once the training is complete, you will see the following screen:
-
+
e. Now, click on the `Infer` button to navigate to the `InferView`. The `InferView` will prompt you to record your voice. Once the recording is complete, it will perform inference with the trained model and display the result of the inference.
-
+
That's it! Hopefully, it identified your voice correctly.
diff --git a/docs/tutorials/web/ep-webnn.md b/docs/tutorials/web/ep-webnn.md
index fe1c1d729daf0..f04dd7870d7cb 100644
--- a/docs/tutorials/web/ep-webnn.md
+++ b/docs/tutorials/web/ep-webnn.md
@@ -74,59 +74,59 @@ To use WebNN EP, you just need to make 3 small changes:
WebNN API and WebNN EP are in actively development, you might consider installing the latest nightly build version of ONNX Runtime Web (onnxruntime-web@dev) to benefit from the latest features and improvements.
-## Keep tensor data on WebNN MLBuffer (IO binding)
+## Keep tensor data on WebNN MLTensor (IO binding)
-By default, a model's inputs and outputs are tensors that hold data in CPU memory. When you run a session with WebNN EP with 'gpu' or 'npu' device type, the data is copied to GPU or NPU memory, and the results are copied back to CPU memory. Memory copy between different devices as well as different sessions will bring much overhead to the inference time, WebNN provides a new opaque device-specific storage type MLBuffer to address this issue.
-If you get your input data from a MLBuffer, or you want to keep the output data on MLBuffer for further processing, you can use IO binding to keep the data on MLBuffer. This will be especially helpful when running transformer based models, which usually runs a single model multiple times with previous output as the next input.
+By default, a model's inputs and outputs are tensors that hold data in CPU memory. When you run a session with WebNN EP with 'gpu' or 'npu' device type, the data is copied to GPU or NPU memory, and the results are copied back to CPU memory. Memory copy between different devices as well as different sessions will bring much overhead to the inference time, WebNN provides a new opaque device-specific storage type MLTensor to address this issue.
+If you get your input data from a MLTensor, or you want to keep the output data on MLTensor for further processing, you can use IO binding to keep the data on MLTensor. This will be especially helpful when running transformer based models, which usually runs a single model multiple times with previous output as the next input.
-For model input, if your input data is a WebNN storage MLBuffer, you can [create a MLBuffer tensor and use it as input tensor](#create-input-tensor-from-a-mlbuffer).
+For model input, if your input data is a WebNN storage MLTensor, you can [create a MLTensor tensor and use it as input tensor](#create-input-tensor-from-a-mltensor).
For model output, there are 2 ways to use the IO binding feature:
-- [Use pre-allocated MLBuffer tensors](#use-pre-allocated-mlbuffer-tensors)
+- [Use pre-allocated MLTensor tensors](#use-pre-allocated-mltensor-tensors)
- [Specify the output data location](#specify-the-output-data-location)
Please also check the following topic:
-- [MLBuffer tensor life cycle management](#mlbuffer-tensor-life-cycle-management)
+- [MLTensor tensor life cycle management](#mltensor-tensor-life-cycle-management)
-**Note:** The MLBuffer necessitates a shared MLContext for IO binding. This implies that the MLContext should be pre-created as a WebNN EP option and utilized across all sessions.
+**Note:** The MLTensor necessitates a shared MLContext for IO binding. This implies that the MLContext should be pre-created as a WebNN EP option and utilized across all sessions.
-### Create input tensor from a MLBuffer
+### Create input tensor from a MLTensor
-If your input data is a WebNN storage MLBuffer, you can create a MLBuffer tensor and use it as input tensor:
+If your input data is a WebNN storage MLTensor, you can create a MLTensor tensor and use it as input tensor:
```js
const mlContext = await navigator.ml.createContext({deviceType, ...});
-const inputMLBuffer = await mlContext.createBuffer({
+const inputMLTensor = await mlContext.createTensor({
dataType: 'float32',
dimensions: [1, 3, 224, 224],
- usage: MLBufferUsage.WRITE_TO,
+ usage: MLTensorUsage.WRITE,
});
-mlContext.writeBuffer(mlBuffer, inputArrayBuffer);
-const inputTensor = ort.Tensor.fromMLBuffer(mlBuffer, {
+mlContext.writeTensor(inputMLTensor, inputArrayBuffer);
+const inputTensor = ort.Tensor.fromMLTensor(inputMLTensor, {
dataType: 'float32',
dims: [1, 3, 224, 224]
});
```
-Use this tensor as model inputs(feeds) so that the input data will be kept on MLBuffer.
+Use this tensor as model inputs(feeds) so that the input data will be kept on MLTensor.
-### Use pre-allocated MLBuffer tensors
+### Use pre-allocated MLTensor tensors
-If you know the output shape in advance, you can create a MLBuffer tensor and use it as output tensor:
+If you know the output shape in advance, you can create a MLTensor tensor and use it as output tensor:
```js
-// Create a pre-allocated buffer and the corresponding tensor. Assuming that the output shape is [10, 1000].
+// Create a pre-allocated MLTensor and the corresponding ORT tensor. Assuming that the output shape is [10, 1000].
const mlContext = await navigator.ml.createContext({deviceType, ...});
-const myPreAllocatedBuffer = await mlContext.createBuffer({
+const myPreAllocatedMLTensor = await mlContext.createTensor({
dataType: 'float32',
dimensions: [10, 1000],
- usage: MLBufferUsage.READ_FROM,
+ usage: MLTensorUsage.READ,
});
-const myPreAllocatedOutputTensor = ort.Tensor.fromMLBuffer(myPreAllocatedBuffer, {
+const myPreAllocatedOutputTensor = ort.Tensor.fromMLTensor(myPreAllocatedMLTensor, {
dataType: 'float32',
dims: [10, 1000]
});
@@ -140,17 +140,17 @@ const results = await mySession.run(feeds, fetches);
```
-By specifying the output tensor in the fetches, ONNX Runtime Web will use the pre-allocated buffer as the output buffer. If there is a shape mismatch, the `run()` call will fail.
+By specifying the output tensor in the fetches, ONNX Runtime Web will use the pre-allocated MLTensor as the output tensor. If there is a shape mismatch, the `run()` call will fail.
### Specify the output data location
-If you don't want to use pre-allocated MLBuffer tensors for outputs, you can also specify the output data location in the session options:
+If you don't want to use pre-allocated MLTensor tensors for outputs, you can also specify the output data location in the session options:
```js
const mySessionOptions1 = {
...,
- // keep all output data on MLBuffer
- preferredOutputLocation: 'ml-buffer'
+ // keep all output data on MLTensor
+ preferredOutputLocation: 'ml-tensor'
};
const mySessionOptions2 = {
@@ -158,7 +158,7 @@ const mySessionOptions2 = {
// alternatively, you can specify the output location for each output tensor
preferredOutputLocation: {
'output_0': 'cpu', // keep output_0 on CPU. This is the default behavior.
- 'output_1': 'ml-buffer' // keep output_1 on MLBuffer buffer
+ 'output_1': 'ml-tensor' // keep output_1 on MLTensor tensor
}
};
```
@@ -169,18 +169,18 @@ See [API reference: preferredOutputLocation](https://onnxruntime.ai/docs/api/js/
## Notes
-### MLBuffer tensor life cycle management
+### MLTensor tensor life cycle management
-It is important to understand how the underlying MLBuffer is managed so that you can avoid memory leaks and improve buffer usage efficiency.
+It is important to understand how the underlying MLTensor is managed so that you can avoid memory leaks and improve tensor usage efficiency.
-A MLBuffer tensor is created either by user code or by ONNX Runtime Web as model's output.
-- When it is created by user code, it is always created with an existing MLBuffer using `Tensor.fromMLBuffer()`. In this case, the tensor does not "own" the MLBuffer.
+A MLTensor tensor is created either by user code or by ONNX Runtime Web as model's output.
+- When it is created by user code, it is always created with an existing MLTensor using `Tensor.fromMLTensor()`. In this case, the tensor does not "own" the MLTensor.
- - It is user's responsibility to make sure the underlying buffer is valid during the inference, and call `mlBuffer.destroy()` to dispose the buffer when it is no longer needed.
- - Avoid calling `tensor.getData()` and `tensor.dispose()`. Use the MLBuffer directly.
- - Using a MLBuffer tensor with a destroyed MLBuffer will cause the session run to fail.
-- When it is created by ONNX Runtime Web as model's output (not a pre-allocated MLBuffer tensor), the tensor "owns" the buffer.
+ - It is user's responsibility to make sure the underlying MLTensor is valid during the inference, and call `mlTensor.destroy()` to dispose the MLTensor when it is no longer needed.
+ - Avoid calling `tensor.getData()` and `tensor.dispose()`. Use the MLTensor tensor directly.
+ - Using a MLTensor tensor with a destroyed MLTensor will cause the session run to fail.
+- When it is created by ONNX Runtime Web as model's output (not a pre-allocated MLTensor tensor), the tensor "owns" the MLTensor.
- - You don't need to worry about the case that the buffer is destroyed before the tensor is used.
- - Call `tensor.getData()` to download the data from the MLBuffer to CPU and get the data as a typed array.
- - Call `tensor.dispose()` explicitly to destroy the underlying MLBuffer when it is no longer needed.
+ - You don't need to worry about the case that the MLTensor is destroyed before the tensor is used.
+ - Call `tensor.getData()` to download the data from the MLTensor to CPU and get the data as a typed array.
+ - Call `tensor.dispose()` explicitly to destroy the underlying MLTensor when it is no longer needed.
diff --git a/images/EP_context_node.png b/images/EP_context_node.png
new file mode 100644
index 0000000000000..953bcf353558a
Binary files /dev/null and b/images/EP_context_node.png differ
diff --git a/images/EP_context_nodes_with_different_eps.png b/images/EP_context_nodes_with_different_eps.png
new file mode 100644
index 0000000000000..c7b986d0f9c89
Binary files /dev/null and b/images/EP_context_nodes_with_different_eps.png differ
diff --git a/images/Onnx_weight_sharing.png b/images/Onnx_weight_sharing.png
new file mode 100644
index 0000000000000..b3c277903ddfb
Binary files /dev/null and b/images/Onnx_weight_sharing.png differ
diff --git a/images/Ort_Qnn_Ep_weight_sharing.png b/images/Ort_Qnn_Ep_weight_sharing.png
new file mode 100644
index 0000000000000..e8fa37d1bb2a4
Binary files /dev/null and b/images/Ort_Qnn_Ep_weight_sharing.png differ
diff --git a/images/Qnn_weight_sharing.png b/images/Qnn_weight_sharing.png
new file mode 100644
index 0000000000000..d415c3bfc57ca
Binary files /dev/null and b/images/Qnn_weight_sharing.png differ
diff --git a/images/nashville.jpg b/images/nashville.jpg
new file mode 100644
index 0000000000000..da40173230e0c
Binary files /dev/null and b/images/nashville.jpg differ
diff --git a/src/app.html b/src/app.html
index 5f79324942486..cdfdad8b3f2dc 100644
--- a/src/app.html
+++ b/src/app.html
@@ -36,6 +36,11 @@
},
propertyConfiguration: {
// Properties Plugin configuration
+ gpcDataSharingOptIn: false,
+ callback: {
+ userConsentDetails: _getWcpUserConsentDetails
+ },
+
env: 'PROD' // Environment can be set to PPE or PROD as needed.
},
webAnalyticsConfiguration: {
@@ -77,6 +82,7 @@
}
};
+ var siteConsent = null;
WcpConsent.init(
'en-US',
'cookie-banner',
@@ -91,6 +97,24 @@
WcpConsent.themes.light
);
+ function _getWcpUserConsentDetails() {
+ if (siteConsent) {
+ return siteConsent.getConsent();
+ }
+
+ // The exact value that you return here is dependent on your site, team and how
+ // use any data that is stored (work with you privacy team to determine what the
+ // correct "defaults" (true or false) should be for each item when the code is
+ // unable to determine (via WCP) if or what the user has (or has not) consented
+ // to.
+ return {
+ Required: [true], // Most likely `true`
+ Analytics: [true],
+ SocialMedia: [true],
+ Advertising: [false]
+ };
+ }
+
function onConsentChanged(categoryPreferences) {
if (categoryPreferences.Analytics) {
// Google Analytics
diff --git a/src/images/logos/autodesk-logo.png b/src/images/logos/autodesk-logo.png
new file mode 100644
index 0000000000000..cb7d223734dbb
Binary files /dev/null and b/src/images/logos/autodesk-logo.png differ
diff --git a/src/images/logos/goodnotes-logo.png b/src/images/logos/goodnotes-logo.png
new file mode 100644
index 0000000000000..86ee9ccee519a
Binary files /dev/null and b/src/images/logos/goodnotes-logo.png differ
diff --git a/src/routes/blogs/+page.svelte b/src/routes/blogs/+page.svelte
index 8dda46876bcdc..bbc36db183912 100644
--- a/src/routes/blogs/+page.svelte
+++ b/src/routes/blogs/+page.svelte
@@ -366,6 +366,12 @@
}
];
let blogsCommunity = [
+ {
+ title:'Running Phi-3 Mistral 7B LLMs on Raspberry Pi 5: A Step-by-Step Guide',
+ date: 'September 5, 2024',
+ link: 'https://medium.com/@vadikus/running-phi-3-mistral-7b-llms-on-raspberry-pi-5-a-step-by-step-guide-185e8102e35b',
+ blurb: 'Learn how to run Phi-3 Mistral 7B on Raspberry Pi 5 using the ONNX Runtime Gen AI library.'
+ },
{
title:
'Deploying a Production-Ready RAG Server: A Comprehensive Guide with LlamaIndex',
diff --git a/src/routes/blogs/nimbleedge-x-onnxruntime/+page.svx b/src/routes/blogs/nimbleedge-x-onnxruntime/+page.svx
index 7dc2f326cb3f5..48efc63953143 100644
--- a/src/routes/blogs/nimbleedge-x-onnxruntime/+page.svx
+++ b/src/routes/blogs/nimbleedge-x-onnxruntime/+page.svx
@@ -32,7 +32,7 @@ url: 'https://onnxruntime.ai/blogs/nimbleedge-x-onnxruntime'
[NimbleEdge](https://www.nimbleedge.com/) is an on-device Machine Learning (ML) platform that enables real-time personalization in mobile apps, executing data capture, processing and ML inference on end users' mobile devices vs. on cloud. Using mobile compute efficiently to deliver optimal performance with minimal device resource usage is a key priority for NimbleEdge. For this, NimbleEdge leverages various ML inference runtimes, including, prominently, **ONNX Runtime**.
-In this blog post, we'll explore how on-device compute can be leveraged for cost-efficient, privacy-preserving real-time ML in mobile apps, and how NimbleEdge leverages ONNX Runtime to enable this. We also share results from NimbleEdge's on-device deployment with Dream11, India's largest fantasy gaming platform with 200Mn+ users.
+In this blog post, we'll explore how on-device compute can be leveraged for cost-efficient, privacy-preserving real-time ML in mobile apps, and how NimbleEdge leverages ONNX Runtime to enable this. We also share results from NimbleEdge’s on-device deployment with one of India’s largest fantasy gaming platforms with hundreds of millions of users.
### **Introduction**
@@ -102,17 +102,17 @@ For inference execution, NimbleEdge utilizes a number of runtimes, prominently i
Through the capabilities listed here, NimbleEdge's comprehensive on-device ML platform enables high performance real-time ML deployments in days vs. months.
-### **Case Study: Real time ranking of fantasy sports contests for Dream11**
+### **Case Study: Real time ranking of fantasy sports contests for leading Indian fantasy gaming co**
-Dream11 is an Indian fantasy sports platform (like Fanduel/ Draftkings in USA) with 200M+ users, and a peak concurrency of ~15 million users. Dream11 offers thousands of fantasy contests across dozens of matches from 10+ sports, with each contest varying in contest entry amount, win %, and participant count.
+Fantasy Gaming co (name obscured for confidentiality) is an Indian fantasy sports platform (like Fanduel/ Draftkings in USA) with hundreds of millions of users, and a peak concurrency of several million users. Fantasy Gaming co offers thousands of fantasy contests across dozens of matches from 10+ sports, with each contest varying in contest entry amount, win %, and no. of participants.
-To streamline the user journey, Dream11 was running a recommendation system that delivered personalized contest recommendations to users, based on historical interactions. Dream11 analyzed customer clickstream data, and identified that incorporating in-session user interactions in the recommender systems would significantly improve quality of recommendations vs. leveraging batch predictions generated hourly.
+To streamline the user journey, Fantasy Gaming co was running a recommendation system that delivered personalized contest recommendations to users, based on historical interactions. They analyzed customer clickstream data, and identified that incorporating in-session user interactions in the recommender systems would significantly improve quality of recommendations vs. leveraging batch predictions generated hourly.
-Due to this, Dream11 was keen to deploy real-time, session-aware recommendations, but implementation was challenging due to the aforementioned challenges in real-time ML on cloud. Hence, Dream11 turned to on-device ML with NimbleEdge for implementing real-time personalized contest recommendations.
+Due to this, Fantasy Gaming co was keen to deploy real-time, session-aware recommendations, but implementation was challenging due to the aforementioned challenges in real-time ML on cloud. Hence, Fantasy Gaming co turned to on-device ML with NimbleEdge for implementing real-time personalized contest recommendations.
**Results**
-With NimbleEdge, Dream11 is now able to generate features and predictions based on real-time user interactions, resulting in improved relevance of recommendations for millions of users. Additionally, inference was delivered at millisecond latency, with minimal battery and CPU usage impact!
+With NimbleEdge, Fantasy Gaming co is now able to generate features and predictions based on real-time user interactions, resulting in improved relevance of recommendations for millions of users. Additionally, inference was delivered at millisecond latency, with minimal battery and CPU usage impact!
**No. of inferences:** `7B+`
diff --git a/src/routes/blogs/pytorch-on-the-edge/+page.svelte b/src/routes/blogs/pytorch-on-the-edge/+page.svelte
index 83ab6d2d49db6..d0a9d765cd5f1 100644
--- a/src/routes/blogs/pytorch-on-the-edge/+page.svelte
+++ b/src/routes/blogs/pytorch-on-the-edge/+page.svelte
@@ -179,9 +179,9 @@ fun run(audioTensor: OnnxTensor): Result {
- By: Natalie Kershaw + By: Natalie Kershaw and - Prasanth Pulavarthi
@@ -217,12 +217,12 @@ fun run(audioTensor: OnnxTensor): Result { anywhere that is outside of the cloud, ranging from large, well-resourced personal computers to small footprint devices such as mobile phones. This has been a challenging task to accomplish in the past, but new advances in model optimization and software like - ONNX Runtime + ONNX Runtime make it more feasible - even for new generative AI and large language models like Stable Diffusion, Whisper, and Llama2. -There are several factors to keep in mind when thinking about running a PyTorch model on the @@ -292,7 +292,7 @@ fun run(audioTensor: OnnxTensor): Result { -
We mentioned ONNX Runtime several times above. ONNX Runtime is a compact, standards-based @@ -305,7 +305,7 @@ fun run(audioTensor: OnnxTensor): Result { format that doesn't require the PyTorch framework and its gigabytes of dependencies. PyTorch has thought about this and includes an API that enables exactly this - torch.onnxtorch.onnx. ONNX is an open standard that defines the operators that make up models. The PyTorch ONNX APIs take the Pythonic PyTorch code and turn it into a functional graph that captures the operators that are needed to run the model without Python. As with everything @@ -318,7 +318,7 @@ fun run(audioTensor: OnnxTensor): Result { The popular Hugging Face library also has APIs that build on top of this torch.onnx functionality to export models to the ONNX format. Over 130,000 models130,000 models are supported making it very likely that the model you care about is one of them.
@@ -328,7 +328,7 @@ fun run(audioTensor: OnnxTensor): Result { and web browsers) via various languages (from C# to JavaScript to Swift). -You don't have to export the fifth model, ClipTokenizer, as it is available in ONNX Runtime extensionsONNX Runtime extensions, a library for pre and post processing PyTorch models.
@@ -353,7 +353,7 @@ fun run(audioTensor: OnnxTensor): Result { To run this pipeline of models as a .NET application, we build the pipeline code in C#. This code can be run on CPU, GPU, or NPU, if they are available on your machine, using ONNX Runtime's device-specific hardware accelerators. This is configured with theExecutionProviderTargetExecutionProviderTarget below.
You can build the application and run it on Windows with the detailed steps shown in this tutorialtutorial.
@@ -374,7 +374,7 @@ fun run(audioTensor: OnnxTensor): Result {Running a PyTorch model locally in the browser is not only possible but super simple with - the transformers.js library. Transformers.js uses ONNX Runtime Web as its backend. Many models are already converted to ONNX and served by the tranformers.js CDN, making inference in the browser a matter of writing @@ -407,7 +407,7 @@ fun run(audioTensor: OnnxTensor): Result { All components of the Whisper Tiny model (audio decoder, encoder, decoder, and text sequence generation) can be composed and exported to a single ONNX model using the Olive frameworkOlive framework. To run this model as part of a mobile application, you can use ONNX Runtime Mobile, which supports Android, iOS, react-native, and MAUI/Xamarin.
@@ -420,7 +420,7 @@ fun run(audioTensor: OnnxTensor): Result {The relevant snippet of a example Android mobile appAndroid mobile app that performs speech transcription on short samples of audio is shown below:
You can read the full Speaker Verification tutorialSpeaker Verification tutorial, and build and run the application from sourcebuild and run the application from source.
diff --git a/src/routes/components/customers.svelte b/src/routes/components/customers.svelte index a5da8146bea27..6c6c7dce06171 100644 --- a/src/routes/components/customers.svelte +++ b/src/routes/components/customers.svelte @@ -8,32 +8,36 @@ import antgroupLogo from '../../images/logos/antgroup-logo.png'; import algoriddimLogo from '../../images/logos/algoriddim-logo.png'; import ATLASLogo from '../../images/logos/ATLAS-logo.png'; + import autodeskLogo from '../../images/logos/autodesk-logo.png'; import bazaarvoiceLogo from '../../images/logos/bazaarvoice-logo.png'; import camoLogo from '../../images/logos/camo-logo.png'; import cephableLogo from '../../images/logos/cephable-logo.png'; import clearbladeLogo from '../../images/logos/clearblade-logo.png'; import deezerLogo from '../../images/logos/deezer-logo.png'; + import goodnotesLogo from '../../images/logos/goodnotes-logo.png'; + import huggingfaceLogo from '../../images/logos/huggingface-logo.png'; import hypefactorsLogo from '../../images/logos/hypefactors-logo.png'; import infarmLogo from '../../images/logos/infarm-logo.png'; import intelLogo from '../../images/logos/intel-logo.png'; import intelligenzaEticaLogo from '../../images/logos/intelligenza-etica-logo.png'; - import navitaireAmadeusLogo from '../../images/logos/navitaire-amadeus-logo.png'; - import PeakSpeedLogo from '../../images/logos/PeakSpeed_logo.png'; + import navitaireLogo from '../../images/logos/navitaire-amadeus-logo.png'; + import nvidiaLogo from '../../images/logos/nvidia.png'; + import opennlpLogo from '../../images/logos/opennlp-logo.png'; + import oracleLogo from '../../images/logos/oracle-logo.png'; + import peakspeedLogo from '../../images/logos/PeakSpeed_logo.png'; import piecesLogo from '../../images/logos/pieces-logo.png'; + import ptwLogo from '../../images/logos/ptw-logo.png'; import redisLogo from '../../images/logos/redis-logo.png'; - import RockchipLogo from '../../images/logos/Rockchip-logo.png'; + import rockchipLogo from '../../images/logos/Rockchip-logo.png'; import samtecLogo from '../../images/logos/samtec-logo.png'; import sasLogo from '../../images/logos/sas-logo.png'; import teradataLogo from '../../images/logos/teradata-logo.png'; import topazlabsLogo from '../../images/logos/topazlabs-logo.png'; - import ueLogo from '../../images/logos/ue-logo.png'; + import unrealengineLogo from '../../images/logos/ue-logo.png'; import usdaLogo from '../../images/logos/usda-logo.png'; import vespaLogo from '../../images/logos/vespa-logo.png'; import writerLogo from '../../images/logos/writer-logo.png'; import xilinxLogo from '../../images/logos/xilinx-logo.png'; - import huggingfaceLogo from '../../images/logos/huggingface-logo.png'; - import nvidiaLogo from '../../images/logos/nvidia.png'; - import oracleLogo from '../../images/logos/oracle-logo.png'; const testimonials = [ { @@ -61,6 +65,11 @@ src: ATLASLogo, alt: 'ATLAS' }, + { + href: './testimonials#Autodesk', + src: autodeskLogo, + alt: 'Autodesk' + }, { href: './testimonials#Bazaarvoice', src: bazaarvoiceLogo, @@ -86,6 +95,11 @@ src: deezerLogo, alt: 'Deezer' }, + { + href: './testimonials#Goodnotes', + src: goodnotesLogo, + alt: 'GoodNotes' + }, { href: './testimonials#Hugging%20Face', src: huggingfaceLogo, @@ -113,7 +127,7 @@ }, { href: './testimonials#Navitaire', - src: navitaireAmadeusLogo, + src: navitaireLogo, alt: 'Navitaire' }, { @@ -121,6 +135,11 @@ src: nvidiaLogo, alt: 'NVIDIA' }, + { + href: './testimonials#Apache%20OpenNLP', + src: opennlpLogo, + alt: 'Apache OpenNLP' + }, { href: './testimonials#Oracle', src: oracleLogo, @@ -128,7 +147,7 @@ }, { href: './testimonials#Peakspeed', - src: PeakSpeedLogo, + src: peakspeedLogo, alt: 'Peakspeed' }, { @@ -136,6 +155,11 @@ src: piecesLogo, alt: 'Pieces' }, + { + href: './testimonials#PTW%20Dosimetry', + src: ptwLogo, + alt: 'PTW Dosimetry' + }, { href: './testimonials#Redis', src: redisLogo, @@ -143,7 +167,7 @@ }, { href: './testimonials#Rockchip', - src: RockchipLogo, + src: rockchipLogo, alt: 'Rockchip' }, { @@ -168,7 +192,7 @@ }, { href: './testimonials#Unreal%20Engine', - src: ueLogo, + src: unrealengineLogo, alt: 'Unreal Engine' }, { diff --git a/src/routes/components/footer.svelte b/src/routes/components/footer.svelte index b030524976742..e6b855d0ca129 100644 --- a/src/routes/components/footer.svelte +++ b/src/routes/components/footer.svelte @@ -9,7 +9,7 @@