Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model loading failed: [TypeError: dims[1] must be an integer, got: undefined] #2

Open
lakpriya1s opened this issue Oct 8, 2024 · 7 comments
Assignees
Labels
bug Something isn't working help wanted Extra attention is needed
Milestone

Comments

@lakpriya1s
Copy link

lakpriya1s commented Oct 8, 2024

Describe the bug
I'm trying to use this library on my bare react native app, but after downloading the models, I'm getting this error "Model loading failed: [TypeError: dims[1] must be an integer, got: undefined]"

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
Load model without any error.

Screenshots
Screenshot 2024-10-09 at 04 31 27
simulator_screenshot_FD4548F8-5B6A-4941-AADF-853CF6E413DF

Smartphone (please complete the following information):

  • Device: [iPhone16]
  • OS: [iOS18]
  • Version [1]

Additional context

[
  {
    "name": "Phi-3-mini",
    "model": "microsoft/Phi-3-mini-4k-instruct-onnx-web",
    "onnx_path": "onnx/model_q4.onnx",
    "options": {"externalData": true}
  },
  {
    "name": "Llama-160M",
    "model": "Felladrin/onnx-Llama-160M-Chat-v1",
    "onnx_path": "onnx/decoder_model_merged.onnx"
  },
  {
    "name": "Minueza-32M",
    "model": "Felladrin/onnx-Minueza-32M-UltraChat",
    "onnx_path": "onnx/decoder_model_merged.onnx"
  },
  {
    "name": "distilgpt2",
    "model": "Xenova/distilgpt2_onnx-quantized",
    "onnx_path": "decoder_model.onnx"
  },
  {
    "name": "tiny-mamba",
    "model": "Xenova/tiny-mamba-onnx",
    "onnx_path": "onnx/model.onnx"
  }
]
import React, {useState} from 'react';
import {
  StyleSheet,
  Text,
  Button,
  TextInput,
  SafeAreaView,
  View,
} from 'react-native';
import RNFS, {
  DownloadBeginCallbackResult,
  DownloadProgressCallbackResult,
} from 'react-native-fs';
import {Pipeline} from 'react-native-transformers';
import presets from './presets.json';

export default function App() {
  const [progress, setProgress] = useState<number>(0);
  const [input, setInput] = useState<string>('We love local LLM');
  const [output, setOutput] = useState<string>();

  const loadModel = async (preset: {
    name: string;
    model: string;
    onnx_path: string;
    options?: any;
  }) => {
    console.log('Loading model...');
    setProgress(0);

    try {
      await Pipeline.TextGeneration.init(preset.model, preset.onnx_path, {
        verbose: true,
        fetch: async (url: string) => {
          console.log('Downloading... ' + url);

          const localPath = `${RNFS.CachesDirectoryPath}/${url
            .split('/')
            .pop()!}`;

          if (await RNFS.exists(localPath)) {
            console.log('Model already downloaded at: ' + localPath);
            setProgress(1);
            return localPath;
          }

          const downloadResumable = RNFS.downloadFile({
            fromUrl: url,
            toFile: localPath,
            background: true,
            discretionary: true,
            cacheable: true,
            progress: (res: DownloadProgressCallbackResult) => {
              let progressPercent =
                (res.bytesWritten / res.contentLength) * 100;
              setProgress(progressPercent);
            },
          });

          await downloadResumable.promise;

          console.log('Downloaded model to ' + localPath);
          setProgress(1);
          return localPath;
        },
        ...preset.options,
      });

      console.log('Model loaded successfully');
    } catch (error) {
      console.error('Model loading failed:', error);
    }
  };

  const AutoComplete = () => {
    if (!input) return;
    Pipeline.TextGeneration.generate(input, setOutput);
  };

  return (
    <SafeAreaView style={styles.container}>
      <Text>Select a model</Text>
      {presets.map(preset => (
        <Button
          key={preset.name}
          title={preset.name}
          onPress={() => loadModel(preset)}
        />
      ))}
      <Text>Input: </Text>
      <TextInput
        value={input}
        onChangeText={setInput}
        style={styles.input}
        placeholder="Type something..."
      />
      <Text>Output: {output || 'No output yet'}</Text>

      {/* Display progress */}
      <View style={styles.progressContainer}>
        <Text>{`Download Progress: ${(progress * 100).toFixed(2)}%`}</Text>
      </View>

      <Button title="Run" onPress={AutoComplete} />
    </SafeAreaView>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    alignItems: 'center',
    justifyContent: 'center',
    padding: 16,
  },
  input: {
    borderWidth: 1,
    borderColor: 'gray',
    padding: 8,
    marginVertical: 8,
    width: '80%',
  },
  progressContainer: {
    marginVertical: 8,
  },
});

@lakpriya1s lakpriya1s added the bug Something isn't working label Oct 8, 2024
@lakpriya1s
Copy link
Author

Now I get ERROR Model loading failed: [Error: Can't load a model: failed to load model]

@PierreCapo
Copy link

@lakpriya1s same error here happening with Expo on my side: Error: Can't load a model: failed to load model

@daviddaytw
Copy link
Owner

Hi @lakpriya1s @PierreCapo,

Thank you so much for bringing this issue to my attention! I'll be diving into it this week.

Please keep in mind that the library is still in its early stages, so some hiccups like this are expected. Your feedback is very valuable, helping me make the project more stable every day!

Best,
David

@daviddaytw daviddaytw self-assigned this Oct 14, 2024
@PierreCapo
Copy link

Thanks for your work David 🙏

@daviddaytw
Copy link
Owner

Hi @lakpriya1s @PierreCapo,

I just checked the example app because I thought it might be due to some API changes on Huggingface or Onnx-runtime. It turns out it's working fine on my machine.

Screenshot 2024-10-27 at 5 13 15 PM

Since you said you're using it in your react-native app. Could you share a demo repository and the instructions on how you set up your environment so I can take a look?

@daviddaytw daviddaytw added this to the v1.0.0 milestone Dec 12, 2024
@Flamebamboo
Copy link

@lakpriya1s same error here happening with Expo on my side: Error: Can't load a model: failed to load model

Hey im also having the same issue have you found a solution for it? Thanks

@daviddaytw
Copy link
Owner

Hi @Flamebamboo,

I'm more than happy to help, but I cannot reproduce the error. However, one thing that comes to my mind that might caused the issue is loading the model while it has not yet been fully downloaded. Can you check if that's the case?

Best,
David

@daviddaytw daviddaytw added the help wanted Extra attention is needed label Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants