Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zip up Rule directories and unzip on demand #252

Open
NSoiffer opened this issue Feb 14, 2024 · 3 comments
Open

Zip up Rule directories and unzip on demand #252

NSoiffer opened this issue Feb 14, 2024 · 3 comments

Comments

@NSoiffer
Copy link
Owner

Each language file is about 500k. As the number of languages gets bigger, this is means a substantial size increase for MathCAT.

There is a huge amount of compression that can happen when these are zipped up. Typically, only one or two languages will get used, so zipping up each language directory can save a lot of space.

Implementation ideas:

  • Directory structure is Languages/xx/xx.zip and Braille/yy/yy.zip
  • Need some way to know the Speech Styles that are supported. Maybe add a ._Rules.txt file that has the rules listed, one per line (SimpleSpeech, ClearSpeak).
  • When SetPreference sets a language, it unzips the file for language if the files don't exist. This moves the unzip time to when the preference is set (probably via the user saying "ok" or "apply") from the time it needs the Rule files. It also means the up-to-date checks don't need to change.
  • Potentially I can get rid of build.rs so that rule changes are quick during development. The zipping could move to a github action and/or shell script.
  • The WASM code probably needs to change
@brichwin
Copy link
Contributor

I can see this is at least partially implemented. Should I be able to set the Language preference from a web assembly and then process MathML?

I've made a stab at creating a basic wasm/javascript interface: https://github.com/brichwin/MathCATForWeb/

If I try to set the Language preference to en (or other value like es), the set_preference call executes w/o error and other calls to set_preferences execute. However, a call to set_mathml will fail with:
[Error] Error processing MathML: – "Couldn't open zip file Rules/Languages/en/Auto.zip: Didn't find Auto.zip in dir Rules/Languages/en in zip archive." (anonymous function) (index.html:39)

If I don't set the Language preference then calls to set_mathml, get_spoken_text, and get_braille work as expected.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>MathCAT Web</title>
</head>
<body>
    <script type="module">
        import init, { wasm_set_rules_dir, wasm_get_version, wasm_set_preference, wasm_set_mathml, wasm_get_spoken_text, wasm_get_braille } from './pkg/mathcat_web.js';

        async function set_mathcat_pref(pref, value) {
          try {
            await wasm_set_preference(pref, value);
            console.log(`${pref} Preference set to ${value}`);
          } catch (error) {
            console.error(`Error setting ${pref} preference to ${value}: ${error}`);
          }
        }

        async function run() {
            await init();

            console.log('MathCAT Version:', wasm_get_version());

            try {
                await wasm_set_rules_dir('Rules');
                console.log('Rules directory set');

                await set_mathcat_pref('Language', 'en');
                await set_mathcat_pref('SpeechStyle', 'ClearSpeak');

                try {
                    const canonicalMathML = await wasm_set_mathml(`<math><mrow><mi>x</mi><mo>=</mo><msup><mi>y</mi><mn>2</mn></msup></mrow></math>`);
                    console.log('Canonical MathML:', canonicalMathML);
                    const spokenText = await wasm_get_spoken_text();
                    console.log('Spoken Text:', spokenText);
                } catch (error) {
                    console.error('Error processing MathML:', error);
                }

                try {
                    const braille = await wasm_get_braille('');
                    console.log('Braille:', braille);
                } catch (error) {
                    console.error('Error getting braille:', error);
                }

            } catch (error) {
                console.error('Error setting rules directory:', error);
                console.error('Cannot continue, rules directory must be set before calling other MathCAT functions.')
            }
        }

        run();
    </script>
</body>
</html>

@brichwin
Copy link
Contributor

brichwin commented Jul 20, 2024

Ah, okay. I'm a little bit more awake now and realized it doesn't look like my calls to set_preference are working appropriately.

  • If try setting preferences to ClearSpeak Verbose, I still get speech text as if on ClearSpeak Medium.
  • If I try setting a bogus preference name, then I get an appropriate unknown MathCAT preference error
  • If I try setting SpeechStyle to SimpleSpeak, then I get the Didn't find auto.zip error again when calling set_mathml

Since I can compile the MathCATDemo project and that works I'm guessing that my MathCATForWeb isn't configuring the build of MathCAT correctly to trigger all the right conditional compiles.

@NSoiffer
Copy link
Owner Author

Looks like I didn't check in the latest thing I found I could do in Cargo.toml: I think you want to add
[patch.crates-io]
MathCAT= { path = "../path/to/MathCAT" }

Because I didn't publish the latest version which has the zip file fixes, you can't use directly refer to 0.6.4 from another project. But this "patch" feature allows you to get around that by telling Rust not to looking in crates.io for a published version. I'm not 100% sure I have the syntax right. I copied and modified it from some Rust documentation. That might help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants