Skip to content

Commit

Permalink
Merge branch 'main' into 670-type-safe-entry-parsing
Browse files Browse the repository at this point in the history
  • Loading branch information
bjthehun authored Mar 22, 2024
2 parents 46e5da0 + 8758433 commit af8b0f3
Show file tree
Hide file tree
Showing 9 changed files with 22 additions and 20 deletions.
2 changes: 1 addition & 1 deletion cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"stats-helper": "ts-node src/statistics-helper-app.ts",
"slicer": "ts-node src/slicer-app.ts",
"benchmark-helper": "ts-node src/benchmark-helper-app.ts",
"benchmark": "npm run build && node dist/src/benchmark-app.js",
"benchmark": "ts-node src/benchmark-app.ts",
"summarizer": "ts-node src/summarizer-app.ts",
"export-quads": "ts-node src/export-quads-app.ts",
"build": "tsc --project ."
Expand Down
2 changes: 1 addition & 1 deletion cli/src/flowr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import { standardReplOutput } from './repl/commands'
import type { Server } from './repl/server/net'
import { NetServer, WebSocketServerWrapper } from './repl/server/net'
import { defaultConfigFile, setConfigFile } from '@eagleoutice/flowr/config'
import { flowrVersion } from '../../src/util/version'
import { flowrVersion } from '@eagleoutice/flowr/util/version'
import { version } from '../package.json'

const scriptsText = Array.from(Object.entries(scripts).filter(([, { type }]) => type === 'master script'), ([k,]) => k).join(', ')
Expand Down
2 changes: 1 addition & 1 deletion cli/src/repl/commands/version.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { ReplCommand, ReplOutput } from './main'
import { RShell } from '@eagleoutice/flowr/r-bridge'
import { guard } from '@eagleoutice/flowr/util/assert'
import { flowrVersion } from '../../../../src/util/version'
import { flowrVersion } from '@eagleoutice/flowr/util/version'

type Version = `${number}.${number}.${number}`

Expand Down
3 changes: 1 addition & 2 deletions scripts/extract-all-stats.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,7 @@ for ((i = 0; i < ${#CONTAINED[@]}; i += size)); do

mkdir -p "${dirBasis}/${chunkNumber}/"
# shellcheck disable=SC2068
npm run stats -- ${chunk[@]} --output-dir "${dirBasis}/${chunkNumber}/" > "${dirBasis}/${chunkNumber}/full.log" 2>&1 &
npm run stats --prefix ../cli -- ${chunk[@]} --output-dir "${dirBasis}/${chunkNumber}/" > "${dirBasis}/${chunkNumber}/full.log" 2>&1 &

# printf '%s\n' "${chunk[@]}"
done

3 changes: 1 addition & 2 deletions scripts/extract-top-stats.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ done

DIR="./statistics-out/top-$(date +%Y-%m-%d-%H-%M-%S)/"
mkdir -p "${DIR}"
npm run stats -- "${arr[@]}" --output-dir "${DIR}" |& tee -a "./full.log"

npm run stats --prefix ../cli -- "${arr[@]}" --output-dir "${DIR}" |& tee -a "./full.log"
4 changes: 2 additions & 2 deletions test/performance/run-suite.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,14 @@ echo "done."
FILES_DIR="$(pwd)/files/"

## run the benchmark script for each file
CMD=(npm run benchmark -- --parallel "${PARALLEL}" --runs "${RUNS}" --output "${RAW_OUTPUT}" "${FILES_DIR}")
CMD=(npm run benchmark --prefix ../../../cli -- --parallel "${PARALLEL}" --runs "${RUNS}" --output "${RAW_OUTPUT}" "${FILES_DIR}")

echo -e " * Running: \"${CMD[*]}\"...\033[33m"
"${CMD[@]}"
echo -e "\033[0m * Done (written to ${RAW_OUTPUT})."
echo " * Summarizing results to ${OUTPUT_FILE}${ULTIMATE_SUMMARY_SUFFIX}..."

CMD=(npm run summarizer -- --input "${RAW_OUTPUT}" --output "${OUTPUT_FILE}" --graph)
CMD=(npm run summarizer --prefix ../../../cli -- --input "${RAW_OUTPUT}" --output "${OUTPUT_FILE}" --graph)
echo -e " * Running: \"${CMD[*]}\"...\033[33m"
"${CMD[@]}"
echo -e "\033[0m * Done (written to ${OUTPUT_FILE}${ULTIMATE_SUMMARY_SUFFIX})."
Expand Down
18 changes: 10 additions & 8 deletions wiki/Overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ To generate a slice you need to provide two things:
1. A [slicing criterion](https://github.com/Code-Inspect/flowr/wiki/Terminology#slicing-criterion): the location of a single variable or several variables of interest to slice for, like "`12@product`"
2. The path to an R&nbsp;file that should be sliced.

For example, you can run
For example, from the `cli` directory, you can run

```shell
npm run slicer -- --criterion "12@product" "test/testfiles/example.R"
Expand All @@ -150,7 +150,7 @@ npm run slicer -- --criterion "12@product" "test/testfiles/example.R"
This slices for the first use of the variable `product` in line&nbsp;12 of the source file at `test/testfiles/example.R` (see the [slicing criterion](https://github.com/Code-Inspect/flowr/wiki/Terminology#slicing-criterion) definition for more information).
By default, the resulting slice is output to the standard output.

For more options see,
For more options, run the following from the `cli` directory:

```shell
npm run slicer -- --help
Expand All @@ -172,13 +172,13 @@ Within the original [thesis](https://github.com/Code-Inspect/flowr/wiki/Thesis),
The corresponding *benchmark* script ultimately allows to do the same thing as the *slicing* script, but 1) parallel for many files and 2) for a wider selection of slicing points.
By default, it starts by collecting all variables in a script, producing a slice for each of them.

For example, to run the benchmark on 500 randomly picked files of the folder `<folder>` using 8 threads and writing the output to `<output.json>`, you can use:
For example, to run the benchmark on 500 randomly picked files of the folder `<folder>` using 8 threads and writing the output to `<output.json>`, you can run this from the `cli` directory:

```shell
npm run benchmark -- --limit 500 --parallel 8 --output "<output.json>" "<folder>"
```

For more options see,
For more options, run the following from the `cli` directory:

```shell
npm run benchmark -- --help
Expand All @@ -187,7 +187,7 @@ npm run benchmark -- --help
#### Summarizing the Benchmark Results

The resulting JSON file can be rather larger (starting off with a couple of hundred megabytes). Therefore, you probably want to summarize the results of the benchmark.
For this, you can make use of the *summarizer* script like this:
For this, you can make use of the *summarizer* script from within the `cli` directory like this:

```shell
npm run summarizer -- "<output.json>"
Expand All @@ -200,7 +200,7 @@ Please note that the summarizer may require a long time as it parses, normalizes

As the ultimate summary is much quicker, you can re-run it by specifically adding the `--ultimate-only` flag (although this is only really of use if you modify what should be summarized within the source code of *flowR*).

For more options see,
For more options, run the following from the `cli` directory:

```shell
npm run summarizer -- --help
Expand All @@ -210,15 +210,17 @@ npm run summarizer -- --help

If you want to reproduce the statistics as presented in my [master's thesis](http://dx.doi.org/10.18725/OPARU-50107), see the corresponding [wiki page](https://github.com/Code-Inspect/flowr/wiki/Thesis#how-to-reproduce-the-statistics-from-the-masters-thesis).

See
For more information, run the following from the `cli` directory:

```shell
npm run stats -- --help
```

### Export RDF N-Quads

If you know what the RDF N-Quads refer to, then you are good to go! If not, you do not have to worry - you probably do not need them (they are for a graph search that is based on *flowR*). See the following for more information:
If you know what the RDF N-Quads refer to, then you are good to go! If not, you do not have to worry - you probably do not need them (they are for a graph search that is based on *flowR*).

For more information, run the following from the `cli` directory:

```shell
npm run export-quads -- --help
Expand Down
2 changes: 1 addition & 1 deletion wiki/Setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ After cloning the repository,<a href="#note2" id="note2ref"><sup>&lt;2&gt;</sup>
npm i
```

After that, you should be fine! You may test *flowR* by running:
After that, you should be fine! You may test *flowR*'s command-line interface by running the following from the `cli` directory:

```shell
npm run slicer -- --criterion "12@product" test/testfiles/example.R
Expand Down
6 changes: 4 additions & 2 deletions wiki/Thesis.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,13 @@ The (sorted) results should be versioned and can be found in [scripts/top-r-down
```

4. Now you can run the statistics program on the downloaded sources.
You can do this in two ways (check `npm run stats -- --help` for more information).
You can do this in two ways (run `npm run stats -- --help` from the `cli` directory for more information).
In any case, the extraction may take a long time, so be prepared for that!
Furthermore, you may want to store the output of the tool as it provides additional information.

- **On the complete folder**

First, make sure you are currently in the `cli` directory to allow running command-line interface commands.

```shell
npm run stats -- --input <location-of-source folders> --output-dir <output-dir>
Expand All @@ -64,7 +66,7 @@ The (sorted) results should be versioned and can be found in [scripts/top-r-down
of course limited to only those files that have been processed so far.

5. Afterward, your output folder should contain several folders with the recorded stats of all extracted features.
To make sense of them, you need to use the post-processor, which prints the summarized information to the command-line:
To make sense of them, you need to use the command-line interface's post-processor, which prints the summarized information to the command-line:
```shell
npm run stats -- --post-process "./statistics-out/cran-500" --no-ansi > "./statistics-out/cran-500/cran-500-summary.log"
Expand Down

0 comments on commit af8b0f3

Please sign in to comment.