ayyy it works and passes, test, mypy, ruff, docs

.pre-commit-config.yaml

@@ -52,7 +52,7 @@ repos:
         files: '^\.github/dependabot\.ya?ml$'
 
   - repo: https://github.com/charliermarsh/ruff-pre-commit
-    rev: v0.8.6
+    rev: v0.9.1
     hooks:
       - id: ruff
         args: [--fix, --exit-non-zero-on-fix, --no-cache]

README.md

@@ -39,45 +39,44 @@ pip install neural-pipeline-search
 
 Using `neps` always follows the same pattern:
 
-1. Define a `run_pipeline` function capable of evaluating different architectural and/or hyperparameter configurations
+1. Define a `evaluate_pipeline` function capable of evaluating different architectural and/or hyperparameter configurations
    for your problem.
 1. Define a search space named `pipeline_space` of those Parameters e.g. via a dictionary
-1. Call `neps.run` to optimize `run_pipeline` over `pipeline_space`
+1. Call `neps.run(evaluate_pipeline, pipeline_space)`
 
 In code, the usage pattern can look like this:
 
 ```python
 import neps
 import logging
 
+logging.basicConfig(level=logging.INFO)
 
 # 1. Define a function that accepts hyperparameters and computes the validation error
-def run_pipeline(
-        hyperparameter_a: float, hyperparameter_b: int, architecture_parameter: str
-) -> dict:
+def evaluate_pipeline(lr: float, alpha: int, optimizer: str) -> float:
     # Create your model
-    model = MyModel(architecture_parameter)
+    model = MyModel(lr=lr, alpha=alpha, optimizer=optimizer)
 
     # Train and evaluate the model with your training pipeline
-    validation_error = train_and_eval(
-        model, hyperparameter_a, hyperparameter_b
-    )
+    validation_error = train_and_eval(model)
     return validation_error
 
 
-# 2. Define a search space of parameters; use the same parameter names as in run_pipeline
+# 2. Define a search space of parameters; use the same parameter names as in evaluate_pipeline
 pipeline_space = dict(
-    hyperparameter_a=neps.Float(
-        lower=0.001, upper=0.1, log=True  # The search space is sampled in log space
+    lr=neps.Float(
+        lower=1e-5,
+        upper=1e-1,
+        log=True,   # Log spaces
+        prior=1e-3, # Incorporate you knowledge to help optimization
     ),
-    hyperparameter_b=neps.Integer(lower=1, upper=42),
-    architecture_parameter=neps.Categorical(["option_a", "option_b"]),
+    alpha=neps.Integer(lower=1, upper=42),
+    optimizer=neps.Categorical(choices=["sgd", "adam"])
 )
 
 # 3. Run the NePS optimization
-logging.basicConfig(level=logging.INFO)
 neps.run(
-    run_pipeline=run_pipeline,
+    evaluate_pipeline=evaluate_pipeline,
     pipeline_space=pipeline_space,
     root_directory="path/to/save/results",  # Replace with the actual path.
     max_evaluations_total=100,
@@ -94,8 +93,6 @@ Discover how NePS works through these examples:
 
 - **[Utilizing Expert Priors for Hyperparameters](neps_examples/efficiency/expert_priors_for_hyperparameters.py)**: Learn how to incorporate expert priors for more efficient hyperparameter selection.
 
-- **[Architecture Search](neps_examples/basic_usage/architecture.py)**: Dive into (hierarchical) architecture search in NePS.
-
 - **[Additional NePS Examples](neps_examples/)**: Explore more examples, including various use cases and advanced configurations in NePS.
 
 ## Contributing

docs/_code/api_generator.py

@@ -3,7 +3,6 @@
 # https://mkdocstrings.github.io/recipes/
 """
 
-
 import logging
 from pathlib import Path
 

docs/getting_started.md

@@ -4,103 +4,59 @@ Getting started with NePS involves a straightforward yet powerful process, cente
 This approach ensures flexibility and efficiency in evaluating different architecture and hyperparameter configurations
 for your problem.
 
-NePS requires Python 3.8 or higher. You can install it via pip or from source.
+NePS requires Python 3.10 or higher.
+You can install it via `pip` or from [source](https://github.com/automl/neps/).
 
 ```bash
 pip install neural-pipeline-search
 ```
 
 ## The 3 Main Components
-1. **Execute with [`neps.run()`](./reference/neps_run.md)**:
-Optimize your `run_pipeline=` over the `pipeline_space=` using this function.
-For a thorough overview of the arguments and their explanations, check out the detailed documentation.
-
-2. **Define a [`run_pipeline=`](./reference/run_pipeline.md) Function**:
-This function is essential for evaluating different configurations.
-You'll implement the specific logic for your problem within this function.
-For detailed instructions on initializing and effectively using `run_pipeline=`, refer to the guide.
-
-3. **Establish a [`pipeline_space=`](./reference/pipeline_space.md)**:
-Your search space for defining parameters.
-You can structure this in various formats, including dictionaries, YAML, or ConfigSpace.
-The guide offers insights into defining and configuring your search space.
-
-By following these steps and utilizing the extensive resources provided in the guides, you can tailor NePS to meet
-your specific requirements, ensuring a streamlined and effective optimization process.
-
-## Basic Usage
-In code, the usage pattern can look like this:
-
+1. **Establish a [`pipeline_space=`](reference/pipeline_space.md)**:
 ```python
-import neps
-import logging
-
-
-def run_pipeline(  # (1)!
-        hyperparameter_a: float,
-        hyperparameter_b: int,
-        architecture_parameter: str,
-) -> dict:
-    # insert here your own model
-    model = MyModel(architecture_parameter)
-
-    # insert here your training/evaluation pipeline
-    validation_error, training_error = train_and_eval(
-        model, hyperparameter_a, hyperparameter_b
-    )
+pipeline_space={
+    "some_parameter": (0.0, 1.0),   # float
+    "another_parameter": (0, 10),   # integer
+    "optimizer": ["sgd", "adam"],   # categorical
+    "epoch": neps.Integer(lower=1, upper=100, is_fidelity=True),
+    "learning_rate": neps.Float(lower=1e-5, uperr=1, log=True),
+    "alpha": neps.Float(lower=0.1, upper=1.0, prior=0.99, prior_confidence="high")
+}
 
-    return {
-        "loss": validation_error,  # ! (2)
-        "info_dict": {
-            "training_error": training_error
-            # + Other metrics
-        },
-    }
+```
+2. **Define an `evaluate_pipeline()` function**:
 
+```python
+def evaluate_pipeline(some_parameter: float,
+                 another_parameter: float,
+                 optimizer: str, epoch: int,
+                 learning_rate: float, alpha: float) -> float:
+    model = make_model(...)
+    loss = eval_model(model)
+    return loss
+```
 
-pipeline_space = {  # (3)!
-    "hyperparameter_b": neps.Integer(1, 42, is_fidelity=True),  # ! (4)
-    "hyperparameter_a": neps.Float(1e-3, 1e-1, log=True)  # ! (5)
-    "architecture_parameter": neps.Categorical(["option_a", "option_b", "option_c"]),
-}
+1. **Execute with [`neps.run()`](reference/neps_run.md)**:
 
-if __name__ == "__main__":
-    logging.basicConfig(level=logging.INFO)
-    neps.run(
-        run_pipeline=run_pipeline,
-        pipeline_space=pipeline_space,
-        root_directory="path/to/save/results",  # Replace with the actual path.
-        max_evaluations_total=100,
-        optimizer="hyperband"  # Optional specifies optimizer to use
-        # otherwise NePs decides based on your data.
-    )
+```python
+neps.run(evaluate_pipeline, pipeline_space)
 ```
 
-1.  Define a function that accepts hyperparameters and computes the validation error.
-2.  Return a dictionary with the objective to minimize and any additional information.
-3.  Define a search space of the parameters of interest; ensure that the names are consistent with those defined in the run_pipeline function.
-4.  Use `is_fidelity=True` for a multi-fidelity approach.
-5.  Use `log=True` for a log-spaced hyperparameter.
+---
 
-!!! tip
-
-    Please visit the [full reference](./reference/neps_run.md) for a more comprehensive walkthrough of defining budgets,
-    optimizers, YAML configuration, parallelism, and more.
+You can find a longer walk through in the [reference](reference/neps_run.md)!
 
 ## Examples
 Discover the features of NePS through these practical examples:
 
-* **[Hyperparameter Optimization (HPO)](./examples/template/basic_template.md)**:
+* **[Hyperparameter Optimization (HPO)](examples/template/basic.md)**:
 Learn the essentials of hyperparameter optimization with NePS.
 
-* **[Architecture Search with Primitives](./examples/basic_usage/architecture.md)**:
-Dive into architecture search using primitives in NePS.
-
-* **[Multi-Fidelity Optimization](./examples/efficiency/multi_fidelity.md)**:
+* **[Multi-Fidelity Optimization](examples/efficiency/multi_fidelity.md)**:
 Understand how to leverage multi-fidelity optimization for efficient model tuning.
 
-* **[Utilizing Expert Priors for Hyperparameters](./examples/template/priorband_template.md)**:
+* **[Utilizing Expert Priors for Hyperparameters](examples/template/priorband.md)**:
 Learn how to incorporate expert priors for more efficient hyperparameter selection.
 
-* **[Additional NePS Examples](./examples/index.md)**:
+* **[Additional NePS Examples](examples/index.md)**:
 Explore more examples, including various use cases and advanced configurations in NePS.

docs/index.md

@@ -2,7 +2,7 @@
 
 [![PyPI version](https://img.shields.io/pypi/v/neural-pipeline-search?color=informational)](https://pypi.org/project/neural-pipeline-search/)
 [![Python versions](https://img.shields.io/pypi/pyversions/neural-pipeline-search)](https://pypi.org/project/neural-pipeline-search/)
-[![License](https://img.shields.io/pypi/l/neural-pipeline-search?color=informational)](LICENSE)
+[![License](https://img.shields.io/pypi/l/neural-pipeline-search?color=informational)](https://github.com/automl/neps/blob/master/LICENSE)
 [![Tests](https://github.com/automl/neps/actions/workflows/tests.yaml/badge.svg)](https://github.com/automl/neps/actions)
 
 Welcome to NePS, a powerful and flexible Python library for hyperparameter optimization (HPO) and neural architecture search (NAS) with its primary goal: **make HPO and NAS usable for deep learners in practice**.
@@ -30,9 +30,9 @@ In addition to the features offered by traditional HPO and NAS libraries, NePS s
     Check out:
 
     * [Reference documentation](./reference/neps_run.md) for a quick overview.
-    * [API](./api/neps/api.md) for a more detailed reference.
+    * [API](api/neps/api.md) for a more detailed reference.
     * [Colab Tutorial](https://colab.research.google.com/drive/11IOhkmMKsIUhWbHyMYzT0v786O9TPWlH?usp=sharing) walking through NePS's main features.
-    * [Examples](./examples) for basic code snippets to get started.
+    * [Examples](examples/index.md) for basic code snippets to get started.
 
 ## Installation
 
@@ -46,10 +46,10 @@ pip install neural-pipeline-search
 
 Using `neps` always follows the same pattern:
 
-1. Define a `run_pipeline` function capable of evaluating different architectural and/or hyperparameter configurations
+1. Define a `evalute_pipeline` function capable of evaluating different architectural and/or hyperparameter configurations
    for your problem.
 1. Define a search space named `pipeline_space` of those Parameters e.g. via a dictionary
-1. Call `neps.run` to optimize `run_pipeline` over `pipeline_space`
+1. Call `neps.run` to optimize `evalute_pipeline` over `pipeline_space`
 
 In code, the usage pattern can look like this:
 
@@ -59,7 +59,7 @@ import logging
 
 
 # 1. Define a function that accepts hyperparameters and computes the validation error
-def run_pipeline(
+def evalute_pipeline(
         hyperparameter_a: float, hyperparameter_b: int, architecture_parameter: str
 ) -> dict:
     # Create your model
@@ -72,7 +72,7 @@ def run_pipeline(
     return validation_error
 
 
-# 2. Define a search space of parameters; use the same parameter names as in run_pipeline
+# 2. Define a search space of parameters; use the same parameter names as in evalute_pipeline
 pipeline_space = dict(
     hyperparameter_a=neps.Float(
         lower=0.001, upper=0.1, log=True  # The search space is sampled in log space
@@ -84,7 +84,7 @@ pipeline_space = dict(
 # 3. Run the NePS optimization
 logging.basicConfig(level=logging.INFO)
 neps.run(
-    run_pipeline=run_pipeline,
+    evalute_pipeline=evalute_pipeline,
     pipeline_space=pipeline_space,
     root_directory="path/to/save/results",  # Replace with the actual path.
     max_evaluations_total=100,
@@ -95,20 +95,18 @@ neps.run(
 
 Discover how NePS works through these examples:
 
-- **[Hyperparameter Optimization](./examples/basic_usage/hyperparameters.py)**: Learn the essentials of hyperparameter optimization with NePS.
+- **[Hyperparameter Optimization](examples/basic_usage/hyperparameters.md)**: Learn the essentials of hyperparameter optimization with NePS.
 
-- **[Multi-Fidelity Optimization](./examples/efficiency/multi_fidelity.py)**: Understand how to leverage multi-fidelity optimization for efficient model tuning.
+- **[Multi-Fidelity Optimization](examples/efficiency/multi_fidelity.md)**: Understand how to leverage multi-fidelity optimization for efficient model tuning.
 
-- **[Utilizing Expert Priors for Hyperparameters](./examples/efficiency/expert_priors_for_hyperparameters.py)**: Learn how to incorporate expert priors for more efficient hyperparameter selection.
+- **[Utilizing Expert Priors for Hyperparameters](examples/efficiency/expert_priors_for_hyperparameters.md)**: Learn how to incorporate expert priors for more efficient hyperparameter selection.
 
-- **[Architecture Search](./examples/basic_usage/architecture.py)**: Dive into (hierarchical) architecture search in NePS.
-
-- **[Additional NePS Examples](./examples/)**: Explore more examples, including various use cases and advanced configurations in NePS.
+- **[Additional NePS Examples](examples/index.md)**: Explore more examples, including various use cases and advanced configurations in NePS.
 
 ## Contributing
 
-Please see the [documentation for contributors](./dev_docs/contributing/).
+Please see the [documentation for contributors](dev_docs/contributing.md).
 
 ## Citations
 
-For pointers on citing the NePS package and papers refer to our [documentation on citations](./citations.md).
+For pointers on citing the NePS package and papers refer to our [documentation on citations](citations.md).

docs/reference/analyse.md

@@ -77,7 +77,7 @@ NePS will also generate a summary CSV file for you.
 
 
 The `config_data.csv` contains all configuration details in CSV format, ordered by ascending `loss`.
-Details include configuration hyperparameters, any returned result from the `run_pipeline` function, and metadata information.
+Details include configuration hyperparameters, any returned result from the `evalute_pipeline` function, and metadata information.
 
 The `run_status.csv` provides general run details, such as the number of sampled configs, best configs, number of failed configs, best loss, etc.
 
@@ -122,7 +122,7 @@ tblogger.log(
 
 !!! tip
 
-    The logger function is primarily designed for use within the `run_pipeline` function during the training of the neural network.
+    The logger function is primarily designed for use within the `evalute_pipeline` function during the training of the neural network.
 
 ??? example "Quick Reference"
 

docs/reference/cli.md

@@ -32,17 +32,17 @@ Executes the optimization based on the provided configuration. This command serv
 
 - `-h, --help` (Optional): show this help message and exit
 - `--run-args` (Optional): Path to the YAML configuration file.
-- `--run-pipeline` (Optional): Optional: Provide the path to a Python file and a function name separated by a colon, e.g., 'path/to/module.py:function_name'. If provided, it overrides the run_pipeline setting from the YAML configuration.
+- `--evaluate-pipeline` (Optional): Optional: Provide the path to a Python file and a function name separated by a colon, e.g., 'path/to/module.py:function_name'. If provided, it overrides the evaluate_pipeline setting from the YAML configuration.
 - `--pipeline-space` (Optional): Path to the YAML file defining the search space for the optimization. This can be provided here or defined within the 'run_args' YAML file.
 - `--root-directory` (Optional): The directory to save progress to. This is also used to synchronize multiple calls for parallelization.
-- `--overwrite-working-directory` (Optional): If set, deletes the working directory at the start of the run. This is useful, for example, when debugging a run_pipeline function.
+- `--overwrite-working-directory` (Optional): If set, deletes the working directory at the start of the run. This is useful, for example, when debugging a evalute_pipeline function.
 - `--post-run-summary` (Optional): Provide a summary of the results after running.
 - `--no-post-run-summary` (Optional): Do not provide a summary of the results after running.
 - `--max-evaluations-total` (Optional): Total number of evaluations to run.
 - `--max-evaluations-per-run` (Optional): Number of evaluations a specific call should maximally do.
 - `--continue-until-max-evaluation-completed` (Optional): If set, only stop after max-evaluations-total have been completed. This is only relevant in the parallel setting.
 - `--max-cost-total` (Optional): No new evaluations will start when this cost is exceeded. Requires returning a cost
-  in the run_pipeline function.
+  in the evalute_pipeline function.
 - `--ignore-errors` (Optional): If set, ignore errors during the optimization process.
 - `--loss-value-on-error` (Optional): Loss value to assume on error.
 - `--cost-value-on-error` (Optional): Cost value to assume on error.

docs/reference/declarative_usage.md

@@ -11,7 +11,7 @@ Please check [`neps.run()`][neps.api.run] for complete information on the argume
 Below is a YAML configuration example for NePS covering the required arguments.
 The arguments match those that you can pass to [`neps.run()`][neps.api.run].
 
-In general, you can encode any of the [parameters][neps.space.parameters] into a YAML format.
+In general, you can encode any [`Parameter`][neps.space.Parameter] into a YAML format.
 
 === "config.yaml"
 
@@ -52,17 +52,17 @@ In general, you can encode any of the [parameters][neps.space.parameters] into a
     import neps
     import yaml
 
-    def run_pipeline(learning_rate, optimizer, epochs, batch_size):
+    def evaluate_pipeline(learning_rate, optimizer, epochs, batch_size):
         model = initialize_model()
         training_loss = train_model(model, optimizer, learning_rate, epochs)
         evaluation_loss = evaluate_model(model)
-        return {"loss": evaluation_loss, "training_loss": training_loss}
+        return {"objective_value_to_minimize": evaluation_loss, "training_loss": training_loss}
 
     if __name__ == "__main__":
         with open("path/config.yaml") as f:
             settings = yaml.safe_load(f)
 
-        neps.run(run_pipeline, **settings)
+        neps.run(evaluate_pipeline, **settings)
     ```
 
 !!! tip "Merging multiple yaml files"
@@ -74,12 +74,12 @@ In general, you can encode any of the [parameters][neps.space.parameters] into a
     ```python
     import neps
 
-    def run_pipeline(...):
+    def evaluate_pipeline(...):
         ...
 
     if __name__ == "__main__":
         settings = neps.load_yamls("path/to/your/config.yaml", "path/to/your/optimizer.yaml")
-        neps.run(run_pipeline, **settings)
+        neps.run(evaluate_pipeline, **settings)
     ```