Add Doc Strings to Config Files #465

ParagEkbote · 2024-12-19T10:24:01Z

As discussed in #247, I've added docstrings to the config class of the required files and updated some of them for enhanced clarity.

Please let me know if any further corrections are needed and I will make the necessary changes.

partially fixes #247

cc: @clefourrier

…gEkbote/lighteval into Document-Custom-Model-Files

src/lighteval/models/transformers/adapter_model.py

examples/model_configs/base_model.yaml

clefourrier

Very nice first draft!
In general, you need to homogeneize a bit (and remove useless words, I explained in the first class but you can apply it to all :)

examples/model_configs/serverless_model.yaml

examples/model_configs/quantized_model.yaml

examples/model_configs/peft_model.yaml

examples/model_configs/base_model.yaml

src/lighteval/models/endpoints/endpoint_model.py

src/lighteval/models/endpoints/tgi_model.py

src/lighteval/models/transformers/adapter_model.py

…gEkbote/lighteval into Document-Custom-Model-Files

Co-authored-by: Clémentine Fourrier <[email protected]>

…gEkbote/lighteval into Document-Custom-Model-Files

NathanHB

thanks for the doc ! Only need to adress a the few comment and this will be good to merge :)

examples/model_configs/quantized_model.yaml

examples/model_configs/serverless_model.yaml

Co-authored-by: Nathan Habib <[email protected]>

HuggingFaceDocBuilderDev · 2024-12-31T10:31:09Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

albertvillanova

The syntax of the docstrings is not right. Please note, that the docstring parser is very strict with the syntax conventions: tabs, colons, parentheses,...

You have the syntax examples in the docs-builder repo: https://github.com/huggingface/doc-builder

Also the formatter raises errors: please run make style to align with the linter/formatter conventions.

…gEkbote/lighteval into Document-Custom-Model-Files

ParagEkbote · 2024-12-31T11:30:02Z

I will update this PR as draft to fix the syntax errors and update the docstrings correctly as mentioned before.

cc: @albertvillanova

ParagEkbote · 2025-01-11T16:04:53Z

Could you please review the changes?

cc: @albertvillanova

clefourrier · 2025-01-23T07:57:16Z

src/lighteval/models/transformers/adapter_model.py

@@ -27,7 +27,7 @@
 import torch
 from transformers import AutoModelForCausalLM, PreTrainedTokenizer

-from lighteval.models.transformers.transformers_model import TransformersModel, TransformersModelConfig
+from lighteval.models.transformers.base_model import BaseModel, BaseModelConfig


This path is incorrect

clefourrier · 2025-01-23T07:57:48Z

src/lighteval/models/transformers/adapter_model.py

+    """
+    Integrates the adapter models with a pre-trained base model.
+    """


To be informative, it would be good to explain what adapter weights are.

clefourrier · 2025-01-23T07:59:00Z

src/lighteval/models/transformers/adapter_model.py

+        Args:
+            config(AdapterModelConfig): An instance of AdapterModelConfig.
+            env_config(EnvConfig): An instance of EnvConfig.
+
+        Returns: AutoModelForCasualLM


This is not adding new information, you should remove it

clefourrier · 2025-01-23T07:59:19Z

src/lighteval/models/transformers/delta_model.py

+    """
+    This class is used to manage the configuration class for delta models.
+    """


Explain what delta weights are

clefourrier · 2025-01-23T07:59:36Z

src/lighteval/models/transformers/delta_model.py

+         Args:
+        config(AdapterModelConfig): An instance of AdapterModelConfig.
+        env_config(EnvConfig): An instance of EnvConfig.
+
+        Returns: AutoModelForCasualLM


The args and returns are not adding new ifnformation, remove

clefourrier · 2025-01-23T08:00:26Z

src/lighteval/models/transformers/adapter_model.py

+            env_configs(EnvConfig): An instance of EnvConfig.
+
+        Returns:
+            Any: Result of the configuration initialization.


This is not informative, expand or remove

clefourrier · 2025-01-23T08:00:54Z

src/lighteval/models/endpoints/tgi_model.py

+
+    Attributes:
+    inference_server_address (str, required):Endpoint address of the inference server hosting the model.
+    inference_server_auth (str, required): Authentication credentials or tokens required to access the server.


I think it should only be a token

clefourrier · 2025-01-23T08:01:42Z

src/lighteval/models/endpoints/openai_model.py

@@ -251,7 +284,7 @@ def _loglikelihood_tokens(
            ), "Only single token continuations are supported when using openai API."

            for i in range(len(dataset)):
-                logit_bias = {tok: 100 for tok in dataset[i].tokenized_continuation}
+                logit_bias = dict.fromkeys(dataset[i].tokenized_continuation, 100)


Please leave initial formulation

clefourrier · 2025-01-23T08:02:00Z

src/lighteval/models/endpoints/openai_model.py

            override_bs (int, optional): Override the batch size for generation. Defaults to None.

        Returns:
-            list[GenerativeResponse]: list of generated responses.
+            list [GenerativeResponse]: list of generated responses.


Remove extra space

clefourrier · 2025-01-23T08:03:09Z

examples/model_configs/transformers_model.yaml

@@ -1,6 +1,6 @@
 model:
  base_params:
-    model_args: "pretrained=HuggingFaceTB/SmolLM-1.7B,revision=main" # pretrained=model_name,trust_remote_code=boolean,revision=revision_to_use,model_parallel=True ...
+    model_args: "pretrained=HuggingFaceH4/zephyr-7b-beta,revision=main" # pretrained=model_name,trust_remote_code=boolean,revision=revision_to_use,model_parallel=True ...


Don't change the default model

davidberenstein1957 · 2025-01-23T15:45:23Z

@ParagEkbote great job!

ParagEkbote added 4 commits December 19, 2024 10:19

Add docstrings and comments.

9334097

Merge branch 'huggingface:main' into Document-Custom-Model-Files

72534f6

Add docstrings for config class.

f35ad57

Merge branch 'Document-Custom-Model-Files' of https://github.com/Para…

e9d7bb0

…gEkbote/lighteval into Document-Custom-Model-Files

ParagEkbote commented Dec 20, 2024

View reviewed changes

src/lighteval/models/transformers/adapter_model.py Show resolved Hide resolved

ParagEkbote commented Dec 20, 2024

View reviewed changes

examples/model_configs/base_model.yaml Outdated Show resolved Hide resolved

ParagEkbote marked this pull request as ready for review December 20, 2024 10:18

ParagEkbote changed the title ~~Add Documentation to Config Files~~ Add Doc Strings to Config Files Dec 20, 2024

Merge branch 'main' into Document-Custom-Model-Files

291a92c

clefourrier reviewed Dec 26, 2024

View reviewed changes

ParagEkbote and others added 6 commits December 26, 2024 21:23

Merge branch 'main' into Document-Custom-Model-Files

4f25938

Add proper spacing between comments.

bf50f2d

Merge branch 'Document-Custom-Model-Files' of https://github.com/Para…

1a96a1f

…gEkbote/lighteval into Document-Custom-Model-Files

Re-write comments as per review

3ca0960

Co-authored-by: Clémentine Fourrier <[email protected]>

Update spacing in yaml file.

bf65e27

Merge branch 'Document-Custom-Model-Files' of https://github.com/Para…

9b753af

…gEkbote/lighteval into Document-Custom-Model-Files

NathanHB reviewed Dec 30, 2024

View reviewed changes

examples/model_configs/quantized_model.yaml Outdated Show resolved Hide resolved

examples/model_configs/serverless_model.yaml Outdated Show resolved Hide resolved

Re-write comments as per review-2

caae117

Co-authored-by: Nathan Habib <[email protected]>

albertvillanova requested changes Dec 31, 2024

View reviewed changes

ParagEkbote added 2 commits December 31, 2024 10:45

Make style.

095f277

Merge branch 'Document-Custom-Model-Files' of https://github.com/Para…

b10f9da

…gEkbote/lighteval into Document-Custom-Model-Files

ParagEkbote marked this pull request as draft December 31, 2024 11:30

ParagEkbote added 5 commits January 9, 2025 16:12

make style.

e3f6f13

styling improvements.

79a3e35

Update Docstrings and fix formatting.

4f7ee64

Merge branch 'main' of https://github.com/ParagEkbote/lighteval

dda8266

Update docstrings to files.

5d225b3

ParagEkbote marked this pull request as ready for review January 11, 2025 16:00

ParagEkbote requested a review from albertvillanova January 11, 2025 16:06

ParagEkbote added 2 commits January 17, 2025 21:25

Merge branch 'main' into Document-Custom-Model-Files

262b1cd

Merge branch 'main' into Document-Custom-Model-Files

4c33274

clefourrier approved these changes Jan 23, 2025

View reviewed changes

Merge branch 'main' into Document-Custom-Model-Files

5815119

clefourrier requested changes Jan 23, 2025

View reviewed changes

ParagEkbote added 2 commits January 23, 2025 21:36

Merge branch 'main' into Document-Custom-Model-Files

73af85b

Merge branch 'main' into Document-Custom-Model-Files

ab38bd5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Doc Strings to Config Files #465

Add Doc Strings to Config Files #465

ParagEkbote commented Dec 19, 2024 •

edited

Loading

clefourrier left a comment

NathanHB left a comment

HuggingFaceDocBuilderDev commented Dec 31, 2024

albertvillanova left a comment •

edited

Loading

ParagEkbote commented Dec 31, 2024

ParagEkbote commented Jan 11, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

clefourrier Jan 23, 2025

davidberenstein1957 commented Jan 23, 2025

Add Doc Strings to Config Files #465

Are you sure you want to change the base?

Add Doc Strings to Config Files #465

Conversation

ParagEkbote commented Dec 19, 2024 • edited Loading

clefourrier left a comment

Choose a reason for hiding this comment

NathanHB left a comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Dec 31, 2024

albertvillanova left a comment • edited Loading

Choose a reason for hiding this comment

ParagEkbote commented Dec 31, 2024

ParagEkbote commented Jan 11, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidberenstein1957 commented Jan 23, 2025

ParagEkbote commented Dec 19, 2024 •

edited

Loading

albertvillanova left a comment •

edited

Loading