-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
added the download model from url feature #2423
Open
codezeros
wants to merge
9
commits into
lllyasviel:main
Choose a base branch
from
codezeros:download_model_from_url_feature
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
d7ad69f
added the download_model_from_url_feature
codezeros 47eddef
Merge branch 'main' into download_model_from_url_feature
codezeros af13c46
removed unwanted comments and reduced codebase simpler
codezeros b07057a
fixed the codebase formatted
codezeros 8421440
fixed the codebase formatted remaining
codezeros eb35281
added disable download tab using argument
codezeros 4820f31
fixed for previous commit
codezeros 56d0af5
added translations to en.json
codezeros 35ec483
Merge remote-tracking branch 'refs/remotes/origin/main' into download…
codezeros File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -51,4 +51,6 @@ user_path_config-deprecated.txt | |
/package-lock.json | ||
/.coverage* | ||
/auth.json | ||
/model_config_path.json | ||
.DS_Store | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# download_models.py | ||
from modules.model_loader import load_file_from_url | ||
from modules.shared_module import read_model_config_path | ||
|
||
def download_models(url, selected, file_name=None): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this can be migrated into modules/util.py |
||
""" | ||
This function downloads models from a given URL and saves them to a specified path. | ||
|
||
'url' is the URL from which the model will be downloaded. | ||
|
||
'selected' is the key to get the path from the 'model_paths' dictionary where the downloaded model will be saved. | ||
|
||
'file_name' is an optional parameter. If provided, the downloaded file will be saved with this name. If not provided, the original file name from the URL will be used. | ||
|
||
The function first reads the 'model_config_path.json' file to get the 'model_paths' dictionary. | ||
|
||
The function then gets the path where the model will be saved from the 'model_paths' dictionary using the 'selected' key. | ||
|
||
The function then tries to download the file from the URL and save it to the path. If the download is successful, a success message is returned. If the download fails, an error message is returned. | ||
""" | ||
model_paths = read_model_config_path("./model_config_path.json") | ||
codezeros marked this conversation as resolved.
Show resolved
Hide resolved
|
||
path = model_paths.get(selected) | ||
|
||
try: | ||
|
||
load_file_from_url(url, model_dir=path, progress=True, file_name=file_name) | ||
success_message = f"Download successful! Model saved to {path}." | ||
codezeros marked this conversation as resolved.
Show resolved
Hide resolved
|
||
except Exception as e: | ||
success_message = f"Download failed! please check url if it is correct." | ||
|
||
return success_message | ||
codezeros marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
""" | ||
This script contains functions for handling folders and paths. | ||
|
||
Importing the 'os' module which provides a way of using operating system dependent functionality. | ||
The 'os' module provides a portable way of using operating system dependent functionality such as reading or writing to the file system, starting or killing processes, etc. | ||
|
||
Importing the 'json' module which provides a way of working with JSON data. | ||
The 'json' module provides a way of encoding and decoding JSON data. | ||
""" | ||
import os | ||
import json | ||
|
||
def get_folders_and_paths(root_folder): | ||
""" | ||
This function takes a root folder as input and returns a dictionary containing all the folders and their paths in the root folder and its subdirectories. | ||
|
||
'root_folder' is the path to the root folder. | ||
|
||
'folder_data' is a dictionary that will contain the folders and their paths. | ||
|
||
The function iterates over all the items in the root folder. If an item is a directory, its name and path are added to the 'folder_data' dictionary. | ||
|
||
The function is called recursively to handle subdirectories. | ||
""" | ||
folder_data = {} | ||
|
||
for folder_name in os.listdir(root_folder): | ||
folder_path = os.path.join(root_folder, folder_name) | ||
if os.path.isdir(folder_path): | ||
folder_data[folder_name] = folder_path | ||
|
||
subfolder_data = get_folders_and_paths(folder_path) | ||
folder_data.update(subfolder_data) | ||
|
||
return folder_data | ||
|
||
def save_to_json(data, json_file): | ||
""" | ||
This function takes data and a json file as input and writes the data to the json file. | ||
|
||
'data' is the data to be written to the json file. | ||
|
||
'json_file' is the path to the json file. | ||
|
||
The data is written to the json file with an indentation of 4 spaces. | ||
""" | ||
with open(json_file, 'w') as f: | ||
json.dump(data, f, indent=4) | ||
codezeros marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
def get_model_paths(): | ||
""" | ||
This function gets the paths of all the models in the 'models' directory and its subdirectories and saves them to a json file. | ||
|
||
The function first gets the absolute path of the script's directory. | ||
|
||
The root folder is set to the 'models' directory in the script's directory. | ||
|
||
If the root folder does not exist, an error message is printed and the function returns. | ||
|
||
The function then gets all the folders and their paths in the root folder and its subdirectories. | ||
|
||
The function then iterates over all the folders and their paths. If a folder name contains a path separator, the folder is a subdirectory. The function then updates the 'folder_data' dictionary to contain the subdirectory and its path and adds the parent directory to the 'items_to_delete' list. | ||
|
||
The function then deletes all the items in the 'items_to_delete' list from the 'folder_data' dictionary. | ||
|
||
The function then saves the 'folder_data' dictionary to a json file. | ||
|
||
The function then prints a message indicating that the folder data has been saved to the json file. | ||
""" | ||
script_directory = os.path.dirname(os.path.abspath(__file__)) | ||
|
||
root_folder = os.path.join(script_directory, "../models/") | ||
|
||
if not os.path.exists(root_folder): | ||
print("Error: The specified folder does not exist.") | ||
return | ||
|
||
folder_data = get_folders_and_paths(root_folder) | ||
codezeros marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
items_to_delete = [] | ||
for folder_name, folder_path in folder_data.items(): | ||
if os.path.sep in folder_name: | ||
parent_folder_name, subfolder_name = folder_name.split(os.path.sep, 1) | ||
parent_folder_path = folder_data[parent_folder_name] | ||
folder_data[subfolder_name] = os.path.join(parent_folder_path, folder_name) | ||
items_to_delete.append(folder_name) | ||
|
||
for item in items_to_delete: | ||
del folder_data[item] | ||
|
||
json_file_name = "model_config_path.json" | ||
codezeros marked this conversation as resolved.
Show resolved
Hide resolved
|
||
json_file_path = os.path.join('./', json_file_name) | ||
|
||
save_to_json(folder_data, json_file_path) | ||
|
||
print(f"Folder data has been saved to {json_file_path}.") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# shared_module.py | ||
import json | ||
|
||
def read_model_config_path(json_file_path): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not needed, use config here |
||
""" | ||
This function reads a JSON file and returns its content. | ||
|
||
'json_file_path' is the path to the JSON file. | ||
|
||
The function opens the JSON file in read mode, loads its content into the 'model_paths' variable, and then returns 'model_paths'. | ||
""" | ||
with open(json_file_path, 'r') as f: | ||
model_paths = json.load(f) | ||
return model_paths |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not needed