Update dependencies #14

MostlyKIGuess · 2025-01-19T17:40:23Z

security fixes

Signed-off-by: Chihurumnaya Ibiam <[email protected]>

chimosky · 2025-01-19T22:30:42Z

Did you test your changes to be sure nothing breaks with these versions?

MostlyKIGuess · 2025-01-20T16:25:00Z

Did you test your changes to be sure nothing breaks with these versions?

while checking I realized , most of them are not needed, I only kept the necessary ones , will push the commit for that. I saw the mail and updated dependencies, want to fix this before this tuesday.

- add command-line interface; -include unit test for functionality

MostlyKIGuess · 2025-01-20T17:19:29Z

Welp the master branch wasn't working, so I made a lot of changes, almost revamping. But shouldn't we just merge the documents of this along with https://sugar-docs-ai.streamlit.app/. But I am willling to update this as well. I just saw the mail regarding dep issues and thought would fix it.

MostlyKIGuess · 2025-01-20T17:20:16Z

issue with current master is that ollama and httpx can't be run simultaneity they break each other's dependencies.

chimosky · 2025-01-20T18:05:11Z

issue with current master is that ollama and httpx can't be run simultaneity they break each other's dependencies.

Are you referring to aiohttp and ollama or langchain-ollama?

Also @kshitijdshah99 why do we have two ollama dependencies?

MostlyKIGuess · 2025-01-20T18:09:40Z

issue with current master is that ollama and httpx can't be run simultaneity they break each other's dependencies.

Are you referring to aiohttp and ollama or langchain-ollama?

Also @kshitijdshah99 why do we have two ollama dependencies?

ollama and httpx, check ollama/ollama-python#356

kshitijdshah99 · 2025-01-20T18:39:37Z

issue with current master is that ollama and httpx can't be run simultaneity they break each other's dependencies.

Are you referring to aiohttp and ollama or langchain-ollama?

Also @kshitijdshah99 why do we have two ollama dependencies?

Both these dependencies serve different purpose. langchain-ollama is a package which provides interaction between Langchain and Ollama to build Langchain workflows. On other hand ollama is for extracting and managing different models like llama3.1, Mistral or Codellama we downloaded locally on our system.

MostlyKIGuess · 2025-01-20T18:45:19Z

issue with current master is that ollama and httpx can't be run simultaneity they break each other's dependencies.

Are you referring to aiohttp and ollama or langchain-ollama?

Also @kshitijdshah99 why do we have two ollama dependencies?

Both these dependencies serve different purpose. langchain-ollama is a package which provides interaction between Langchain and Ollama to build Langchain workflows. On other hand ollama is for extracting and managing different models like llama3.1, Mistral or Codellama we downloaded locally on our system.

We should just pull from HF instead this way we can remove ollama. What say?

kshitijdshah99 · 2025-01-20T19:36:27Z

I'll suggest if @chimosky agrees we can try solving this issue first instead of directly skipping to HuggingFace because then we would have to adjust many code files.

chimosky · 2025-01-20T21:28:37Z

I'll suggest if @chimosky agrees we can try solving this issue first instead of directly skipping to HuggingFace because then we would have to adjust many code files.

If we can do without a dependency, then we should.

harshkasat · 2025-02-04T14:48:21Z

requirements.txt

 langchain
+transformers==4.45.2
 langchain-ollama


can you remove langchain-ollama because we're not using ollama in his pr

harshkasat · 2025-02-04T15:00:16Z

Are there any criteria for open-source models? I'm curious, especially since new models like DeepPeek and Qwen perform well in coding and mathematics with their instruction models.

chimosky · 2025-02-04T16:47:28Z

rag_agent.py

+        print(f"An error occurred: {e}")
+
+if __name__ == "__main__":
+    main()


You should remove the No EOF at end of file.

chimosky · 2025-02-04T16:48:51Z

rag_agent.py

+def main():
+    parser = argparse.ArgumentParser(description="Pippy's AI-Coding Assistant")
+    parser.add_argument('--model', type=str, choices=[
+        'bigscience/bloom-1b1',
+        'facebook/opt-350m',
+        'EleutherAI/gpt-neo-1.3B'
+    ], default='bigscience/bloom-1b1', help='Model name to use for text generation')
+    parser.add_argument('--docs', nargs='+', default=[
+        './docs/Pygame Documentation.pdf',
+        './docs/Python GTK+3 Documentation.pdf',
+        './docs/Sugar Toolkit Documentation.pdf'
+    ], help='List of document paths to load into the vector store')
+    args = parser.parse_args()
+
+    try:
+        agent = RAG_Agent(model=args.model)
+        agent.retriever = agent.setup_vectorstore(args.docs)

+        while True:
+            question = input("Enter your question: ").strip()
+            if not question:
+                print("Please enter a valid question.")
+                continue
+            response = agent.run(question)
+            print("Response:", response)
+    except Exception as e:
+        print(f"An error occurred: {e}")


Considering this is an API, we definitely wouldn't use command line args for it.

chimosky · 2025-02-04T16:50:43Z

Are there any criteria for open-source models? I'm curious, especially since new models like DeepPeek and Qwen perform well in coding and mathematics with their instruction models.

The only thing we cared about was how the model responded based on the prompt, which was adjusted of course.

kshitijdshah99 · 2025-02-06T13:54:39Z

@MostlyKIGuess I saw you are using very smaller models from HF which is a little less preferable....and of course needless to say that 8B models are taking lot of time for inference so it becomes necessary to use smaller ones. Have you find out some middle way to solve this problem? I tried running Llama-8B model from HF but it's consuming lot's of RAM and space too while running.

kshitijdshah99 · 2025-02-06T14:00:06Z

That's the reason why I preferred Ollama over HF suggest something if we can prevent this. AFAIK the model was working great when integrated with the Pippy-Activity locally.

kshitijdshah99 · 2025-02-06T14:09:51Z

@chimosky it seems these guys have fixed this interdependency issue. See the latest messages. Probably we can work with the latest httpx version if we want to.

MostlyKIGuess · 2025-02-06T16:04:59Z

@MostlyKIGuess I saw you are using very smaller models from HF which is a little less preferable....and of course needless to say that 8B models are taking lot of time for inference so it becomes necessary to use smaller ones. Have you find out some middle way to solve this problem? I tried running Llama-8B model from HF but it's consuming lot's of RAM and space too while running.

I did that just to test the pipeline, by no means I promote using that and your point of using ollama isn't in good faith, as ollama using quantization on the model to run it, the models in themselves remain the same, we can do the same with huggingface, I am by no means in support of huggingface, it's just convenient because we get access to more models, with ollama there's a dep hole. I will be working on the reviews today and try to remove more bloat.

MostlyKIGuess · 2025-02-06T16:20:47Z

I also noticed;

  from langchain.embeddings import HuggingFaceEmbeddings
Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.

MostlyKIGuess · 2025-02-06T16:20:58Z

I will be fixing that shortly today

kshitijdshah99 · 2025-02-06T16:30:55Z

@MostlyKIGuess I saw you are using very smaller models from HF which is a little less preferable....and of course needless to say that 8B models are taking lot of time for inference so it becomes necessary to use smaller ones. Have you find out some middle way to solve this problem? I tried running Llama-8B model from HF but it's consuming lot's of RAM and space too while running.

I did that just to test the pipeline, by no means I promote using that and your point of using ollama isn't in good faith, as ollama using quantization on the model to run it, the models in themselves remain the same, we can do the same with huggingface, I am by no means in support of huggingface, it's just convenient because we get access to more models, with ollama there's a dep hole. I will be working on the reviews today and try to remove more bloat.

Bro ik and am convinced with your purpose of opening this PR. The prime reason for supporting Ollama is bcz I have integrated it with fastAPI to the UI and also it's faster for the inference that's why I am not preferring HF. The primary reason of dependency conflict is not there anymore hence I don't feel the need of migrating the code. That's my point. Ollama like HF has many open source models (including deepseek) so that's not going to be an issue in future.

kshitijdshah99 · 2025-02-06T16:31:54Z

I also noticed;

  from langchain.embeddings import HuggingFaceEmbeddings
Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.

I opted using CPU bcz I hadn't one in my system yes we can improve it.

harshkasat · 2025-02-06T16:40:58Z

I will suggest using collab if you want to leverage GPU or make ssh or tunnel to use GPU in vs code

- update requirements for compatibility

harshkasat · 2025-02-06T16:43:56Z

are there any TODO fixes ? I also want to work on this.

MostlyKIGuess · 2025-02-06T16:44:31Z

I will suggest using collab if you want to leverage GPU or make ssh or tunnel to use GPU in vs code

I have resources actually personally to test it out.

MostlyKIGuess · 2025-02-06T16:44:49Z

are there any TODO fixes ? I also want to work on this.

maybe review the changes? will push in like 5-10 mins? would be a great help

MostlyKIGuess · 2025-02-06T17:42:44Z

Getting a 9.96 gb Image now with total GPU support!

harshkasat · 2025-02-06T17:44:26Z

with model?

MostlyKIGuess · 2025-02-06T17:47:34Z

with model?

no no, this is just the image generated container, model would add space depending upon which we use, I have commented 2 and kept the lowest parameter one because I wanted to just test the pipeline.

MostlyKIGuess · 2025-02-06T17:50:19Z

https://hub.docker.com/repository/docker/mostlyk/sugar-ai/general

harshkasat · 2025-02-06T18:11:29Z

I also find some low-level design issues:

Single Responsibility Principle: since RagAgent does a lot of task embed, storing, generate, I think we can create a base class and follow abstraction,
Dependency Injection Violation: agent.retriever = agent.setup_vectorstore(args.docs) modifidying retriever object after creating.
there might be other issues but I find this ones.

MostlyKIGuess · 2025-02-06T20:27:41Z

I also find some low-level design issues:

1. Single Responsibility Principle: since RagAgent does a lot of task embed, storing, generate, I think we can create a base class and follow abstraction,

2. Dependency Injection Violation: `agent.retriever = agent.setup_vectorstore(args.docs)` modifidying retriever object after creating.
   there might be other issues but I find this ones.

For this, I haven't actually worked mainly on this project , I did make another similar project it is called sugar docs ai in my repositories, well at least as of now, the immediate action was to lower the size and fix the dep loophole, perhaps we can create a ticket and start working on these too?

chimosky · 2025-02-06T22:09:03Z

I haven't reviewed your changes yet, please write better commit messages, the ones I'm seeing above aren't helpful.

This PR was originally for updating dependencies and it's become more than that, how about you open a new PR with the changes that have nothing to do with updating dependencies and write better commit messages.

MostlyKIGuess · 2025-02-07T16:45:46Z

I haven't reviewed your changes yet, please write better commit messages, the ones I'm seeing above aren't helpful.

This PR was originally for updating dependencies and it's become more than that, how about you open a new PR with the changes that have nothing to do with updating dependencies and write better commit messages.

alright, will close this then!

chimosky and others added 2 commits November 7, 2024 14:08

More dpeendecy changes

a1908c7

Signed-off-by: Chihurumnaya Ibiam <[email protected]>

update dependencies

315e491

Refactor RAG_Agent to use HuggingFace model

c42b1c3

- add command-line interface; -include unit test for functionality

chimosky force-pushed the main branch from a1908c7 to e4dc7db Compare January 27, 2025 16:31

harshkasat reviewed Feb 4, 2025

View reviewed changes

chimosky reviewed Feb 4, 2025

View reviewed changes

MostlyKIGuess added 2 commits February 6, 2025 22:12

enhance pipeline configuration

a444c42

- update requirements for compatibility

Dockerfile added

ce68fff

MostlyKIGuess added 2 commits February 6, 2025 22:18

readme updated

6d0599e

docker fixes

7208a94

MostlyKIGuess force-pushed the dev branch from df75289 to 7208a94 Compare February 6, 2025 17:41

MostlyKIGuess closed this Feb 7, 2025

Update dependencies #14

Update dependencies #14

Conversation

MostlyKIGuess commented Jan 19, 2025

chimosky commented Jan 19, 2025

MostlyKIGuess commented Jan 20, 2025

MostlyKIGuess commented Jan 20, 2025

MostlyKIGuess commented Jan 20, 2025

chimosky commented Jan 20, 2025

MostlyKIGuess commented Jan 20, 2025

kshitijdshah99 commented Jan 20, 2025

MostlyKIGuess commented Jan 20, 2025

kshitijdshah99 commented Jan 20, 2025

chimosky commented Jan 20, 2025

harshkasat Feb 4, 2025

Choose a reason for hiding this comment

harshkasat commented Feb 4, 2025

chimosky Feb 4, 2025

Choose a reason for hiding this comment

chimosky Feb 4, 2025

Choose a reason for hiding this comment

chimosky commented Feb 4, 2025

kshitijdshah99 commented Feb 6, 2025

kshitijdshah99 commented Feb 6, 2025

kshitijdshah99 commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

kshitijdshah99 commented Feb 6, 2025

kshitijdshah99 commented Feb 6, 2025 • edited Loading

harshkasat commented Feb 6, 2025

harshkasat commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

harshkasat commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

harshkasat commented Feb 6, 2025

MostlyKIGuess commented Feb 6, 2025

chimosky commented Feb 6, 2025

MostlyKIGuess commented Feb 7, 2025

kshitijdshah99 commented Feb 6, 2025 •

edited

Loading