Yudhistira Jinawi Agung | yudh

[agungyudhis]

My assignment repo https://github.com/agungyudhis/gen-ai-assignment/tree/main

Some Questions

I'm having trouble giving current time context to LLM. On my first attempt, I created a tool to get the current date, but several times, the tool didn't get called. Then, I added the date directly to the user input prompt template. It works fine, but I still need to know the best practice.

After attending the workshop, I became very interested in vector databases and LLM. My college projects used text embedding and classification using BERT. Could the model for tool calling be replaced with BERT, and the generative model only used at the start and the end? If this can be done, then I think this can save API usage.

My idea is:

1. Use a generative model to generate steps,
2. Use BERT to determine tools,
3. The output data from the tools is fed into the generative model.

Thank you very much for your time and dedication, Kak Sam.

[onlyphantom]

Hi Yudhistira, first of all congratulations on the job well done. Excellent work and I can tell how much time you spent on it, which is such a positive sign.

Technically, if you have a fairly limited set of tools (n) and train your BERT as a classifier to classify possible tool-use (n + 1 possibilities, since there is also the possibility of the query being outside the scope of existing tools), then you can have this logic chain that run the BERT classifier + LLM to generate final output yes (this is where the generative tool does things like format into table, add a narrative etc, turn it into natural language answers etc).

Thank you again for being such an excellent participant. Marked as passed and will be issuing your certificate soon!

[agungyudhis]

Thank you very much, Kak Sam. 🙏

I was wondering about using [CLS] output on BERT as text embedding. So, every docstring of tools is embedded and (maybe) saved to a vector database. If you want to do a tool calling, the prompt is embedded, and then use k-NN to find the tools. However, I'll try to use BERT as a classifier too.