Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Features: gpt-4-vision, Configurable OpenAI Endpoint, Model, and System Prompt #8

Open
wants to merge 46 commits into
base: main
Choose a base branch
from

Conversation

Josh-XT
Copy link

@Josh-XT Josh-XT commented Mar 17, 2024

New Features

  • Replaced the getResponse function that used the completions endpoint with the divinci model with a new function that uses the OpenAI chat completions endpoint. This enables the use of gpt-3.5-turbo, gpt-4, and gpt-4-vision.
  • gpt-4-vision support added when sending an image instead of doing the stable diffusion image to image.

New first start up configuration (Resolves #7 )

  • Added configuration to choose the OpenAI endpoint. Defaults to https://api.openai.com/v1 so that no one has to look for it.) The purpose for this is to allow people to use third party OpenAI style endpoint services like ezlocalai as drop in replacements for OpenAI API.
  • Added configuration to choose the Model. Default is set to gpt-4-vision-preview for the latest gpt-4 vision model.
  • Added configuration to choose the System Prompt. Default is set to The assistant is Noa, the user's AR AI companion.

Clean up & Updates

  • Remove the duplicate getResponse function that was in ChatGptFragment and referenced the new function with parentActivity.getResponse.
  • Android studio did some updates. I ran the app on my phone successfully.

Recommendations post-merge

  • Further recommendations on this would be to completely remove all references of Stable Diffusion, or maybe make it some separate toggle for use when you don't want to send the image to the OpenAI model.
  • Add some kind of way for users to edit the settings. I figured out how to expose the ability to fill in the information and save it, but I didn't get further.

Other notes

I am a very experienced software engineer, but this is the first time I've ever touched Kotlin. I can't promise the highest quality here, but the added functionality not only makes the app usable again, it vastly improves its capabilities.

I was also informed in Discord that there is a Flutter app being built to replace the Kotlin one, but I was already well down this road and figured that it will make a good entrypoint for other developers to build off of this Noa app in Kotlin for their own applications. I wouldn't fully abandon this one, I would simply rename it noa-for-android-kotlin or something similar to serve as an example for developers to build on.

There may be things I forgot to mention that I had to change along the way to make things work. My fork will continue to change and improve as I find better ways to do these things in Kotlin or ways to improve.

@josuah
Copy link
Member

josuah commented Mar 17, 2024

I wouldn't fully abandon this one

Some developers would prefer to develop something in Kotlin, others in Flutter, others in React Native...

It is good to have as many examples of how to pair with one of these devices and use various back-ends, services, databases along with Brilliant Labs devices... Good to have that going regardless of the fate of this repo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Configurable OpenAI Endpoints and Models
2 participants