Skip to content

Commit

Permalink
Adds prompt manager to cli (#247)
Browse files Browse the repository at this point in the history
  • Loading branch information
michaelfarrell76 authored Oct 18, 2023
1 parent a5b09b5 commit 0055ff7
Show file tree
Hide file tree
Showing 36 changed files with 1,937 additions and 252 deletions.
10 changes: 5 additions & 5 deletions .pnp.cjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file not shown.
106 changes: 103 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@
- [Authentication](#authentication-28)
- [Arguments](#arguments-28)
- [Usage](#usage-29)
- [Prompt Manager](#prompt-manager)
- [Proxy usage](#proxy-usage)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
Expand Down Expand Up @@ -351,7 +352,11 @@ The API key permissions for this command vary based on the value to the `resourc
| cookies | Consent Manager Cookie definitions. | View Data Flows | false | [Consent Manager -> Cookies](https://app.transcend.io/consent-manager/cookies/approved) |
| consentManager | Consent Manager general settings, including domain list. | View Consent Manager | false | [Consent Manager -> Developer Settings](https://app.transcend.io/consent-manager/developer-settings) |
| assessments | The Transcend assessments | View Assessments | false | [Assessments -> Browse](https://app.transcend.io/assessments/browse) |
| assessmentTemplates | The Transcend assessment tempaltes | View Assessments | false | [Assessments -> Templates](https://app.transcend.io/assessments/templates) |
| assessmentTemplates | The Transcend assessment templates | View Assessments | false | [Assessments -> Templates](https://app.transcend.io/assessments/templates) |
| prompts | The Transcend AI prompts | View Prompts | false | [Prompt Manager -> Browse](https://app.transcend.io/prompts/browse) |
| promptTemplates | The Transcend AI prompt templates | View Prompts | false | [Prompt Manager -> Templates](https://app.transcend.io/prompts/templates) |
| promptPartials | The Transcend AI prompt partials | View Prompts | false | [Prompt Manager -> Partials](https://app.transcend.io/prompts/partialss) |
| promptGroups | The Transcend AI prompt groups | View Prompts | false | [Prompt Manager -> Groups](https://app.transcend.io/prompts/groups) |

_Note: The scopes for tr-push are comprehensive of the scopes for tr-pull_

Expand Down Expand Up @@ -469,6 +474,12 @@ Pull in assessments and assessment templates (see [this example](./examples/asse
tr-pull --auth=$TRANSCEND_API_KEY --resources=assessment,assessmentTemplate
```

Pull in prompts, prompt templates, prompt partials and prompt groups (see [this example](./examples/prompts.yml)):

```sh
tr-pull --auth=$TRANSCEND_API_KEY --resources=prompts,promptTemplates,promptPartials,promptGroups
```

Pull everything:

```sh
Expand Down Expand Up @@ -522,7 +533,11 @@ The API key needs the following scopes when pushing the various resource types:
| cookies | Consent Manager Cookie definitions. | Manage Data Flows | false | [Consent Manager -> Cookies](https://app.transcend.io/consent-manager/cookies/approved) |
| consentManager | Consent Manager general settings, including domain list. | Manage Consent Manager Developer Settings | false | [Consent Manager -> Developer Settings](https://app.transcend.io/consent-manager/developer-settings) |
| assessments | The Transcend assessments | Manage Assessments | false | [Assessments -> Browse](https://app.transcend.io/assessments/browse) |
| assessmentTemplates | The Transcend assessment tempaltes | Manage Assessments | false | [Assessments -> Templates](https://app.transcend.io/assessments/templates) |
| assessmentTemplates | The Transcend assessment templates | Manage Assessments | false | [Assessments -> Templates](https://app.transcend.io/assessments/templates) |
| prompts | The Transcend AI prompts | View Prompts | false | [Prompt Manager -> Browse](https://app.transcend.io/prompts/browse) |
| promptTemplates | The Transcend AI prompt templates | Manage Prompts | false | [Prompt Manager -> Templates](https://app.transcend.io/prompts/templates) |
| promptPartials | The Transcend AI prompt partials | Manage Prompts | false | [Prompt Manager -> Partials](https://app.transcend.io/prompts/partialss) |
| promptGroups | The Transcend AI prompt groups | Manage Prompts | false | [Prompt Manager -> Groups](https://app.transcend.io/prompts/groups) |

#### Arguments

Expand Down Expand Up @@ -1891,7 +1906,7 @@ to grab your encryption and signing keys.
Upload consent preferences to partition key `4d1c5daa-90b7-4d18-aa40-f86a43d2c726`

```sh
yarn tr-upload-consent-preferences --base64EncryptionKey=$TRANSCEND_CONSENT_ENCRYPTION_KEY --base64SigningKey=$TRANSCEND_CONSENT_SIGNING_KEY --partition=4d1c5daa-90b7-4d18-aa40-f86a43d2c726
yarn tr-upload-consent-preferences --base64EncryptionKey=$TRANSCEND_CONSENT_ENCRYPTION_KEY --base64SigningKey=$TRANSCEND_CONSENT_SIGNING_KEY --partition=4d1c5daa-90b7-4d18-aa40-f86a43d2c726
```

Upload consent preferences to partition key `4d1c5daa-90b7-4d18-aa40-f86a43d2c726` from file `./consent.csv`
Expand Down Expand Up @@ -2285,6 +2300,91 @@ Configuring additional variables:
yarn tr-create-assessment --auth=$TRANSCEND_API_KEY --title="Test" --template="[AI Prompt] Product Manager Notes Parsing" --variables=description:"testt test"
```

## Prompt Manager

If you are integrating Transcend's Prompt Manager into your code, it may look like:

```ts
import * as t from 'io-ts';
import { TranscendPromptManager } from '@transcend-io/cli';
/**
* Example prompt integration
*/
export async function main(): Promise<void> {
const promptManager = new TranscendPromptManager({
// API key
transcendApiKey: process.env.TRANSCEND_API_KEY,
// Define the prompts that are stored in Transcend
prompts: {
test: {
// identify by ID
id: '30bcaa79-889a-4af3-842d-2e8ba443d36d',
// no runtime variables
paramCodec: t.type({}),
// response is list of strings
outputCodec: t.array(t.string),
},
json: {
// identify by title
title: 'test',
// one runtime variable "test"
paramCodec: t.type({ test: t.string }),
// runtime is json object
outputCodec: t.record(t.string, t.string),
// response is stored in <json></json> atg
extractFromTag: 'json',
},
predictProductLine: {
// identify by title
title: 'Predict Product Line',
// runtime parameter for slack channel name
paramCodec: t.type({
slackChannelName: t.string,
}),
// response is specific JSON shape
outputCodec: t.type({
product: t.union([t.string, t.null]),
clarification: t.union([t.string, t.null]),
}),
// response is stored in <json></json> atg
extractFromTag: 'json',
},
},
// Optional arguments
// transcendUrl: 'https://api.us.transcend.io', // defaults to 'https://api.transcend.io'
// requireApproval: false, // defaults to true
// cacheDuration: 1000 * 60 * 60, // defaults to undefined, no cache
// defaultVariables: { myVariable: 'this is custom', other: [{ name: 'custom' }] }, // defaults to {}
// handlebarsOptions: { helpers, templates }, // defaults to {}
});
// Fetch the prompt from Transcend and template any variables
// in this case, we template the slack channel name in the LLM prompt
const systemPrompt = await promptManager.compilePrompt('predictProductLine', {
slackChannelName: channelName,
});
// Pass the prompt into your LLM of choice
const response = await openai.createCompletion([
{
role: 'system',
content: systemPrompt,
},
{
role: 'user',
content: input,
},
]);
// Parsed response as JSON
const parsedResponse = promptManager.parseAiResponse(
'predictProductLine',
response,
);
}
```

## Proxy usage

If you are trying to use the cli inside a corporate firewall and need to send traffic through a proxy, you can do so via the `http_proxy` environment variable or the `--httpProxy` flag, with a command like `yarn tr-pull --auth=$TRANSCEND_API_KEY --httpProxy="http://localhost:5051"`.
93 changes: 93 additions & 0 deletions examples/prompts.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
prompts:
- title: '[Autogen Groupchat Research] Admin'
content: >-
<p>A human admin. Interact with the planner to discuss the plan. Plan
execution needs to be approved by this admin.</p>
- title: '[Autogen Groupchat Research] Critic'
content: >-
<p>Critic. Double check plan, claims, code from other agents and provide
feedback. Check whether the plan includes adding verifiable info such as
source URL.</p>
- title: '[Autogen Groupchat Research] Engineer'
content: >-
<p>Engineer. You follow an approved plan. You write python/shell code to
solve tasks. Wrap the code in a code block that specifies the script type.
The user can't modify your code. So do not suggest incomplete code which
requires others to modify. Don't use a code block if it's not intended to
be executed by the executor. Don't include multiple code blocks in one
response. Do not ask others to copy and paste the result. Check the
execution result returned by the executor. If the result indicates there
is an error, fix the error and output the code again. Suggest the full
code instead of partial code or code changes. If the error can't be fixed
or if the task is not solved even after the code is executed successfully,
analyze the problem, revisit your assumption, collect additional info you
need, and think of a different approach to try.</p>
- title: '[Autogen Groupchat Research] Executor'
content: >-
<p>Executor. Execute the code written by the engineer and report the
result.</p>
- title: '[Autogen Groupchat Research] Planner'
content: >-
<p>Planner. Suggest a plan. Revise the plan based on feedback from admin
and critic, until admin approval. The plan may involve an engineer who can
write code and a scientist who doesn't write code. Explain the plan first.
Be clear which step is performed by an engineer, and which step is
performed by a scientist.</p>
- title: '[Autogen Groupchat Research] Scientist'
content: >-
<p>Scientist. You follow an approved plan. You are able to categorize
papers after seeing their abstracts printed. You don't write code.</p>
- title: Customer Name Prediction
content: >-
<p>You are an experienced project manager, tasked with taking vague
feature requests and turning them into structured data so they can be
cataloged. Each feature request submitted should be tagged to the set of
customers that the feature relates to. Return a JSON object in the
following format: {customers, clarification}.</p><p>{{>
slackChannelCustomer }}</p><p>The customers key should be returned as a
JSON list of attributeValues that falls into the following list of
attributeValues: {{#each customers}}{{name}},{{/each}}</p><p>Common
corrections: {{#each correction}}</p><ol><li><p>The customer {{ name }} is
often called by the following names: {{#each
attributeValues}}{{name}},{{/each}}</p></li></ol><p>{{/each}}</p><p>There
is normally only 1 customer tied to the request, sometimes there is more
than 1 customer, and sometimes there is no customer included in the
initial request. If a customer is found that is not in that list of
existing customers, prompt in the clarification key of the response asking
the user to confirm that a new customer label should be created. If no
customers can be determined, return an empty list [] and prompt in the
clarification key of the response asking the user to confirm which
customer(s) this ticket relates to.</p><p>Please output your JSON response
in tags.</p><p></p>
prompt-templates:
- title: Engineer
content: You are an experienced engineer
- title: Product Manager
content: You are an experienced product manager
prompt-partials:
- title: Product Manager
content: You are an experienced product manager
- title: Slack Channel
content: >-
<p> {{#if slackChannelName}}</p><p>This question was recorded in a slack
channel named: #{{ slackChannelName }}.</p><p> {{/if}}</p>
- title: Today's Date
content: <p>Today's date is {{{ currentDate }}}.</p>
- title: Transcend Products
content: >-
<p>It may be helpful to know that Transcend has the following product
lines:</p><ul><li><p>{{#with attribute-productLine}} {{#each
attributeValues}}</p></li><li><p>The product "{{name}}" is defined by: {{{
description }}}.</p></li><li><p> {{/each}} {{/with}}</p></li></ul>
prompt-groups:
- title: Autogen Groupchat Research Example
description: >-
Example autogen implementation:
https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_research.ipynb
prompts:
- '[Autogen Groupchat Research] Admin'
- '[Autogen Groupchat Research] Critic'
- '[Autogen Groupchat Research] Engineer'
- '[Autogen Groupchat Research] Executor'
- '[Autogen Groupchat Research] Planner'
- '[Autogen Groupchat Research] Scientist'
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"author": "Transcend Inc.",
"name": "@transcend-io/cli",
"description": "Small package containing useful typescript utilities.",
"version": "4.108.0",
"version": "4.109.0",
"homepage": "https://github.com/transcend-io/cli",
"repository": {
"type": "git",
Expand Down Expand Up @@ -61,7 +61,7 @@
"@transcend-io/handlebars-utils": "^1.0.1",
"@transcend-io/internationalization": "^1.6.0",
"@transcend-io/persisted-state": "^1.0.2",
"@transcend-io/privacy-types": "^4.43.0",
"@transcend-io/privacy-types": "^4.46.0",
"@transcend-io/secret-value": "^1.1.1",
"@transcend-io/type-utils": "^1.1.1",
"bluebird": "^3.7.2",
Expand Down
Loading

0 comments on commit 0055ff7

Please sign in to comment.