Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker build #3157

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Worker build #3157

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
12ff79f
fix worker lint
chitalian Jan 22, 2025
3ec47bd
fix cost test
chitalian Jan 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions bifrost/packages/cost/providers/mappings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { openAIProvider } from "./openai";
import { anthropicProvider } from "./anthropic";
import { costs as awsBedrockCosts } from "./awsBedrock";
import { costs as deepseekCosts } from "./deepseek";
import { costs as xCosts } from "./x";

const openAiPattern = /^https:\/\/api\.openai\.com/;
const anthropicPattern = /^https:\/\/api\.anthropic\.com/;
Expand Down Expand Up @@ -52,6 +53,8 @@ const firecrawl = /^https:\/\/api\.firecrawl\.dev/;
const awsBedrock = /^https:\/\/bedrock-runtime\.[a-z0-9-]+\.amazonaws\.com\/.*/;
// https://api.deepseek.com
const deepseek = /^https:\/\/api\.deepseek\.com/;
// https://api.x.ai
const x = /^https:\/\/api\.x\.ai/;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: regex pattern for x.ai should include optional 'api.' subdomain for consistency with other patterns like openRouter


export const providersNames = [
"OPENAI",
Expand All @@ -78,6 +81,7 @@ export const providersNames = [
"FIRECRAWL",
"AWS",
"DEEPSEEK",
"X",
] as const;

export type ProviderName = (typeof providersNames)[number];
Expand Down Expand Up @@ -127,6 +131,11 @@ export const providers: {
pattern: cloudflareAiGatewayPattern,
provider: "CLOUDFLARE",
},
{
pattern: x,
provider: "X",
costs: xCosts,
},
Comment on lines +134 to +138
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: X provider configuration should be placed after AWS and before DEEPSEEK to maintain alphabetical ordering with other providers

{
pattern: twoYFV,
provider: "2YFV",
Expand Down
9 changes: 9 additions & 0 deletions costs/__tests__/ensureOnlyOne.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,15 @@ WHEN (request_response_rmt.model ILIKE 'o1-mini-2024-09-12') THEN 3000 * request
ELSE 0
END
)
WHEN (request_response_rmt.provider = 'X') THEN (
CASE
WHEN (request_response_rmt.model ILIKE 'grok-beta') THEN 5000 * request_response_rmt.prompt_tokens + 15000 * request_response_rmt.completion_tokens
WHEN (request_response_rmt.model ILIKE 'grok-vision-beta') THEN 5000 * request_response_rmt.prompt_tokens + 15000 * request_response_rmt.completion_tokens
WHEN (request_response_rmt.model ILIKE 'grok-2-1212') THEN 2000 * request_response_rmt.prompt_tokens + 10000 * request_response_rmt.completion_tokens
WHEN (request_response_rmt.model ILIKE 'grok-2-vision-1212') THEN 2000 * request_response_rmt.prompt_tokens + 10000 * request_response_rmt.completion_tokens
ELSE 0
END
)
WHEN (request_response_rmt.provider = 'TOGETHER') THEN (
CASE
WHEN (request_response_rmt.model ILIKE 'allenai/OLMo-7B-Instruct') THEN 200 * request_response_rmt.prompt_tokens + 200 * request_response_rmt.completion_tokens
Expand Down
9 changes: 9 additions & 0 deletions valhalla/jawn/src/packages/cost/providers/mappings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { openAIProvider } from "./openai";
import { anthropicProvider } from "./anthropic";
import { costs as awsBedrockCosts } from "./awsBedrock";
import { costs as deepseekCosts } from "./deepseek";
import { costs as xCosts } from "./x";

const openAiPattern = /^https:\/\/api\.openai\.com/;
const anthropicPattern = /^https:\/\/api\.anthropic\.com/;
Expand Down Expand Up @@ -52,6 +53,8 @@ const firecrawl = /^https:\/\/api\.firecrawl\.dev/;
const awsBedrock = /^https:\/\/bedrock-runtime\.[a-z0-9-]+\.amazonaws\.com\/.*/;
// https://api.deepseek.com
const deepseek = /^https:\/\/api\.deepseek\.com/;
// https://api.x.ai
const x = /^https:\/\/api\.x\.ai/;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: regex pattern for x.ai should include optional 'api.' prefix for consistency with openRouter pattern


export const providersNames = [
"OPENAI",
Expand All @@ -78,6 +81,7 @@ export const providersNames = [
"FIRECRAWL",
"AWS",
"DEEPSEEK",
"X",
] as const;

export type ProviderName = (typeof providersNames)[number];
Expand Down Expand Up @@ -127,6 +131,11 @@ export const providers: {
pattern: cloudflareAiGatewayPattern,
provider: "CLOUDFLARE",
},
{
pattern: x,
provider: "X",
costs: xCosts,
},
Comment on lines +134 to +138
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: X provider entry should be placed at the end of the providers array to maintain alphabetical ordering with other providers

{
pattern: twoYFV,
provider: "2YFV",
Expand Down
2 changes: 1 addition & 1 deletion valhalla/jawn/src/tsoa-build/private/routes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,7 @@ const models: TsoaRoute.Models = {
// WARNING: This file was auto-generated with tsoa. Please do not modify it. Re-run tsoa to re-generate this file: https://github.com/lukeautry/tsoa
"ProviderName": {
"dataType": "refAlias",
"type": {"dataType":"union","subSchemas":[{"dataType":"enum","enums":["OPENAI"]},{"dataType":"enum","enums":["ANTHROPIC"]},{"dataType":"enum","enums":["AZURE"]},{"dataType":"enum","enums":["LOCAL"]},{"dataType":"enum","enums":["HELICONE"]},{"dataType":"enum","enums":["AMDBARTEK"]},{"dataType":"enum","enums":["ANYSCALE"]},{"dataType":"enum","enums":["CLOUDFLARE"]},{"dataType":"enum","enums":["2YFV"]},{"dataType":"enum","enums":["TOGETHER"]},{"dataType":"enum","enums":["LEMONFOX"]},{"dataType":"enum","enums":["FIREWORKS"]},{"dataType":"enum","enums":["PERPLEXITY"]},{"dataType":"enum","enums":["GOOGLE"]},{"dataType":"enum","enums":["OPENROUTER"]},{"dataType":"enum","enums":["WISDOMINANUTSHELL"]},{"dataType":"enum","enums":["GROQ"]},{"dataType":"enum","enums":["COHERE"]},{"dataType":"enum","enums":["MISTRAL"]},{"dataType":"enum","enums":["DEEPINFRA"]},{"dataType":"enum","enums":["QSTASH"]},{"dataType":"enum","enums":["FIRECRAWL"]},{"dataType":"enum","enums":["AWS"]},{"dataType":"enum","enums":["DEEPSEEK"]}],"validators":{}},
"type": {"dataType":"union","subSchemas":[{"dataType":"enum","enums":["OPENAI"]},{"dataType":"enum","enums":["ANTHROPIC"]},{"dataType":"enum","enums":["AZURE"]},{"dataType":"enum","enums":["LOCAL"]},{"dataType":"enum","enums":["HELICONE"]},{"dataType":"enum","enums":["AMDBARTEK"]},{"dataType":"enum","enums":["ANYSCALE"]},{"dataType":"enum","enums":["CLOUDFLARE"]},{"dataType":"enum","enums":["2YFV"]},{"dataType":"enum","enums":["TOGETHER"]},{"dataType":"enum","enums":["LEMONFOX"]},{"dataType":"enum","enums":["FIREWORKS"]},{"dataType":"enum","enums":["PERPLEXITY"]},{"dataType":"enum","enums":["GOOGLE"]},{"dataType":"enum","enums":["OPENROUTER"]},{"dataType":"enum","enums":["WISDOMINANUTSHELL"]},{"dataType":"enum","enums":["GROQ"]},{"dataType":"enum","enums":["COHERE"]},{"dataType":"enum","enums":["MISTRAL"]},{"dataType":"enum","enums":["DEEPINFRA"]},{"dataType":"enum","enums":["QSTASH"]},{"dataType":"enum","enums":["FIRECRAWL"]},{"dataType":"enum","enums":["AWS"]},{"dataType":"enum","enums":["DEEPSEEK"]},{"dataType":"enum","enums":["X"]}],"validators":{}},
},
// WARNING: This file was auto-generated with tsoa. Please do not modify it. Re-run tsoa to re-generate this file: https://github.com/lukeautry/tsoa
"Provider": {
Expand Down
3 changes: 2 additions & 1 deletion valhalla/jawn/src/tsoa-build/private/swagger.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1180,7 +1180,8 @@
"QSTASH",
"FIRECRAWL",
"AWS",
"DEEPSEEK"
"DEEPSEEK",
"X"
]
},
"Provider": {
Expand Down
9 changes: 9 additions & 0 deletions web/packages/cost/providers/mappings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { openAIProvider } from "./openai";
import { anthropicProvider } from "./anthropic";
import { costs as awsBedrockCosts } from "./awsBedrock";
import { costs as deepseekCosts } from "./deepseek";
import { costs as xCosts } from "./x";

const openAiPattern = /^https:\/\/api\.openai\.com/;
const anthropicPattern = /^https:\/\/api\.anthropic\.com/;
Expand Down Expand Up @@ -52,6 +53,8 @@ const firecrawl = /^https:\/\/api\.firecrawl\.dev/;
const awsBedrock = /^https:\/\/bedrock-runtime\.[a-z0-9-]+\.amazonaws\.com\/.*/;
// https://api.deepseek.com
const deepseek = /^https:\/\/api\.deepseek\.com/;
// https://api.x.ai
const x = /^https:\/\/api\.x\.ai/;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: regex pattern should include optional 'api.' prefix for consistency with other patterns like openRouter


export const providersNames = [
"OPENAI",
Expand All @@ -78,6 +81,7 @@ export const providersNames = [
"FIRECRAWL",
"AWS",
"DEEPSEEK",
"X",
] as const;

export type ProviderName = (typeof providersNames)[number];
Expand Down Expand Up @@ -127,6 +131,11 @@ export const providers: {
pattern: cloudflareAiGatewayPattern,
provider: "CLOUDFLARE",
},
{
pattern: x,
provider: "X",
costs: xCosts,
},
Comment on lines +134 to +138
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: X provider entry should be placed at the end of the providers array for consistency with providersNames order

{
pattern: twoYFV,
provider: "2YFV",
Expand Down
6 changes: 3 additions & 3 deletions worker/src/lib/RequestWrapper.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -654,7 +654,7 @@ export async function getProviderKeyFromPortalKey(
.from("decrypted_provider_keys")
.select("decrypted_provider_key")
.eq("id", providerKeyId.data?.id ?? "")
.eq("soft_delete", "false")
.eq("soft_delete", false)
.single();

return map(mapPostgrestErr(providerKey), (x) => ({
Expand Down Expand Up @@ -689,7 +689,7 @@ export async function getProviderKeyFromProxy(
.from("helicone_proxy_keys")
.select("*")
.eq("id", proxyKeyId)
.eq("soft_delete", "false")
.eq("soft_delete", false)
.single(),
supabaseClient
.from("helicone_proxy_key_limits")
Expand Down Expand Up @@ -721,7 +721,7 @@ export async function getProviderKeyFromProxy(
.from("decrypted_provider_keys")
.select("decrypted_provider_key")
.eq("id", storedProxyKey.data.provider_key_id)
.eq("soft_delete", "false")
.eq("soft_delete", false)
.single();

if (providerKey.error || !providerKey.data?.decrypted_provider_key) {
Expand Down
9 changes: 9 additions & 0 deletions worker/src/packages/cost/providers/mappings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { openAIProvider } from "./openai";
import { anthropicProvider } from "./anthropic";
import { costs as awsBedrockCosts } from "./awsBedrock";
import { costs as deepseekCosts } from "./deepseek";
import { costs as xCosts } from "./x";

const openAiPattern = /^https:\/\/api\.openai\.com/;
const anthropicPattern = /^https:\/\/api\.anthropic\.com/;
Expand Down Expand Up @@ -52,6 +53,8 @@ const firecrawl = /^https:\/\/api\.firecrawl\.dev/;
const awsBedrock = /^https:\/\/bedrock-runtime\.[a-z0-9-]+\.amazonaws\.com\/.*/;
// https://api.deepseek.com
const deepseek = /^https:\/\/api\.deepseek\.com/;
// https://api.x.ai
const x = /^https:\/\/api\.x\.ai/;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: URL pattern regex should include end-of-string anchor ($) to prevent partial matches


export const providersNames = [
"OPENAI",
Expand All @@ -78,6 +81,7 @@ export const providersNames = [
"FIRECRAWL",
"AWS",
"DEEPSEEK",
"X",
] as const;

export type ProviderName = (typeof providersNames)[number];
Expand Down Expand Up @@ -127,6 +131,11 @@ export const providers: {
pattern: cloudflareAiGatewayPattern,
provider: "CLOUDFLARE",
},
{
pattern: x,
provider: "X",
costs: xCosts,
},
Comment on lines +134 to +138
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: X provider configuration should include modelDetails property for consistency with other major providers like OpenAI and Anthropic

{
pattern: twoYFV,
provider: "2YFV",
Expand Down
Loading