Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda): Create split SPAs #1059

Closed
wants to merge 90 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
913a52c
update id broken
tiffanyvu Jan 16, 2025
000b553
ad dlog
tiffanyvu Jan 16, 2025
4508cac
log
tiffanyvu Jan 16, 2025
2e9a427
log
tiffanyvu Jan 16, 2025
83f8839
adjust logic
tiffanyvu Jan 16, 2025
e65da1c
hm
tiffanyvu Jan 16, 2025
19992a6
why
tiffanyvu Jan 16, 2025
048b091
log getpackage
tiffanyvu Jan 16, 2025
0fa8543
logs
tiffanyvu Jan 16, 2025
4805e7c
revert?
tiffanyvu Jan 16, 2025
d6490e7
log os
tiffanyvu Jan 16, 2025
33372ed
log
tiffanyvu Jan 16, 2025
5cc3ab7
log
tiffanyvu Jan 16, 2025
af2292a
client.get stuck
tiffanyvu Jan 17, 2025
aaa4ca4
fix
tiffanyvu Jan 17, 2025
0ac9739
fix
tiffanyvu Jan 17, 2025
f8a22c2
log error
tiffanyvu Jan 17, 2025
e1fb35f
what
tiffanyvu Jan 17, 2025
2472b32
revert
tiffanyvu Jan 17, 2025
0325b22
Merge branch 'main' into split-spas
tiffanyvu Jan 17, 2025
7771b76
revert
tiffanyvu Jan 17, 2025
83e96f9
wip
tiffanyvu Jan 21, 2025
e67222b
validate id
tiffanyvu Jan 21, 2025
ff1f1e3
zod error
tiffanyvu Jan 21, 2025
4dc1c82
await
tiffanyvu Jan 21, 2025
6e953c8
fix query
tiffanyvu Jan 21, 2025
c5d7ab0
fix query again
tiffanyvu Jan 21, 2025
5230203
fix query
tiffanyvu Jan 21, 2025
9a5cdad
try keyword
tiffanyvu Jan 21, 2025
8c20893
try
tiffanyvu Jan 21, 2025
0157327
query
tiffanyvu Jan 22, 2025
723e162
test
tiffanyvu Jan 22, 2025
15a65a9
log
tiffanyvu Jan 22, 2025
cbd5bba
last index
tiffanyvu Jan 22, 2025
1c61dc9
logic
tiffanyvu Jan 22, 2025
d0de3c0
test
tiffanyvu Jan 22, 2025
1d6c96c
cleanup
tiffanyvu Jan 22, 2025
8df846c
edit names and sinkmainprocessor
tiffanyvu Jan 22, 2025
7a109ec
weird
tiffanyvu Jan 22, 2025
0614c16
log
tiffanyvu Jan 22, 2025
dd43f02
fix
tiffanyvu Jan 22, 2025
0df0678
huh
tiffanyvu Jan 22, 2025
0986323
log
tiffanyvu Jan 22, 2025
05a145b
bug
tiffanyvu Jan 23, 2025
a383819
admin change schema
tiffanyvu Jan 23, 2025
8e7fd3c
missing in schema
tiffanyvu Jan 23, 2025
1e12fd2
idToBeUpdated
tiffanyvu Jan 23, 2025
5dd8135
add changelog type
tiffanyvu Jan 23, 2025
d95f0e4
zod typo?
tiffanyvu Jan 23, 2025
222b860
facepalm
tiffanyvu Jan 23, 2025
7851592
admin package activity
tiffanyvu Jan 23, 2025
b567d50
log changelog pushing, add success response
tiffanyvu Jan 23, 2025
749f159
debug changelog
tiffanyvu Jan 23, 2025
b1a1eaa
log docs
tiffanyvu Jan 23, 2025
48f83c7
change split spa changelog logic
tiffanyvu Jan 23, 2025
228b08c
add tests wip
tiffanyvu Jan 23, 2025
7555130
clean up and fix error bubbling
tiffanyvu Jan 24, 2025
3d4af3c
reference baseschema id shape
tiffanyvu Jan 24, 2025
670f68a
fix import errors and tests wip
tiffanyvu Jan 24, 2025
2b132e5
test after refactor bug
tiffanyvu Jan 24, 2025
96b7073
test fix
tiffanyvu Jan 24, 2025
4edbfd9
fix
tiffanyvu Jan 24, 2025
33f835e
rm logs and update comments
tiffanyvu Jan 27, 2025
2e6d724
Merge branch 'main' into split-spas
tiffanyvu Jan 27, 2025
af412e6
change body parsing, update timestamps, mod admin schema
tiffanyvu Jan 27, 2025
cf01a04
fix timestamp
tiffanyvu Jan 27, 2025
c9697e5
hm
tiffanyvu Jan 27, 2025
33e6045
was it this line
tiffanyvu Jan 27, 2025
3292ccd
test change
tiffanyvu Jan 27, 2025
516f3d7
revert
tiffanyvu Jan 27, 2025
dc61b29
log not incrementing
tiffanyvu Jan 27, 2025
061c83a
log fix
tiffanyvu Jan 27, 2025
3547aca
log hits
tiffanyvu Jan 27, 2025
6033628
m not showing in hits
tiffanyvu Jan 27, 2025
1987dfc
look for m
tiffanyvu Jan 27, 2025
8ca08d2
query size?
tiffanyvu Jan 27, 2025
6b6fce0
syntax fix
tiffanyvu Jan 27, 2025
793068d
rm logs and test admin schema change again
tiffanyvu Jan 27, 2025
629cb7e
revert
tiffanyvu Jan 27, 2025
6f1b0b3
import error
tiffanyvu Jan 27, 2025
8407bc6
import again
tiffanyvu Jan 27, 2025
75e80e1
revert
tiffanyvu Jan 27, 2025
04f9c20
remove
tiffanyvu Jan 27, 2025
1cc0423
Merge branch 'main' into split-spas
tiffanyvu Jan 28, 2025
48b4822
topic name not defined
tiffanyvu Jan 28, 2025
933dbb6
rm unnecessary packageId check and wip tests
tiffanyvu Jan 28, 2025
6754110
reduce query size
tiffanyvu Jan 28, 2025
6593357
reduce query size?
tiffanyvu Jan 28, 2025
576e228
change order of pushing?
tiffanyvu Jan 28, 2025
98f710d
remove query size
tiffanyvu Jan 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 21 additions & 5 deletions lib/lambda/sinkChangelog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
transformUpdateValuesSchema,
transformDeleteSchema,
transformedUpdateIdSchema,
transformedSplitSPASchema,
} from "./update/adminChangeSchemas";
import { getPackageChangelog } from "libs/api/package";

Expand Down Expand Up @@ -64,26 +65,41 @@ const processAndIndex = async ({
// Parse the kafka record's value
const record = JSON.parse(decodeBase64WithUtf8(value));

// query all changelog entries for this ID and create copies of all entries with new ID
if (record.isAdminChange) {
const schema = transformDeleteSchema(offset).or(
transformUpdateValuesSchema(offset).or(transformedUpdateIdSchema),
);
const schema = transformDeleteSchema(offset)
.or(transformUpdateValuesSchema(offset))
.or(transformedUpdateIdSchema)
.or(transformedSplitSPASchema);

const result = schema.safeParse(record);

if (result.success) {
if (result.data.adminChangeType === "update-id") {
// Push doc with package being soft deleted
docs.forEach((log) => {
const recordOffset = log.id.split("-").at(-1);

docs.push({
...log,
id: `${result.data.id}-${recordOffset}`,
packageId: result.data.id,
});
});
// Get all changelog entries for this ID and create copies of all entries with new ID
const packageChangelogs = await getPackageChangelog(result.data.idToBeUpdated);

packageChangelogs.hits.hits.forEach((log) => {
tiffanyvu marked this conversation as resolved.
Show resolved Hide resolved
const recordOffset = log._id.split("-").at(-1);
docs.push({
...log._source,
id: `${result.data.id}-${recordOffset}`,
packageId: result.data.id,
});
});
} else if (result.data.adminChangeType === "split-spa") {
// Push doc with new split package
docs.push(result.data);

// Get all changelog entries for this ID and create copies of all entries with new ID
const packageChangelogs = await getPackageChangelog(result.data.idToBeUpdated);

packageChangelogs.hits.hits.forEach((log) => {
Expand Down
4 changes: 3 additions & 1 deletion lib/lambda/sinkMainProcessors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@ import {
deleteAdminChangeSchema,
updateValuesAdminChangeSchema,
updateIdAdminChangeSchema,
splitSPAAdminChangeSchema,
} from "./update/adminChangeSchemas";

const removeDoubleQuotesSurroundingString = (str: string) => str.replace(/^"|"$/g, "");
const adminRecordSchema = deleteAdminChangeSchema
.or(updateValuesAdminChangeSchema)
.or(updateIdAdminChangeSchema);
.or(updateIdAdminChangeSchema)
.or(splitSPAAdminChangeSchema);

type OneMacRecord = {
id: string;
Expand Down
32 changes: 32 additions & 0 deletions lib/lambda/submit/getNextSplitSPAId.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import { search } from "libs/opensearch-lib";
import { getDomainAndNamespace } from "libs/utils";
import { cpocs } from "lib/packages/shared-types/opensearch";

export const getNextSplitSPAId = async (spaId: string) => {
const { domain, index } = getDomainAndNamespace("main");
const query = {
query: {
regexp: {
"id.keyword": `${spaId}-[A-Z]`,
},
},
};
// Get existing split SPAs for this package id
const { hits } = await search(domain, index, query);
// Extract suffixes from existing split SPA IDs
// If there are no split SPAs yet, start at the ASCII character before "A" ("@")
// Convert to ASCII char codes to get latest suffix
const latestSuffixCharCode = hits.hits.reduce((maxCharCode: number, hit: cpocs.ItemResult) => {
const suffix = hit._source.id.toString().split("-").at(-1) ?? "@";
return Math.max(maxCharCode, suffix.charCodeAt(0));
}, "@".charCodeAt(0));

// Increment letter but not past "Z"
// "A-Z" is 65-90 in ASCII
if (latestSuffixCharCode >= 90) {
throw new Error("This package can't be further split.");
}
const nextSuffix = String.fromCharCode(latestSuffixCharCode + 1);

return `${spaId}-${nextSuffix}`;
};
86 changes: 86 additions & 0 deletions lib/lambda/submit/submitSplitSPA.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { handler } from "./submitSplitSPA";
import { APIGatewayEvent } from "node_modules/shared-types";
import { TEST_CHIP_SPA_ITEM, TEST_MED_SPA_ITEM } from "mocks";

vi.mock("libs/handler-lib", () => ({
response: vi.fn((data) => data),
}));

// vi.mock("./getNextSplitSPAId", () => ({
// getNextSplitSPAId: vi.fn(),
// }));

describe("handler", () => {
beforeEach(() => {
vi.clearAllMocks();
process.env.topicName = "test-topic";
});

it("should return 400 if event body is missing", async () => {
const event = {} as APIGatewayEvent;
const result = await handler(event);
expect(result?.statusCode).toEqual(400);
});

// it("should return 400 if package ID is not provided", async () => {
// const emptyBody = {
// body: JSON.stringify({ packageId: undefined }),
// } as APIGatewayEvent;
// const result = await handler(emptyBody);
// expect(result?.statusCode).toEqual(400);
// });

it("should return 404 if package ID is not found", async () => {
const invalidPackage = {
body: JSON.stringify({ packageId: "MD-25-9999" }),
} as unknown as APIGatewayEvent;
const result = await handler(invalidPackage);
expect(result?.statusCode).toEqual(404);
});

it("should throw an error if not Medicaid SPA", async () => {
const chipSPAPackage = {
body: { packageId: TEST_CHIP_SPA_ITEM._id },
} as unknown as APIGatewayEvent;
const result = await handler(chipSPAPackage);
const expectedResult = {
statusCode: 400,
body: { message: "Record must be a Medicaid SPA" },
};
expect(result).toEqual(expectedResult);
});

it("should return 400 if package ID not provided", async () => {
const invalidPackage = {
body: JSON.stringify({}),
} as unknown as APIGatewayEvent;
const result = await handler(invalidPackage);
expect(result?.statusCode).toEqual(400);
});

it("should fail to split a package with no topic name", async () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  it("should fail to split a package with no topic name", async () => {
    process.env.topicName = "";
    const noActionevent = {
      body: JSON.stringify({
        packageId: TEST_MED_SPA_ITEM._id,
      }),
    } as APIGatewayEvent;

    expect(async () => await handler(noActionevent)).rejects.toThrow();
    expect(async () => await handler(noActionevent)).rejects.toThrowError(
      "Topic name is not defined",
    );
  });

process.env.topicName = "";
const noActionevent = {
body: JSON.stringify({
packageId: TEST_MED_SPA_ITEM._id,
}),
} as APIGatewayEvent;

const result = await handler(noActionevent);
const expectedResult = {
statusCode: 500,
body: { message: "Topic name is not defined" },
};
expect(result).toStrictEqual(expectedResult);
});

// it("should create a split SPA", async () => {
// const medSPAPackage = {
// body: { packageId: TEST_MED_SPA_ITEM._id },
// } as unknown as APIGatewayEvent;
// console.log(medSPAPackage, "HELLO??");
// const result = await handler(medSPAPackage);
// expect(result?.body).toEqual('{"message":"Record must be a Medicaid SPA"}');
// });
});
95 changes: 95 additions & 0 deletions lib/lambda/submit/submitSplitSPA.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import { response } from "libs/handler-lib";
import { APIGatewayEvent } from "aws-lambda";
import { getPackage } from "libs/api/package";
import { produceMessage } from "libs/api/kafka";
import { ItemResult } from "shared-types/opensearch/main";
import { events } from "shared-types/events";
import { getNextSplitSPAId } from "./getNextSplitSPAId";
import { z } from "zod";

const sendSubmitSplitSPAMessage = async (currentPackage: ItemResult) => {
const topicName = process.env.topicName as string;
if (!topicName) {
throw new Error("Topic name is not defined");
}
const newId = await getNextSplitSPAId(currentPackage._id);

Check failure on line 15 in lib/lambda/submit/submitSplitSPA.ts

View workflow job for this annotation

GitHub Actions / coverage-report

lambda/submit/submitSplitSPA.test.ts > handler > should fail to split a package with no topic name

Error: Topic name is not defined ❯ sendSubmitSplitSPAMessage lambda/submit/submitSplitSPA.ts:15:9 ❯ Module.handler lambda/submit/submitSplitSPA.ts:92:7 ❯ lambda/submit/submitSplitSPA.test.ts:70:20

Check failure on line 15 in lib/lambda/submit/submitSplitSPA.ts

View workflow job for this annotation

GitHub Actions / test

lambda/submit/submitSplitSPA.test.ts > handler > should fail to split a package with no topic name

Error: Topic name is not defined ❯ sendSubmitSplitSPAMessage lambda/submit/submitSplitSPA.ts:15:9 ❯ Module.handler lambda/submit/submitSplitSPA.ts:92:7 ❯ lambda/submit/submitSplitSPA.test.ts:70:20

Check failure on line 15 in lib/lambda/submit/submitSplitSPA.ts

View workflow job for this annotation

GitHub Actions / test

lambda/submit/submitSplitSPA.test.ts > handler > should fail to split a package with no topic name

Error: Topic name is not defined ❯ sendSubmitSplitSPAMessage lambda/submit/submitSplitSPA.ts:15:9 ❯ Module.handler lambda/submit/submitSplitSPA.ts:92:7 ❯ lambda/submit/submitSplitSPA.test.ts:70:20
if (!newId) {
throw new Error("Error getting next Split SPA Id");
}

// ID and changeMade are excluded; the rest of the object has to be spread into the new package
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good comment 👍🏻

const {
id: _id,
changeMade: _changeMade,
origin: _origin,
...remainingFields
} = currentPackage._source;

await produceMessage(
topicName,
newId,
JSON.stringify({
id: newId,
idToBeUpdated: currentPackage._id,
...remainingFields,
makoChangedDate: Date.now(),
changedDate: Date.now(),
origin: "OneMAC",
changeMade: "OneMAC Admin has added a package to OneMAC.",
changeReason: "Per request from CMS, this package was added to OneMAC.",
isAdminChange: true,
adminChangeType: "split-spa",
}),
);

return response({
statusCode: 200,
body: { message: `New Medicaid Split SPA ${newId} has been created.` },
});
};

const splitSPAEventBodySchema = z.object({
packageId: events["new-medicaid-submission"].baseSchema.shape.id,
});

export const handler = async (event: APIGatewayEvent) => {
if (!event.body) {
return response({
statusCode: 400,
body: { message: "Event body required" },
});
}
try {
const body = typeof event.body === "string" ? JSON.parse(event.body) : event.body;
const { packageId } = splitSPAEventBodySchema.parse(body);

const currentPackage = await getPackage(packageId);
if (!currentPackage || currentPackage.found == false) {
return response({
statusCode: 404,
body: { message: "No record found for the given id" },
});
}

if (currentPackage._source.authority !== "Medicaid SPA") {
return response({
statusCode: 400,
body: { message: "Record must be a Medicaid SPA" },
});
}

return sendSubmitSplitSPAMessage(currentPackage);
} catch (err) {
console.error("Error has occured modifying package:", err);
if (err instanceof z.ZodError) {
return response({
statusCode: 400,
body: { message: err.errors },
});
}
return response({
statusCode: 500,
body: { message: err.message || "Internal Server Error" },
});
}
};
18 changes: 18 additions & 0 deletions lib/lambda/update/adminChangeSchemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ export const updateIdAdminChangeSchema = z
})
.and(z.record(z.string(), z.any()));

export const splitSPAAdminChangeSchema = z
.object({
id: z.string(),
adminChangeType: z.literal("split-spa"),
idToBeUpdated: z.string(),
})
.and(z.record(z.string(), z.any()));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this essentially allow the schema to be a wildcard? Any sort of data can be passed and accepted


export const transformDeleteSchema = (offset: number) =>
deleteAdminChangeSchema.transform((data) => ({
...data,
Expand All @@ -48,3 +56,13 @@ export const transformedUpdateIdSchema = updateIdAdminChangeSchema.transform((da
id: `${data.id}`,
timestamp: Date.now(),
}));

export const transformedSplitSPASchema = splitSPAAdminChangeSchema.transform((data) => ({
...data,
event: "split-spa",
packageId: data.id,
id: `${data.id}`,
timestamp: Date.now(),
makoChangedDate: Date.now(),
changedDate: Date.now(),
}));
5 changes: 2 additions & 3 deletions lib/lambda/update/updatePackage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,17 +101,15 @@ const sendUpdateIdMessage = async ({
origin: _origin,
...remainingFields
} = currentPackage._source;

if (updatedId === currentPackage._id) {
return response({
statusCode: 400,
body: { message: "New ID required to update package" },
});
}

// check if a package with this new ID already exists
const packageExists = await getPackage(updatedId);
if (packageExists?.found) {
if (packageExists) {
return response({
statusCode: 400,
body: { message: "This ID already exists" },
Expand All @@ -132,6 +130,7 @@ const sendUpdateIdMessage = async ({
}

await sendDeleteMessage(currentPackage._id);

await produceMessage(
topicName,
updatedId,
Expand Down
3 changes: 2 additions & 1 deletion lib/packages/shared-types/opensearch/changelog/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ export type Document = Omit<AppkDocument, "event"> &
| "withdraw-rai"
| "update-values"
| "update-id"
| "delete";
| "delete"
| "split-spa";
};

export type Response = Res<Document>;
Expand Down
11 changes: 11 additions & 0 deletions lib/stacks/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,17 @@ export class Api extends cdk.NestedStack {
indexNamespace,
},
},
{
id: "submitSplitSPA",
entry: join(__dirname, "../lambda/submit/submitSplitSPA.ts"),
environment: {
topicName,
brokerString,
osDomain: `https://${openSearchDomainEndpoint}`,
indexNamespace,
},
provisionedConcurrency: 2,
},
{
id: "getSystemNotifs",
entry: join(__dirname, "../lambda/getSystemNotifs.ts"),
Expand Down
3 changes: 2 additions & 1 deletion react-app/src/features/package/admin-changes/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@
export const AC_Update: FC<opensearch.changelog.Document> = () => {
return <p>Coming Soon</p>;
};

export const AdminChange: FC<opensearch.changelog.Document> = (props) => {
const [label, Content] = useMemo(() => {
switch (props.event) {
Expand All @@ -61,10 +60,12 @@
}
case "legacy-admin-change":
return [props.changeType || "Manual Update", AC_LegacyAdminChange];
case "split-spa":
return ["Package Added", AC_LegacyAdminChange];
default:
return [BLANK_VALUE, AC_Update];
}
}, [props.actionType, props.changeType]);

Check warning on line 68 in react-app/src/features/package/admin-changes/index.tsx

View workflow job for this annotation

GitHub Actions / lint

React Hook useMemo has missing dependencies: 'props.event' and 'props.raiWithdrawEnabled'. Either include them or remove the dependency array

return (
<AccordionItem key={props.id} value={props.id}>
Expand Down
Loading