Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(issue and respond to RAIs): Implement post sub actions for issuing and responding to RAIs #197

Merged
merged 103 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
5917321
Show issue rai for all records for cms user
mdial89f Nov 7, 2023
76081ac
basic routing setup
mdial89f Nov 7, 2023
9943c17
base
mdial89f Nov 7, 2023
9970b9c
form work
mdial89f Nov 7, 2023
e164a3e
nothing
mdial89f Nov 7, 2023
f2d7ab6
basic func
mdial89f Nov 7, 2023
c045e36
sink
mdial89f Nov 7, 2023
1a92952
Merge branch 'master' into issuerai
mdial89f Nov 7, 2023
8b520a5
yut
mdial89f Nov 7, 2023
b4bbbeb
brians lib
mdial89f Nov 7, 2023
d8c4af4
fix thing
mdial89f Nov 7, 2023
43eaf99
bringing in brian
mdial89f Nov 7, 2023
00eff86
add functionality for sinking actions to opensearch
13bfrancis Nov 8, 2023
053441f
remove action-type from schema (not needed)
13bfrancis Nov 8, 2023
0afb047
fix breadcrumb type error
13bfrancis Nov 8, 2023
2f6e865
finished up flow (fingers crossed)
13bfrancis Nov 8, 2023
9976017
fix typescript error for tombstone
13bfrancis Nov 8, 2023
a90e233
get the last item in array
mdial89f Nov 8, 2023
f195a1f
update with timestamp from frontend
mdial89f Nov 8, 2023
0e1919c
update status
mdial89f Nov 8, 2023
cc18f29
move to using a transaction with rollback etc
mdial89f Nov 8, 2023
f840a61
Merge branch 'issuerai' into idm
mdial89f Nov 8, 2023
e3a2575
REVERTME push raw onemac records to other index
mdial89f Nov 8, 2023
15ae9ad
fix thing
mdial89f Nov 8, 2023
297db86
fix for attachments ullish
mdial89f Nov 8, 2023
639c859
increase timeout
mdial89f Nov 8, 2023
99f0fe0
stop custom compaction for onemac
mdial89f Nov 9, 2023
665fc5e
fix deletes
mdial89f Nov 9, 2023
f17f645
udpates per brian
mdial89f Nov 9, 2023
37cf5c1
pass action on enable thing
mdial89f Nov 9, 2023
f9aaa09
asdf
mdial89f Nov 9, 2023
c1a2418
Change how we get state
mdial89f Nov 9, 2023
790bcf2
put id in the payload, as it doesnt come back from safe parse
mdial89f Nov 9, 2023
89e0e0b
sending stuff to onemac
mdial89f Nov 9, 2023
fc11fbc
yut
mdial89f Nov 9, 2023
2d211f4
rework. change state cms to request response
mdial89f Nov 10, 2023
954460c
set sinkSeatool to start putting the timestamps in the right place
mdial89f Nov 10, 2023
c73c518
fixes
mdial89f Nov 10, 2023
044cebd
breadcrumbs
mdial89f Nov 10, 2023
1011c96
turn modal on
mdial89f Nov 10, 2023
77a4673
cleanup
mdial89f Nov 10, 2023
bf61cfe
show rais
mdial89f Nov 10, 2023
71f962b
remove check
mdial89f Nov 10, 2023
69e3555
rename to make more sense
mdial89f Nov 10, 2023
5a8d824
update getAttachmentUrl to allow download of rai attachments
mdial89f Nov 10, 2023
1cdd707
change format
mdial89f Nov 10, 2023
ef4c53c
rm debug
mdial89f Nov 10, 2023
8d0b2b3
handle old rais
mdial89f Nov 10, 2023
df4af5a
ok
mdial89f Nov 10, 2023
b8e7f26
k
mdial89f Nov 10, 2023
8469bdf
add a thing to catch rais with no requested date
mdial89f Nov 10, 2023
e62c436
functional ui etc
mdial89f Nov 10, 2023
54f0e59
add seatool status
mdial89f Nov 11, 2023
04fcfdd
correction
mdial89f Nov 11, 2023
17704dd
pushing the logic on package actions further
mdial89f Nov 11, 2023
71e7cae
start showing response data and fix some issues
mdial89f Nov 11, 2023
2bac638
working with reindex
mdial89f Nov 14, 2023
7be46d0
danger
mdial89f Nov 14, 2023
b08492c
set explicit dep
mdial89f Nov 14, 2023
5eb91b3
Add explicit dependency so we dont start sinking before weve setup th…
mdial89f Nov 14, 2023
f009f48
frontend stuff
mdial89f Nov 15, 2023
b91f54a
yut
mdial89f Nov 15, 2023
ac8d709
get actie
mdial89f Nov 15, 2023
ef09525
fix get package actions
mdial89f Nov 15, 2023
3953ffb
functional, but the timestamps on response is getting jacked
mdial89f Nov 15, 2023
9dc65db
Merge branch 'master' into engine
mdial89f Nov 15, 2023
79b929e
open collapsed
mdial89f Nov 15, 2023
c8e1e77
enumerte and stuff
mdial89f Nov 16, 2023
d0b579d
updates
mdial89f Nov 16, 2023
5dfe958
https://qmacbis.atlassian.net/browse/OY2-26267 fix
mdial89f Nov 16, 2023
ac23765
fixes etc
mdial89f Nov 16, 2023
5c0481b
Add submitter name
mdial89f Nov 16, 2023
f246cb5
Whered he go Whered whooooo go. REVERT ME
mdial89f Nov 16, 2023
58d46d4
Fix logic
mdial89f Nov 16, 2023
716258b
who are we to limit ourselves. how arrogant
mdial89f Nov 16, 2023
7bb46f3
correct helper
mdial89f Nov 16, 2023
04b6d9c
Fix attachmenturl thing to include response attachments
mdial89f Nov 16, 2023
4ad7864
some fixes per padma
mdial89f Nov 16, 2023
e819786
Merge branch 'master' into engine
mdial89f Nov 16, 2023
8f4a6d1
drop these cases
mdial89f Nov 16, 2023
e48d5ca
fixes
mdial89f Nov 16, 2023
c367670
Let all pending statuses issue an rai
mdial89f Nov 16, 2023
29624ab
response fixes
mdial89f Nov 16, 2023
e195c6c
yut
mdial89f Nov 16, 2023
e00e3b2
modals for issue
mdial89f Nov 17, 2023
3a93f46
modals for response
mdial89f Nov 17, 2023
d4241b4
remove dev thing
mdial89f Nov 17, 2023
dcc0996
i think this looks bad and is redundant
mdial89f Nov 17, 2023
c073dd9
language
mdial89f Nov 17, 2023
b8bf36a
11
mdial89f Nov 17, 2023
a269b9a
error handling and fixes
mdial89f Nov 17, 2023
73e4796
fix 13
mdial89f Nov 17, 2023
14097df
add package details consistently
mdial89f Nov 17, 2023
647ba66
put a sentence back
mdial89f Nov 17, 2023
65cad99
language
mdial89f Nov 17, 2023
86566a7
fix up role logic.. user helper could probably go for some refactoring
mdial89f Nov 17, 2023
02dbf8e
Update text at bottom
mdial89f Nov 17, 2023
6d08e42
fix broken disable bit
mdial89f Nov 18, 2023
ac24da2
use blank value
mdial89f Nov 17, 2023
97452b0
remove
mdial89f Nov 17, 2023
c26db03
update faq links
mdial89f Nov 17, 2023
edea356
drop unneeded markup
mdial89f Nov 17, 2023
69e80a8
little bit of cleanup
mdial89f Nov 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions src/libs/opensearch-lib.ts
Original file line number Diff line number Diff line change
Expand Up @@ -117,4 +117,40 @@ export async function getItem(host:string, index:string, id:string){
} catch(e) {
console.log({e})
}
}

export async function createIndexIfNotExists(host:string, index:string) {
client = client || (await getClient(host));
try {
const indexExists = await client.indices.exists({ index, });
if (!indexExists.body) {
const createResponse = await client.indices.create({
index,
});

console.log('Index created:', createResponse);
} else {
console.log('Index already exists.');
}
} catch (error) {
console.error('Error creating index:', error);
throw error;
}
}

export async function updateFieldMapping(host:string, index:string, properties: object) {
client = client || (await getClient(host));
try {
const response = await client.indices.putMapping({
index: index,
body: {
properties,
},
});

console.log('Field mapping updated:', response);
} catch (error) {
console.error('Error updating field mapping:', error);
throw error;
}
}
1 change: 0 additions & 1 deletion src/packages/shared-types/action-types/withdraw-record.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,4 @@ import { z } from "zod";
export const withdrawRecordSchema = z.object({
raiWithdrawEnabled: z.boolean(),
});

export type WithdrawRecord = z.infer<typeof withdrawRecordSchema>;
1 change: 1 addition & 0 deletions src/packages/shared-types/actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ export enum Action {
ENABLE_RAI_WITHDRAW = "enable-rai-withdraw",
DISABLE_RAI_WITHDRAW = "disable-rai-withdraw",
ISSUE_RAI = "issue-rai",
RESPOND_TO_RAI = "respond-to-rai",
}
204 changes: 140 additions & 64 deletions src/packages/shared-types/onemac.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,88 @@ const onemacAttachmentSchema = z.object({
key: z.string().nullish(),
uploadDate: z.number().nullish(),
});
export type OnemacAttachmentSchema = z.infer<typeof onemacAttachmentSchema>;

export const raiSchema = z.object({
id: z.string(),
requestedDate: z.number(),
responseDate: z.number().nullish(),
withdrawnDate: z.number().nullish(),
attachments: z.array(onemacAttachmentSchema).nullish(),
additionalInformation: z.string().nullable().default(null),
submitterName: z.string(),
submitterEmail: z.string(),
});
export type RaiSchema = z.infer<typeof raiSchema>;

export interface RaiData {
[key: number]: {
requestedDate?: string;
responseDate?: string;
withdrawnDate?: string;
response?: {
additionalInformation: string;
submitterName: string | null;
submitterEmail: string | null;
attachments: any[] | null; // You might want to specify the type of attachments
};
request?: {
additionalInformation: string;
submitterName: string | null;
submitterEmail: string | null;
attachments: any[] | null; // You might want to specify the type of attachments
};
};
}

export const transformRaiIssue = (id: string) => {
return raiSchema.transform((data) => ({
id,
rais: {
[data.requestedDate]: {
request: {
attachments:
data.attachments?.map((attachment) => {
return handleAttachment(attachment);
}) ?? null,
additionalInformation: data.additionalInformation,
submitterName: data.submitterName,
submitterEmail: data.submitterEmail,
},
},
},
}));
};
export type RaiIssueTransform = z.infer<ReturnType<typeof transformRaiIssue>>;

export const transformRaiResponse = (id: string) => {
return raiSchema.transform((data) => ({
id,
rais: {
[data.requestedDate]: {
response: {
attachments:
data.attachments?.map((attachment) => {
return handleAttachment(attachment);
}) ?? null,
additionalInformation: data.additionalInformation,
submitterName: data.submitterName,
submitterEmail: data.submitterEmail,
},
},
},
}));
};
export type RaiResponseTransform = z.infer<
ReturnType<typeof transformRaiResponse>
>;

export const onemacSchema = z.object({
additionalInformation: z.string().nullable().default(null),
submitterName: z.string(),
submitterEmail: z.string(),
attachments: z.array(onemacAttachmentSchema).nullish(),
raiWithdrawEnabled: z.boolean().optional(),
raiWithdrawEnabled: z.boolean().default(false),
raiResponses: z
.array(
z.object({
Expand All @@ -30,75 +105,76 @@ export const onemacSchema = z.object({
});

export const transformOnemac = (id: string) => {
return onemacSchema.transform((data) => ({
id,
attachments:
data.attachments?.map((attachment) => {
// this is a legacy onemac attachment
let bucket = "";
let key = "";
let uploadDate = 0;
if ("bucket" in attachment) {
bucket = attachment.bucket as string;
}
if ("key" in attachment) {
key = attachment.key as string;
}
if ("uploadDate" in attachment) {
uploadDate = attachment.uploadDate as number;
}
if (bucket == "") {
const parsedUrl = s3ParseUrl(attachment.url || "");
if (!parsedUrl) return null;
bucket = parsedUrl.bucket;
key = parsedUrl.key;
uploadDate = parseInt(attachment.s3Key?.split("/")[0] || "0");
}

return {
title: attachment.title,
filename: attachment.filename,
uploadDate,
bucket,
key,
return onemacSchema.transform((data) => {
const transformedData = {
id,
attachments:
data.attachments?.map((attachment) => {
return handleAttachment(attachment);
}) ?? null,
raiWithdrawEnabled: data.raiWithdrawEnabled,
additionalInformation: data.additionalInformation,
submitterEmail: data.submitterEmail,
submitterName: data.submitterName === "-- --" ? null : data.submitterName,
origin: "oneMAC",
rais: {} as RaiData,
};
if (data.raiResponses) {
data.raiResponses.forEach((raiResponse, index) => {
// We create an rai keyed off the index, because we don't know which rai it was in response to. Best we can do.
transformedData["rais"][index] = {
responseDate: raiResponse.submissionTimestamp.toString(),
response: {
additionalInformation: raiResponse.additionalInformation || "",
submitterName: null,
submitterEmail: null,
attachments:
raiResponse.attachments?.map((attachment) => {
return handleAttachment(attachment);
}) ?? null,
},
};
}) ?? null,
raiWithdrawEnabled: data.raiWithdrawEnabled,
raiResponses:
data.raiResponses?.map((response) => {
return {
additionalInformation: response.additionalInformation,
submissionTimestamp: response.submissionTimestamp,
attachments:
response.attachments?.map((attachment) => {
const uploadDate = parseInt(
attachment.s3Key?.split("/")[0] || "0"
);
const parsedUrl = s3ParseUrl(attachment.url || "");
if (!parsedUrl) return null;
const { bucket, key } = parsedUrl;

return {
...attachment,
uploadDate,
bucket,
key,
};
}) ?? null,
};
}) ?? null,
additionalInformation: data.additionalInformation,
submitterEmail: data.submitterEmail,
submitterName: data.submitterName === "-- --" ? null : data.submitterName,
origin: "oneMAC",
}));
});
}
return transformedData;
});
};

export type OneMacSink = z.infer<typeof onemacSchema>;
export type OneMacTransform = z.infer<ReturnType<typeof transformOnemac>>;
export type OneMacRecordsToDelete = Omit<
{
[Property in keyof OneMacTransform]: undefined;
[Property in keyof OneMacTransform]: null;
},
"id"
"id" | "rais"
> & { id: string };

function handleAttachment(attachment: OnemacAttachmentSchema) {
let bucket = "";
let key = "";
let uploadDate = 0;
if ("bucket" in attachment) {
bucket = attachment.bucket as string;
}
if ("key" in attachment) {
key = attachment.key as string;
}
if ("uploadDate" in attachment) {
uploadDate = attachment.uploadDate as number;
}
if (bucket == "") {
const parsedUrl = s3ParseUrl(attachment.url || "");
if (!parsedUrl) return null;
bucket = parsedUrl.bucket;
key = parsedUrl.key;
uploadDate = parseInt(attachment.s3Key?.split("/")[0] || "0");
}

return {
title: attachment.title,
filename: attachment.filename,
uploadDate,
bucket,
key,
};
}
6 changes: 4 additions & 2 deletions src/packages/shared-types/opensearch.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { SeaToolTransform } from "./seatool";
import { OneMacTransform } from "./onemac";
import { OneMacTransform, RaiIssueTransform } from "./onemac";
import { Action } from "./actions";

export type OsHit<T> = {
Expand Down Expand Up @@ -32,7 +32,9 @@ export type OsResponse<T> = {
aggregations?: OsAggResult;
};

export type OsMainSourceItem = OneMacTransform & SeaToolTransform;
export type OsMainSourceItem = OneMacTransform &
SeaToolTransform &
RaiIssueTransform;
export type OsMainSearchResponse = OsResponse<OsMainSourceItem>;
export type SearchData = OsHits<OsMainSourceItem>;
export type ItemResult = OsHit<OsMainSourceItem> & {
Expand Down
Loading
Loading