Add ability to manually kick-off subrecipient file generation #497

as1729 · 2024-10-24T13:50:30Z

This PR exists because we noticed that partners are unable to get the subrecipients treasury report. In looking further, it appears that our filters for "valid" subrecipients is looking directly at the reporting period's start/end-dates but not looking at the month in which the reporting period's processing month. For example: Q3 begins and ends on 7/2 to 9/30, however, the reporting-month for q3 is 10/1 to 10/31. Therefore we need to look for subrecipients created in the month of October in order to build the subrecipients report.

This PR does the following:

Update to use the correct start/end-dates for filtering subrecipients
Scope subrecipients to organizationId. Noticed a bug where we were sending all organization's subrecipients to the file.
Give USDR_ADMIN's ability to regenerate subrecipient file manually.
Terraform permissions for graphql lambda to upload subrecipients file

…ting period

…ubrecipients are scoped to organization

github-actions · 2024-10-24T13:53:34Z

QA Summary

Pusher: @as1729, Action: pull_request_target, Workflow: Continuous Integration

See our documentation for tips on how to resolve failing QA checks.

QA Check	Result
🌐 Web Tests	✅
🔗 API Tests	✅
🐍 Python Tests	✅
📏 ESLint	✅
🧼 Ruff	✅
🧹 TFLint	✅

Test Coverage

View the workflow summary for individual coverage reports if this comment is truncated.

Coverage report for api suite

St	File	% Stmts	% Branch	% Funcs	% Lines	Uncovered Line #s
🟡	All files	51.03	33.83	57.27	51.51
🔴	src	0	100	0	0
🔴	server.ts	0	100	0	0	6-13
🟢	src/directives/requireAuth	100	100	100	100
🟢	requireAuth.ts	100	100	100	100
🟡	src/directives/skipAuth	50	100	0	50
🟡	skipAuth.ts	50	100	0	50	13
🔴	src/functions	0	100	0	0
🔴	graphql.ts	0	100	0	0	15-27
🔴	src/functions/processValidationJson	38.09	50	44.44	37.34
🟢	processValidationJson.scenarios.ts	100	100	100	100
🔴	processValidationJson.ts	37.34	50	44.44	36.58	59-98,118-119,157-165,177-178,193-196,201-254,267-268,282-321,333-336,344
🔴	src/lib	13.12	9.57	11.7	13.43
🟡	auth.ts	62.96	48.48	57.14	65.38	60-61,77-78,84-85,101-102,124,131,134,139-146,170,174
🔴	aws.ts	25.42	18.75	25	25.42	53-58,74-97,121-123,150-171,186-272
🟢	constants.ts	100	100	100	100
🔴	db.ts	45.45	50	50	45.45	15-35,41,43,50
🔴	ec-codes.ts	0	100	100	0	1
🟢	logger.ts	100	100	100	100
🔴	persist-upload.js	0	0	0	0	16-295
🔴	preconditions.ts	0	0	0	0	2-3
🔴	records.js	0	0	0	0	12-214
🔴	templateRules.ts	0	0	0	0
🔴	tracer.ts	0	100	100	0	5-14
🔴	validate-upload.js	0	0	0	0	18-790
🟢	validation-error.ts	83.33	100	50	83.33	22
🔴	validation-rules.js	0	0	0	0	6-194
🟡	src/services/agencies	67.34	50	80	67.34
🟢	agencies.scenarios.ts	100	100	100	100
🟡	agencies.ts	65.21	50	75	65.21	40-51,60-64,97-98,104,113-121
🟡	src/services/expenditureCategories	78.57	66.66	88.88	78.57
🟢	expenditureCategories.scenarios.ts	100	100	100	100
🟡	expenditureCategories.ts	77.77	66.66	88.88	77.77	30-34,49-52,60,91
🟡	src/services/inputTemplates	77.77	66.66	85.71	77.77
🟢	inputTemplates.scenarios.ts	100	100	100	100
🟡	inputTemplates.ts	76.92	66.66	85.71	76.92	25-29,39-40,50,85
🟡	src/services/organizations	75	90.9	50	75
🟢	organizations.scenarios.ts	100	100	100	100
🟡	organizations.ts	73.97	90.9	44.44	73.97	34-35,53-57,92,164-194,202,220-247
🟢	src/services/outputTemplates	82.85	66.66	85.71	82.85
🟢	outputTemplates.scenarios.ts	100	100	100	100
🟢	outputTemplates.ts	82.35	66.66	85.71	82.35	26-30,40-41,51,114
🟡	src/services/passage	74.07	62.5	100	74.07
🟡	passage.ts	74.07	62.5	100	74.07	18-19,65-76
🟡	src/services/projects	80	100	62.5	80
🟢	projects.scenarios.ts	100	100	100	100
🟡	projects.ts	78.57	100	62.5	78.57	45-51
🟢	src/services/reportingPeriodCertifications	100	100	100	100
🟢	reportingPeriodCertifications.scenarios.ts	100	100	100	100
🟢	reportingPeriodCertifications.ts	100	100	100	100
🟡	src/services/reportingPeriods	70.58	60	57.89	71.64
🟢	reportingPeriods.scenarios.ts	100	100	100	100
🟡	reportingPeriods.ts	68.75	60	50	69.84	15-27,44-48,58-59,74-77,93,116,124,165,188-209
🟢	src/services/subrecipientUploads	88.88	83.33	85.71	88.88
🟢	subrecipientUploads.scenarios.ts	100	100	100	100
🟢	subrecipientUploads.ts	86.36	83.33	80	86.36	64,94-99
🟢	src/services/subrecipients	88.67	81.81	92.3	88.67
🟢	subrecipients.scenarios.ts	100	100	100	100
🟢	subrecipients.ts	84.21	81.81	83.33	84.21	63,69,106-107,113-118
🟡	src/services/uploadValidations	57.14	100	14.28	57.14
🟢	uploadValidations.scenarios.ts	100	100	100	100
🟡	uploadValidations.ts	53.84	100	14.28	53.84	10,16,30,38,45-48
🟢	src/services/uploads	93.2	75	88.88	93.2
🟢	uploads.scenarios.ts	100	100	100	100
🟢	uploads.ts	90.78	75	75	90.78	40,107,135-149
🟢	src/services/users	85.61	82	88.88	85.61
🟢	users.scenarios.ts	100	100	100	100
🟢	users.ts	84.61	82	84.21	84.61	220,237,253,275-277,286-290,308-309,323-326,344-346,354-355,360,369-375
🟢	src/services/validationRuleses	85.71	100	71.42	85.71
🟢	validationRuleses.scenarios.ts	100	100	100	100
🟢	validationRuleses.ts	84.61	100	71.42	84.61	43-48

Coverage report for web suite

St	File	% Stmts	% Branch	% Funcs	% Lines	Uncovered Line #s
🔴	All files	17.56	22.71	14.97	17.24
🟢	api/src/lib	100	100	100	100
🟢	constants.ts	100	100	100	100
🔴	web/src	28.57	18.75	66.66	28.57
🔴	App.tsx	0	0	0	0	3-36
🟢	Routes.tsx	100	100	100	100
🟡	auth.ts	50	50	100	50	19-24
🔴	entry.client.tsx	0	0	100	0	10-22
🔴	web/src/auth	7.14	0	4.16	7.14
🔴	localAuth.ts	9.09	0	8.33	9.09	39-68,76-80
🔴	passageAuth.ts	5	0	0	5	22-25,31-60
🔴	web/src/components/Agency/Agencies	0	100	0	0
🔴	Agencies.tsx	0	100	0	0	9-21
🔴	web/src/components/Agency/AgenciesCell	0	100	0	0
🔴	AgenciesCell.tsx	0	100	0	0	8-39
🔴	web/src/components/Agency/Agency	0	0	0	0
🔴	Agency.tsx	0	0	0	0	10-78
🔴	web/src/components/Agency/AgencyCell	0	100	0	0
🔴	AgencyCell.tsx	0	100	0	0	7-27
🔴	web/src/components/Agency/AgencyForm	0	0	0	0
🔴	AgencyForm.tsx	0	0	0	0	25-45
🔴	web/src/components/Agency/EditAgencyCell	0	100	0	0
🔴	EditAgencyCell.tsx	0	100	0	0	10-59
🔴	web/src/components/Agency/NewAgency	0	100	0	0
🔴	NewAgency.tsx	0	100	0	0	9-35
🟢	web/src/components/Navigation	100	60	100	100
🟢	Navigation.tsx	100	60	100	100	24-68
🔴	web/src/components/Organization/EditOrganizationCell	0	100	0	0
🔴	EditOrganizationCell.tsx	0	100	0	0	13-64
🔴	web/src/components/Organization/EditOrganizationForm	0	0	0	0
🔴	EditOrganizationForm.tsx	0	0	0	0	27-41
🔴	web/src/components/Organization/NewOrganization	0	100	0	0
🔴	NewOrganization.tsx	0	100	0	0	9-37
🔴	web/src/components/Organization/NewOrganizationForm	0	0	0	0
🔴	NewOrganizationForm.tsx	0	0	0	0	25-54
🔴	web/src/components/Organization/Organization	0	0	0	0
🔴	Organization.tsx	0	0	0	0	10-70
🔴	web/src/components/Organization/OrganizationCell	0	100	0	0
🔴	OrganizationCell.tsx	0	100	0	0	7-28
🔴	web/src/components/Organization/OrganizationPickListsCell	40	0	27.27	36.36
🟡	OrganizationPickListsCell.mock.ts	50	100	0	100
🔴	OrganizationPickListsCell.stories.tsx	0	0	0	0	6-32
🟡	OrganizationPickListsCell.tsx	64.28	0	50	58.33	14-16,50-76
🔴	web/src/components/Organization/Organizations	0	100	0	0
🔴	Organizations.tsx	0	100	0	0	9-21
🔴	web/src/components/Organization/OrganizationsCell	0	100	0	0
🔴	OrganizationsCell.tsx	0	100	0	0	8-37
🔴	web/src/components/OutputTemplate/EditOutputTemplateCell	0	100	0	0
🔴	EditOutputTemplateCell.tsx	0	100	0	0	18-81
🔴	web/src/components/OutputTemplate/NewOutputTemplate	0	0	0	0
🔴	NewOutputTemplate.tsx	0	0	0	0	17-126
🔴	web/src/components/OutputTemplate/OutputTemplate	0	0	0	0
🔴	OutputTemplate.tsx	0	0	0	0	17-97
🔴	web/src/components/OutputTemplate/OutputTemplateCell	0	100	0	0
🔴	OutputTemplateCell.tsx	0	100	0	0	17-47
🔴	web/src/components/OutputTemplate/OutputTemplateForm	0	0	0	0
🔴	OutputTemplateForm.tsx	0	0	0	0	18-63
🔴	web/src/components/OutputTemplate/OutputTemplates	0	0	0	0
🔴	OutputTemplates.tsx	0	0	0	0	18-94
🔴	web/src/components/OutputTemplate/OutputTemplatesCell	0	100	0	0
🔴	OutputTemplatesCell.tsx	0	100	0	0	18-52
🔴	web/src/components/ReportingPeriod/EditReportingPeriodCell	0	100	0	0
🔴	EditReportingPeriodCell.tsx	0	100	0	0	13-74
🔴	web/src/components/ReportingPeriod/NewReportingPeriod	0	100	0	0
🔴	NewReportingPeriod.tsx	0	100	0	0	9-35
🔴	web/src/components/ReportingPeriod/ReportingPeriod	0	0	0	0
🔴	ReportingPeriod.tsx	0	0	0	0	12-101
🔴	web/src/components/ReportingPeriod/ReportingPeriodCell	0	100	0	0
🔴	ReportingPeriodCell.tsx	0	100	0	0	7-33
🔴	web/src/components/ReportingPeriod/ReportingPeriodForm	0	0	0	0
🔴	ReportingPeriodForm.tsx	0	0	0	0	18-43
🟡	web/src/components/ReportingPeriod/ReportingPeriods	71.42	38.46	55.55	71.42
🟡	ReportingPeriods.tsx	67.74	44.44	41.66	67.74	47-52,59-60,66,81,116-133
🟢	columns.tsx	81.81	25	83.33	81.81	36-40
🟡	web/src/components/ReportingPeriod/ReportingPeriodsCell	55	0	55.55	47.05
🟢	ReportingPeriodsCell.mock.ts	100	100	100	100
🔴	ReportingPeriodsCell.stories.tsx	0	0	0	0	6-32
🟢	ReportingPeriodsCell.tsx	100	100	100	100
🔴	web/src/components/Subrecipient/SubrecipientTableUploadLinksDisplay	0	0	0	0
🔴	SubrecipientTableUploadLinksDisplay.stories.tsx	0	100	100	0	5-82
🔴	SubrecipientTableUploadLinksDisplay.tsx	0	0	0	0	14-42
🔴	web/src/components/Subrecipient/Subrecipients	0	0	0	0
🔴	Subrecipients.tsx	0	100	0	0	5-8
🔴	columns.tsx	0	0	0	0	7-93
🔴	web/src/components/Subrecipient/SubrecipientsCell	0	100	0	0
🔴	SubrecipientsCell.tsx	0	100	0	0	7-60
🟢	web/src/components/TableBuilder	88.88	88.88	77.77	88.57
🟡	DebouncedInput.tsx	80	100	66.66	77.77	21,32
🟡	Filter.tsx	75	100	50	75	10
🟢	TableBuilder.tsx	87.5	66.66	75	87.5	40
🟢	TableHeader.tsx	100	91.66	100	100	13
🟢	TableRow.tsx	100	100	100	100
🟡	web/src/components/TemplateUploadReportingPeriodCell	55	0	55.55	47.05
🟢	TemplateUploadReportingPeriodCell.mock.ts	100	100	100	100
🔴	TemplateUploadReportingPeriodCell.stories.tsx	0	0	0	0	11-37
🟢	TemplateUploadReportingPeriodCell.tsx	100	100	100	100
🔴	web/src/components/TreasuryGeneration/DownloadTreasuryFiles	0	100	0	0
🔴	DownloadTreasuryFiles.tsx	0	100	0	0	6-30
🔴	web/src/components/TreasuryGeneration/NewTreasuryGeneration	0	100	0	0
🔴	NewTreasuryGeneration.tsx	0	100	0	0	8-39
🔴	web/src/components/TreasuryGeneration/NewTreasuryGenerationForm	0	0	0	0
🔴	NewTreasuryGenerationForm.tsx	0	0	0	0	20-31
🔴	web/src/components/Upload/EditUploadCell	0	100	0	0
🔴	EditUploadCell.tsx	0	100	0	0	10-66
🔴	web/src/components/Upload/NewUpload	0	100	0	0
🔴	NewUpload.tsx	0	100	0	0	7-35
🔴	web/src/components/Upload/Upload	0	0	0	0
🔴	Upload.stories.tsx	0	100	100	0	5-93
🔴	Upload.tsx	0	0	0	0	16-119
🔴	web/src/components/Upload/UploadCell	0	100	0	0
🔴	UploadCell.tsx	0	100	0	0	7-60
🔴	web/src/components/Upload/UploadForm	0	0	0	0
🔴	UploadForm.tsx	0	0	0	0	23-108
🔴	web/src/components/Upload/UploadValidationButtonGroup	0	0	0	0
🔴	UploadValidationButtonGroup.stories.tsx	0	100	0	0	5-47
🔴	[UploadValidationButtonGroup.tsx](https:...[Comment body truncated]

github-actions · 2024-10-24T13:55:10Z

Terraform Summary

Pusher: @as1729, Action: pull_request_target, Workflow: Continuous Integration

Step	Result
🖌 Terraform Format & Style	✅
⚙️ Terraform Initialization	✅
🤖 Terraform Validation	✅
📖 Terraform Plan	✅

Hint: If "Terraform Format & Style" failed, run terraform fmt -recursive from the terraform/ directory and commit the results.

Output

Validation Output

Success! The configuration is valid.

Plan Summary

CHANGE	RESOURCE
add	`aws_s3_object.origin_dist_artifact["static/js/app.4f06d289.js"]`
	`aws_s3_object.origin_dist_artifact["static/js/app.4f06d289.js.LICENSE.txt"]`
update	`aws_ecs_service.console`
	`aws_s3_object.origin_dist_artifact["200.html"]`
	`aws_s3_object.origin_dist_artifact["build-manifest.json"]`
	`aws_s3_object.origin_dist_artifact["chunk-references.json"]`
	`aws_s3_object.origin_dist_artifact["index.html"]`
	`module.lambda_function-cpfCreateArchive.aws_lambda_function.this[0]`
	`module.lambda_function-cpfValidation.aws_lambda_function.this[0]`
	`module.lambda_function-email-presigned-url.aws_lambda_function.this[0]`
	`module.lambda_function-graphql.aws_iam_policy.additional_inline[0]`
	`module.lambda_function-graphql.aws_lambda_function.this[0]`
	`module.lambda_function-processValidationJson.aws_lambda_function.this[0]`
	`module.lambda_function-subrecipientTreasuryReportGen.aws_lambda_function.this[0]`
	`module.lambda_function-treasuryProjectFileGeneration.aws_lambda_function.this[0]`
recreate	`aws_ecs_task_definition.console`
	`aws_s3_object.lambda_artifact-graphql`
	`aws_s3_object.lambda_artifact-processValidationJson`
	`aws_s3_object.lambda_artifact-python`
	`module.lambda_function-cpfCreateArchive.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]`
	`module.lambda_function-cpfValidation.aws_lambda_permission.current_version_triggers["S3BucketNotification"]`
	`module.lambda_function-email-presigned-url.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]`
	`module.lambda_function-graphql.aws_lambda_permission.current_version_triggers["APIGateway"]`
	`module.lambda_function-processValidationJson.aws_lambda_permission.current_version_triggers["S3BucketNotification"]`
	`module.lambda_function-subrecipientTreasuryReportGen.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]`
	`module.lambda_function-treasuryProjectFileGeneration.aws_lambda_permission.current_version_triggers["S3BucketNotification"]`
	`module.lambda_function-treasuryProjectFileGeneration.aws_lambda_permission.current_version_triggers["StepFunctionTrigger"]`
delete	`aws_s3_object.origin_dist_artifact["static/js/app.08f5b35d.js"]`
	`aws_s3_object.origin_dist_artifact["static/js/app.08f5b35d.js.LICENSE.txt"]`

as1729 added 2 commits October 24, 2024 07:44

feat: allow graphql lambda to upload subrecipient files

000c977

feat: add backend service to upload valid subrecipients for the repor…

9fcaabc

…ting period

github-actions bot added infra javascript terraform labels Oct 24, 2024

fix: ensure subrecipients are chosen based on closing-month. ensure s…

09fa8a7

…ubrecipients are scoped to organization

as1729 enabled auto-merge (squash) October 24, 2024 13:53

as1729 requested a review from TylerHendrickson October 24, 2024 13:53

fix: mock aws

77b8151

TylerHendrickson approved these changes Oct 24, 2024

View reviewed changes

as1729 merged commit 25a4836 into main Oct 24, 2024
22 checks passed

as1729 deleted the as/add-manual-subrecipient-generation branch October 24, 2024 14:42

as1729 mentioned this pull request Oct 24, 2024

feat: add front end button to upload subrecipients #498

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to manually kick-off subrecipient file generation #497

Add ability to manually kick-off subrecipient file generation #497

as1729 commented Oct 24, 2024 •

edited

Loading

github-actions bot commented Oct 24, 2024 •

edited

Loading

github-actions bot commented Oct 24, 2024 •

edited

Loading

Add ability to manually kick-off subrecipient file generation #497

Add ability to manually kick-off subrecipient file generation #497

Conversation

as1729 commented Oct 24, 2024 • edited Loading

github-actions bot commented Oct 24, 2024 • edited Loading

QA Summary

Test Coverage

github-actions bot commented Oct 24, 2024 • edited Loading

Terraform Summary

Output

as1729 commented Oct 24, 2024 •

edited

Loading

github-actions bot commented Oct 24, 2024 •

edited

Loading

github-actions bot commented Oct 24, 2024 •

edited

Loading