Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP get billing preview when holiday stop api is called #2243

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

johnduffell
Copy link
Member

@johnduffell johnduffell commented May 9, 2024

The holiday stop system creates (small) negative charges every time a subscription is suspended, one for each day that is not to be delivered.
These charges are added just before the fulfilment file is generated, and the dates are set up so that the credit due to the customer will be invoiced on the bill run when their next payment is due.

The date must be accurate - if it's correct then the bill run will collate all the charges and reduce the invoice value appropriately. If it's wrong then a negative invoice will be generated on the incorrect date, and the full invoice will be generated on the correct date. The negative invoice will just sit there for ever and the user will be overcharged, unless they notice and speak with a CSR who can manually refund the amount due.

The issue in this case is that the date calculation is not foolproof - at present the code does not work for older, migrated paper subscriptions which have a pending price rise amendment. At the moment, with several print price rises in progress, this is occurring quite often.

This is because the code assumes that either the charge effective date indicates the billing day, or the charge trigger day will indicate which date should be pulled from the subscription e.g. customer acceptance date as the relevant date. It chooses between the two based on the processedThroughDate.

The issue in this case is that the processedThroughDate is empty when the subscription is in pending price rise (as the latest charge has not been processed at all). The code defaults to the charge trigger day logic, which then incorrectly looks at the contract dates on the subscription, rather than on the amendment. These older subscriptions tend to have irrelevant dates to the current charges and amendments, and it ends up just plain wrong.

Possible fixes:

  1. get it to fetch the amendment instead of the contract and use the dates from there
  2. get it to use the invoicing api which seemed to be prototyped in the past for this
  3. do a billing preview and use that instead of predicting ourselves (as demonstrated by the invoicing-api in option 2)
  4. allow negative invoices to be generated in this/all cases, and revive the negative invoice to credit balance step functions - currently these are broken, although the step function is still running daily

This PR is the first part of implementing 3 (billing preview)
It fetches the billing preview for that account and reads in the dates.

Next steps:

  1. refactor the SubscriptionData logic to current practices (add names to apply methods, prefer case classes and standalone functions over traits, pass traits instead of function types around)
  2. pull the data through into the SubscriptionData logic, and group it by date, then we can work out the invoice after each suspended issue and log whether it matches the existing
  3. monitor the logs over the coming weeks
  4. if we are happy with the new logic, switch over and delete the old.

@johnduffell johnduffell requested a review from graham228221 May 9, 2024 20:45
@@ -171,12 +179,12 @@ object Handler extends Logging {
}
case "hsr" :: Nil =>
httpMethod match {
case "POST" => stepsToCreate(getAccessToken, getSubscription, getAccount) _
case "POST" => stepsToCreate(getAccessToken, getSubscription, getAccount, getBillingPreview, today) _
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coincidentally @tjmw has got a bit of refactoring on this to get rid of the underscores and neaten up the types. So we might end up with a small conflict in this area.

billingPreview <- getBillingPreview
.getBillingPreview(accessToken, subscription.accountNumber, today.plusMonths(13))
.toApiGatewayOp(s"get account ${subscription.accountNumber}")
_ = logger.info("billingPreview: " + billingPreview)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this stage it just logs the response rather than passing it in to the subscriptiondata. we could try to paralellise the getAccount and getBillingPreview if we want to improve performance.

)
implicit val decode: Decoder[InvoiceItem] = (c: HCursor) =>
for {
chargeDate <- c.downField("chargeDate").as[String].map(_.takeWhile(_ != ' ')).map(LocalDate.parse)
Copy link
Member Author

@johnduffell johnduffell May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zuora returns a simplified date-time kind of this format yyyy-mm-dd hh:mm:ss that doesn't parse as a LocalDate, I did this as we're only interested in the date part anyway.

config: Config,
backend: SttpBackend[Identity, Any],
): GetBillingPreview =
GetBillingPreviewLive.billingPreviewGetResponse(config.zuoraConfig, backend)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could inline this, I'm not sure why the convention is to have one function that calls another

Copy link
Contributor

@graham228221 graham228221 May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove Mario's invoicing-api experiment (testInProdPreviewPublications) as part of this? This is currently erroring due to a problem with invoicing-api credentials, but if it were working it would also be requesting a billing-preview.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed in #2504


val preview = op.getBillingPreview(accessToken, ACCOUNT_TO_USE, LocalDate.now.plusMonths(13))
info("preview is: " + preview)
preview.toOption.get.length shouldBe 26
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this line checking, out of interest?

we might want to be aware that this is a real account in the api sandbox, and its state may change (it currently has two active subscriptions, but one or both could be cancelled any time, or more could be acquired).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it's good to flag this - this is more like a manual test, as I couldn't think of a quick way to make it reliable. It should be left as "ignore" unless it's being used.
If we can make it reliable then that's even better and we should do that (although even then, it's an Effects test and we don't have them automated yet)

To answer your question it's just checking that 26 items were returned for the 13 months (rather than an error). so not really a completely useful assertion.

Copy link
Contributor

@graham228221 graham228221 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to approve as is - we might look at removing Mario's testInProd infrastructure in a next steps PR

# Conflicts:
#	handlers/holiday-stop-api/src/main/scala/com/gu/holiday_stops/Handler.scala
@johnduffell johnduffell changed the title get billing preview when holiday stop api is called WIP get billing preview when holiday stop api is called Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants