Overdue System

In this tutorial, we look a little bit deeper into Kill Bill’s payment retry system and overdue system. The main Subscription User Guide already explains some basics about these systems. This tutorial will show you how they work in practice.

Prerequisites

Ensure that you have configured Kill Bill, Kaui and the database as explained in the Getting Started document.
Ensure that you have set up a tenant configured with API key bob and API secret lazar.
Ensure that the org.killbill.server.test.mode=true property is set as explained in the Configuration Guide. This property allows moving the clock through time.
Ensure that you have gone through the Subscription User Guide and are familiar with catalogs, invoices, payments and other concepts.
Ensure that you have cURL installed. If you are on Windows, we recommend that you use Git Bash to run the cURL commands.

Overdue and Payment Retry System Overview

Before we dive into the details, let us understand some high-level concepts.

Kill Bill has two distinct built-in mechanisms. These are the Payment Retry System and the Overdue System. Though separate, these two systems work in close conjunction with each other.

The Payment Retry System is a system that retries failed payments as per a defined schedule. The schedule can be defined via a property in the Kill Bill configuration file or can be uploaded on a per-tenant basis.

The Overdue System defines the flow that the system must follow when an account is overdue (that is, has an unpaid balance). It can be configured via an XML file. The XML file can be specified via a property in the Kill Bill configuration file or can be uploaded on a per-tenant basis. The XML file defines the various states that the account must go through, the change in the user’s entitlements when the account is transitioned from one state to the other and the period after which the state is re-evaluated.

It is important to understand that the Payment Retry System and the Overdue System are independent of each other. So, the payment retry system attempts payments as per its retry schedule while the overdue system moves the account from one state to the other as per its configuration.

Having said that, most business cases would require to retry payments several times before blocking the user’s access and thus would require using both systems in conjunction with one another.

Although the payment retry system and overdue systems need to be configured independently of each other, it makes sense to align payment retries with overdue states. This allows retrying the payment on the boundary of each overdue state and thus prevents transitioning from one state to the next if the payment gets retried successfully. Further, to ensure that the payment is always retried before the overdue system, it makes sense to configure a payment retry 1 day before each overdue state.

Scenario

Let us now define a test scenario and understand how to configure the payment retry system and overdue system.

Let’s assume you want to build a website where your customers can subscribe to movies (similar to Netflix). For simplicity’s sake, we will assume you only provide one $10/month plan for unlimited streaming. The plan will contain a 10 days trial period during which customers have unlimited access. After the trial period, the customer should be invoiced with $10. We would also want to take some actions if the user fails to pay:

The system should be configured to retry failed payments 4 times: the first payment retry should happen 1 day after the payment failure, the next attempt should happen 8 days after previous attempt, the third attempt should happen 4 days after the previous attempt and the last one should happen 7 days after the previous attempt
The system should notify the user when an invoice has not been paid after 10 days (during which time the system already automatically retried 2 times)
The system should block the entitlements associated with the account after 14 days
The system should automatically cancel the subscriptions after 21 days

Additionally, we want to prevent users from changing plans (e.g. upgrade) until they clear their balance.

So, the scenario specified above requires configuring 3 overdue states. The account should be moved to the first overdue state 10 days after an unpaid invoice, it should move to the second overdue state 14 days after the unpaid invoice and it should be moved to the final overdue state 21 days after the unpaid invoice.

Further, the scenario above also specifies the payment retries in such a way that they align with the overdue states. So, it requires the second payment attempt to be made after 9 days (just before the first overdue state), the third payment attempt after 13 days (before the second overdue state) and the final payment attempt after 20 days (before the last overdue state). Thus, we are configuring the system to retry payments 1 day before each overdue state.

Let us now understand how to to configure both the Payment Retry System and the Overdue system in order to implement this scenario.

Configuration of the System

Catalog Configuration

First of all, you will need to create a catalog. Your catalog will contain a plan entry that specifies two phases, one for the trial period of 10 days and one for the recurring monthly charge of $10:

<?xml version="1.0" encoding="UTF-8"?>
<catalog xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="CatalogSchema.xsd ">
    <effectiveDate>2013-02-08T00:00:00+00:00</effectiveDate>
    <catalogName>Movies</catalogName>
    <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
    <currencies>
        <currency>USD</currency>
    </currencies>
    <products>
        <product name="Movies">
            <category>BASE</category>
        </product>
    </products>
    <rules>
        <changePolicy>
            <changePolicyCase>
                <policy>IMMEDIATE</policy>
            </changePolicyCase>
        </changePolicy>
        <cancelPolicy>
            <cancelPolicyCase>
                <policy>IMMEDIATE</policy>
            </cancelPolicyCase>
        </cancelPolicy>
    </rules>
    <plans>
      <plan name="movies-monthly">
          <product>Movies</product>
          <initialPhases>
             <phase type="TRIAL">
                 <duration>
                     <unit>DAYS</unit>
                     <number>10</number>
                 </duration>
                 <fixed>
                   <fixedPrice>
                   </fixedPrice>
                 </fixed>
             </phase>
          </initialPhases>
          <finalPhase type="EVERGREEN">
              <duration>
                  <unit>UNLIMITED</unit>
              </duration>
              <recurring>
                  <billingPeriod>MONTHLY</billingPeriod>
                  <recurringPrice>
                      <price>
                          <currency>USD</currency>
                          <value>10.00</value>
                      </price>
                  </recurringPrice>
              </recurring>
          </finalPhase>
      </plan>
    </plans>
    <priceLists>
        <defaultPriceList name="DEFAULT">
            <plans>
                <plan>movies-monthly</plan>
            </plans>
        </defaultPriceList>
    </priceLists>
</catalog>

You can upload the catalog by hitting the following endpoint:

curl -v \
-u 'admin:password' \
-H "X-Killbill-ApiKey: bob" \
-H "X-Killbill-ApiSecret: lazar" \
-H "Content-Type: text/xml" \
-H "X-Killbill-CreatedBy: stephane" \
-X POST \
-d @/VALID_PATH/catalog.xml \
"http://127.0.0.1:8080/1.0/kb/catalog/xml"

In order to understand more about catalogs, you can refer to the Subscription Billing document.

Overdue Configuration

Next, you need to define the overdue configuration XML file.

Creating Overdue.xml File

Based on the scenario defined above, the overdue configuration will define the three states the account will go through:

The account will enter the WARNING state 10 days after the first unpaid invoice (Trial period will last for 10 days, after which a $10 invoice will be generated. 10 days after this (20 days after the subscription creation), the account needs to enter the WARNING state)
4 days later, the account will move to the BLOCKED state
7 days later, the account will move to the CANCELLATION state.

You can create an XML file corresponding to this overdue configuration as follows:

<overdueConfig>
   <accountOverdueStates>
       <initialReevaluationInterval>
           <unit>DAYS</unit><number>10</number>
       </initialReevaluationInterval>
       <state name="CANCELLATION">
           <condition>
               <timeSinceEarliestUnpaidInvoiceEqualsOrExceeds>
                   <unit>DAYS</unit><number>21</number>
               </timeSinceEarliestUnpaidInvoiceEqualsOrExceeds>
           </condition>
           <externalMessage>Reached CANCELATION</externalMessage>
           <subscriptionCancellationPolicy>END_OF_TERM</subscriptionCancellationPolicy>
       </state>
       <state name="BLOCKED">
           <condition>
               <timeSinceEarliestUnpaidInvoiceEqualsOrExceeds>
                   <unit>DAYS</unit><number>14</number>
               </timeSinceEarliestUnpaidInvoiceEqualsOrExceeds>
           </condition>
           <externalMessage>Reached BLOCKED</externalMessage>
           <blockChanges>true</blockChanges>
           <disableEntitlementAndChangesBlocked>false</disableEntitlementAndChangesBlocked>
           <autoReevaluationInterval>
               <unit>DAYS</unit><number>7</number>
           </autoReevaluationInterval>
       </state>
       <state name="WARNING">
           <condition>
               <timeSinceEarliestUnpaidInvoiceEqualsOrExceeds>
                   <unit>DAYS</unit><number>10</number>
               </timeSinceEarliestUnpaidInvoiceEqualsOrExceeds>
           </condition>
           <externalMessage>Reached WARNING</externalMessage>
           <blockChanges>true</blockChanges>
           <disableEntitlementAndChangesBlocked>false</disableEntitlementAndChangesBlocked>
           <autoReevaluationInterval>
               <unit>DAYS</unit><number>4</number>
           </autoReevaluationInterval>
       </state>
   </accountOverdueStates>
</overdueConfig>

In more detail:

initialReevaluationInterval is set to 10 (if no payment is posted, the overdue state is recomputed 10 days later to match timeSinceEarliestUnpaidInvoiceEqualsOrExceeds)
WARNING:
- timeSinceEarliestUnpaidInvoiceEqualsOrExceeds is set to 10 days as specified by the scenario
- blockChanges is set to true to prevent users from changing plans
- disableEntitlementAndChangesBlocked is set to false, since we still want the user to have access to the service
- autoReevaluationInterval is set to 4 days to make sure that if there is no payment (which would trigger a re-evaluation of the overdue state), the state will be correctly recomputed and transition to BLOCKED 4 days later
BLOCKED:
- blockChanges is set to true to prevent users from changing plans
- timeSinceEarliestUnpaidInvoiceEqualsOrExceeds is set to 14 days as specified by the scenario
- disableEntitlementAndChangesBlocked is also set to false. This configuration also has an impact on the billing so setting it to true would block the billing (and entitlement) from that date forward.
- autoReevaluationInterval is set to 8 days to make sure that if there is no payment (no overdue trigger), the state will be correctly recomputed and transition to CANCELLATION 8 days later
CANCELLATION:
- subscriptionCancellationPolicy is set to END_OF_TERM to indicate that subscriptions should be cancelled in such a way that we do not generate credit for the account (no proration)

Some Important Notes

The definition order of the states in the XML configuration file is important: You must have the first state at the bottom and then all the way up to the last state (as shown in our example).
The overdue state names (name attribute of state tag) should not contain spaces. In case a space is specified in the state name, an error will occur while uploading the overdue XML file.

Uploading Overdue Configuration

The path of the overdue configuration file can be specified as a property in the Kill Bill Configuration File as follows:

org.killbill.overdue.uri=file:///<path>/overdue.xml

Alternately, you can upload the overdue configuration on a per-tenant basis by hitting the following endpoint:

curl -v \
-u 'admin:password' \
-H "X-Killbill-ApiKey: bob" \
-H "X-Killbill-ApiSecret: lazar" \
-H "Content-Type: text/xml" \
-H "X-Killbill-CreatedBy: stephane" \
-X POST \
-d @/VALID_PATH/overdue.xml \
"http://127.0.0.1:8080/1.0/kb/overdue/xml"

Payment Retries System Configuration

The scenario defined above also requires the payments to be attempted 4 times with a gap of 1,8, 4 and 7 days between attempts. Thus, the payment retry system needs to be configured as per this schedule.

The payment retry schedule can be configured as a property in the Kill Bill Configuration File as follows:

org.killbill.payment.retry.days=1,8,4,7

Alternately, you can upload the payment retry schedule on a per-tenant basis by hitting the following endpoint:

curl -v \
-u admin:password \
-H "X-Killbill-ApiKey: bob" \
-H 'X-Killbill-ApiSecret: lazar' \
-H "Content-Type: text/plain" \
-H 'X-Killbill-CreatedBy: stephane' \
-X POST \
--data-binary '{"org.killbill.payment.retry.days":"1,8,4,7"}' \
"http://127.0.0.1:8080/1.0/kb/tenants/uploadPerTenantConfig"

Additionally, if you need more granularity in how you want to retry payments, you can implement your own logic in a Payment Control Plugin (which goes beyond the scope of this tutorial).

Some Customer Flows

Let us now take a look at some typical customer flows and understand how the payment retry system and overdue system will function.

Flow 1 - Good Customer, No issue with payment

Let us first understand what happens when there is no issue with the payment:

Immediately after the customer subscribes, a $0 invoice is created to indicate the customer is in TRIAL. 10 days later, the customer moves out of TRIAL and the system generates a $10 invoice for the month. At this point, the system attempts to make a payment which is successful.

So, in this case, the payment retry system DOES NOT kick-in.

Flow 2 - Good Customer, fixes payment issue during retry flow

Next, suppose the customer’s credit card does not have enough funds to cover the payment initially but the customer fixes this issue later on:

As before, the system attempts a payment after the trial period. Since this payment fails, the payment retry system is activated. It retries a payment 1 day later (on 2021-08-07). The customer then fixes his/her credit card issue on 2021-08-09. The payment retry system which is scheduled to run again 8 days later runs on 2021-08-15. At this point, the payment is successful, so no more payments are attempted.

Flow 3 - Good Customer, fixes payment issue during overdue flow

In the next scenario, let us see what happens when the customer fixes his/her credit card issue after the overdue flow has kicked in:

As before, the system attempts a payment after the trial period which fails. The payment retry system then attempts the payment after 1 day(on 2021-08-07) and again after 8 days(on 2021-08-15). On 2021-08-16, 10 days have elapsed since the first unpaid invoice. So, the overdue system is activated which moves the account to the WARNING state. The customer then fixes his/her credit card issue on 2021-08-18.The payment retry system (which has already made 2 payment attempts), is scheduled to make the third payment attempt 4 days after the previous payment attempt. The previous payment attempt was on 2021-08-15 so the payment retry system attempts a payment on 2021-08-19. Since the payment issue is now fixed, the payment is successful. So, the account is moved back to the CLEAR state.

Flow 4 - Bad Customer, does not fix payment issue

Let us now consider what happens when the customer is a bad customer and does not fix the payment issue: