Driven by the entitlement and subscription modules, along with the catalog configuration, the invoice subsystem is core to Kill Bill. This manual will walk you through a couple of use-cases, to help you understand how it works under the covers and how to make sense of the data on disk.

Overview

Before we dig into examples, let’s review some of the characteristics of the invoicing algorithm. The first thing to recall is that there are different ways to invoice customers and this is specified inside the catalog configuration. For the sake of simplicity we will discard usage-billing altogether here and focus on recurring subscriptions (along with fixed-term charges).

Invoicing for recurring subscriptions also comes in two different fashions, or so called billing modes. Such a mode is defined at the Plan level-- with a possible default at the catalog level — and there are 2 options:

  • IN_ADVANCE: In such a configuration, each recurring subscription is charged at the begining of the period (in advance), and as a consequence, this can lead to pro-ration credits upon certain cancellation or change of Plan (special policies can be configured to make the system generate or not such pro-ration credits).

  • IN_ARREAR: In such a configuration, each recurring subscription is charged at the end of the period (in arrear), and as a consequence there is never any pro-ration credit generated because the system always know what to bill for (things can’t change back in time).

In the rest of this documentation, we will assume an IN_ADVANCE billing mode because this is the most common and most complex to deal with.

Let’s revisit some of the terminology:

  • chargedThroughDate or CTD: We call chargedThroughDate the last day of the recurring period associated with a given subscription that has been charged for; there is one such date for each subscription in the system.

  • targetDate: We call targetDate the date up to which we want the invoiving system to bill for. Let’s assume a monthly subscription and a current date of 2012-05-01 (which corresponds to our PHASE event in our example below), and let’s look at the impact of the targetDate:

    • targetDate < 2012-05-01 : Nothing happens

    • 2012-05-01 ⇐ targetDate < 2012-06-01 : System would bill for period [2012-05-01, 2012-06-01), and CTD = 2012-06-01

    • 2012-06-01 ⇐ targetDate < 2012-07-01 : System would bill for period [2012-05-01, 2012-06-01) and [2012-06-01, 2012-07-01) and CTD = 2012-07-01

Kill Bill always invoice at the account level, meaning each time an invoice is being generated, this is for a given Kill Bill Account (matching a given customer) and so the resulting invoice will include potentially different items matching different subscriptions associated with this particular account. In the most normal case, invoices are generated by reacting to specific events and we 'll see examples below of the system reacting to specific bus events or (future) notifications.

Initial subscription creation

Let’s assume a new user subscribes to the shotgun-monthly plan on 2012-04-01.

The plan is defined as follows:

<plan name="shotgun-monthly">
    <product>Shotgun</product>
    <initialPhases>
        <phase type="TRIAL">
            <duration>
                <unit>DAYS</unit>
                <number>30</number>
            </duration>
        </phase>
    </initialPhases>
    <finalPhase type="EVERGREEN">
        <duration>
            <unit>UNLIMITED</unit>
            <number>-1</number>
        </duration>
        <recurring>
            <billingPeriod>MONTHLY</billingPeriod>
            <recurringPrice>
                <price>
                    <currency>USD</currency>
                    <value>249.95</value>
                </price>
            </recurringPrice>
        </recurring>
    </finalPhase>
</plan>

Upon subscription creation, a new entry is added to the bundles and subscriptions tables, which look like this:

MySQL [killbill]> select * from bundles\G
*************************** 1. row ***************************
            record_id: 1
                   id: a7a1370e-1fa4-4c32-abd5-223e1da97339
         external_key: externalKey
           account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 last_sys_update_date: 2012-04-01 00:01:15
original_created_date: 2012-04-01 00:01:14
           created_by: website-through-http-client
         created_date: 2012-04-01 00:01:14
           updated_by: SubscriptionBaseTransition
         updated_date: 2012-04-01 00:01:15
    account_record_id: 1
     tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from subscriptions\G
*************************** 1. row ***************************
           record_id: 1
                  id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
           bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
            category: BASE
          start_date: 2012-04-01 00:01:14
   bundle_start_date: 2012-04-01 00:01:14
charged_through_date: 2012-04-01 00:00:00
            migrated: 0
          created_by: website-through-http-client
        created_date: 2012-04-01 00:01:15
          updated_by: SubscriptionBaseTransition
        updated_date: 2012-04-01 00:01:15
   account_record_id: 1
    tenant_record_id: 0
1 row in set (0.00 sec)

Associated with the subscription are a couple of subscription events, one CREATE marking the beginning of the subscription at the effective date 2012-04-01 00:01:14 and one PHASE event marking the date at which the EVERGREEN phase starts:

MySQL [killbill]> select * from subscription_events\G
*************************** 1. row ***************************
              record_id: 1
                     id: 55d5cc5e-2c58-4560-9042-f3c7115d6ccd
             event_type: API_USER
              user_type: CREATE
         effective_date: 2012-04-01 00:01:14
        subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
              plan_name: shotgun-monthly
             phase_name: shotgun-monthly-trial
        price_list_name: DEFAULT
billing_cycle_day_local: 0
              is_active: 1
             created_by: website-through-http-client
           created_date: 2012-04-01 00:01:15
             updated_by: website-through-http-client
           updated_date: 2012-04-01 00:01:15
      account_record_id: 1
       tenant_record_id: 0
*************************** 2. row ***************************
              record_id: 2
                     id: 8751c48e-686b-4eea-b959-52676e1bb9da
             event_type: PHASE
              user_type: NULL
         effective_date: 2012-05-01 00:01:14
        subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
              plan_name: NULL
             phase_name: shotgun-monthly-evergreen
        price_list_name: NULL
billing_cycle_day_local: 0
              is_active: 1
             created_by: website-through-http-client
           created_date: 2012-04-01 00:01:15
             updated_by: website-through-http-client
           updated_date: 2012-04-01 00:01:15
      account_record_id: 1
       tenant_record_id: 0
2 rows in set (0.00 sec)

The subscription service has also recorded a future notification effective when the EVERGREEN phase starts:

MySQL [killbill]> select * from notifications\G
*************************** 1. row ***************************
                record_id: 1
               class_name: org.killbill.billing.subscription.engine.core.SubscriptionNotificationKey
               event_json: {"eventId":"8751c48e-686b-4eea-b959-52676e1bb9da","seqId":0}
               user_token: f291917d-ce03-428f-9e58-e538db057d37
             created_date: 2012-04-01 00:01:15
           creating_owner: 127.0.0.1
         processing_owner: NULL
processing_available_date: NULL
         processing_state: AVAILABLE
              error_count: 0
              search_key1: 1
              search_key2: 0
               queue_name: subscription-service:subscription-events
           effective_date: 2012-05-01 00:01:14
        future_user_token: 892a1fdf-45b5-404d-a492-99f612ba8b55
1 row in set (0.00 sec)

The entitlement subsystem has also a record of the start of the entitlement, in the blocking_states table (on older Kill Bill versions, this was not present so you could still see some data were this is missing and this is fine, the system knows how to handle this case):

MySQL [killbill]> select * from blocking_states\G
*************************** 1. row ***************************
        record_id: 1
               id: 18696a69-bcb0-40c4-98b5-9c13bc00307e
     blockable_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
             type: SUBSCRIPTION
            state: ENT_STARTED
          service: entitlement-service
     block_change: 0
block_entitlement: 0
    block_billing: 0
   effective_date: 2012-04-01 00:01:15
        is_active: 1
     created_date: 2012-04-01 00:01:14
       created_by: website-through-http-client
     updated_date: 2012-04-01 00:01:14
       updated_by: website-through-http-client
account_record_id: 1
 tenant_record_id: 0
1 row in set (0.00 sec)

Upon subscription creation, a bus event is triggered and caught by the invoicing subsystem, which invoices the account with a target date of 2012-04-01. To do so, it computes the billing events from these subscription events and blocking states (they are effectively markers between billable periods). In our case, these billing events are:

DefaultBillingEvent{type=CREATE, effectiveDate=2012-04-01T00:01:14.000Z, planPhaseName=shotgun-monthly-trial, subscriptionId=d9c7bb57-675e-4419-a340-5f6b4fd612f4, totalOrdering=1}
DefaultBillingEvent{type=PHASE, effectiveDate=2012-05-01T00:01:14.000Z, planPhaseName=shotgun-monthly-evergreen, subscriptionId=d9c7bb57-675e-4419-a340-5f6b4fd612f4, totalOrdering=2}

The target date being 2012-04-01, only the first one matters. Based on the catalog configuration, the following invoice and invoice item are generated (an invoice has always 1 or more invoice items associated with it):

MySQL [killbill]> select * from invoices\G
*************************** 1. row ***************************
        record_id: 1
               id: 5c6369d2-cd18-489f-9fe5-748e72f9938e
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
     invoice_date: 2012-04-01
      target_date: 2012-04-01
         currency: USD
           status: COMMITTED
         migrated: 0
   parent_invoice: 0
       created_by: SubscriptionBaseTransition
     created_date: 2012-04-01 00:01:15
account_record_id: 1
 tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from invoice_items\G
*************************** 1. row ***************************
        record_id: 1
               id: 19667140-fa16-48e0-b04e-579b9972f612
             type: FIXED
       invoice_id: 5c6369d2-cd18-489f-9fe5-748e72f9938e
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
  subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
      description: shotgun-monthly-trial
        plan_name: shotgun-monthly
       phase_name: shotgun-monthly-trial
       usage_name: NULL
       start_date: 2012-04-01
         end_date: NULL
           amount: 0.000000000
             rate: NULL
         currency: USD
   linked_item_id: NULL
       created_by: SubscriptionBaseTransition
     created_date: 2012-04-01 00:01:15
account_record_id: 1
 tenant_record_id: 0

There is only a single FIXED item with a start date of 2012-04-01.

Upon invoice generation, an event is triggered and caught by the payment subsystem, which triggers a payment for that invoice (using the default payment method on the account).

What happens is that the payment subsystem calls the createPurchaseWithPaymentControl API and specifies INVOICE_PAYMENT_CONTROL_PLUGIN as the control plugin to use (you can add your own via the system property org.killbill.payment.invoice.plugin). This plugin is responsible to compute the payment amount (typically the invoice balance) and to insert a row in the invoice_payments table (success is first set to false, to implement a two-phase commit strategy). The payment is then delegated to the payment system (a payment and/or a transaction are recorded if necessary). Upon success, the entry is updated with the metadata from the payment (the plugin could have decided to pay less than the requested amount for example) and the success flag is set to true. In case of failure, success would remain set to false and the next payment retry date would be computed, based on the system configuration.

While the link between invoice and payments is encapsulated in the invoice_payments table, there is one level of indirection with the payments table through the payment_attempts table, to manage aborted payments and retries:

  • Aborted payments: In a situation where the invoice was already paid (or there is a $0 balance), the invoice control plugin would abort the payment. In such situations, we would end up with a row in the payment_attempts table with an ABORTED state and no row in the invoice_payments, payments and payment_transactions tables.

  • Payment Retries: In a situation where we see a payment failure (e.g. insufficient funds), a payment will be associated with multiple transactions (all sharing the same transaction external key and typically in a PAYMENT_FAILURE status). Each of these transactions will be associated with an attempt in a RETRIED state.

Note also that the payment_attempts entry is linked to the invoice via the plugin property IPCD_INVOICE_ID (which points to the invoice id).

In our scenario, no payment was actually processed, since the invoice amount is zero (trial). Hence the ABORTED state:

MySQL [killbill]> select * from payment_attempts\G
*************************** 1. row ***************************
               record_id: 1
                      id: 16f869b1-c5c9-41ed-a776-87f3ce4e5bb5
              account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
       payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
    payment_external_key: 6bd135f7-8a7d-4448-9ce9-3889055af9e3
          transaction_id: NULL
transaction_external_key: ae53501e-c9dd-45e3-8ec6-78da4e9f8d99
        transaction_type: PURCHASE
              state_name: ABORTED
                  amount: NULL
                currency: USD
             plugin_name: __INVOICE_PAYMENT_CONTROL_PLUGIN__
       plugin_properties: ZV  <[{"IPCD_INVOICE_ID":"5c6369d2-cd18-489f-9fe5-748e72f9938e"}]
              created_by: PaymentRequestProcessor
            created_date: 2012-04-01 00:01:15
              updated_by: PaymentRequestProcessor
            updated_date: 2012-04-01 00:01:15
       account_record_id: 1
        tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from invoice_payments\G
Empty set (0.00 sec)

MySQL [killbill]> select * from payments\G
Empty set (0.00 sec)

Note: if the payment didn’t go through because a custom payment control plugin aborted the payment (after the built-in INVOICE_PAYMENT_CONTROL_PLUGIN), there would be a row in invoice_payments (with the success flag set to false and a NULL payment_id) and a corresponding ABORTED row in payment_attempts.

See below for an example of an actual payment and what would happen in case of payment failures.

Reference time and fixed offset timezone

Each account in Kill Bill has a reference time associated with it. It is the reference_time value associated with the accounts row (specified at account creation time, default value is the created_date). Additionally, it is associated with a fixed offset timezone, a special timezone used by the system for dates manipulation which ignores changes like DST (you cannot change it either). This timezone depends on the account timezone (UTC if none specified) and if DST was in effect at the reference time (this lets us handle gracefully DST gaps throughout the year with respect to invoicing, handling subscription changes, etc.).

For example, if the account timezone is America/Los_Angeles and the reference time is 2015-03-07T10:00:01.000Z, DST was not in effect and the fixed offset timezone is -08:00. If the reference time however is 2015-03-08T10:00:01.000Z, DST was in effect and the fixed offset timezone is -07:00.

While most subsystems (entitlement, subscription, etc.) work at the time level (i.e. you could have several upgrades during the day), invoicing works at the day boundary level (Kill Bill doesn’t invoice for granularities smaller than a day). Both of these parameters help us convert LocalDate (i.e. a specify day in a year, like 2012-04-01) to a DateTime (i.e. a specific point in time, in a specific timezone), and vice versa. The former conversion is required for instance when computing the next notification time (based on the invoice item end date for instance). The latter can happen during invoice generation, when transforming a BillingEvent effective DateTime to a LocalDate for the invoice item (service period).

A nice side effect of using this fixed reference time is that most system-driven operations will always happen at the same time during the day for a given account. It also helps spreading the load on the system since the distribution of these reference times should be uniform.

Phase transition

Let’s fast forward the time to 2012-05-02.

The notification for the phase event is processed by the subscription subsystem. There is nothing to be done in that case (in other scenarios, add-ons may need to be cancelled or a future phase event may need to be computed): it simply sends a message on the bus letting the system know about the phase transition.

The invoicing subsystem picks it up and re-compute the billing events:

DefaultBillingEvent{type=CREATE, effectiveDate=2012-04-01T00:01:14.000Z, planPhaseName=shotgun-monthly-trial, subscriptionId=d9c7bb57-675e-4419-a340-5f6b4fd612f4, totalOrdering=1}
DefaultBillingEvent{type=PHASE, effectiveDate=2012-05-01T00:01:14.000Z, planPhaseName=shotgun-monthly-evergreen, subscriptionId=d9c7bb57-675e-4419-a340-5f6b4fd612f4, totalOrdering=2}

Nothing has changed but since the target date is now 2012-05-01, both events need to be taken into account. The invoice subsystem recomputes all invoice items since the beginning of time, and come up with a FIXED item (trial period) and a RECURRING item (for the service period 2012-05-01 to 2012-06-01). Because the FIXED item is already present in the database, only the second one is persisted on disk, on a new invoice:

MySQL [killbill]> select * from invoices order by record_id desc limit 1\G
*************************** 1. row ***************************
        record_id: 2
               id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
     invoice_date: 2012-05-02
      target_date: 2012-05-01
         currency: USD
           status: COMMITTED
         migrated: 0
   parent_invoice: 0
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:14:43
account_record_id: 1
 tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from invoice_items order by record_id desc limit 1\G
*************************** 1. row ***************************
        record_id: 2
               id: 2326d3ff-e90d-43f0-b611-6c028bb88c71
             type: RECURRING
       invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
  subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
      description: shotgun-monthly-evergreen
        plan_name: shotgun-monthly
       phase_name: shotgun-monthly-evergreen
       usage_name: NULL
       start_date: 2012-05-01
         end_date: 2012-06-01
           amount: 249.950000000
             rate: 249.950000000
         currency: USD
   linked_item_id: NULL
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:14:43
account_record_id: 1
 tenant_record_id: 0
1 row in set (0.00 sec)

The subscription charged_through_date is updated to 2012-06-01:

MySQL [killbill]> select * from subscriptions\G
*************************** 1. row ***************************
           record_id: 1
                  id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
           bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
            category: BASE
          start_date: 2012-04-01 00:01:14
   bundle_start_date: 2012-04-01 00:01:14
charged_through_date: 2012-06-01 00:00:00
            migrated: 0
          created_by: website-through-http-client
        created_date: 2012-04-01 00:01:15
          updated_by: SubscriptionBaseTransition
        updated_date: 2012-05-02 00:14:44
   account_record_id: 1
    tenant_record_id: 0

The system will also generate a new (future) notification on 2012-06-01. This invoice notification will be the trigger for the next invoice generation:

MySQL [killbill]>  select * from notifications\G
*************************** 1. row ***************************
                record_id: 2
               class_name: org.killbill.billing.invoice.notification.NextBillingDateNotificationKey
               event_json: {"uuidKey":"d9c7bb57-675e-4419-a340-5f6b4fd612f4","targetDate":"2012-06-01T00:00:00.000Z","isDryRunForInvoiceNotification":false}
               user_token: 892a1fdf-45b5-404d-a492-99f612ba8b55
             created_date: 2012-05-02 00:14:44
           creating_owner: 127.0.0.1
         processing_owner: NULL
processing_available_date: NULL
         processing_state: AVAILABLE
              error_count: 0
              search_key1: 1
              search_key2: 0
               queue_name: invoice-service:next-billing-date-queue
           effective_date: 2012-06-01 00:00:00
        future_user_token: aa2c96e2-71b4-4149-abdc-2889256c2b34
1 row in set (0.00 sec)

After the invoice is generated, an event is sent to the bus, which makes the payment subsystem react to it:

MySQL [killbill]> select * from invoice_payments order by record_id desc limit 1\G
*************************** 1. row ***************************
                record_id: 2
                       id: e6e534e1-2ffa-4d5e-bcac-6905d4d26f61
                     type: ATTEMPT
               invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
               payment_id: b0e61973-a921-413d-a04b-84e36e3ad6bf
             payment_date: 2012-05-02 00:14:44
                   amount: 249.950000000
                 currency: USD
       processed_currency: USD
        payment_cookie_id: 943d005c-5f89-4664-88f5-c65f39a3a9c8
linked_invoice_payment_id: NULL
                  success: 1
               created_by: PaymentRequestProcessor
             created_date: 2012-05-02 00:14:44
        account_record_id: 1
         tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_attempts order by record_id desc limit 1\G
*************************** 1. row ***************************
               record_id: 2
                      id: 090fa541-7b69-42b2-bec7-a16f3c616071
              account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
       payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
    payment_external_key: d04ce5ad-e667-4113-8eb3-6d7f87f92bca
          transaction_id: 8b671a2e-6556-4aa8-8464-ef1cb99e5189
transaction_external_key: 943d005c-5f89-4664-88f5-c65f39a3a9c8
        transaction_type: PURCHASE
              state_name: SUCCESS
                  amount: NULL
                currency: USD
             plugin_name: __INVOICE_PAYMENT_CONTROL_PLUGIN__
       plugin_properties: ZV  <[{"IPCD_INVOICE_ID":"fa759cb6-6702-4a1c-85a3-9df7b101d3bc"}]
              created_by: PaymentRequestProcessor
            created_date: 2012-05-02 00:14:44
              updated_by: PaymentRequestProcessor
            updated_date: 2012-05-02 00:14:44
       account_record_id: 1
        tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payments\G
*************************** 1. row ***************************
              record_id: 1
                     id: b0e61973-a921-413d-a04b-84e36e3ad6bf
             account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
      payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
           external_key: d04ce5ad-e667-4113-8eb3-6d7f87f92bca
             state_name: PURCHASE_SUCCESS
last_success_state_name: PURCHASE_SUCCESS
             created_by: PaymentRequestProcessor
           created_date: 2012-05-02 00:14:44
             updated_by: PaymentRequestProcessor
           updated_date: 2012-05-02 00:14:44
      account_record_id: 1
       tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_transactions\G
*************************** 1. row ***************************
               record_id: 1
                      id: 8b671a2e-6556-4aa8-8464-ef1cb99e5189
              attempt_id: 090fa541-7b69-42b2-bec7-a16f3c616071
transaction_external_key: 943d005c-5f89-4664-88f5-c65f39a3a9c8
        transaction_type: PURCHASE
          effective_date: 2012-05-02 00:14:44
      transaction_status: SUCCESS
                  amount: 249.950000000
                currency: USD
        processed_amount: 249.950000000
      processed_currency: USD
              payment_id: b0e61973-a921-413d-a04b-84e36e3ad6bf
      gateway_error_code:
       gateway_error_msg:
              created_by: PaymentRequestProcessor
            created_date: 2012-05-02 00:14:44
              updated_by: PaymentRequestProcessor
            updated_date: 2012-05-02 00:14:44
       account_record_id: 1
        tenant_record_id: 0
1 row in set (0.00 sec)

In this case, a PURCHASE (i.e. auto-capture) payment was performed. The invoice_payments entry is linked to the payments entry via payment_id and to the transactions table via payment_cookie_id (which is the transaction external key).

If the payment didn’t go through the first time (e.g. insufficient funds on the credit card), and the system was configured to retry the payments 8 days after, the data would look like this on a successful retry:

MySQL [killbill]> select * from invoice_payments order by record_id desc limit 1\G
*************************** 1. row ***************************
                record_id: 2
                       id: 6a45a92e-72ee-4415-9c5a-8066d4448cc5
                     type: ATTEMPT
               invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
               payment_id: d0a6b1c4-44b0-4e84-8883-4ec4cf8a3b2a
             payment_date: 2012-05-09 00:00:49
                   amount: 249.950000000
                 currency: USD
       processed_currency: USD
        payment_cookie_id: 6a5b6c15-0a8b-43ae-8b82-f6d66568eb8f
linked_invoice_payment_id: NULL
                  success: 1
               created_by: PaymentRequestProcessor
             created_date: 2012-05-02 00:14:44
        account_record_id: 1
         tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_attempts where record_id > 1\G
*************************** 1. row ***************************
               record_id: 2
                      id: be5f3706-105d-4874-9857-2b0d197b7ff3
              account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
       payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
    payment_external_key: 73035f59-9364-408f-a41f-d89b3483cd26
          transaction_id: 4935d163-7f3a-4b5d-8ad2-13dcb6d4b540
transaction_external_key: 6a5b6c15-0a8b-43ae-8b82-f6d66568eb8f
        transaction_type: PURCHASE
              state_name: RETRIED
                  amount: NULL
                currency: USD
             plugin_name: __INVOICE_PAYMENT_CONTROL_PLUGIN__
       plugin_properties: ZV  <[{"IPCD_INVOICE_ID":"fa759cb6-6702-4a1c-85a3-9df7b101d3bc"}]
              created_by: PaymentRequestProcessor
            created_date: 2012-05-01 00:00:46
              updated_by: PaymentRequestProcessor
            updated_date: 2012-05-01 00:00:46
       account_record_id: 1
        tenant_record_id: 0
*************************** 2. row ***************************
               record_id: 3
                      id: 672bc1a2-189a-4615-9619-544977cca8ea
              account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
       payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
    payment_external_key: 73035f59-9364-408f-a41f-d89b3483cd26
          transaction_id: 49ff12b5-7dfc-408d-956e-3d5335818738
transaction_external_key: 6a5b6c15-0a8b-43ae-8b82-f6d66568eb8f
        transaction_type: PURCHASE
              state_name: SUCCESS
                  amount: NULL
                currency: USD
             plugin_name: __INVOICE_PAYMENT_CONTROL_PLUGIN__
       plugin_properties: ZV  <[{"IPCD_INVOICE_ID":"fa759cb6-6702-4a1c-85a3-9df7b101d3bc"}]
              created_by: payment-service-retry
            created_date: 2012-05-09 00:00:49
              updated_by: payment-service-retry
            updated_date: 2012-05-09 00:00:49
       account_record_id: 1
        tenant_record_id: 0
2 rows in set (0.00 sec)

MySQL [killbill]> select * from payments\G
*************************** 1. row ***************************
              record_id: 1
                     id: d0a6b1c4-44b0-4e84-8883-4ec4cf8a3b2a
             account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
      payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
           external_key: 73035f59-9364-408f-a41f-d89b3483cd26
             state_name: PURCHASE_SUCCESS
last_success_state_name: PURCHASE_SUCCESS
             created_by: PaymentRequestProcessor
           created_date: 2012-05-01 00:00:46
             updated_by: payment-service-retry
           updated_date: 2012-05-09 00:00:49
      account_record_id: 1
       tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_transactions\G
*************************** 1. row ***************************
               record_id: 1
                      id: 4935d163-7f3a-4b5d-8ad2-13dcb6d4b540
              attempt_id: be5f3706-105d-4874-9857-2b0d197b7ff3
transaction_external_key: 6a5b6c15-0a8b-43ae-8b82-f6d66568eb8f
        transaction_type: PURCHASE
          effective_date: 2012-05-01 00:00:46
      transaction_status: PAYMENT_FAILURE
                  amount: 249.950000000
                currency: USD
        processed_amount: 0.000000000
      processed_currency: USD
              payment_id: d0a6b1c4-44b0-4e84-8883-4ec4cf8a3b2a
      gateway_error_code: 500
       gateway_error_msg: Insufficient funds
              created_by: PaymentRequestProcessor
            created_date: 2012-05-01 00:00:46
              updated_by: PaymentRequestProcessor
            updated_date: 2012-05-01 00:00:46
       account_record_id: 1
        tenant_record_id: 0
*************************** 2. row ***************************
               record_id: 2
                      id: 49ff12b5-7dfc-408d-956e-3d5335818738
              attempt_id: 672bc1a2-189a-4615-9619-544977cca8ea
transaction_external_key: 6a5b6c15-0a8b-43ae-8b82-f6d66568eb8f
        transaction_type: PURCHASE
          effective_date: 2012-05-09 00:00:49
      transaction_status: SUCCESS
                  amount: 249.950000000
                currency: USD
        processed_amount: 249.950000000
      processed_currency: USD
              payment_id: d0a6b1c4-44b0-4e84-8883-4ec4cf8a3b2a
      gateway_error_code:
       gateway_error_msg:
              created_by: payment-service-retry
            created_date: 2012-05-09 00:00:49
              updated_by: payment-service-retry
            updated_date: 2012-05-09 00:00:49
       account_record_id: 1
        tenant_record_id: 0
2 rows in set (0.00 sec)

A few things to notice:

  • There is a single invoice_payments entry pointing to a single payments entry

  • There are two payment_attempts, one RETRIED and one SUCCESS, pointing to two transactions in PAYMENT_FAILURE and SUCCESS respectfully

  • The transaction external key is shared for both transactions

Invoice item adjustment

Let’s consider the case where the administrator item adjusts for $10 the recurring item (for the service period 2012-05-01 to 2012-06-01). The second invoice now has 2 new items:

MySQL [killbill]> select * from invoice_items where invoice_id = 'fa759cb6-6702-4a1c-85a3-9df7b101d3bc'\G
*************************** 1. row ***************************
        record_id: 2
               id: 2326d3ff-e90d-43f0-b611-6c028bb88c71
             type: RECURRING
       invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
  subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
      description: shotgun-monthly-evergreen
        plan_name: shotgun-monthly
       phase_name: shotgun-monthly-evergreen
       usage_name: NULL
       start_date: 2012-05-01
         end_date: 2012-06-01
           amount: 249.950000000
             rate: 249.950000000
         currency: USD
   linked_item_id: NULL
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:14:43
account_record_id: 1
 tenant_record_id: 0
*************************** 2. row ***************************
        record_id: 3
               id: e702518e-2da1-4d1a-8939-316a2fef4df3
             type: ITEM_ADJ
       invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: NULL
  subscription_id: NULL
      description: Invoice item adjustment
        plan_name: NULL
       phase_name: NULL
       usage_name: NULL
       start_date: 2012-05-02
         end_date: 2012-05-02
           amount: -10.000000000
             rate: NULL
         currency: USD
   linked_item_id: 2326d3ff-e90d-43f0-b611-6c028bb88c71
       created_by: kaui-through-http-client
     created_date: 2012-05-02 00:30:41
account_record_id: 1
 tenant_record_id: 0
*************************** 3. row ***************************
        record_id: 4
               id: 74d1d3eb-d4b7-4b45-93ac-01805156d3db
             type: CBA_ADJ
       invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: NULL
  subscription_id: NULL
      description: Adjustment (account credit)
        plan_name: NULL
       phase_name: NULL
       usage_name: NULL
       start_date: 2012-05-02
         end_date: 2012-05-02
           amount: 10.000000000
             rate: NULL
         currency: USD
   linked_item_id: NULL
       created_by: kaui-through-http-client
     created_date: 2012-05-02 00:30:41
account_record_id: 1
 tenant_record_id: 0
3 rows in set (0.00 sec)

The ITEM_ADJ of $-10 points to the recurring item (see linked_item_id). Because the balance of the invoice was $0, a credit item (CBA_ADJ) of $10 is also added and will be available when the next invoice is being generated.

Refund with invoice item adjustment

A variation of the invoice item adjustment is refund with invoice item adjustment, i.e. refund the customer instead of generating a credit.

This time, there would only be a single ITEM_ADJ (no CBA_ADJ item):

MySQL [killbill]> select * from invoice_items where invoice_id = 'fa759cb6-6702-4a1c-85a3-9df7b101d3bc'\G
*************************** 1. row ***************************
        record_id: 2
               id: 2326d3ff-e90d-43f0-b611-6c028bb88c71
             type: RECURRING
       invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
  subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
      description: shotgun-monthly-evergreen
        plan_name: shotgun-monthly
       phase_name: shotgun-monthly-evergreen
       usage_name: NULL
       start_date: 2012-05-01
         end_date: 2012-06-01
           amount: 249.950000000
             rate: 249.950000000
         currency: USD
   linked_item_id: NULL
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:14:43
account_record_id: 1
 tenant_record_id: 0
*************************** 2. row ***************************
        record_id: 3
               id: aadfc291-0981-4e20-b231-8ae047e5514b
             type: ITEM_ADJ
       invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: NULL
  subscription_id: NULL
      description: NULL
        plan_name: NULL
       phase_name: NULL
       usage_name: NULL
       start_date: 2012-05-02
         end_date: 2012-05-02
           amount: -10.000000000
             rate: NULL
         currency: USD
   linked_item_id: 2326d3ff-e90d-43f0-b611-6c028bb88c71
       created_by: kaui-through-http-client
     created_date: 2012-05-02 00:15:25
account_record_id: 1
 tenant_record_id: 0
2 rows in set (0.01 sec)

Note that the refund would have actually happened first and the invoice_items and invoice_payments tables would have been updated upon success.

Here is the state of the payment related tables:

MySQL [killbill]> select * from invoice_payments where record_id > 2\G
*************************** 1. row ***************************
                record_id: 3
                       id: 35159bde-7bf0-4b77-a506-43aa1f37a29d
                     type: REFUND
               invoice_id: fa759cb6-6702-4a1c-85a3-9df7b101d3bc
               payment_id: b0e61973-a921-413d-a04b-84e36e3ad6bf
             payment_date: 2012-05-02 00:15:25
                   amount: -10.000000000
                 currency: USD
       processed_currency: USD
        payment_cookie_id: 212ce168-526e-4f23-85a5-ffaa67b189a8
linked_invoice_payment_id: e6e534e1-2ffa-4d5e-bcac-6905d4d26f61
                  success: 1
               created_by: kaui-through-http-client
             created_date: 2012-05-02 00:15:25
        account_record_id: 1
         tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_attempts where record_id > 2\G
*************************** 1. row ***************************
               record_id: 3
                      id: 45e7adfc-7549-4f81-8562-8c6907485275
              account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
       payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
    payment_external_key: d04ce5ad-e667-4113-8eb3-6d7f87f92bca
          transaction_id: 0d6b12b6-b54a-4092-a977-9d950214d4e4
transaction_external_key: 212ce168-526e-4f23-85a5-ffaa67b189a8
        transaction_type: REFUND
              state_name: SUCCESS
                  amount: 10.000000000
                currency: USD
             plugin_name: __INVOICE_PAYMENT_CONTROL_PLUGIN__
       plugin_properties: ZV j s[{"IPCD_REFUND_WITH_ADJUSTMENTS:"true"},�'IDS_AMOU`"{"2326d3ff-e90d-43f0-b611-6c028bb88c71":null}}]
              created_by: kaui-through-http-client
            created_date: 2012-05-02 00:15:25
              updated_by: kaui-through-http-client
            updated_date: 2012-05-02 00:15:25
       account_record_id: 1
        tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payments\G
*************************** 1. row ***************************
              record_id: 1
                     id: b0e61973-a921-413d-a04b-84e36e3ad6bf
             account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
      payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
           external_key: d04ce5ad-e667-4113-8eb3-6d7f87f92bca
             state_name: REFUND_SUCCESS
last_success_state_name: REFUND_SUCCESS
             created_by: PaymentRequestProcessor
           created_date: 2012-05-02 00:14:44
             updated_by: kaui-through-http-client
           updated_date: 2012-05-02 00:15:25
      account_record_id: 1
       tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_transactions where record_id > 1\G
*************************** 1. row ***************************
               record_id: 2
                      id: 0d6b12b6-b54a-4092-a977-9d950214d4e4
              attempt_id: 45e7adfc-7549-4f81-8562-8c6907485275
transaction_external_key: 212ce168-526e-4f23-85a5-ffaa67b189a8
        transaction_type: REFUND
          effective_date: 2012-05-02 00:15:25
      transaction_status: SUCCESS
                  amount: 10.000000000
                currency: USD
        processed_amount: 10.000000000
      processed_currency: USD
              payment_id: b0e61973-a921-413d-a04b-84e36e3ad6bf
      gateway_error_code:
       gateway_error_msg:
              created_by: kaui-through-http-client
            created_date: 2012-05-02 00:15:25
              updated_by: kaui-through-http-client
            updated_date: 2012-05-02 00:15:25
       account_record_id: 1
        tenant_record_id: 0
1 row in set (0.00 sec)

A few things to notice:

  • There is a new invoice_payments entry of type REFUND, linking to the attempt through linked_invoice_payment_id

  • There is still a single payments entry, but the state is now REFUND_SUCCESS

  • The payment has an additional payment_transactions entry of type REFUND, linking to a new payment_attempts entry

Change plan

Let’s assume the user changes on 2012-05-02 to the blowdart-monthly plan.

The plan is defined as follows:

<plan name="blowdart-monthly">
    <product>Blowdart</product>
    <initialPhases>
        <phase type="TRIAL">
            <duration>
                <unit>DAYS</unit>
                <number>30</number>
            </duration>
        </phase>
        <phase type="DISCOUNT">
            <duration>
                <unit>MONTHS</unit>
                <number>6</number>
            </duration>
            <recurring>
                <billingPeriod>MONTHLY</billingPeriod>
                <recurringPrice>
                    <price>
                        <currency>USD</currency>
                        <value>9.95</value>
                    </price>
                </recurringPrice>
            </recurring>
        </phase>
    </initialPhases>
    <finalPhase type="EVERGREEN">
        <duration>
            <unit>UNLIMITED</unit>
        </duration>
        <recurring>
            <billingPeriod>MONTHLY</billingPeriod>
            <recurringPrice>
                <price>
                    <currency>USD</currency>
                    <value>29.95</value>
                </price>
            </recurringPrice>
        </recurring>
    </finalPhase>
</plan>

For this scenario, we assume a START_OF_SUBSCRIPTION change alignment. Conceptually, the timeline for the subscriptions are as follows:

  • shotgun-monthly: [TRIAL 2012-04-01 → 2012-05-01][EVERGREEN 2012-05-01 → …​]

  • blowdart-monthly: [TRIAL 2012-04-01 → 2012-05-01][DISCOUNT 2012-05-01 → 2012-11-01][EVERGREEN 2012-11-01 → …​]

With a START_OF_SUBSCRIPTION change alignment, both timelines align on the start of the subscription (2012-04-01). On 2012-05-02, the target phase is hence the DISCOUNT one. If we had chosen a CHANGE_OF_PLAN alignment instead, the blowdart-monthly timeline would have been aligned on the date of the change (2012-05-02) and the target phase would have been the TRIAL one.

Two new subscription events are recorded, one for the change and one for the future phase change:

MySQL [killbill]> select * from subscription_events where record_id > 2\G
*************************** 1. row ***************************
              record_id: 3
                     id: c3a87783-b6fc-41bd-acc9-130aba08dc5c
             event_type: API_USER
              user_type: CHANGE
         effective_date: 2012-05-02 00:37:59
        subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
              plan_name: blowdart-monthly
             phase_name: blowdart-monthly-discount
        price_list_name: DEFAULT
billing_cycle_day_local: 0
              is_active: 1
             created_by: website-through-http-client
           created_date: 2012-05-02 00:37:59
             updated_by: website-through-http-client
           updated_date: 2012-05-02 00:37:59
      account_record_id: 1
       tenant_record_id: 0
*************************** 2. row ***************************
              record_id: 4
                     id: 0f448b11-b705-4074-b6a6-ecc62cc2b305
             event_type: PHASE
              user_type: NULL
         effective_date: 2012-11-01 00:01:14
        subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
              plan_name: NULL
             phase_name: blowdart-monthly-evergreen
        price_list_name: NULL
billing_cycle_day_local: 0
              is_active: 1
             created_by: website-through-http-client
           created_date: 2012-05-02 00:37:59
             updated_by: website-through-http-client
           updated_date: 2012-05-02 00:37:59
      account_record_id: 1
       tenant_record_id: 0
2 rows in set (0.00 sec)

And a new invoice is generated with 3 items:

MySQL [killbill]> select * from invoices order by record_id desc limit 1\G
*************************** 1. row ***************************
        record_id: 3
               id: 742a1a60-58f8-4ed2-b9db-b1816a741f44
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
     invoice_date: 2012-05-02
      target_date: 2012-05-02
         currency: USD
           status: COMMITTED
         migrated: 0
   parent_invoice: 0
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:37:59
account_record_id: 1
 tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from invoice_items where record_id > 4\G
*************************** 1. row ***************************
        record_id: 5
               id: d6b71f66-d54a-4cfc-bb73-ba55c91d8429
             type: RECURRING
       invoice_id: 742a1a60-58f8-4ed2-b9db-b1816a741f44
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: a7a1370e-1fa4-4c32-abd5-223e1da97339
  subscription_id: d9c7bb57-675e-4419-a340-5f6b4fd612f4
      description: blowdart-monthly-discount
        plan_name: blowdart-monthly
       phase_name: blowdart-monthly-discount
       usage_name: NULL
       start_date: 2012-05-02
         end_date: 2012-06-01
           amount: 9.630000000
             rate: 9.950000000
         currency: USD
   linked_item_id: NULL
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:37:59
account_record_id: 1
 tenant_record_id: 0
*************************** 2. row ***************************
        record_id: 6
               id: 592d7318-8fea-4e3b-b843-f6d444857132
             type: REPAIR_ADJ
       invoice_id: 742a1a60-58f8-4ed2-b9db-b1816a741f44
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: NULL
  subscription_id: NULL
      description: Adjustment (subscription change)
        plan_name: NULL
       phase_name: NULL
       usage_name: NULL
       start_date: 2012-05-02
         end_date: 2012-06-01
           amount: -239.950000000
             rate: NULL
         currency: USD
   linked_item_id: 2326d3ff-e90d-43f0-b611-6c028bb88c71
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:37:59
account_record_id: 1
 tenant_record_id: 0
*************************** 3. row ***************************
        record_id: 7
               id: 3413d964-90d3-4aa0-8b0e-406d02eb231e
             type: CBA_ADJ
       invoice_id: 742a1a60-58f8-4ed2-b9db-b1816a741f44
       account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
 child_account_id: NULL
        bundle_id: NULL
  subscription_id: NULL
      description: Adjustment (account credit)
        plan_name: NULL
       phase_name: NULL
       usage_name: NULL
       start_date: 2012-05-02
         end_date: 2012-05-02
           amount: 230.320000000
             rate: NULL
         currency: USD
   linked_item_id: NULL
       created_by: SubscriptionBaseTransition
     created_date: 2012-05-02 00:37:59
account_record_id: 1
 tenant_record_id: 0
3 rows in set (0.00 sec)

The REPAIR_ADJ item on the new invoice points to the RECURRING item on the second invoice that is being adjusted (repaired). The amount repaired is only $239.95 because of the previous item adjustment.

Because the new rate is only $9.95, the account has extra credit of $239.95 - $9.63 (pro-rated) = $230.32 (new CBA_ADJ item).

Note that the previous invoices have not been updated in that case.

A new payment attempt is triggered, but aborted since the invoice balance is $0:

MySQL [killbill]> select * from invoice_payments order by record_id desc limit 1\G
*************************** 1. row ***************************
                record_id: 3
                       id: e7f6b1d0-3823-4c5b-9226-12327af64492
                     type: ATTEMPT
               invoice_id: 742a1a60-58f8-4ed2-b9db-b1816a741f44
               payment_id: NULL
             payment_date: 2012-05-02 00:37:59
                   amount: 0.000000000
                 currency: USD
       processed_currency: USD
        payment_cookie_id: bac67833-ac7c-458d-8e2a-659c0cdfb0b6
linked_invoice_payment_id: NULL
                  success: 0
               created_by: PaymentRequestProcessor
             created_date: 2012-05-02 00:37:59
        account_record_id: 1
         tenant_record_id: 0
1 row in set (0.00 sec)

MySQL [killbill]> select * from payment_attempts order by record_id desc limit 1\G
*************************** 1. row ***************************
               record_id: 3
                      id: 0703a3de-9216-4601-95cd-8540e0687bf5
              account_id: 8e4f353f-ddbb-4155-b52d-9fe77b8e96e3
       payment_method_id: c046e5be-e632-444a-905f-c4bc0c5c0086
    payment_external_key: dff4c517-b229-4796-afe2-112aa4ba35e5
          transaction_id: NULL
transaction_external_key: bac67833-ac7c-458d-8e2a-659c0cdfb0b6
        transaction_type: PURCHASE
              state_name: ABORTED
                  amount: NULL
                currency: USD
             plugin_name: __INVOICE_PAYMENT_CONTROL_PLUGIN__
       plugin_properties: ZV  <[{"IPCD_INVOICE_ID":"742a1a60-58f8-4ed2-b9db-b1816a741f44"}]
              created_by: PaymentRequestProcessor
            created_date: 2012-05-02 00:37:59
              updated_by: PaymentRequestProcessor
            updated_date: 2012-05-02 00:37:59
       account_record_id: 1
        tenant_record_id: 0
1 row in set (0.00 sec)

Invoicing internals

The invoice subsystem computes new invoices in several stages.

First, it loads all existing invoices on disk for the account being processed, and creates a tree per subscription (SubscriptionItemTree) containing all of these invoice items. Each node in the tree represents all invoice items for a given service period. The service period of a parent node overlaps all service periods of its children.

For example, let’s assume a subscription was billed in advance from 2012-05-01 to 2012-06-01, and a change plan occured on 2012-05-07. There would be 3 items on disk, one recurring for the first plan (2012-05-01 to 2012-06-01), a second recurring for the second plan (2012-05-07 to 2012-06-01) and a repair adjustment (from 2012-05-07 to 2012-06-01) associated with the initial item.

The tree would look like this:

  A
 /
 B
/
C

A: [2012-05-01,2012-06-01]
B: [2012-05-01,2012-06-01](A)
C: [2012-05-07,2012-06-01](A,C)

A is the root node, it doesn’t contain any item and the service period is the largest period containing all items in the tree. Node B represents the initial recurring period (the node has one ADD — or A — invoice item). B has one child node — C — which has two items, the ADD item representing the new recurring item, and the CANCEL item, pointing to the ADD item in the node B, representing the repair.

The next step is to build the tree: resulting invoice items for each service period are computed and item adjustments processed (if any). In this case, 2 items would be generated (one from 2012-05-01 to 2012-05-07 and one from 2012-05-07 to 2012-06-01). The original tree is then replaced (flatten phase) by a shallow one containing these items (it’s really a list at this point):

 A
/
Bc

A: [2012-05-01,2012-06-01]
B: [2012-05-01,2012-05-07](C)
c: [2012-05-07,2012-06-01](C)

The items already invoiced are also reversed and labeled as C (CANCEL) before the next step.

Billing events are then processed and a list of proposed invoice items is generated. Each item will either be matched to an (existing) item in the flattened list, trigger a repair (in case of new change plan for instance) or kept as-is.

For example, if a proposed RECURRING item from 2012-05-01 to 2012-05-07 is merged, the flattened list becomes:

 A
/
Bc

A: [2012-05-01,2012-06-01]
B: [2012-05-01,2012-05-07]
c: [2012-05-07,2012-06-01](C)

Note that node B is empty now, the existing item has been removed (matched).

If a change plan would occur on 2012-05-08 and a proposed RECURRING item from 2012-05-07 to 2012-05-08 is merged, the list would become:

 A
/
B-c
 /
 D

A: [2012-05-01,2012-06-01]
B: [2012-05-01,2012-05-07]
c: [2012-05-07,2012-06-01](C)
D: [2012-05-07,2012-05-08](A)

A new node D, child of c, is inserted representing that period. After the RECURRING from 2012-05-08 to 2012-06-01 is merged, the list would be unchanged (no match found and no repair to create) and the RECURRING item kept on the side.

Finally, one last time, the tree is built and the computed invoice items become the resulting items, i.e. the items for the resulting invoice. In our case, two items are generated: the RECURRING kept as-is (outside the tree), and a repair from 2012-05-08 to 2012-06-01.

Keeping invoices in DRAFT mode

In some situations, we would like Kill Bill to generate the invoices, but leave them in a DRAFT mode.

Examples:

  • Case where we need to append invoice items after generation

  • Case where we want payment system to ignore the invoice until it has been COMMITTED

This behavior can be configured at the account level through the AUTO_INVOICING_DRAFT tag: when an Account is tagged with AUTO_INVOICING_DRAFT, any automatic invoice generated by the system (such as when generating invoices for recurring and usage subscriptions) will end up in DRAFT mode. Generating invoices through API calls (e.g. adding external charges) will not be affected by this tag.

Implementation expectation

Any DRAFT invoice containing RECURRING or USAGE items will be taken into consideration by the system when computing subsequent invoices, regardless of the AUTO_INVOICING_DRAFT tag.

As an example, let’s assume a monthly recurring subscription billed on the 1st of the month:

  • AUTO_INVOICING_DRAFT is added

  • January 1st : DRAFT invoice generated with a RECURRING item from January 1st to February 1st

  • AUTO_INVOICING_DRAFT is removed

  • February 1st : COMMITTED invoice generated with a RECURRING item from February 1st to March 1st — although the previous invoice has not been committed yet

In other words, the feature assumes that at some point such DRAFT invoices will be committed by the user. Not doing so will lead to users not being billed for those specific periods.

Invoice Date

The invoice date is the one from when the DRAFT invoice was generated — the date at which this invoice gets committed is not reflected. In particular, this may have some impact if the payment associated with such invoice fails, and dunning kicks-in: the timeSinceEarliestUnpaidInvoiceEqualsOrExceeds overdue configuration option will be based on the original invoice creation date (DRAFT).

Re-using invoices in DRAFT mode

The account level AUTO_INVOICING_REUSE_DRAFT control tag will make the invoicing system reuse an existing DRAFT invoice instead of creating a new invoice when generating invoices for recurring and usage subscriptions, or when generating parent summary invoices:

  • If there is no existing DRAFT invoice, the system would default to normal behavior and create a new invoice

  • If there is one existing DRAFT invoice, the system would use this DRAFT invoice instead of creating a new one

  • If there is more than one, the system will use the earliest DRAFT invoice

Similar to the AUTO_INVOICING_DRAFT feature, it is the responsibility of the user to to commit such DRAFT invoices.

Parked Accounts

Because invoicing drives payments, any issue in the system or misconfiguration leading to wrong invoices could have a direct impact on your customers. In order to avoid bad invoicing (and potentially invalid customer payment) we have introduced a mechanism to PARK the accounts. Once an account has been parked, no more invoicing will happen until its state has been fixed (or until safety bounds have been removed). A account can become parked in the following scenarios:

First, the system now has a safety bound mechanism on the upper limit of invoice items generated for a given subscription on a given day. This safety bound is controlled by a system property org.killbill.invoice.maxDailyNumberOfItemsSafetyBound whose default value is set to 15 but that can also be disabled by setting it to -1. The safety bound will kick-in if a customer tries to buy too many things during a single day or in situations where the customer has not been invoiced for a while and the system is catching up (could happen if the account was tagged with AUTO_INVOICING_OFF and the tag gets removed).

Second, the invoicing system will automatically park an account when an illegal state is reached during invoicing. This could be because the data associated with the account looks corrupted (which could happen because of previous bugs that did corrupt the data, manual data manipulation, or existing bug in the system).

The way to get out of that state is to fix the issue that led to that state in the first place and trigger a new invoice generation. If this issue arose because of a bug, fixing the issue may require voiding invoices creating the issues.

For cases where there are many parked accounts, we have added an administrative api to go through all parked accounts and issue an invoice generation for each of those, only leaving in the parked state, accounts whose state has not been fixed yet. If you think the issue that led into that state is not a problem with the account state (data) but a bug in the system, please report to the mailing list.

Summary

The Kill Bill invoicing system generates invoices on a per account level based on some system trigger:

  • Bus event: Those are usually subscription events like when a subscription starts, when there is a phase transition (trial → evergreen), upon plan change or cancellation (there are also other bus events in the system that could trigger an invoice generation, but not discussed for sake of simplicity).

  • Future notfications: The recurring piece of the invoiving is based on the invoice system inserting future notfications on the right date (so as to be called back at the right time and generate the next invoice).

Note also that invoice generation can be triggered through API but this is not the default use case.

The computation of the next invoice (and matching invoice items) is based upon the following:

  • A billing mode (e.g IN_ADVANCE)

  • A set of billing events (mostly aggregated across subscritpion_events and blocking_states tables)

  • A targetDate specifying upon which point to bill (and which billing events to consider)

The invoicing algorithm recomputes the full view at each invocation and compares the existing view on disk (what was already billed) with the new set of proposed items, and based on the difference generates the next invoice (or nothing if there is no change since the previous invocation). If a new invoice was generated, it also inserts a new future notification and updates the chargedThroughDate for all subscriptions.

In terms of payments, the source of truth resides in the payments (and associated payments_transactions) tables. However, the payment view of the invoicing subsystem relies on the invoice_payments table and is used to compute things like invoice and account balance: The invoice_payments table is updated using a 2-phase commit algorithm where a row is first inserted with a status set to false and then upon payment completion is updated to true.