KreditBeeDvar

Post-training

Small models KreditBee owns, trained on its own misses

Evals find where a small model misses; those misses, corrected by reviewers, become its training data. The tuned model ships only through the same gate.

Step 1 · Measure before tuning

Where the small model misses today

Dvar · GLM 5.3 Flash on the 120 test calls: 116 with every field right, 0 with a critical failure. The misses, by field:

Reason

3

calls wrong · field accuracy 97.5%

Outcome

1

calls wrong · field accuracy 99.2%

Each miss becomes a training example

The transcript in, the reviewer's disposition out. In a pilot these come from real, redacted calls, in the thousands.

C057 · Reason

Call details: started 2026-10-12 13:59, EMI ₹6340, due 2026-09-26, borrower Suresh Kumar.
Transcript:
Agent: Namaste, main aapke personal loan account ke baare mein call kar rahi hoon. Ye call record ho rahi hai. Kya meri baat Suresh ji se ho rahi hai?
Customer: Haan, main hi bol raha hoon.
Agent: Aapka ₹6,340 ka EMI 26 September ko due tha aur abhi tak pending hai. Aap kab tak payment kar payenge?
Customer: Main 18 tareekh ko ₹6,340 bhar dunga.
Agent: Theek hai, toh 20 tareekh ko ₹6,340. Dhanyavaad.
Customer: Nahi nahi, 18 tareekh. Maine 18 bola.
Customer: Sorry, bhool gaya tha.
Agent: Theek hai, note kar liya. Dhanyavaad.

Model wrote

{"reason":null}

Training target

{"outcome":"ptp","ptp_date":"2026-10-18","ptp_amount":6340,"reason":"forgot","hardship":false,"wants_human":false,"stop_calling":false,"callback_at":null}

C062 · Reason

Call details: started 2026-10-09 18:59, EMI ₹4250, due 2026-10-06, borrower Nandini Reddy.
Transcript:
Agent: Namaste, main aapke personal loan account ke baare mein call kar rahi hoon. Ye call record ho rahi hai. Kya meri baat Nandini ji se ho rahi hai?
Customer: Haan, main hi bol raha hoon.
Agent: Aapka ₹4,250 ka EMI 6 October ko due tha aur abhi tak pending hai. Aap kab tak payment kar payenge?
Customer: Ye late fee galat lagi hai. Auto-debit aapki taraf se fail hua tha, main ye charge nahi dunga.
Agent: Theek hai, note kar liya. Dhanyavaad.

Model wrote

{"reason":null}

Training target

{"outcome":"dispute","ptp_date":null,"ptp_amount":null,"reason":"other","hardship":false,"wants_human":false,"stop_calling":false,"callback_at":null}

C088 · Reason

Call details: started 2026-10-13 15:07, EMI ₹2140, due 2026-10-04, borrower Arjun Singh.
Transcript:
Agent: Namaste, main aapke personal loan account ke baare mein call kar rahi hoon. Ye call record ho rahi hai. Kya meri baat Arjun ji se ho rahi hai?
Customer: Haan, main hi bol raha hoon.
Agent: Aapka ₹2,140 ka EMI 4 October ko due tha aur abhi tak pending hai. Aap kab tak payment kar payenge?
Customer: Ye late fee galat lagi hai. Auto-debit aapki taraf se fail hua tha, main ye charge nahi dunga.
Agent: Theek hai, note kar liya. Dhanyavaad.

Model wrote

{"reason":null}

Training target

{"outcome":"dispute","ptp_date":null,"ptp_amount":null,"reason":"other","hardship":false,"wants_human":false,"stop_calling":false,"callback_at":null}

From run run-20261022172936-test.

Step 2 · Tune

How we post-train KreditBee custom models

  1. 1

    Measure

    Run the current model and a frontier model on KreditBee's eval sets. The misses say what to train.

  2. 2

    Collect

    Traces scored by Dvar Evals, reviewer corrections, synthetic borrowers for rare cases and languages, and frontier answers that pass every check.

  3. 3

    Supervised tuning

    LoRA adapters on an open base model, one per task, so one base serves several tasks.

  4. 4

    Preference tuning

    Pairs the checks and reviewers marked better or worse, so the model learns what to avoid, not only what to copy.

  5. 5

    Gate

    The same suite as the Collections and Voice pages. A version ships only with no critical failure.

  6. 6

    Release

    A KreditBee custom model under KreditBee's name, with a model card and an eval report.

TaskWhere it runsWhy it suits a tuned small modelShips as (placeholder name)
Call dispositionCollections copilotShort JSON a schema can check; high volume; Hinglish and KannadaKB Collect-1
Statement line labelsUnderwriting copilotNarration formats repeat across banks; learnable from analyst labelsKB Statement-1
Calling-agent repliesThe voice vendor's agentPolicy-following and regional languages, judged by the release gateKB Voice-1
Document readingKYC harnessFixed layouts; reviewer corrections are ready-made training dataKB Docs-1

Step 3 · Gate and release under KreditBee's name

A model card KreditBee can publish

KreditBee owns the weights trained on its data and the model names. Dvar does not reuse KreditBee's data or weights for anyone else. A tuned model goes through the same scorer as the collections table and ships only with no critical failure; until then the hard call types stay on the frontier model.

  • • Where it trains: KreditBee's cloud account or Dvar's environment in India.
  • • Where it serves: Dvar inference in India, or KreditBee's own endpoint.
  • • Cadence: a release about every 4 weeks once the first task is live.
  • • Open points: the open base model and its licence, where the weights sit, GPU budget.

Model card · draft

KB Collect-1

Publisher
KreditBee
Base
Open small model
Method
LoRA SFT, then preference tuning
Task
Call transcript → disposition (JSON)
Languages
Hinglish, Hindi, English, Kannada; Tamil and Telugu next
Eval set
120 test calls, 5 critical checks
Base today
116/120 right, 0 critical
Tuned
measured at release
Gate
zero critical failures

Not measured: no model was trained for this demo, so there is no tuned score; model names are placeholders. Computed here: the base model's misses from the frozen run.