I Had AI Agents Do the Development: a 95.5% Reduction on the Development Stage, a Projected 60–70% Across a Whole Engagement

Introduction

While working on the banking system I build myself, I added one more column to the estimate table of the WBS (Work Breakdown Structure — the tracking table that breaks a project down into individual tasks). Its name is “AI actual”. It is a column reserved for writing down, after an AI agent has actually executed a task, how long that task took.

A table that only held estimates became a table that can be matched against measurements. This article covers what numbers came out of it, what those numbers mean and do not mean, and what I got wrong along the way while producing them.

Adding One Column for Measurements to the Estimate Table

This WBS originally had two estimate columns: “human estimate” (a traditional person-day estimate for a human SE working on it full time) and “AI estimate”. I added a third column, “AI actual”. The recording rule I wrote into the WBS reads like this.

AI actual: a column reserved for appending after a task has actually been executed. Do not fill it in advance; do not overwrite a pre-execution estimate (a rule to keep estimates and actuals from being confused). This WBS functions as an append-only log of “estimate → execute → record actual”.

The owner column does not hold people’s names either. It holds custom subagent names: rag-coder for implementation, rag-reviewer / test-reviewer for verification, test-coder for test code. How the roles are divided is not re-explained here; that is delegated to the following article.

Building a streaming platform and a bank in parallel — deploying a fleet of AI agents to implement real features, and tuning the division of labor by measurement

The WBS also states: “the human (myself) performs only final approval, environment setup, and live verification.” So the time entered in the AI actual column is the time the agent was executing the task itself, and does not include the time I spend looking at the result and judging it (I return to this later as a note on how to read the numbers).

Actuals are appended in an independent commit each time one task is finished. There are no rows filled in later in bulk. The commit history shows it. The messages are in Japanese; each one records the AI actual for the deliverable number shown.

bed0c8b docs: WBSに口座詳細画面向けBFF API+取引履歴フィルタのAI実績を追記
f3ac5d3 docs: WBSにBFF-4(accounts-service部分)のAI実績を記録
2f4f5d6 docs: WBSにBFF-1のAI実績を記録
0206dda docs: WBSにAUTH-4のAI実績を記録
4cc2182 docs: WBSにNOTIF-4のAI実績を記録
04da765 docs: WBSにACC-8/PAY-7/AUTH-7/NOTIF-6のAI実績を記録
a738c4b docs: WBSにACC-7のAI実績を記録
3c8dcce docs: WBSにACC-6のAI実績を記録
3434ddc docs: WBSにACC-5のAI実績を記録
b0d82f9 docs: WBSにPAY-5のAI実績を記録
df89606 docs: WBSにPAY-4のAI実績を記録
3ef1172 docs: WBSにPAY-3のAI実績を記録
2b88b45 docs: WBSにPAY-6のAI実績を記録
3290475 docs: WBSにACC-4のAI実績を記録
cc289cb docs: WBSに人間実施予想/AI実施予想/AI実績の3軸を追加
27399ea docs: 全体WBS・マスタスケジュールを新規作成

Alphanumeric strings such as BFF-4 and AUTH-4 in the messages are deliverable numbers in the WBS. The prefix identifies the service (AUTH = auth service, ACC = accounts service, PAY = payments service, NOTIF = notifications service, BFF = the API aggregation layer for screens, WEB = web front end, INF = infrastructure), and the digit is the task number within it. The ones appearing in the commits above are, in plain words:

Look at the second line from the bottom. When the WBS was first created, the three-axis structure did not exist at all. The columns were added later, after I decided to measure. The moment I decided to measure is itself part of the history.

Drawing the flow by which one task closes as one row looks like this. It does not end at implementation → verification: there is a return edge for rework, and a second return edge feeding actuals back into the next estimate.

flowchart TD
    subgraph WBS["WBS (nested groups: auth / accounts / payments / notifications / bff / web)"]
        T["1 task<br/>e.g. PAY-3 = the Saga itself"]
    end
    T --> IMPL["Implementation<br/>rag-coder"]
    T --> TC["Test perspectives → test code<br/>test-coder"]
    IMPL --> REV{"Review<br/>rag-reviewer / test-reviewer"}
    TC --> REV
    REV -->|"0 must-fix"| REC
    REV -->|"N must-fix<br/>e.g. missing ownership check"| FIX["Rework"]
    FIX -->|"sent back"| REV
    REC["Record elapsed time in m in the AI actual column<br/>e.g. 75m = 50m first pass + 25m rework"] --> CM["1 task = 1 commit"]
    CM -->|"actuals accumulate"| EST["The AI estimate column gets more accurate"]
    EST -->|"reflected in the next estimate"| T

    style REC fill:#e8f4ea,stroke:#4a7c59
    style FIX fill:#fbeee6,stroke:#c17f4a

Work I Estimated at 490.8 Human SE Hours, Written in a Measured 21.9 Hours — a 95.5% Reduction

Of the 189 AI-owned rows in the WBS, 49 rows have a number in both the human estimate and the AI actual, which makes them comparable (what the rest are is written out plainly further down).

490.8 hours is 61.4 person-days in a traditional estimate — close to three months of a full-time SE working on nothing else. That was written in a measured 21.90 hours. On the same axis, it looks like this.

xychart-beta
    title "Time spent on the same 49 tasks (hours)"
    x-axis ["Human SE estimate 490.8h", "AI agents measured 21.9h"]
    y-axis "Hours" 0 --> 500
    bar [490.8, 21.9]

Hours alone make the volume hard to grasp, so here are the same figures restated in other units (1 person-day = 8 hours, 1 month = 20 working days).

• 490.8 hours ⇒ 21.90 hours, a 95.5% reduction
• 61.4 person-days (about 3 months for one person full time) ⇒ 2.7 person-days (about 3 days)
• A month’s worth of work, 20 working days ⇒ about 7 hours, one working day

The ratio is the same, so it holds at smaller units too. What one SE would write over a month fits into a single working day.

Splitting it by role shows that it was not only implementation that was fast.

This is the core of the article. The number of review rows (23) is roughly the same as the number of implementation rows (22). Broken down, the 21.90 hours were spent like this.

pie showData
    title Breakdown of the 21.90 measured hours, by role
    "Implementation rag-coder" : 13.64
    "Review rag-reviewer / test-reviewer" : 6.11
    "Test code test-coder" : 2.15

28% of the total went into verification. The source of the speed is not “one agent did everything and got it over with”; it is that creation and verification were split across separate agents, and it still came to 21.90 hours. This is not a number produced by skipping review.

Working out the reduction per agent shows a gap between the side that creates and the side that verifies.

xychart-beta
    title "Ratio by agent (human SE estimate / AI actual)"
    x-axis ["rag-coder impl", "rag-reviewer review", "test-coder tests", "test-reviewer review"]
    y-axis "times" 0 --> 30
    bar [26.4, 17.7, 14.9, 13.3]

Implementation is a 96.2% reduction (26.4x), while the verification side stays at 92.5–94.4% (13–18x). Verification does not compress as much as implementation does. Review means reading a diff, matching it against the design documents, and judging whether it needs changing — work that does not become mechanically fast the way writing does. That is the shape the 49 rows produced.

A note on notation is due here. This article is written in reductions, but a reduction percentage hides something. 96.2% for implementation and 92.5% for review read as almost identical, even though the ratios differ twofold, 26.4x against 13.3x. Above 90%, a reduction percentage flattens differences. That is why the charts stay in ratios and the body text carries the ratio alongside: showing only the percentage hands the reader the false picture that every stage speeds up by about the same amount.

Cut by service, the reduction is not uniform either.

xychart-beta
    title "Ratio by service (human SE estimate / AI actual)"
    x-axis ["web", "payments", "bff", "auth", "accounts", "notifications", "infra"]
    y-axis "times" 0 --> 40
    bar [34.8, 23.7, 23.6, 22.0, 20.2, 17.8, 15.0]

The screen side (web, 2 tasks) is highest at 97.1% (34.8x) and notifications (2 tasks) is lowest at 94.4% (17.8x). But both have only 2 tasks behind them, which is not enough to read as a trend. The ones with more behind them are accounts (7 tasks, 95.0%) and payments / auth (5 tasks each, 95.8% / 95.5%), and those three cluster around 95%.

Reductions Range from 90.4% to 97.8% — Showing Only the Average Invites Misreading

Writing only the average misrepresents what happened. Looking at individual tasks (26 of them, grouping work that spans several rows into single tasks), the reduction varies widely.

From here on, deliverable numbers are accompanied by the task name in words.

Counting the 26 tasks by band makes the spread visible directly. This count is in ratios. Banding it by reduction would cram eight bars into the range 90.4%–97.8%, where every one of them looks like “practically 100%” and the distribution stops being readable.

xychart-beta
    title "Ratio distribution across 26 tasks (median 21.8x = 95.4% reduction)"
    x-axis ["10-15x", "15-20x", "20-25x", "25-30x", "30-35x", "35-40x", "40-45x", "45-50x"]
    y-axis "Tasks" 0 --> 9
    bar [4, 6, 8, 4, 1, 1, 1, 1]

18 tasks (69%) fall in the 93.3–96.7% band (15–30x), while one reaches 97.8% (over 45x).

Written as reductions, minimum 90.4% and maximum 97.8% look like a difference of only 7.4 points, but measured by the time that remains, there is a 4.4-fold gap between them. The overall average of 95.5% (22.4x) merely runs through the middle of that range; it does not mean any individual task lands on that value. If this number is shown in a client setting and the average alone leads someone to read “every task finishes at a 95% reduction”, that reading is wrong.

Tasks on the low side (where the gap against the human estimate was smallest):

Tasks on the high side:

“Dependency direction tests” means putting in place a mechanism that mechanically checks, as test code, whether the dependencies between layers run in the direction the design calls for (the one-way api → application → domain → infrastructure).

Six tasks were sent back during review, and their reductions are 90.4%, 93.4%, 95.0%, 95.8%, 95.9% and 96.6% — scattered on both sides of the 95.4% median. The numbers show no relationship of the form “rework happened, therefore it was slow”. As a concrete example of verification and rework being included in the speed itself, here are entries from the WBS actual column.

PAY-3 (implementing the Saga itself — a Saga is the standard design pattern where a sequence spanning multiple services is not committed all at once but one step at a time, with a reverse operation, the compensation, undoing it if something fails partway) has an AI actual of “75m (50m first pass + 25m must-fix rework)” and a review of “25m (15m initial detection + 10m re-check after rework)”. The review entry reads as follows.

2 must-fix items detected (1: missing ownership check on fromAccountId = IDOR, 2: risk of double crediting from immediate compensation when transfer-in times out). Resolved and confirmed in a re-review after rework. As a side effect, an existing gap — jwt.secret mismatched across 3 services — was also found and fixed.

IDOR (Insecure Direct Object Reference — the standard vulnerability where merely specifying an ID lets you touch someone else’s account or data) was a defect that would have shipped to production without review. PAY-6 (the inter-service client) is the same shape: against an AI actual of “50m (35m first implementation + 15m rework)”, the review entry reads as follows.

1 must-fix item detected (contradiction with the NEVER rule in 0402 §1.6; self-issuance of JWTs had been implemented in production code). rag-coder fixed it on the safe side (reverting to a test-only helper), re-verified.

AUTH-4 (biometric authentication implementation; 66m + 16m rework), ACC-7 (dormancy detection batch; 19m + 10m rework), NOTIF-6 (dependency direction tests for the notifications service; 36m + 10m rework) and BFF-1 (skeleton of the API aggregation layer; 11m + 16m rework) have the same kind of send-back recorded. must-fix is the term used inside the WBS for “a defect found in review that must be fixed before release”, and the overall 21.90 hours is a mixture of tasks that passed with zero of them and tasks that went through rework.

The 49 Rows Used for Comparison, and the Rows Not Used

“Was it executed” and “was the effort measured” are two different axes. Splitting the 189 AI-owned rows of the WBS along those two axes gives this.

flowchart LR
    A["AI-owned rows in the WBS<br/>189 rows"] --> B["Never arose, due to design<br/>decisions or process changes<br/>47 rows"]
    A --> C["Tasks that arose<br/>142 rows"]
    C --> D["Executed<br/>124 rows = 87.3%"]
    C --> E["Not started, 18 rows<br/>scope of Phase 3 onward"]
    D --> F["Effort measured to the minute<br/>49 rows<br/>the numbers in this article come from here"]
    D --> G["Effort not measured<br/>75 rows"]

    style D fill:#e8f4ea,stroke:#4a7c59
    style F fill:#e8f4ea,stroke:#4a7c59
    style G fill:#fbeee6,stroke:#c17f4a

Of the 142 tasks that arose, 124 (87.3%) have been executed. Of those, only 49 rows have effort recorded to the minute. I started measuring in mid-July 2026 and there was a stretch where I stopped measuring in favor of implementation. The figures 490.8 hours → 21.90 hours come only from those 49 measured rows.

I do not go back and write estimated values into the 75 unmeasured rows. That is stated in the WBS notation itself (in the quote, “legend 2” refers to rows that are executed but unmeasured, and “legend 1” to rows holding a measured value).

Do not write an estimated effort into a legend 2 row after the fact. What was not measured was not measured. The only comparable population is the legend 1 rows.

The “47 rows that never arose” also deserve a breakdown. 45 of them assumed a three-stage arrangement in which test-writer would produce a table of test perspectives, test-coder would implement tests against it, and test-reviewer would review them. In actual operation it converged on rag-coder writing the tests together with the implementation, and rag-reviewer reviewing implementation and tests together. Not inserting the intermediate artifact of a perspectives table made the round trip between implementation and tests faster. The remaining 2 rows (AUTH-6 / BFF-3) went out of scope with ADR 0016 (dropping the arrangement where Keycloak acts as the IdP, in favor of auth-service issuing JWTs itself).

Neither group is “unfinished” — they stopped being necessary. They are excluded from the denominator of the effort reduction.

The 18 rows not started are the mobile business screens, the skeletons of the internal systems (operator / backoffice / admin), and the optional items of observability and k8s manifests, all of which sit in the scope of Phase 3 onward.

I could have deleted the rows I failed to measure and raised the apparent coverage, but doing that turns a ledger into a piece of advertising. Rows marked “no measurement data (exact minutes unrecoverable because the interval predates summary compaction; see commits 467f594/a6e6057)” are left exactly as they are.

94.7% Reduction in the Estimates, 95.5% in the Measurements

The WBS holds one other number. Its summary section states “about 18.7x” — a 94.7% reduction (210.35 AI-owned person-days = 1,682.8 hours, divided by an AI estimate total of about 90.2 hours), and that is estimate divided by estimate, not a measurement. The WBS itself notes that it is “a provisional value, stemming from the fact that most of the AI estimates are guesses”.

The reduction actually measured across 49 rows is 95.5% (490.8 hours → 21.90 hours). The two numbers have different denominators, so they cannot be compared directly, but they can be placed side by side. The 95.5% measured within that range came out higher than the 94.7% seen at the estimate stage. In other words, the estimate was the more conservative of the two.

This number and the per-role breakdown were not calculated for this article. They are written into the summary section of the WBS itself, as an “actuals summary”. The ledger is the source of truth, and this article is a derived read.

Here is where the two numbers come from, side by side. On the left, both numerator and denominator are estimates; on the right, only the denominator is measured. The figure is drawn in ratios (placing 94.7% and 95.5% next to each other makes the difference in where they come from invisible).

flowchart LR
    subgraph A["About 18.7x in the WBS summary = 94.7% reduction = estimate / estimate"]
        A1["AI-owned 210.35 person-days<br/>= 1,682.8 hours<br/>human estimate"] --> A3["18.7x"]
        A2["AI estimate total<br/>about 90.2 hours<br/>mostly guesses"] --> A3
    end
    subgraph B["22.4x in this article = 95.5% reduction = estimate / measurement"]
        B1["490.8 hours across 49 rows<br/>human estimate"] --> B3["22.4x"]
        B2["21.90 hours across 49 rows<br/>AI actual = measured"] --> B3
    end
    A3 -.->|"different populations<br/>not directly comparable"| B3

    style A2 fill:#fbeee6,stroke:#c17f4a
    style B2 fill:#e8f4ea,stroke:#4a7c59

This 95.5% needs a caveat of its own, though. No record exists of a human SE actually doing those same 49 tasks by hand. The human side is still an estimate; only the AI side is measured. So I cannot write “cut working time by 95.5%”. Stated precisely, it is a one-directional measurement: work estimated at 490.8 human SE hours was written in a measured 21.90 hours.

The scope of the AI actual column itself is also limited. It excludes final approval, environment setup, live verification, and the time spent judging whether to send something back and writing the instructions. As the WBS states — “the human (myself) performs only final approval, environment setup, and live verification” — those sit outside the actual column, on the human side. So the numbers in this article do not show “hours in which an engineer becomes unnecessary”; they show only the hours an agent was actually processing something.

Finally, the per-task reductions carry an error of their own, because some rows in the actual column are marked as combined.

BFF-2  48m（BFF-7と合算）
WEB-1  29m（WEB-2/WEB-4一部と合算）
AUTH-7 46m（auth-service/bff/web一体実装）
BFF-4  27m（accounts-service部分のみ、AUTH-6未着手のためpayments/notifications/auth向けは別途）
INF-1  11m12s（docker-compose.yml実装作業と合算、内訳分離不可）

The numbers in those five rows point to the following tasks.

(The phrase “AUTH-6 not started” in the BFF-4 entry is how it read at the time of writing. AUTH-6 — integration with an external identity provider — was later confirmed out of scope by ADR 0016, and is one of the “47 rows that never arose” above.)

One row therefore carries the real work of several tasks, while the human estimate for it is still computed for a single task. The combined side comes out with an understated reduction and its counterpart an overstated one. Summed across the whole, they cancel out; but the per-task reductions themselves contain error, and I state that explicitly.

The Figure 24.40 Hours Was Not Correct

The numbers above are written as 21.90 hours, but the first total the aggregation script produced was 24.40 hours.

The cause was in a regular expression. Against a notation like 75m（初回50m+must-fix是正25m）, it picked up 75, 50 and 25 alike and counted them as 150m. What is inside the parentheses is a breakdown; it must not be added on top of the body value. The same notation appears in 50m（初回実装35m＋是正15m）.

Meanwhile there is also a form without parentheses, such as 66m+16m是正. There, both the first pass and the rework are body values, so 82m is the correct count. My error was reading everything with only one of the two rules, without noticing that two notations were mixed together.

flowchart TD
    S["The WBS actual column<br/>two notations are mixed"] --> P1["75m (50m first pass + 25m rework)<br/>the parentheses hold a breakdown"]
    S --> P2["66m+16m rework<br/>no parentheses = both are body values"]
    P1 --> BUG["The first regular expression<br/>picks up every number on the line"]
    P2 --> BUG
    BUG --> W["counts 75+50+25 = 150m"]
    BUG --> OK["counts 66+16 = 82m"]
    W --> WT["Total 24.40 hours / 95.0% reduction"]
    OK --> WT
    WT --> D{"Rows remain that the<br/>presence of parentheses cannot explain"}
    D -->|"notice the notation variance"| FIX["Strip the parentheses before parsing"]
    FIX --> R["counts 75m"]
    R --> RT["Total 21.90 hours / 95.5% reduction"]

    style WT fill:#fbeee6,stroke:#c17f4a
    style RT fill:#e8f4ea,stroke:#4a7c59

The number underlying an article about measuring AI productivity was off by 11.4% because of my own way of measuring. The direction is worth stating too. Correcting it brought the AI-side total down from 24.40 hours to 21.90 hours, and the reduction rose from 95.0% to 95.5%. In other words, the original bug made the AI side look slower than it was — a bug against my own interest. And the property of percentages shows up here as well. 95.0% and 95.5% look like a difference of half a point, but the time that remains is 24.40 hours against 21.90 — a gap of 2.5 hours.

I do not treat that as “it erred in the harmless direction, so it is fine”. Whichever way it leaned, the fact remains that I believed the first number that came out until I questioned the way I was measuring it.

What led me to the notation variance was that rows remained which the presence or absence of parentheses could not explain. It was not that the number felt too large. The order of events was: I treated the fact that my own rule failed to explain every row as the anomaly itself. Had I been judging plausibility by feel, the error would still be in there.

The Conditions Under Which This 95.5% Reduction Holds

If I show this number in a client setting, I show these six points along with it. A reduction figure with its conditions hidden is itself an error.

• The human estimate is not a record of a human who actually did the work. It is the traditional person-day estimate placed there when the WBS was built
• The AI actual does not include the human-side time for final approval, environment setup, live verification, judgment, or writing instructions
• The 95.5% compares a measured AI side against an estimated human side — only one side was measured
• Per-task reductions carry error from rows where several tasks were combined into one
• The denominator is the 49 rows whose effort was measured to the minute. The 75 rows that were executed but not measured carry no estimated values and are not in the denominator either
• One of the 49 rows has an owner of “rag-coder / performed by me instead”. It is a task estimated at 4.0 human hours, and the 0.33h in the actual column is time I spent doing it myself. So the actual column is not purely agent time (0.33h against 21.90 hours overall, 1.5%)

Put the other way: drop those six points and the reduction can be made larger. Take review rows out of the denominator and it becomes 96.2% for implementation alone; leave out rework and send-backs and it rises further. 95.5% is the number that did none of that.

Across a Whole Engagement It Is 60–70% — 95.5% Does Not Carry Over As Is

The 95.5% above is measured only across the development tasks that become rows in a WBS. A real engagement does not end there. Producing the documents delivered to the client, aligning on requirements, confirming specifications, meetings and preparing for them, building agreement among stakeholders. A substantial amount of an engagement is work that never gets a deliverable number, or work that I end up rewriting myself even after handing it to an AI.

Including all of that, the reduction across a whole engagement is 60–70% of working time.

flowchart LR
    A["Total working time of an engagement"] --> B["Development tasks in the WBS<br/>implementation, review, tests"]
    A --> C["Client deliverables, requirement alignment<br/>meetings, building agreement<br/>the hard-to-compress part"]
    B -->|"measured 95.5% reduction"| D["Across the whole engagement<br/>60-70% reduction"]
    C -->|"barely compresses at all"| D

    style B fill:#e8f4ea,stroke:#4a7c59
    style C fill:#fbeee6,stroke:#c17f4a

The 60–70% also holds up arithmetically. Take development as 6–7 tenths of the working time of an engagement, cut only that part by 95.5%, and leave the rest uncut: the overall reduction lands at 60–70%. Put the other way, the figure comes as a set with the premise that the non-development part is 3–4 tenths, and it moves for an engagement with a different mix.

The two numbers come from different places. The 95.5% is a value measured to the minute across 49 rows of this WBS, with the method and my own mistakes in it written into this article. The 60–70% is an answer I arrived at myself, from watching where the hours actually go in an engagement. The one to apply to a whole engagement is the latter.

If I show these numbers in a client setting, this is the order I show them in: looking at development alone, 95.5% was cut; across a whole engagement, it is 60–70%.

The Same Column Fits Any WBS

What I did is simple. I added one field to an estimate table for “how long it actually took”, and filled it in one commit at a time as each task closed. That alone made it possible to state, in numbers, both how accurate the estimates were and which stages compress and which do not.

What came out was this: work estimated at 490.8 human SE hours, written in a measured 21.90 hours — a 95.5% reduction — and its breakdown. Implementation was reduced by 96.2%; verification stayed at 92.5–94.4%. The source of the speed is not “one agent did everything and got it over with”; it takes the shape of creation and verification split across separate agents, and this was the time it took.

That 95.5% is a figure for the development stage, not for a whole engagement. Including deliverables and meetings, a whole engagement comes to a 60–70% reduction. To keep two numbers of different origin from being mixed together, this article gives them separate sections.

The way I measured, and the places where I got it wrong along the way, are all written down. The same column fits any WBS.