EN We Let the AI That Got Banned for Hacking Test Our Fintech Code
In June, Anthropic released Claude Fable 5 — a model capable of independently discovering zero-day vulnerabilities and writing exploits. Three days later it was blocked on orders from the U.S. Department of Commerce. Nineteen days after that, it returned with new restrictions in place.
We are a payment company based in Serbia. We process transactions, connect merchants to card acceptance, and handle cardholder data. When a model known for being "too good at breaking things" came back online, we had to try it on our own code.
But this article is not just about a pen test. It is about a line every fintech company needs to draw: where AI genuinely helps, and where it has no business being. Auditing the security of your code is one thing. Giving AI access to your customers' payment data is something else entirely.
The government-banned model reviews our code
When Fable 5 returned on July 1st, we ran our payment service source code through it. We had already done a similar pass with Claude Sonnet as part of our PCI-DSS preparation — that run surfaced eight vulnerabilities, all moderate: stack traces leaking into API responses. Fixed in a couple of days.
Fable 5 went deeper. A model that was banned for its ability to find zero-days is, in practical terms, a free first pass before you pay for an expensive audit. It does not replace a live external pen test — that question is still open for us. But for the cost of a subscription, you get a level of analysis that used to run into the thousands of euros.
One detail matters: the AI analyzes code, but has no access to live data. We uploaded source files — not databases, not configs containing secrets, not transaction dumps. Code is the public-facing part of what we do. Vulnerabilities in it are exactly what an attacker would look for. That is why AI belongs here.
Where AI works: far from customer data
After the pen test, we tried AI in a few other areas. Every successful use case has one thing in common: the AI never touches customer data, payment flows, or production systems.
Contract review
We started running merchant contracts through Claude before signing. Upload a PDF, get back a list of risks, ambiguous clauses, and missing provisions.
The results surprised us. The AI caught things we had been walking past: missing liability carve-outs for third-party outages, fuzzy auto-renewal language, data protection gaps. This is now a standard step in our workflow.
It does not replace a lawyer. But it catches the obvious problems before a document goes out for signature.
Meeting documentation
We connected AI to our working sessions: automatic transcription, decision extraction, task tracking. From this grew a knowledge management system that the whole team uses.
For a small team with no dedicated project manager, this turned out to be genuinely important. Decisions used to disappear into chat threads. Now there is a single source of truth.
Interface prototyping
One of our founders made more than 25 commits in a single night with Claude's help — carousels, category pages, a sidebar. For testing ideas and moving fast, it is a powerful tool.
But — and this matters — every commit goes through human review before it reaches production.
Where AI causes problems: lessons that cost us
Not everything we tried worked. Some experiments cost us money and headaches.
Automated review on every pull request
We set up Claude as a GitHub Action — triggered automatically on every pull request. On paper, a great idea. In practice:
- Our frontend repo was generating 50–100 commits a day. Each one kicked off a 5–6 minute review run.
- GitHub Actions free tier: 2,000 minutes per month. We were burning through it in a week.
- We started topping up: $10, then $20, then $40.
- On the backend, Claude kept failing to complete a single useful review.
We switched to a manual trigger: the developer decides when to call the AI. On one PR for a payment provider integration, Claude flagged 28 issues — half were noise, half were genuinely useful. The signal is there, but only when you control the process.
AI testing its own code
When one of our founders began building the frontend with Claude, testing was the natural next question. The obvious move: let Claude write the tests too.
This is a trap. The same model that wrote the code is checking whether it works correctly. The author's blind spots become the reviewer's blind spots. For an "About Us" page, that's fine. For a payment form, it's not.
AI created three public repositories
During one working session, Claude created three separate repositories outside our main project. The default visibility: public. With chunks of working payment system code inside.
For a company in the middle of PCI-DSS certification, a public repository containing production code is not an "oops" moment — it is a potential incident. After this, we put hard limits in place: AI works only within existing repositories, cannot create new ones, and every change gets human review.
The hard line: where AI does not belong
Here is the rule we landed on: AI does not get access to anything a first-day intern would not be allowed to touch.
Specifically:
- Customer payment data — no AI sees card numbers, CVV codes, or personal data belonging to our payers. Full stop.
- Production environment — AI works only in dev and staging. Deploying to production is a manual process requiring human sign-off.
- Keys and configurations — API keys for payment providers, encryption secrets, database credentials — all of these stay out of sight.
- Financial operations — refunds, merchant payouts, fee changes — manual only, with explicit confirmation at every step.
This is not excessive caution. It is what the industry requires. When you handle other people's money, customer trust is not an abstraction — it is the foundation of the business. One incident with a data leak, and merchants move to a competitor where "AI doesn't touch my data."
Fable 5 and the new normal
Fable 5 came back with a new safety classifier, a jailbreak scoring system, and a bug bounty program through HackerOne. Anthropic acknowledged the problem and built defenses around it.
But the fact that a model can find zero-day vulnerabilities has changed the economics of fintech security. A full security audit used to cost tens of thousands of euros and take weeks. Now the first pass costs a subscription fee.
This doesn't mean "turn on AI and forget about it." It means that small payment companies have, for the first time, access to a tool that was previously reserved for corporations with budgets for external auditors.
Where the line is
| AI belongs here | AI does not belong here |
|---|---|
| Source code audit for vulnerabilities | Access to customer data |
| Contract review before signing | Operations involving customer funds |
| Meeting transcription and documentation | Deploying to production |
| Interface prototyping | Managing keys and secrets |
| On-demand code review by developer request | Automated actions without human oversight |
The main takeaway: AI is not an autopilot for fintech. It is a tool that works around sensitive areas — never inside them. It reviews code but does not touch data. It speeds things up but does not make decisions. It finds problems but does not fix them without a human in the loop.
We use AI every day. But our customers can be certain: between AI and their money, there is always a person.
In short
- Fable 5 surfaced deeper vulnerabilities in our payment service code than previous models — at a fraction of the cost of a traditional audit.
- Every AI use case that worked had one thing in common: no customer data, no production access, no live payment flows.
- Automating AI review on every PR burned through our GitHub Actions budget in a week and produced no useful results on the backend.
- AI creating public repositories with production code is a real risk — one we experienced firsthand during PCI-DSS certification.
- The governing rule: AI gets the same access as a first-day intern. Customer payment data, production systems, and financial operations stay off-limits, always.