Regex or a neural net: parsing text without torching your budget

What this choice is and why it’s about money

Picture this: someone drops a box of letters on your desk. The job is to pull a name and an address out of each one. You open it up and find the letters come in two kinds.

The first kind is strict-format forms. Always “Name:” at the top, always “Address:” below it. Same in every single one. No surprises. You don’t even have to think here — you put a nitpicker robot on it: “grab everything after the word Name and a colon.” It’s fast, free, and never gets tired.

The second kind is handwritten postcards from grandmas. One has the name in the corner. Another has it way at the bottom. A third drew a little map instead of an address: “turn left behind the shop, ask by the oak tree.” The nitpicker robot quietly sobs in the corner here. This needs a living, thinking person — someone who’ll read it, get the meaning, and figure out even the bit about the oak tree.

In vibecoding it’s exactly the same. The nitpicker robot is regex Regex (a regular expression) is a short template-rule that tells the computer to find chunks of a certain shape in text: ‘find everything that looks like a date’ or ‘grab the text after the word Price.’ It only works when the shape repeats. (simple rules). The thinking person is a neural net (Claude or GPT, take your pick). And the core skill of this lesson is sensing which of the two to call for a given job.

Why you’re the one making this call

You’re a vibecoder — you don’t write regex by hand, the agent does that for you. But deciding who to call stays with you. It lives right inside how you phrase the task. And it drives three things at once:

• money — every neural-net call costs tokens; pushing 10,000 lines through it when a template would’ve handled them is burning budget for nothing;
• speed — rules fire instantly, while the neural net spends its own seconds thinking about each chunk;
• reliability — on tidy text a rule gives the exact same answer every time, while a neural net will every now and then invent something fresh.

The classic beginner mistake: “oh, text, I need to parse it — call the AI.” And it’s a list of a thousand identical lines, where three words in a prompt would’ve done it. Take this lesson to heart and you’ll stop overpaying.

Regex or a neural net: a decision tree from a single question

Before you parse anything, ask yourself one thing: is the text tidy or chaotic? After that it’s simple.

Text is tidy and repeats (90%+ on one template)

Start with simple rules. Two branches here:

• the rules pull out 95% or more — great, you don’t need the neural net at all;
• the rules get less than 95% — bring in the neural net, but only for the leftovers that didn’t parse.

Text is freeform and different every time

Here rules are helpless, no matter how hard you try. Call the neural net right away — this is exactly its job, exactly what it was built for.

Regex versus a neural net: who’s good at what

The regex-plus-neural-net hybrid: the pros’ signature move

Here’s how experienced folks do it. The text rolls down a conveyor belt:

1. Simple rules run over the whole stream and pull out the structure — that catches 95-98%.
2. A confidence check flags the suspicious chunks: too few options here, a missing answer there, a line cut off mid-word over here.
3. The neural net kicks in only on the flagged leftovers — and fixes them.

You don’t call the pricey expert for the whole box of letters. Just for the three postcards where the nitpicker threw up its hands.

Parsing a thousand quiz questions: what it looks like in practice

Say you’ve got an export from an old textbook — a thousand quiz questions, all alike: a number, the question text, options A-D, an “Answer:” line with a letter. You need to pull this into a neat table.

How a beginner does it: “Claude, here’s a file with a thousand questions, sort it all out.” The agent dutifully chews through tens of thousands of lines via the neural net — slow, expensive, and somewhere around question 600 it’ll start fibbing too. Just from fatigue.

How a clued-in vibecoder asks:

I have a file with a thousand quiz questions. They all share one format: a number, the question text, options A, B, C, D, then an Answer line with a letter.
Do this:

First, write a simple rule (regex) that pulls the number, question, options, and answer out of this tidy text. Run it over the whole file.
Count which chunks parsed poorly: fewer than three options, no answer, or a suspiciously short question. Show me the list.
Then parse only those tricky chunks intelligently, by meaning. Leave the rest alone.
At the end, tell me: what percentage the rule handled and how many chunks had to be patched up.

Common mistakes when parsing text

• Running everything through the neural net “just in case.” Text is tidy? That’s like hiring a professor to count tally marks in a notebook. Expensive, slow, and a little insulting to the professor.
• Forcing rules onto live chaos. If the shape dances from chunk to chunk, the nitpicker robot will choke — and here you actually need the neural net.
• Skipping the confidence check. You can’t blindly trust “the rule seems to have worked.” Ask the agent to flag the suspicious chunks instead of silently handing everything over.
• Calling an expensive model where a cheap one would do. For checking the tricky leftovers you use the cheapest neural net (Haiku-class) — for a job that small it’s more than enough.
• Not testing on messy data. Empty lines, a missing answer, weird characters — that’s exactly where everything breaks. Run the “bad” examples ahead of time, before the real launch.