Human on the Podium
Hack the North 2026
How much of a political speech did a machine write? That was the question we spent Hack the North on. Try the live tool.
The idea
Politicians can use AI to write speeches and official documents, and nobody has to say so. We wanted a tool that tells you roughly how much of a text a machine wrote, as a single number like “28% AI”, instead of a flat HUMAN or AI label. Most writing is a mix, and one label hides that.
Making our own data
The plan was to test on real speeches. Problem: nearly all of them were written by people, so there was nothing to test against. So we made a dataset where we knew exactly how much AI was in each record.
We took 200 public appointment records from the Investigative Journalism Foundation: 100 Manitoba Orders in Council and 100 federal Privy Council appointments (half French). A Python pipeline sent each one to GPT-4 with a target for how much to rewrite, measured the result, and retried until it hit the target.
| Set | Records | What we did | Text kept |
|---|---|---|---|
| Provincial | 40% | Full AI rewrite | 24% |
| Provincial | 40% | Light edit | 97% |
| Provincial | 20% | Left alone | 100% |
| Federal | 60% | Moderate edit | 82% |
| Federal | 40% | Deeper rewrite | 64% |
The fiddly part was keeping every name, date, dollar amount and legal citation
intact. Legal text is full of full stops that don’t end sentences (c. P271,
$4,404.38), and 19 records didn’t follow the usual layout, so the pipeline
detects the language of each line instead of trusting fixed phrases. In the end
it ran all 200 with zero failures, about 40 seconds per 100.
What we built
- Live detector. Speak into the mic and it transcribes you, then colours each sentence by how AI it reads, every 80 words or 2.5 second pause.
- Hacker badge. Joshua got the Hack the North badge running and wrote a small API for it in Lua, so our Next.js app can hit it directly and change what the badge shows. It sits on the same WiFi and flashes red when a section is 50%+ AI, green when it isn’t.
- Analyze. Paste a YouTube or CPAC link with a timestamp and it scores 60 seconds of audio.
- Database and Map. All 205 records, searchable, and plotted in 2D/3D by meaning so you can spot near-identical text that scored very differently.
Tech Stack
Built with GPTZero for AI detection, ElevenLabs Scribe for live transcription, Elasticsearch kNN with Jina embeddings for the map, Next.js, SQLite and yt-dlp + ffmpeg for videos.
What we found
Across the 200 rewritten records, the average AI share was 35%. 81 records scored 50% or more, and 74 scored under 10%.
The federal records, which all got edited, mostly land around 60–70%. The provincial ones pile up near zero, even though 40 of them were full rewrites. Only about 20 provincial records scored above 50%. That gap is on us: some of the rewritten data got missed during processing, so not every full rewrite made it into the scored set.
The five real speeches we tried (Stephen Harper, Evan Solomon, the EU State of the Union, and a couple of others) all read as human sentence by sentence, between 0% and 8%.
The Bill Oliver clip
One of those real clips is a New Brunswick MLA who read a chatbot’s reply out loud in the legislature, preamble and all. Score it as one block and you get 28% AI. Score it sentence by sentence and average, and it drops to 8%. The line that literally is the chatbot talking? 6%.
8% AI
Another concern relates to expectations. One of the dangers associated with creating advocacy offices is that citizens often develop expectations that exceed the powers actually granted to those offices. Here's a more natural flowing version of that section that reads like a legislative speech rather than a series of short points. Madam Speaker, one of my concerns with this legislation is the potential gap between what New Brunswickers may reasonably expect this office to accomplish and what the office will actually have the authority to do once it is established.
Tap a sentence to see its score.
What we learned
AI detectors aren’t great with speech. Short clips don’t give them enough to go on, so they often miss AI text, which means a “human” result on a short clip doesn’t prove much. And the same text can score very differently depending on how you feed it in, like the Bill Oliver clip above. So we’d treat a detector score as a hint, not a verdict.
How it went
We didn’t win, but I had a great time talking through the idea with Alex (Co-founder of GPTZero) and the other mentors, and got some really useful feedback. The hacker badge also made our demo so much better. Watching it flash red while someone read out a speech got a much bigger reaction than any screen could :)