IRC Bot - GPT-3

My core friend group hangs out in an IRC chat room. Like any nerd who spends too much time on IRC, you eventually have the irresistible temptation to build your own IRC bot. I succumbed to that temptation.

Owithertz started, like many other bots, as a straightforward command-based IRC bot. It primarily serves as our shared YouTube subscription feed for common interests among our friends: Edutainers, Space, Formula 1, etc.

Then I got access to GPT-3, and inspiration struck. An afternoon later, Owithertz evolved from a dumb command bot to the most fun IRC bot that anyone in our private group chat has ever played with. You can ping Owithertz to chat with it, and it’ll reply back with a GPT-3 response. And, because of my penchant for chaos, Owithertz has a 2% chance to respond to any message in our room. This often leads to hilarity or frustration.

Snippets

Here’s some selected chat snippets. Our usernames are hidden.

The Snark
Owithertz was crafted to have some real snarky personality.

The Trenchcoats
Its selection of materials and prices is amazing.

The Bandcamp
This is the first time I remember being completely shocked by owithertz. It made up that URL out of nowhere.

The Caillou Event
We were poking at some edge cases in GPT-3. Turns out, Caillou is a bit of a monster.

OpenAI GPT-3

In a loose attempt to follow the spirit of the OpenAI guidelines, each bot query that uses the GPT-3 stack follows a contextualization and filter pipeline.

1. Context (Babbage) - Attempts to categorize the input to direct optimized sub-variants of the bot based on the conversation context: are we having a casual chat? Are we asking the bot a direct question? Are we encouraging the bot to be creative?
2. Generate (Davinci) - From the context, the request lands in a different seeded variant of Owithertz optimized for the context.
3. Filter (Babbage) - The output from the generation step is filtered for both OpenAI content rules (less important to me, more important to them) and how additive it is to the conversation.

Frustrations

My frustration with this strategy is that GPT-3 thinks replying with “I don’t know” or summarizing the context of the conversation is an appropriate reply. It isn’t, so I ban and regenerate these replies or just give up (if it’s a random chance reply). Interestingly, the more mechanical steps and variants I add to the pipeline, the less intelligent Owithertz feels, so the current pipeline is a bit shorter than it used to be.

My seed generation is more “intuition-based,” not data or evidence-based, and I’ve had some interesting learns along the way:

• using “is never unfriendly” vs. “friendly”: GPT-3 appears to not pick up on the intended meaning/interpretation of words but rather the probability of them occurring in the text. So using things like “never” in a seed in the context of “never unfriendly” will just make it more likely to use “never” in any context.
• It’s not great at net-new creation. With a general seed optimized for “general creativity,” queries with “come up with a band name for X” will just be...boring or bad. You’d
• It’s shockingly good at providing advice. I once experimented with a seed variant for “business,” and it spoke with a decent business leader’s cognitive ability and intuition. I had too much fun making it write product requirements for fake hardware and software features.
• If you want S-tier generations, you need to be a subject matter expert in writing that seed. However, customizing a seed variant to give you B-tier (or better) replies is relatively easy once you already know how to write good seeds.

And, one more sad anecdote: as OpenAI updates the GPT-3 models, they’ve added more guard rails to their responses, but it’s also led substantive decrease of my ability to build a bot with personality. My latest bot variant using the latest GPT-3 models is too dull and dry. I haven’t figured out how to get the sharp chaotic sassy personality Owithertz used to have back.

A note to OpenAI

This bot is in a very small access-controlled IRC room. Everyone in the room knows it's a bot. We don't poke at uncomfortable things. Don't ban me, please :)