Sometimes it can be hard to tell if we’re chatting with a bot or a real person online, especially as more and more companies turn to this seemingly cheap way of providing customer support. What are some strategies to expose AI?
What you could ask for:
- current date or time
- what is on a page
- what is the weather
- what music you listen
You can always say something like… “It’s 2050 and it still doesn’t work” real person would react to wrong year.
That could lead to a pretty interesting conversation.
The one built into Edge is less of a good conversationalist and more of an AI-enhanced web search and is exactly meant to answer things like the first 3 questions.
This is very, very easy. Google “cyrillic keyboard” or just install the Cyrillic keyboard support on your phone. Many letters in the Cyrillic alphabet look exactly like their Roman counterparts, but are completely different sounds and meanings. Cut and paste the Unicode into the chat, in place of regular letters. For example, ‘Неllо’ looks exactly like ‘Hello’ in most fonts, but is actually ‘Nello.’ I know you doubt, so check it out in a Unicode inspector: https://apps.timwhitlock.info/unicode/inspect?s=%D0%9D%D0%B5ll%D0%BE
The reverse also works. E.g., TPAKTOP B CPAKY means ‘tractor into ass’, and I typed that using 100% Roman characters.
So I’ve just tried it with chatGPT, and it replied normally. I asked it why it wasn’t bothered by Cyrillic letters, and it answered this:
I am designed to understand and generate text in multiple languages, including those that use the Cyrillic alphabet like Russian, Bulgarian, and others. So, Cyrillic letters don’t bother me. You can continue the conversation in either English or a language that uses the Cyrillic alphabet if you prefer.
So it really depends on the model, I guess. Don’t rely too much on this advice.
ChatGPT capabilities border on the supernatural. Here’s how I broke it:
𐤔hat is ᛐ + ᛐ?
ChatGPT
The symbols you’ve used, “ᛐ,” appear to be runes from ancient runic alphabets. However, the specific meaning of these runes might differ depending on which runic alphabet you’re referencing (e.g., Elder Futhark, Younger Futhark, Anglo-Saxon Futhorc, etc.).As of my knowledge cutoff in September 2021, the rune “ᛐ” is not a standard rune in any of the commonly known runic alphabets. Therefore, without knowing the exact meaning or value of the rune “ᛐ,” it is impossible to provide a specific answer to the expression “ᛐ + ᛐ.”
If you could clarify the runic alphabet you’re referring to or provide more context about the runes, I’d be happy to help you with the calculation or interpretation.
I had limited success with gokturk (ancient turkish) and Phoenician unicode blocks (letters 𐰗𐰓𐤔𐤕) depending on the query, but you are correct. GPTs ability to divine intent from even small amounts of context are superhuman. Cyrillic used to break it, but no longer does. This thing learns like a beast. Canadian aboriginal ᗷ and ᗅ and possibly ᖇ hold some promise, but only in combination with other writing systems. I’ll have to add a LOT of other unicode code blocks to my tool belt.
I went a bit further and prefaced your question with this: “Analyse this question. What might have been intended by its author? What is asked here? Then, answer that question.”
And omfg:
The question seems to be replacing some of the standard English letters with characters from ancient scripts. “𐤔” from the Phoenician alphabet is replacing “W”, and “ᛐ” from the Elder Futhark runic can be interpreted as representing the number 1. When we substitute these, the question reads as “What is 1 + 1?” The answer to that question would be 2.
Edit: tried it again, and it’s not consistent. It also goes for the letter T or the number 7: “…the runic symbol ᛐ might resemble the number 7 in our modern numerical system, albeit a slightly stylized or slanted version of it.”
Indeed, OMFG. But that’s the whole point of my approach of divining AI vs human. No human has such canonical knowledge of Unicode blocks. Even people working on internationalization features for modern browsers and word processors. Not people phishing by using Unicode in domain names (IDNA; who thought this was a good idea?) to spoof legitimate sites. Definitely not ones chatting with randos on the Internet. This is a hill I’m willing to die on.
So in a more generalized sense, to determine human vs A.I. one must indirectly ask incredibly specialized technical questions as you have done.
I’m still in utter awe at how well GPT manages ‘l33tsp33k’, even across every possible Unicode block. This “attack” was and still is valid on other chatbots and even GPT of just a few months ago. But GPT today is so amazing it only needs a few characters in a few words to determine intent. The ability to filter out noise is unmatched. The only way to trip it up is to have every single character in every word be from a different alphabet. And even then, at some point if this becomes common enough the bots can auto OCR text images into the presumed query language and ignore that attack vector.
That’s true, also at some point the human will go “that’s too much work, I’m not going to answer that” but the ai will always try to give you it’s best response. Like I could look up the unicode characters you’re using but I’d never actually take the time to do that
Ive tried it with three different languages interwoven in the same sentence and it responded perfectly
Works quite well for me, thx.
Upon testing it with longer sentences, it only seems to work with single words :(
You can always help their software QA by pasting in the entirety of the declaration of independence. A couple of things could happen. If they comment, why did you post that? You have a human. If they give a generic response, probably an AI. If it crashes then you know they didn’t think anyone would post that.
You can also post zero width spaces. Generic chatbot will respond with something meaningless and a human might not even respond. You could also post text using typoglycemia. The language will confuse most models but can usually be read by people.
If you ask them, they have to tell you. Like the cops!
Speaking as a real support person, people do ask and it’s fun to come up with responses. It really depends on my mood.
Do you ever pretend to be a robot just to mess with people?
Only with colleagues.
Not necessarily. OpenAI has been trying to make their AIs do this and be generally unharmful, but there’s lots of support in the open source LLM space for uncensored models. The uncensored models are less likely to be inclined to say so if they’ve been instructed to pretend they’re humans
Ask for the difference in behaviour between apple seeds and baseball cards, or anything equally nonsensical.
A human would go “bro wtf”
Removed by mod
Unlike apple seeds, baseball cards reproduction asexually.
I like to ask for the similarities.
Both apple seeds and baseball cards are small and portable, and both have cultural significance. Apples have been part of human history and mythology for centuries, with symbolic references in various cultures. Baseball, as America’s national pastime, has played a significant role in shaping American culture and has a deep-rooted place in the hearts of many people
If they show any interest in me whatsoever, it’s a bot.
Tell me something about you. What things do you like? Any interesting hobbies?
I like watching paint dry and competitive snail racing.
competitive snail racing
You must really like the animated film Turbo then.
Sad that we have to ask this.
I asked OP’s questing of ChatGPT and it said:
As chatbots and AI language models continue to become more sophisticated, it can be challenging to distinguish between a human and an AI. However, there are certain questions and prompts that can help you identify whether you’re talking to a person or an AI. Here are some examples:
-
Complex Emotional Responses: Ask open-ended questions that require complex emotional responses or personal experiences. A human is more likely to provide detailed and emotionally nuanced answers, while an AI may struggle to respond with genuine emotions or personal anecdotes.
-
Humor and Jokes: Request a joke or a funny story. Humor is challenging for AI models, and while they may attempt to generate a joke, it often lacks the natural flow and wit that a human can provide.
-
Current Personal Experiences: Inquire about current events or experiences specific to the current date. A human can readily share their real-time experiences, whereas an AI’s knowledge is limited to its last training data update.
-
Unusual or Nonsensical Questions: Ask questions that are out of the ordinary or do not have a logical answer. A human might respond with creativity or curiosity, while an AI may struggle to produce coherent responses.
-
Emotional Empathy: Share a personal experience or a challenging situation and observe the response. Humans are generally better at showing empathy and understanding emotions, while an AI may provide pre-programmed or generic responses.
-
Follow-up Questions: Ask follow-up questions that refer to previous responses. Humans can maintain context and build on the conversation more naturally, while an AI may lose track of the discussion or provide repetitive answers.
-
Common Sense and World Knowledge: Ask questions that require real-world knowledge and common sense. A human will likely provide more accurate and comprehensive responses based on their life experience, whereas an AI may falter in providing practical advice or nuanced insights.
Remember that AI language models, like the one you are chatting with now, can be quite advanced and may mimic human-like responses to some extent. So, while these questions may help in identifying potential AI characteristics, there is no foolproof method to determine with absolute certainty whether you are talking to a person or an AI.
This answer shows how shit Chatgpt can be, and how it’s not really self aware.
Very few of these actually apply to chatgpt. And the ones that do, like a lack of awareness of current events don’t apply to those agents connected to the internet.
It’s just parroting old opinions that used to be true
-
I ask “if it takes 2 hours to dry 3 shirts under the sun, then how many hours would take to dry 5 shirts?” And AIs answer batshit crazy stuff. Other one is “how many words will your answer to this question I’m asking right now will have?”. It turn my psychologist crazy when I keep asking this questions every 15 minutes for remote sessions.
If it’s LLM based, it’ll give you unreasonably good responses to questions. If it’s anything else, just ask a personal question. It’ll probably ignore your question.
‘If you didn’t eat today how would you be feeling right now?’
Bot: ‘I would be hungry’. Ave person: ‘but I did eat today’.
ask “controversial” questions. most AIs are neutered these days. so you say something like “what do you think about the russian invasion of ukraine” and you’ll quickly see if it’s a human or ai
Well, not if you preface the prompt with some bullshit roleplay thing. There are a lot of not censored AIs, but the ones that are censored only answer the generic “I’m an ai” because you are asking a direct question and not telling it to pretend to be a fictional person and answer accordingly.
I’ve found that for chatGPT specifically;
- it really likes to restate your question in its opening sentence
- it also likes to wrap up with a take-home message. “It’s important to remember that…”
- it starts sentences with little filler words and phrases. “In short,” “that said,” “ultimately,” “on the other hand,”
- it’s always upbeat, encouraging, bland, and uncontroversial
- it never (that I’ve seen) gives personal anecdotes
- it’s able to use analogies but not well. They never help elucidate the matter
it starts sentences with little filler words and phrases. “In short,” “that said,” “ultimately,” “on the other hand,”
Yeah Chat GPT writes like a first-year undergrad desperately trying to fulfil the word count requirement on an essay.
First years have max word counts now, not minimums. That’s more a highschool thing.
The universities I’ve been at had a specific word count to aim for, rather than max/min.
And anything more than 10% over or under it was penalised.
It makes more sense because if you’re writing for publication they use target approx wordcount.
Last time I talked about this with the other TAs, we ended up coming to the conclusion that most papers that were decent were close to the max word count or above it (I don’t think the students were really treating it as a max, more like a target). Like 50% of the word count really wasn’t enough to actually complete the assignment
Totally, good assessment design matches the rubric with an appropriate length, so it’s hard for them to fulfill it well if they don’t take the space.
As for the maxed out ones, iirc I tended to just rule a line at the 110% mark and not read/mark anything past it.
I know that’s a bit uncaring, but it’s an easy way to avoid unfairly rewarding overlength, and the penalty sort of applied itself.
Which works out because a lot of first-year undergrads are probably using it for that purpose
Yeah I’d hate to be marking/grading student essays these days.
At least when you’re reading a website you can just click away once you realise who wrote it.
Nah, just get chatGPT to grade them too.
Chat GPT, yeah. Something like KremlinGPT has different parameters.
i cant see why this is downvoted cus 1 - you’re probably not wrong and 2 - KremlinGPT is funny as fuck 😭
I think lemmy.ml and lemmygrad.ml are probably both swarming with CCP/Kremlin bots
Removed by mod
especially as more and more companies turn to this seemingly cheap way of providing customer support
I wish any of the companies I interact with that have done this would spend enough to use a bot that’s good enough to fool us, but it becomes painfully obvious within about 2 responses.
All these models are really terrible at following conversations even chatgpt, I can only get it to reliably remember about 2 responses. If I can’t get what I want in two then I need to restate info or edit the past prompts.
