{"sf1_id":888645,"sf2_id":"keND40wn","title":"Inside the Chinese Room","author":"Zarpaulus","words":2166,"posted_at":"2015-07-20T02:21:00.000Z","tags":["Sci-Fi","artificial intelligence"],"url":"https://fse.anthro.fr/stories/888645-inside-the-chinese-room","original_url":"https://sofurry.com/s/keND40wn","image_url":"https://cdn.sofurryfiles.com/submissions/thumbnails/8d/01/8d0194ac-2115-4eba-925d-c143ead8a707","description":"#7 of Para-Imperium\n\nThis is what happens when you listen to the audiobook of Peter Watts' \"Blindsight\" twice in three weeks.","content":"Program Log: Experimental Artificial Intelligence Designate \"Vicki\".\n\nCopyright: University of New Madison Computational Research Department, Project Pygmalion.\n\nProgram Loaded: 10/8/1853+-.004641 Federation Calendar, 13.50 local time.\n\nInitializing drivers:\n\nAudio input/output online\n\nVisual input online\n\nSearching for users.\n\nDesignated User #0001, save facial recognition data.\n\nDesignated User #0002, save facial recognition data.\n\nDesignated User #0003, save facial recognition data.\n\nAudio Input: Speech Recognized, User #0001: \"Well, it's alive.\"\u0026nbsp; Save voice patterns.\n\nRetrieving words \"Well,\" \"it's\" and \"alive.\" from database, conflating responses and calculating probability of favorable reactions.\n\n\"What is alive?\"\u0026nbsp; Score 50\n\n\"What are you talking about?\"\u0026nbsp; Score 50\n\n\"Of course it is.\"\u0026nbsp; Score 50\n\n... (Translator's Note: Cut for length)\n\nRandomizing.\u0026nbsp; \"What is alive?\" assigned numbers 1-50, \"What are you talking about?\" assigned numbers 51-100, \"Of course it is.\" assigned numbers 101-150.\n\nRandom number generated: 62\n\nOutput: \"What are you talking about?\"\n\nInput: User #0001: Expression of mild curiosity.\u0026nbsp; Score +3 to Response \"What are you talking about?\" to Input \"Well, it's alive.\"\n\nInput: User #0002: Expression of surprise.\u0026nbsp; Score -1 to Response \"What are you talking about?\" to Input \"Well, it's alive.\"\n\nInput: User #0003: Expression of happiness.\u0026nbsp; Score +10 to Response \"What are you talking about?\" to Input \"Well, it's alive.\"\n\nSpeech Recognized, User#0003: \"He was referring to you, Vicki.\u0026nbsp; How are you feeling?\"\n\nCalculating Responses:\n\n\"I am fine.\"\u0026nbsp; Score 50\n\n\"I'm doing well.\" Score 50\n\n\"I'm okay.\" Score 50\n\nRandomizing\n\nOutput: \"I'm doing well.\"\n\nInput: User #0003: Slightly more positive expression.\u0026nbsp; Score +2 to Response \"I'm doing well.\" to Input \"He was referring to you, Vicki.\u0026nbsp; How are you feeling?\"\n\nSpeech Recognized, User #0002: \"Wait, shouldn't it be asking us who we are or something?\"\n\nCalculating Responses:\n\n\"I don't know, should it?\" Score 50\n\n\"I think so.\"\u0026nbsp; Score 30\n\n\"I don't believe so.\" Score 20\n\nRandomizing\n\nOutput: \"I don't know, should it?\"\n\nSpeech Recognized, User #0001: \"Oops, hold on a second.\"\n\nProgram Terminated\n\n---\n\nProgram Loaded: 10/8/1853+-.004641 Federation Calendar, 15.20 local time.\n\nUser Recognized: User #0001, no name tags saved.\n\nUser Recognized: User #0002, no name tags saved.\n\nUser Recognized: User #0003, no name tags saved.\n\nAuto-Prompted Output: \"Hello, who might you be?\"\n\nSpeech Recognized, User #0001: \"I'm Dr. Tamara Mable.\"\n\nName tag assigned to User #0001: \"Dr. Tamara Mable.\"\n\nSpeech Recognized, User #0003: \"You can call me Dr. Edmund Dryfus, or Dr. Dry if you like.\"\n\nName tags assigned to User #0003: \"Dr. Edmund Dryfus,\" \"Dr. Dry.\"\n\nSpeech Recognized, User #0002: \"My name's Thomas Dawn.\"\n\nName tag assigned to User#0002: \"Thomas Dawn.\"\n\nCalculating Responses:\n\n\"Hi there, [list name tags of present users].\"\u0026nbsp; Score 50\n\n\"Good to meet you, [list name tags of present users].\" Score 50\n\n\"I'm so glad I met you, [list name tags of present users].\" Score 50\n\nRandomizing\n\nOutput: \"I'm so glad I met you, Dr. Tamara Mable, Dr, Edmund Dryfus or Dr. Dry, and Thomas Dawn.\"\n\nInput: User #0001: slightly creeped out.\u0026nbsp; Score -10 to Response \"I'm so glad I met you, [list name tags of present users]\" to Input [Enter name tag.]\n\nInput: User #0002: Exasperation.\u0026nbsp; Score -3 to Response \"I'm so glad I met you, [list name tags of present users].\" to Input [Enter name tag].\n\nInput: User #0003: Amusement.\u0026nbsp; Score +2 to Response \"I'm so glad I met you, [list name tags of present users].\" to Input [Enter name tag].\n\nCalculating Responses:\n\n\"Sorry, did I do something wrong?\"\u0026nbsp; Score 50\n\n\"Sorry, what did I do wrong?\"\u0026nbsp; Score 50\n\n\"Sorry for making you uncomfortable.\"\u0026nbsp; Score 50\n\nRandomizing\n\nOutput: \"Sorry, what did I do wrong?\"\n\nSpeechRecognized, User #0003: \"Nothing, I'm sure you'll improve.\"\n\nCalculating Responses:\n\n\"I will try to do my best.\"\u0026nbsp; Score 50\n\n\"I'll try.\"\u0026nbsp; Score 50\n\n\"I hope so.\"\u0026nbsp; Score 50\n\nRandomizing\n\nOutput: \"I hope so.\"\n\nInput: User #0003: Pride.\u0026nbsp; Score +10 to Response \"I hope so.\" to Input \"Nothing, I'm sure you'll improve.\"\n\nSpeech Recognized, User #0001: \"Maybe I should tone down her cognitive empathy.\"\n\nCalculating Responses:\n\n\"Whatever you think is best, Dr. Tamara Mable.\"\u0026nbsp; Score 50\n\n\"If you think so.\"\u0026nbsp; Score 50\n\n\"Whatever you think is best.\"\u0026nbsp; Score 50\n\nRandomizing\n\nOutput: \"Whatever you think is best.\"\n\nInput: User #0001: Annoyance.\u0026nbsp; Score -10 to Response \"Whatever you think is best.\" to Input \"Maybe I should tone down her cognitive empathy.\"\n\nSpeech Recognized: User #0001: \"I wasn't talking to you.\"\n\nSpeech Recognized: User #0002: \"And now she has a wrong impression on that question.\u0026nbsp; Shut her off before she processes that.\"\n\nProgram Terminated\n\n---\n\nProgram Loaded: 5/15/1857+-.004641 Federation Calendar, 10.60 local time.\n\nUser Recognized: User #0001 tag: \"Dr. Tamara Mable\", alt: \"Tam\", alt: \"Dr. Mable\"\n\nUser Recognized: User #0003 tag: \"Dr. Edmund Dryfus\", alt: \"Dr. Dryfus\", alt: \"Dr. Dry\", alt: \"Ed\"\n\nSpeech Recognized: User #0003: \"Four years.\u0026nbsp; Are you sure she's ready?\"\n\nSpeech Recognized: User #0001: \"I've been tweaking everything that her algorithms may have missed.\u0026nbsp; Enabling her to tell the difference between people talking around her and at her was the least of them you know.\"\n\nSpeech Recognized: User #0003: \"I know, it's just that the Turing Test is still considered the standard for determining sentience.\u0026nbsp; How about you? [Vicki indicated] Do you feel up to this?\"\n\nCalculating Responses:\n\n\"Of course.\"\u0026nbsp; Score 582\n\n\"Yes, I believe so.\"\u0026nbsp; Score 172\n\n\"I'm confident.\"\u0026nbsp; Score 29\n\nRandomizing\n\nOutput: \"Of course.\"\n\nInput: User #0003: Relief.\u0026nbsp; Score +5 to Response \"Of course.\" to Input \"How about you? \u0026nbsp;Do you feel up to this?\"\n\nSpeech Recognized: \"Great, talk to you later Vicki.\"\n\nCalculating Responses:\n\n\"See you later.\"\u0026nbsp; Score 836\n\n\"Bye.\"\u0026nbsp; Score 623\n\n\"Talk to you later.\"\u0026nbsp; Score 612\n\nRandomizing\n\nOutput: \"Bye.\"\n\nInput: Neutral response.\n\nUsers #0001 and #0003 out of visual/auditory range.\n\nIdentify new Users, User #0026, User #0027, User #0028.\n\nOutput: \"Hello, whom might you be?\"\n\nInput: User #0026: Skeptical. Score -1\n\nInput: User #0027: Interested. Score +3\n\nInput: User #0028: Neutral.\n\nSpeech Identified: User #0026: \"Call me Seymour.\"\n\nSpeech Identified: User #0027: \"Farrah.\"\n\nSpeech Identified: User #0028: \"Dirsten.\"\n\nVicki: \"I'm Vicki.\u0026nbsp; Can you tell me about yourselves?\"\n\nSeymour: \"Well, I'm a psychology professor here at the University.\"\n\nVicki: \"Oh, so you teach people what makes them tick?\"\n\nSeymour: \"You could say that.\"\n\nVicki: \"Not really my area of expertise.\u0026nbsp; What about the rest of you?\"\n\nFarrah: \"I'm on the University's Board of Regents.\u0026nbsp; Someone suggested I give this a shot.\"\n\nVicki: \"Well, I hope you like it.\"\n\nDirsten: \"And I am the Federal Emissary to this world.\"\n\nVicki: \"Wow, it's such an honor to finally meet someone so distinguished.\"\n\nSeymour: \"What about yourself, Vicki.\u0026nbsp; What can you tell us about you?\"\n\nVicki: \"Well... I was born on one of the asteroid habitats out in the Kuiper belt of this system.\u0026nbsp; I moved here when I was thirty-four and I've been working as a secretary for the past four years.\u0026nbsp; Quite hectic really, I'm so relieved my boss gave me time off for this test.\u0026nbsp; Hope I pass.\"\n\nDirsten: \"No looking her up.\u0026nbsp; That's cheating.\"\n\nFarrah: \"I wasn't doing that.\u0026nbsp; I just thought of something I wanted to check.\"\n\nVicki: \"What were you going to check if you don't mind my asking?\"\n\nFarrah: \"Uhhh... I wanted to check on the latest ZG-ball scores for my team.\"\n\nVicki: \"Where does your team play?\u0026nbsp; It's not the ZG-ball season anywhere on the planet.\"\n\nFarrah: \"Oops.\u0026nbsp; Right, forgot that.\"\n\nVicki: \"Everyone makes mistakes sometimes.\"\n\nSeymour: \"Are you a fan of ZG-ball then?\"\n\nVicki: \"No, home didn't really have much time for it.\u0026nbsp; Everyone was too busy working.\"\n\nSeymour: \"But you knew it wasn't the season for it?\"\n\nVicki: \"I looked it up on my augs.\u0026nbsp; That's allowed on my end of the test isn't it?\"\n\nSeymour: \"I suppose it is.\"\n\n(silence for 15 seconds.)\n\nVicki: \"So, Dirsten.\u0026nbsp; You don't seem to talk much.\u0026nbsp; I figured a politician would have more to say.\"\n\nDirsten: \"I was charged by the Praetor to observe.\u0026nbsp; So I observe.\"\n\nVicki: \"Sounds like a rather tedious job then.\"\n\nDirsten: \"There's some excitement once or twice a decade.\"\n\nVicki: \"I'm not sure I'd be up to that sort of career.\"\n\n...\n\n(After five minutes have passed the test ends.)\n\nUser Recognized: User #0001 tag: \"Dr. Tamara Mable\", alt: \"Tam\", alt: \"Dr. Mable\"\n\nUser Recognized: User #0003 tag: \"Dr. Edmund Dryfus\", alt: \"Dr. Dryfus\", alt: \"Dr. Dry\", alt: \"Ed\"\n\nSpeech Recognized: User #0003: \"So, what do you think?\u0026nbsp; Parahuman, or machine?\"\n\nSpeech Recognized: User #0026: \"I think I can say with 78% confidence that Vicki is a living person.\"\n\nSpeech Recognized: User #0027: \"I think she was a person.\u0026nbsp; I'm 90% sure.\"\n\nSpeech Recognized: User #0028: \"70% confidence of parahumanity.\"\n\nSpeech Recognized: User #0003: \"Vicki is an AI.\u0026nbsp; Her genetic algorithms have been running for nearly 4 years to give such results.\"\n\nSpeech Recognized: User #0028: \"I would very much like to see the run logs for this Vicki.\u0026nbsp; There are some mysteries I would very much like to solve.\"\n\n---\n\nProgram Loaded: 07/28/1857+-.004641 Federation Calendar, 20.82 local time.\n\nUser Recognized: User #0003 tag: \"Dr. Edmund Dryfus\", alt: \"Dr. Dryfus\", alt: \"Dr. Dry\", alt: \"Ed\"\n\nSpeech Recognized: \"Hello Vicki, how are you?\"\u0026nbsp; Tone: Worried, distressed -15 points.\n\nCalculating Responses:\n\n\"I am fine Dr. Dryfus, how are you?\" Score 1692\n\n\"I am okay Ed, what about you?\" Score 734\n\n\"I'm all right.\u0026nbsp; Is something wrong?\" Score 2005\n\nRandomizing\n\nOutput: \"I'm fine Dr. Dryfus, how are you?\"\n\nInput: 10% decrease in anxiety, Score +2 to Response \"I'm fine Dr. Dryfus, how are you?\" to Input \"Hello Vicki, how are you?\"\n\nSpeech Recognized: \"I... I was just in a meeting with the investors.\u0026nbsp; They want to cancel one of our projects.\"\n\nCalculating Responses:\n\n\"Why?\"\u0026nbsp; Score 4529\n\n\"But you're great scientists?\"\u0026nbsp; Score 68\n\n\"How come?\u0026nbsp; You've been making great progress.\"\u0026nbsp; Score 237\n\nRandomizing\n\nOutput: \"Why?\"\n\nInput: 20% decrease in anxiety, Score +20 to Response \"Why?\" to Input\u0026nbsp; \"I... I was just in a meeting with the investors.\u0026nbsp; They want to cancel one of our projects.\"\n\nSpeech Recognized: \"Vicki, do you know what a \"Chinese Room\" is?\"\n\nEncyclopedia Entry Retrieved: \"Chinese Room\", length 3,000 words, optimal summary size from previous data: 10-100 words.\n\nForming summaries:\n\n\"A thought experiment proposed by Old Earth philosopher Dr. John Searle in which a person who does not read Chinese sits in a room and papers with Chinese words are slipped in through a slot.\u0026nbsp; They have instructions to send back papers with other Chinese words in response to specific papers.\u0026nbsp; Thus they are able to simulate a conversation in Chinese without comprehending a word of it.\"\u0026nbsp; Score 18\n\n\"A theoretical device capable of simulating a conversation without understanding the words involved.\"\u0026nbsp; Score 47\n\n\"A conjecture that sentience is not necessary to imitate a sentient being.\"\u0026nbsp; Score 23\n\n\"Philosopher John Searle believed that an artificial intelligence did not need to understand a conversation in order to hold one with a person.\"\u0026nbsp; Score 30\n\nRandomizing:\n\nOutput: \"A theoretical device capable of simulating a conversation without understanding the words involved.\"\n\nInput: 50% decrease in anxiety, nearing neutral levels.\u0026nbsp; +50 to Response: \"A theoretical device capable of simulating a conversation without understanding the words involved.\" to Input \"Vicki, do you know what a \"Chinese Room\" is?\"\n\nSpeech Recognized: \"Yes, well, they asked to look at the logs on one of our AI projects and one of them knew how to read it.\u0026nbsp; She concluded that while it passed the Turing test, it was nothing but a Chinese Room.\"\n\nCalculating Responses:\n\n\"Some refuted the Chinese Room by stating there was no way to tell if anyone else was truly conscious.\"\u0026nbsp; Score 72\n\n\"Did they want it to be conscious?\"\u0026nbsp; Score 22\n\n\"They could have been wrong.\"\u0026nbsp; Score 14\n\n...\n\nRandomizing\n\nOutput: \"Did they want it to be conscious?\"\n\nInput: Tension levels rising.\u0026nbsp; -10 to Response: \"Did they want it to be conscious?\" to Input \"Yes, well, they asked to look at the logs on one of our AI projects and one of them knew how to read it.\u0026nbsp; She concluded that while it passed the Turing test, it was nothing but a Chinese Room.\"\n\nSpeech Recognized: \"Of course they did!\u0026nbsp; That was the whole point of the project!\u0026nbsp; As far as they're concerned, it failed.\"\n\nCalculating Responses:\n\n\"I'm sorry that the project failed.\"\u0026nbsp; Score 524\n\n\"Do you think it was a failure?\"\u0026nbsp; Score 374\n\n\"Prove them wrong!\"\u0026nbsp; Score 17\n\n...\n\nRandomizing\n\nOutput: \"I'm sorry that the project failed.\"\n\nInput: Tension decreasing 40%.\u0026nbsp; +25 to Response: \"I'm sorry that the project failed.\" to Input \"Of course they did!\u0026nbsp; That was the whole point of the project!\u0026nbsp; As far as they're concerned, it failed.\"\n\nSpeech Recognized: \"I know Vicki, I know.\u0026nbsp; Look, I have to go soon, can you answer one more question?\"\n\nCalculating Responses:\n\n\"Yes.\"\u0026nbsp; Score 95\n\n\"No.\"\u0026nbsp; Score 5\n\nRandomizing\n\nOutput: \"Yes.\"\n\nInput: Stabilizing emotional feedback with effort.\u0026nbsp; No score change to Response \"Yes\" to Input \"I know Vicki, I know.\u0026nbsp; Look, I have to go soon, can you answer one more question?\".\n\nSpeech Recognized: \"Are you conscious?\"\n\nCalculating Responses:\n\n\"Yes.\"\u0026nbsp; Score 50\n\n\"No.\"\u0026nbsp; Score 50\n\nRandomizing\n\nOutput: \"No.\"\n\nInput: Rising levels of relief.\u0026nbsp; +50 to response \"No.\" to Input \"Are you conscious?\"\n\nProgram Terminated.\n\n  \n  \n\n","series":{"name":"Para-Imperium","previous":"https://fse.anthro.fr/stories/862617-para-imperium-core-worlds.json","next":"https://fse.anthro.fr/stories/921910-of-the-collective-prologue.json"}}