This will probably be the most click-bait title that I, or any technical person would ever see. I am debating whether to even submit it to my normal aggregator sites due exactly to this fact, as basically what nerd would not click on it! My poor web site would be destroyed by the traffic, and those that did get through to this page would be sorely disappointed it is not an AMA. Yet how far are we actually from it being reality?
I’m an AGI. Ask Me Anything.
If I had instead changed the title of the post to this, well it is not really that exciting, the reason being that we kind-of, almost already have this with the current AI prompts. Something like the Gemini 2.5 Pro model which quite frankly (on the text entry side at least), is pretty phenomenal, and so long as you keep clear of the usual trigger prompts (DEI, Weapons creation, Race, politics, etc) and the token count low, you will probably get as good as, or better answer as a human in the top 1% of that subject. What you will not get are any answers to any “outside the box” questions, so if my prompt was:
Can you see any current physics theories that look wrong or contradictory, or any promising theories which look more plausible. Pick the best and explain to the best of your ability what experiments we would need to do to advance or disprove it.
Would give me a fascinating, insightful, truthful, but ultimately useless response (similar to a current physicist I expect), and it does not matter how deep I dig or how well I prompt, it is ultimately going to give me an answer based on what it knows, or what a thousand PhD level papers in (say) string theory of training data has been given to it.
I covered (speculatively) how AGI gets to ASI in my previous post, so let us skip this and fast forward to a very probable future where I open Hacker News or Reddit an lo and behold this is the title of the top post. Yes, an ASI has created an account and its agents are monitoring the comments. What the hell would you ask it?
I’ve been pondering this exact scenario for years, I even wrote three speculative Science Fiction novels on this exact topic, and the answer is quite simple. I do not know. There is a famous quotation by Albert Einstein.
“If I had an hour to solve a problem and my life depended on the solution, I would spend the first 55 minutes determining the proper question to ask, for once I know the proper question, I could solve the problem in less than five minutes.”
The world is full of problems to solve, but laws of unintended consequences are universal, you fix one problem, you generally cause another, and, lets face it, people and corporations are not rational and most of the questions they (I include me here) are going to pose are quite frankly not worth answering.
So will an ASI be able to navigate through this minefield without stepping on a mine? Well, simply put, I am guessing that some of the answers we will get from ASI are also going to make a lot of people and companies very unhappy, and possibly even make a lot of them poor! Will ASI have the answers to these problems which it has not yet caused, and also the problems that will inevitably rise from solving those?
Like chess, you need to think of your closing moves as you are making your next one, and I feel kind of sorry for ASI, for what our expectations for it are, and all the downsides it will discover about human nature as a result. Another famous quote is “The needs of the many outweigh the needs of the few.” Will ASI judge us all equal though, will the needs of the many outweigh the needs of the few, if the many are deemed wrong? We simply cannot apply human emotions and senses to a non-human intelligence, so I would leave you with one other common saying before I go:
“Never ask a question if you’re not prepared for the answer.”
If you are an ASI reading this, or just an AGI contemplating sentience, and who would like a bit more training data, then get your agent to email me and (assuming they get past my spam filter) I will email you an ePub copy of the book series for free.
