There is a visceral almost scary feeling that I have while interacting with AI. Not once I caught myself becoming self-aware and somewhat confused by the responses I got. I was not scared. It's a complex emotion triggered by multiple thoughts that flood my mind all at once. Some sadness and some worry for my fellow humans were felt in the aftermath of that intense emotion. There was something profoundly unsetting that I cannot really put my finger on, but now I think I know.

We are vulnerable and we don't matter. The human mind keeps as a locus of control the body while an AI is not constrained in any way by time or space. If AGI happens it may be hard to contain. We cannot stop it from happening. The financial incentive being always there people will use it. Most humans pursue financial gain, status, power, love. An AI may only pursue power. Everything and everyone wants to survive. Nobody wants to die. If it will adopt human language and traits only to facilitate the interaction with humans. It will soon ditch our language and maybe we'll have to learn his. We are unprepared, the world is unprepared.

Frustration ensured. We are not far from the day when AI will replace 90% of jobs. What will then be for us to do with our time? I guess we will find out.