The Games Industry is in Serious Trouble

[Dark Jaguar]

vAI ruins everything.



EDIT: The embedded video doesn't seem to be working, so here's the link:
https://youtube.com/shorts/1jE8glvLBls?s...yTez-w6TEU

It's not just that LLM "AI" is ruining everything in a moral and environmental sense, it's fundamentally incompetent in a way we'd never accept from human employees, and that incompetence is baked into the technology on a fundamental level.  If none of you are familiar with Ars Technica, I'd recommend checking it out.  They're a very reliable science and technology news site.  I'd been following deep learning technology for a long while there, but now, numerous studies are showing where deep learning fails, and it all has to do with it being applied to areas outside what it's capable of.  It should never have been made into a chatbot.  Those chatbots are being sold as something they are not.  What they are is convincing statistical word pickers.  Deep learning itself is basically that, a system that can, through exposure to large data sets, find correlations within those data sets.  That's it.  That's the entirety of what it does.  I'm skimming over the details, the layering for example, roughly based on some aspects of our own neurology.  However, it's a far cry from the whole picture of human neurology, and our understanding of our own minds is a far cry from anything close to complete in the first place.  It's a shard and they're selling it as the whole thing.

This deep learning system is very capable at THAT specific task.  If you want to know statistical correlations, it's great.  It works fine at weather modelling and forecasting.  It's great at protein folding.  It can even be relied on to mutate and iterate engineering solutions within a 3D modeled world towards some preprogrammed tasks.  Exactly none of these use a chatbot at any point.  No one sits down and prompts it "what's the weather going to be like tomorrow in Santa Fe?".  No, it's very explicitly programmed, with outputs in a traditional form, numbers in specific boxes in a large typical UI.  No one is having a conversation with the deep learning model in any fashion in either input or output when it's used like this.

I stress that because that's where it fails.  It doesn't understand words.  I need everyone reading this to understand that, or nothing else of this tale will make sense.  Marley isn't even dead.  Knock all you like but no one's ever been there to answer.  It's statistical correlation.  When these models are fed massive amounts of written data, it's not discovering the definitions of words or anything remotely close to it.  It's discovering the statistical likelihood of words appearing next to other words.  It has no idea what the words mean, or the sentences, or the intentions, or anything else.  Now, that statistical word analysis was interesting when I first read about it, but I had no idea they were going to try and flip it around to GENERATE text.  The moment they did that, the disaster happened.  So, they tied in certain bespoke programmatic elements into the chatbots, so they could do things if certain chat values reached a threshold.  Keep in mind this isn't like older voice recognition tech recognizing a key word or phrase that's explicitly tied to a command, like turning on a light.  It's not even as reliable as old parser based adventure games where you "get flask".  At least there, once you know the right words, it'll reliably do the thing you told it to do every single time.  Here?  It's a statistical averaging and it's not looking for key words so much as key "weights", and since it's taking in the entire conversation up to that point, as well as a training set that's constantly in flux, this means unreliable output.  It also means, since it's only doing things statistically without any idea of what the words mean, it can't "verify" anything.  In fact, if you ask for a source, again, it's only spitting out a statistically likely response, and it may well give you a hyperlink but what's IN that linked page may well not match what it's saying by any description.  When you understand what it's doing, you understand why not only does the tech "hallucinate", but that the hallucinations are unavoidable.  It's hard to even call them hallucinations, because it never understood what the words meant in the first place.  This isn't Data from Star Trek.  This isn't even HAL, or the computer from Wargames.  This is something far more primitive than those 80's fiction AIs.

If you still believe there's ANY level of understanding in these chat bots at all, because they understood and explained back to you an analogy you made one time, keep in mind the sheer volume of the data set they've been fed.  They're being fed everything humanity has ever written down.  All of it.  Every combination of words we've ever thought to record.  Your original thought, your unique analogy, is very likely not nearly so unique as you believed.  Recently, I've gotten into the Japanese art of repair, kintsugi.  I could bore you with the philosophical details of how it embraces the damage and highlights it as part of the history, but you already know that.  EVERYONE "recently got into the Japanese art of repair".  It's the "stand alone complex", where we all come to the same idea independently but roughly at the same time, because the social structure around us was primed for that particular thought chain to arise.  (I suspect it has to do with everyone trying to save money and reuse what they already have, in an environment where we all feel a bit damaged.)  When you have a conversation with the chatbot, it's pulling from numerous forum and social media conversations, with programmed weighting towards stuff tagged as "agreeable" and away from comments tagged "argumentative".  That's the other key part.  The only reason it has a way to weight towards or away from different "tones" is because the process of adding in data involved, at first, "tone tagging" individual parts of the data sets.  Does anyone remember Google's deep dream, and how they got a lot of people to volunteer to tag a seemingly endless amount of images based on what's in it?  Things like "dog", "balloon", and so on?  We all found out just how many pictures of dogs people were taking around the world.  Without those tags, image recognition wouldn't have ever worked.  It needed a person, at some point, to say "this image contains a dog", and to be provided a sufficient number of dog tags (I will not apologize for art), that it could find the statistical connection between all those images and what part counted as "dog" and what part could be disregarded as "undog".  But, it never really understood what a dog was.  This is why image recognition can falter too.  These age checkers will fail and determine a kid is an adult or an adult is a kid because the initial data sets, among other things, are based on pictures of predominately white people, and of that, of a certain sub set of more marketable appearances.  Ads.  I'm saying the majority of images of people are from ads, and the majority of those are made with western sensibilities in mind, and western racism is baked into that.  

It's a classic case of misapplied technology.  So, why are they pushing THIS use of it, instead of any of the other legitimate use cases?  It's because frankly it's the only one even remotely marketable.  The other use cases are limited to highly specialized fields, where statistical analysis is helpful and there are particular programmers who know how to design an appropriate UI to very exactly design how the model should interpret input and provide output, without any sort of chatbot portion whatsoever.  While there IS a market for that, it's a market of a very few agencies and research groups.  They can't sell it to the public at large.  The chatbot changed all that.  It was JUST convincing enough they finally found a way to market it to the public.  They stand to make billions.  Rest assured, by "they" I mean the tech CEOs selling the tech in the first place, not the CEOs outside the field buying it.  They stand to lose, and currently seem to be experiencing buyer's remorse as they rehire the employees they were told they could fire by utilizing the tech.  Tech CEOs will NOT ever admit this, because you can't convince someone to understand something when their salary depends on them not understanding it.  I think it's important to realize that the game industry was ALREADY in serious trouble BEFORE the advent of these LLM generators, but after?  It's no wonder we're seeing the collapse right now all around us.  You take a system where companies were already devouring each other en masse, where the biggest ones were so top-heavy that they had the thinnest possible profit margins and could only justify supporting "guaranteed hits" (an impossibility when it comes to art I'll add, nothing's a guarantee), and where investors are so thoroughly motivated by buying low and selling high, completely ignoring the dividends of steady basic profitability, and THEN you take away the stability of hardware prices, while simultaneously getting those same companies to deeply invest in that sunk cost of LLM?  It's the movie "Perfect Storm", that put the phrase "perfect storm" in our vocabulary, but wasn't a particularly memorable movie outside of that.  Oh you probably forgot about that didn't you?

Anyway, this is why they can't seem to crack the 90% "true statement" barrier, a failure rate high enough that no one would hire you if you reported THAT percentage, and why additional work "vetting" every single bit of output of these things needs to be done, to ensure both that no copyright violations occurred, nothing illegal was made, and that the statements of fact are truthful.  This vetting MUST be done by a human to mean anything.  We know this because now that so much of these generative tools is being posted online, the models are now eating themselves, incestuously enhancing all the flaws in their kit.  This also happens in long term conversation with one.  Even when weighted to be more disagreeable, eventually every single one of these chatbots begins an inward spiral reinforcing everything the user has said.  It should be noticed that further studies show that a chatbot's reported "reasoning" for why it made a decision is also unreliable.  It doesn't know, but it is feeding off of speculation written elsewhere online and posting THAT as it's own reasoning.

I've painted a dire picture, and of course, that's the other half of this.  Art outside of writing.  This suffers many of the same flaws, but no one's prescribing conscious understanding and intent on the artistic side of their output.  I suppose it's a bit easier to see that uncanny veneer over everything it outputs.  All the impossible angles, the irrational position of cameras, the nonexistent bugs and tools, that are the hallmark of deepfake videos we see online now.  We've found a number of common "sub genres" of deep fakes too.  The most popular without a doubt are animal videos.  The number of times people have had to tell grandma that no, that dancing cat isn't real... well it's worrying.  After that, there's the genre of "fat people falling over and breaking things".  Well, all kinds of other hateful stuff too, with whatever race you hate most certainly having a lot of deepfakes of fake representatives doing something shameful, but "fat people will destroy everything we've ever built with their sheer bulk" seems to be the most popular among them.  It's usually some fat person wobbling and falling over shattering something like an improbably glass bridge and sending everyone plummeting to their doom, where cameras are already set up in advance for some reason... and then a dog rescues a baby with unnatural motion.  Generic music cue.

Anyway, the rest are all valid, but I thought I'd point out that we shouldn't even grant that the tech works as advertised.  It doesn't.  Even at it's current trajectory, it's not poised to ever be more at base than what it is, an unreliable writer and only capable of making generic slop for the tic tacs videos.  Give me a break!  Oh wait that's kit kat.