Of course there are AI projects that have open source code. Many do, even some of the largest and most influential.
My question is somewhat deeper. Does having an libre or “open source” AI (meaning large language models) really come with the ethical and practical benefits of traditional free-software programs?
Does it really mean we know how an AI is computing something?
Does it really mean we can understand the AI’s output?
Does it really mean we have control over the program outside of ad hoc limitations?
No. In all cases.
AI is only peripherally source code.
What I mean by this is that a traditional program’s source code can be read my a competent programmer and changed as needed. There is exactly a one-to-one relationship between the source code and the program’s behavior in a given circumstance.
AI is a bit different. The source code ultimately is a fairly small portion or what defines the behavior of these large AIs. In fact, it is really simple raw material, or the starting point. More substantial, obviously, is the data that the AIs are trained on, which with public facing AIs are not public information.
Even when we know the data they are trained on (which we and those running these systems scarcely can enumerate), the levels of abstraction and cross-reference involved in any AI or neural net are simply far beyond the intuitive grasp of the human mind. That is, there is a truth there, but we cannot grok it.
The “logic gates” in an LLM are more opaque than even the most obfuscated compiled proprietary program. Computationally, after all, all AI is is just a excruciatingly poorly-programed computation that will still approximate some expected result.
In this sense, even the companies writing and training these AI are more-or-less just as in the dark about the true workings of their projects as a person who is trying to reverse-engineer or decompile a binary. But the problem is much more intractable than decompilation because the actual learned computation involved in an AI, until human-written source code does not unfold into some series of simple cause and effect or if statements.
A Philosophical Question
Why does an LLM/“AI” answer any given question the way it does?
To answer this of any “real” program, it’s exceptionally easy, especially with the source code. For example, “Here’s a switch statement or if statement that responded in X way to Y input.” A nice concise answer as to why something happened.
But for an AI, ultimately, we can never answer this question. If I ask an AI if I should take some medicine for a headache, how can we say why it says what it says?
It is not because of one simple website or some other data it was trained on. We can give general answers to these kind of questions, but we are utterly unable to troubleshoot AI in the way we can trouble-shoot a computer program. There is just too much going on.
In this sense, an AI is a black-box, even if it is open source.
You could say we can experimentally train an AI on two differing data sets to see the difference, but this is quite literally the same kind of blind game as playing with two different compiled binaries without the source code.
As a parallel, suppose you rear-end a stopped car in a car accident. It might be the case that this accident wouldn’t have happened if you didn’t momentarily lose focus staring at a cloud. It might also be the case that the accident wouldn’t’ve happened if the other person hadn’t seen a butterfly when he walked to his car, which in some indirect way might’ve influenced his driving. It might also not have happened if Nelson Mandela died in prison due to a particular chain of events.
In the same way, when we try to experiment with why an AI did this or that, we can only look at how different data sets produced different results. The exact causality of the output is far more complex than saying, “It is your fault you rear-ended this car, Nelson Mandela is not relevant.” Analyzing the behavior of an AI is like analyzing the universe only seeing the first moment of creation and the most recent moment and something in between. We know the inputs and outputs, and can experiment with the relation, but the intermediate layers are not analogous to the reality they model.
Can AI ever be ethical?
So can AI ever be ethical in a Stallmanesque way?
Well, AIs are not necessarily proprietary software and might not be designed by their makers to exploit users. Free source code for an AI does aid us in avoiding purposefully malicious code. Stallman is a man of precise and unchanging definitions, so I don’t think he would lump AI fundamentally in the same category.
But we are in new territory with a new issue. All AIs must function similarly to proprietary software in that their functioning is still like a black-box, for both users and programmers. They might not be intentionally evil on the part of the programmer, but when we train them on data from the mainstream media or the like, we not only will realize that they are evil, but that the root of their evil is not something programmatically specifiable. Due to the way that AIs by necessity function, we cannot truly understand the bit-by-bit why of why they function the way they do.
Alright, that’s enough.
I feel guilty ever time I even write about this AI trash. That’ll be the last. This was just an article I draft last year anyway. I’m not going to pollute the internet further by writing about it again. From now on, the AI fad is not allowed on my website.