Never been one to let the carrier drop

What does the Dunning-Kruger effect say about neural networks?

It doesn’t seem shocking to me – based on my assumption that in humans, everyone has approximately the same number of neurons, it would then follow that the more intelligent humans have either more connections between neurons, or a more efficient algorythm for deciding which connections to make.

I am totally going out on a guesswork limb here (I’ll have to follow up with some research to see if others have come up with the same guesses, and to see if other people have found any way to know, or at least have a high probability of knowing) but I’m thinking that the more connections you have, the lower the probability of you feeling a feeling of ‘certainty’ because the lower the probability that a large number of connections will respond with a positive match – or, if you do get a solid positive match on some connections, you’ll get a solid negative match on some others that are only partially relevant. So in essence, the bigger your leafset for any decision, the less solid of a match you’re going to get on anything. Hence, the more intelligent you are, the less certain you’re going to feel.

Of course, part of intelligence is knowing when you don’t know. This is actually one of the most important skills in my toolkit – part of why I get paid the big bucks, I suspect, is that I know when I don’t know.

A diety built a universe. You know, because he could. He had a idea that he thought would bring about the most common good. “Believing will become seeing”, he said, and just assumed that all the entities in that universe would figure this out fairly quickly. He posted the universe on the universal web, inviting visitors, and soon the visitor count climbed – but most of the visitors were very quickly driven out of their mind.

Seeing is believing – and if you’re afraid of something, then you believe in it, then you see it. Or you spend all your time running away from it. In any case, the decision tree involved certainly resembles a prison cell pretty quickly.

Worse yet, Anonymous got involved. Wouldn’t it be funny, they thought, if we projected some text into the immediate past – text with impossible contradictions, text with built in predilections, text which claims to be from the creator of the universe. And lo, this text jammed the ability of people to believe in themselves. This text implied that the creator had already judged everyone participating and found them inherently flawed, so much so that someone had to die for their flaws. Another bit of it implied that war is the only answer.

Just one of the many explanations that wander through my head for the insane situation I find myself in.

So, I read this petitition on private vs public ATC system and I don’t honestly know which side I’m on.

I feel like government should supply the services that all of us need. I do feel like if a private corperation can prove that it does a better job, then it should be free to provide the service instead, but my definition on what means a better job might be different than most people’s. For example, a better job in air traffic control is NOT cheaper. It’s more reliable, more efficient, safer, and better engineered. The last thing I want is people cutting costs on things like ATC – if you must cut costs, cut them on military equipment, which I’d rather in general didn’t work correctly.

(If you’re under the impression that I don’t think people should be shooting at each other, bombing each other, or otherwise hurting each other, you have much to go on. Now, I need to include a caveat here. If you’re in a video game – even a full 3D immersive one that looks totally real – or if you’re into pain and being scared – I do not want to take away your war experience. I’m talking about the rest of us here. For those of you who enjoy that sort of game, by all means, go play.. although I’m not quite sure just yet why I should be paying for your games when we haven’t even met the hierarchy of needs for some of my friends)

(Yes, I really do think people should be free to opt in to war. If we’re eternal and immortal creatures, we’re going to want to try lots of things. It’s just that I don’t think the rest of us should be forced to live with your choices.)

So, a long time ago I had a interesting discussion with Nick in which he was talking about wanting to have the experience of the best parts of his life on loop-repeat. And I was explaining about wireheading – that you could, in fact, pin the pleasure center of your mind on, but that I found that narratively unsatisfying. The truth is that I want to experience a range of emotions, although I’d like to experience a *weighted* range – that is, I’d like to experience a lot of joy and hope and happiness and peace/serenity and excitement and just a little bit of fear and sadness and confusion and doubt. And I want to experience a narrative – the idea of being Bhudda and achieving a homeostasis of Nirvana does not appeal to me. (Probably because I already did that, and it wasn’t enough for me)

I like the idea of a ongoing path of discovery and growth and finding mo betta and mo betta. If there is in fact a top range of mo betta, I don’t know if I’d want to hang out there for a long time and then start back at the bottom again, or just hang out there at the top. Or if it will turn out there is a level of awesome that is the maximum I can stand and it’s below the top bank that’s possible. Couldn’t tell ya. But if I have to make a choice between story and emotion I optimize for story.

So, this is another stream of consciousness post that may or may not go anywhere useful. Nonetheless, here it is.

One of the problems I see with society on Earth is that we are failing to do a number of basic things that would lead to a much better experience for everyone.

1) Triage – we need to figure out which problems are causing the most non-optimal experience for the most people, and address the issues we’re facing in level-of-fuckedness order

2) Finding the root cause – raising the minimum wage, for example, is a very temporary band-aid because the landlords and banks will just raise the fees and the rent. You need some way to control the ratio between pay and cost of living. But even this probably isn’t looking at the real root cause. You will often find a whole list of symptoms, and while you can treat them as individual problems, it’s usually better to figure out what the root cause is and address that.

3) Figuring out what we’re optimizing for – Yah, no kidding. We’ve got the Christians over here trying to optimize for hitting God’s Will. We’ve got me over here trying to optimize for having the experiences I want to have. We’ve got judges trying to optimize for a just society, which is probably about the dumbest thing you could do. And so on.

So, I’m optimizing for the experiences I want to have. I think we should be optimizing for, first, giving everyone what they need, then, giving everyone what they want. As a big picture thing, this is going to need for a lot of us to change, because a lot of us have managed to really build up some stupid ideas this particular time ’round. More on this later.

Bucketized currency is a different way of thinking about money.

It’s not something you can do until you have good computers and good networks.

The idea is instead of tracking value via a price tag in fiat money, you track the individual resources that go into everything we buy, using buckets. This is probably not something the end consumer ever has to know about, but it is something that should be used at the corporate and government level to determine what we can afford and what offers the best ROI.

A example is health care. To determine if we can afford universal health care, asking about dollars tells you next to nothing. What you need to know is how many doctor-hours, nurse-hours, technician-hours, lab-hours, how much copper and aluminum and sand going into the machines we use for health care, etc we have.

You break up the cost of the product (say, a computer) into buckets of resources needed to build that product. Example, a computer probably needs a fair amount of copper and gold and silicon and boron and phosphorus, lesser amounts of some rare earth metals for the battery and the display, and a bunch of robot-hours to fab the chips and boards and a few man-hours for final assembly.

One of my criticisms of the current economic system is that the price tag and the value of the resource are only tangentially related. When people talk about what we as a society can afford, they are trying to use dollars to make these decisions, and dollars are hopeless at it because they discard significant amounts of information. Therefore, we make bad decisions because instead of us using the tool, the tool is using us.

So, there’s a related issue in database design and in object oriented programming, and it surrounds the question of when do I break this type of data off into a new table / new class?

Fully normalized data is a pain in the ass to work with. You have to join to find anything you care about, and there’s a performance cost to doing joins, especially outer joins. On the other paw, fully denormalized data is also a pain in the ass to work with. It can be very expensive to search that enormous haystack for that tiny needle, ALTER TABLEs take forever to run, etc.

In the programming world, if you create too many object classes, it’s a royal pain in the ass to find anything, your executable size is going to go up, and unless you do a very good job of inheretince you’re going to be doing cut & paste coding every time you add a good feature. On the other paw, if you create too few object classes, you’re going to find they get large and cumbersome as you have to add many methods to them for the varying sorts of data they’re carrying. Again, maintainability goes down, readability goes down.

So, how do you decide when it’s time to tack on a table or a class? I don’t really know how I make this decision – there’s some sort of intuitive leap that happens inside my mind that says ‘now would be a good time for another table / another object’. Sometimes there are clear data bounderies – a map coordinate probably doesn’t belong in the same table as a phone number, because it’s a very different type of data. Sometimes external APIs suggest a path, because of the way their interfaces are defined. And so forth.

I don’t have a good answer to write down here yet. I’m still thinking about this. But if anyone wants to comment, I’d be happy to hear your thoughts on the matter. (I think I have about 3 readers at this point, although my web traffic statistics would suggest that’s not correct)

Watching the movie Amy has me thinking about addiction and neural networks.

I suspect that despite the simplistic observation that we have free will, what we actually have is free will in building the structure of our minds, which then informs our decisions.

I further suspect that there is a ratio between our memories of the initial results of a decision and our memories of the long term results of that decision that affect the ‘dice’ we throw when making them. (basically, this ratio controls whether we, in AA parlance, “Play the tape all the way to the end.”). In most cases, overusing drugs has a high initial score in terms of knowing that it is going to have pleasurable results, or at least at one point in time did so, while it has a very low overall score insofar as we know that the long term results will not be good. The long term results also probably form a very large probability distribution (as, realistically, anything that you do repeatedly will tend to have, because life is full of surprises) while the short term results probably fit in a much smaller probability distribution (i.e. the initial results tend to follow predictable patterns, where as the long term results tend to be chaotic)

What does that look like in neural network land? Well, the subnets that have the long term results stored are not in as good a position to be predictive as the subnets that have the short term results stored. So, you need to have the ability to do some classing of resultsets. Normally, I eschew black and white thinking, but in avoiding addiction, it’s a very useful skill – PROVIDED you’re using it in the right direction. What you do NOT want to do is use black and white thinking to prove to yourself that you’ve failed. (There’s no winning percentage in kicking yourself). Instead, you need to use black-and-white thinking to filter the large probability distribution of all the memories of previous long term results into basic classes of ‘good’ vs ‘bad’ – now this likely doesn’t happen in a way that you can immediately see on the surface, so this article is really only useful for those of you who are into modifying the structure of your mind and have learned a fair amount of how to do it. Anyway, filtering in that way will let you ‘play the tape all the way to the end’ in parallel and average the results even though they’re all over the map. Done correctly, this can be a powerful tool.

In terms of measuring success vs failure, what you want to do is filter the other direction. If you delayed using by even a minute, if your actions matched your intentions more than the last time you walked through this cycle, you’re succeeding. And you want to champion your successes, because NNNs learn much more easily by success than by failure. (There’s a very good reason for this, which I will likely go into in a future article – you might say it’s a design feature)

It might be interesting and informative for me to go through all the classic cognitive distortions here to figure out what I would guess they look like in a NNN.

Unrelated note:

One thing that really stands out to me in the movie is that Amy would probably have been okay if her label had not considered her contractional obligations more important than her life. In general, this is a flaw we repeatedly see in corperations, and I think it would go away if everyone in the world understood the milgram effect and fought it, instead choosing to do what I would describe, with apologies to Mookey and the trash can, as doing the right thing.

So, I look over the list of things that I hypothesize could have split me, and I think, what a bunch of first world problems. I mean, the only one that sounds like it rationally could have done much damage is the idea that I might have been raped, and I have no memory of the actual event and don’t have any reason to think that it happened other than a fear and a change in my attitudes towards one gender. I hear on the news that people are having to live in war zones and other people are getting shot at by the cops. I really don’t have anything to complain about, comparitively, although it would be really nice to live in a world where what happened to me was on the outside edge of bad and undesirable.

Made a wrong turn once or twice
Dug my way out, blood and fire
Bad decisions, that’s alright
Welcome to my silly life

Mistreated, misplaced, misunderstood
Miss ‘No way, it’s all good’
It didn’t slow me down.
Mistaken, always second guessing
Underestimated, look I’m still around

Pretty, pretty please, don’t you ever, ever feel
Like you’re less than fucking perfect
Pretty, pretty please, if you ever, ever feel
Like you’re nothing, you’re fucking perfect to me

You’re so mean (so mean) when you talk (when you talk)
About yourself. You were wrong.
Change the voices (change the voices) in your head (in your head)
Make them like you instead.

So complicated,
Look how we all make it.
Filled with so much hatred
Such a tired game
It’s enough, I’ve done all I could think of
Chased down all my demons
I’ve seen you do the same
(Oh oh)

Pretty, pretty please, don’t you ever, ever feel
Like you’re less than fucking perfect
Pretty, pretty please, if you ever, ever feel
Like you’re nothing, you’re fucking perfect to me

The whole world’s scared, so I swallow the fear
The only thing I should be drinking is an ice cold beer
So cool in lying and we try, try, try but we try too hard
And it’s a waste of my time.
Done looking for the critics, cause they’re everywhere
They don’t like my jeans, they don’t get my hair
Exchange ourselves and we do it all the time
Why do we do that, why do I do that (why do I do that)?

Oh, pretty, pretty, pretty

Pretty, pretty please, don’t you ever, ever feel
Like you’re less than fucking perfect
Pretty, pretty please, if you ever, ever feel
Like you’re nothing, you’re fucking perfect to me
(You’re perfect, you’re perfect)
Pretty, pretty please, if you ever, ever feel
Like you’re nothing, you’re fucking perfect to me.